Josh Clark's Blog, page 6

A Protopian Frontier

Take five minutes to watch A Protopian Future, an Ignite talk by Jenny Johnston. She offers a provocation to really think about and describe the future you imagine will come of the things you/we are trying so hard to change right now.

Here, Jenny asks what the world might look like 50 years after nuclear weapons are abolished. Your thing might be something different. You’re probably PUSHING for something to be changed / added / removed in the world; but what future are you PULLING toward? What’s the good and the bad���intentional or unintentional���of the future that you’re designing today?

Protopian stories imagine better futures but not perfect futures. They embrace a kind of messy progress. The reason we’re seeing this protopian surge right now is because humanity is in a weird place. We have this tangle of existential threats in front of us that we’re having a hard time seeing past and certainly our way through…. Protopian stories are powerful tools for reorienting ourselves toward hope and possibility and not dystopian dread.

After you watch Jenny’s video, go check out the farfutures.horizon2045.org project she edited. So great.

A Protopian Frontier

WWDC 2024: Apple Intelligence

John Gruber shares an under-reported tidbit from Apple’s many ���Apple Intelligence��� reveals:

The most unheralded aspect of Apple Intelligence isthat the data centers Apple is building for PrivateCloud Compute are not only carbon neutral, but areoperating entirely on renewable energy sources. That���sextraordinary, and I believe unique in the entire industry.

LLMs are crazy-expensive across many dimensions, including environmental cost. Great to hear at least one company is tackling this head-on.

As usual, John also has lots of other insights on the announcements.

WWDC 2024: Apple Intelligence | Daring Fireball

A Unified Theory of F*cks

The inimitable Mandy Brown reminds us that the f*cks we have to give are a limited resource. Spend them in the right place:

Why love your work? It won���t, of course, love you back.It can���t. Work isn���t a thing that can love. It isn���talive, it isn���t and won���t ever be living. And my answeris: don���t. Don���t give a f*ck about your work.Give all your f*cks to the living. Give a f*ck about thepeople you work with, and the people who receive yourwork���the people who use the tools and products andsystems or, more often than not, are used by them.Give a f*ck about the land and the sea, all the livingthings that are used or used up by the work, that areabandoned or displaced by it, or���if we���re lucky, ifwe���re persistent and brave and willing���are cared forthrough the work. Give a f*ck about yourself, aboutyour own wild and tender spirit, about your peace andespecially about your art. Give every last f*ck youhave to living things with beating hearts and breathinglungs and open eyes, with chloroplasts and myceliaand water-seeking roots, with wings and hands and leaves.Give like every f*ck might be your last.

Because here���s what I���ve learned: if you give yourf*cks to the unliving���if you plant those f*cks in institutionsor systems or platforms or, gods forbid, interest rates���youwill run out of f*cks.

A Unified Theory of F*cks | A Working Library

Illuminate

Illuminate is an experimental project from Google that generates accessible, podcast-style interviews from academic papers:

Illuminate is an experimental technology that usesAI to adapt content to your learning preferences. Illuminategenerates audio with two AI-generated voices in conversation,discussing the key points of select papers. Illuminateis currently optimized for published computer scienceacademic papers.

The service has a waitlist, but you can try out some generated conversations (and I recommend that you do!). The enthusiasm, intonation, and ums & ahs are convincing and feel authentic to the genre that the project mimics. (See also the PDF to Podcast project which does similar things but with flatter voice results.)

But it’s not the seeming authenticity that feels important here. Machine-generated voices���even at this level of fidelity���are nothing new. What’s more interesting is how this project demonstrates what large language models (and now large multimodal models) are truly great at: they are prodigious translators and transformers of symbols, whether those symbols are for language, visuals, or broad concepts. These models can shift those symbols nimbly among formats: from English to Chinese to structured data to speech to UI components to audio to image. These are systems that can understand a concept they are given and then work their alchemy to present that concept in a new medium or language or format.

There are exciting opportunities here for unlocking content that is trapped in unfriendly formats (where the definition of “unfriendly” might be unique to the individual). This application leans into what generative AI is good at (understanding, transforming) around tightly scoped content���and avoids what these models are uneven at: answering questions or building content from scratch. How might this kind of transformation support education efforts, particularly around accessibility and inclusivity?

Illuminate

I Will F***ing Piledrive You If You Mention AI Again

Has breathless AI mania primed you for a ragey rant about overhyped technology and huckster pitchmen? Nikhil Suresh has you covered:

Most organizations cannot ship the most basic applications imaginable with any consistency, and you’re out here saying that the best way to remain competitive is to roll out experimental technology that is an order of magnitude more sophisticated than anything else your IT department runs, which you have no experience hiring for, when the organization has never used a GPU for anything other than junior engineers playing video games with their camera off during standup, and even if you do that all right there is a chance that the problem is simply unsolvable due to the characteristics of your data and business? This isn’t a recipe for disaster, it’s a cookbook for someone looking to prepare a twelve course f***ing catastrophe.

How about you remain competitive by fixing your shit?

There���s no such thing as a quick fix for a broken organization. And there���s no silver bullet for product excellence. AI is capable of amazing things, but you can���t shortcut great execution or ignore its very real downsides.

In another context, I often say, ���High-performing teams have design systems, but having a design system won���t make you a high-performing team.��� The same is true for AI.

There���s only one route to success: get your process and operations in order, understand the technologies you���re using, know their strengths and weaknesses, and above all: start with the right problem to solve.

I Will F***ing Piledrive You If You Mention AI Again | Ludicity

The Future is Built on Solid Foundations

This article was originally published at bradfrost.com.

Looking for help? If your organization needs help integrating AI into your design system operations or making your digital production more effective, that’s what we do! Get in touch.

If you feel like you���re struggling to keep up with the pace of technological change, you���re certainly not alone. We feel this pressure at an individual level, but also at an organizational level. How do digital organizations establish solid, stable foundations while also responsibly exploring the future?

Organizations with solid foundations in place are better equipped to adopt emerging technology. This isn���t particularly revelatory; a person standing on a stable, tall structure peering out over the horizon can better anticipate what���s coming than a person flailing around on the ground trying to extinguish a grease fire. But the fact remains: the sturdier an organization���s critical front-end infrastructure is, the better positioned they are to explore the future.

At Big Medium, we���ve spent more than a decade helping organizations to establish sound foundations while also helping them adopt emerging technology. There���s a ���stability vs innovation��� dynamic that can sometimes feel like competing tension, but we���ve found that stability and innovation are more complementary than competitive.

Design system success is more than assets

High-performing teams have design systems, but having a design system won���t make you a high-performing team.
���Josh Clark

Merely having design system assets in place doesn���t mean it will succeed. It���s the successful orchestration of people, processes, technologies, and assets that really makes this stuff hum.

A design system is more than its kit of parts. People and process are where those assets succeed or fail.

While the assets are important, they���re merely one part of a broader picture. It���s the human systems and the processes by which humans wield technology that overwhelming make or break a design system effort. What we���ve learned is that those same human systems and processes are what truly matter when it comes to innovating and adopting new technologies.

Rounding the corner from building to wielding

It feels like there���s an urgency to get solid foundations in place ��� and fast! ��� because there are bigger fish to fry. AI exploded onto the scene and sucked the air out of the room. While we���re still no doubt in the AI hype cycle, its emergence has lit a fire for teams to get their badges and text fields in order so that everyone can move onto other important initiatives.

Thankfully, AI can help streamline many aspects of design system asset work, freeing up teams��� time and energy to do the important human work that is critical to a design system���s success. Just so it���s said: creating a design system���s assets has never been the end game, but rather an important foundational layer for other important work to stand on.��What you do with the design system is where the excitement and focus should be. Whether it���s the ability to blast out higher-quality work faster than ever, explore new technologies, or spend more time building relationships, the opportunities are many. We want those opportunities for everyone!

Stability and innovation go hand in hand

This week our team ran an AI and Design Systems virtual session where we showed off how we���ve been integrating AI into many aspects of our design system work. When Kevin and Ian showed off how LLMs can help generate and translate component code, people reacted with ���Wow! How are you getting such consistent results?��� The answer is: with solid design system architecture and conventions in place. We���ve been sweating the details and iterating over our design system architecture for over a decade, and we have our conventions consistently implemented and well documented.

We���ve all heard garbage-in, garbage-out when it comes to AI, and the inverse is also true. It turns out that high-quality input yields high-quality output, so the sturdier the design system foundations are the better AI-powered results are going to be. To me it feels like a continuation of future-friendly thinking; there are things we can do today that better prepare us for an unknown future.

Learn more in Paris!

If any of this resonates with you, I���m happy to announce that I���ll be talking about this and more at a Wine and Cheese event (!!!) in Paris the evening of June 26, 2024. And the very next day on June 27, I���ll be delivering a full-day design systems masterclass where I���ll dive into all of the foundational concepts and best practices around creating and maintaining a successful design system.

If you���re in or around Paris, I hope to see you at one (or both!) of these events. We���re at an exciting ��� and disruptive! ��� time in this field, and I���m tremendously excited for the discussions ahead.

Say Hello to Sentient Design

This essay is part of a series about Sentient Design, the already-here future of intelligent interfaces and AI-mediated experiences.

The book

Sentient Design by Josh Clark with Veronika Kindred will be published by Rosenfeld Media.

The talk

Watch Josh Clark's talk ���Sentient Design���

Need help?

If you’re working on strategy, design, or development of AI-powered products, we do that! Get in touch.

What does it mean to weave intelligence into an interface? How do we add AI to digital products in ways that are at once meaningful and responsible? What is the grain of this new design material, and what are its strengths and weaknesses? What are the opportunities and the risks, and how do we balance them?

Hey, those are hard questions! We���re finding ways to answer them with a practice we call Sentient Design.

Sentient Design is the already-here future of intelligent interfaces: experiences that feel almost self-aware in their response to user needs. Sentient Design moves past static info and presentation to embrace UX as a radically adaptive story. These experiences are conceived and compiled in real time based on your intent in the moment���AI-mediated experiences that adapt to people, instead of forcing the reverse.

Sentient Design describes not only the form of this new user experience but also a framework and a philosophy for working with machine intelligence as design material. As our interfaces become more mindful, so must designers.

And! Sentient Design is also a book�����or will be soon. Rosenfeld Media will publish Sentient Design: AI and the New Digital Experience by Josh Clark with Veronika Kindred. Look out for it late this year or early 2025. We’ll get it into your hands as soon as we can!

This is so much more (and in exciting ways, so much less) than chatbots, text prompts, and systems that can write, speak, or draw. In a nutshell, Sentient Design is all about the new opportunities to add intelligence to our interfaces (sometimes just a dash, other times a heaping spoonful). Some of this is so familiar that you might consider it nothing special (content recommendation or prediction), but there are lots of surprising new possibilities here, too. It���s all part of our kit as designers now���and especially powerful in combination. Sentient Design explores all of it.

This essay is a high-level introduction to Sentient Design���s core concepts. To dive straight into the details, check out Josh���s lively Sentient Design talk, which introduces lots of examples and design principles. The talk is available in video, audio, slides, and prose.

You can also keep an eye on our series of Sentient Design articles and talks for detailed examples, design patterns, and strategies. Sign up for our newsletter A Little Big Medium to keep up with all of it, including news about the book.

Why Sentient Design Matters

All around us, there���s something that you feel more than see, an undercurrent that swirls through your everyday. It���s the influence of a million different algorithms supporting, nudging, cajoling… quietly working beneath the surface of nearly every digital experience���and many physical ones, too.

This eddy and flow of machine intelligence infuses everything, serving us in countless ways, often invisibly. More recently, these systems have become more visible partners; everyday folks now interact directly with AI through chat or other experiences. Much of the time, all of this is for the good. Steady leaps in machine learning and AI have made possible a slew of near-magical products and services that ease and improve our lives in ways large and small. The opportunities are everywhere and growing.

But so are the risks and failures. Flaws and mistakes range from the comically trivial (looking at you, autocorrect) to the deadly serious. AI systems have wrecked lives with biased prison sentencing, bungled medical diagnoses, and plane-crashing autopilots. At an even larger scale, we���ve seen AI undermine faith in entire political systems by automating the broadcast of propaganda and hate speech with both a scale and targeted focus that are unprecedented. Others use machine learning to manipulate or monitor us in less profound but equally cynical ways for advertising, surge pricing, surveillance, and more.

These are not entirely or even mostly technology problems. Instead, they reflect a still-developing understanding of how to put AI to work in ways that are not only useful and valuable, but also respectful and responsible.

Data scientists and algorithm engineers have revealed the possible; now design and other fields can join to shape that potential���and reveal the meaningful.

For all of its pervasiveness, our use of AI and machine learning is still in its awkward adolescence. Flashes of maturity mingle with stunning lapses of judgment, and boastful confidence gives way to naive misunderstanding���and sometimes dangerous behavior. These adolescent systems are still gawky and undisciplined, but the same could be said about our approach to building them.

Sentient Design aims to help all of us move through our growing pains and into this next chapter of experience design. That means helping product organizations grow, too. How should makers and managers craft products and services in the era of the algorithm? What shape should those experiences take? How can you take advantage of the powerful opportunities of AI while guarding against its risks and failures? What are the meaningful problems you can solve and new questions you can ask with these tools in your kit?

These are design challenges, not technical. Data scientists and algorithm engineers have revealed the possible; now design and other fields can join to shape that potential���and reveal the meaningful.

What are Sentient Design experiences made of?

Sentient Design refers to intelligent interfaces that are aware of context and intent so that they can be radically adaptive to user needs in the moment. Those are the fundamental attributes of Sentient Design experiences: aware and radically adaptive.

Those core attributes are supported by a handful of common characteristics that inform the manner and interaction of most Sentient Design experiences:

Collaborative. The system is an active (often proactive) partner throughout the user journey, often with independent ability to perform tasks on your behalf.Multimodal. The system works across channels and media, going beyond traditional interfaces to speech, text, touch, physical gesture, etc.Continuous and ambient. The interface is available when it can be helpful and quiet or invisible when it can���t.Deferential. The system suggests instead of imposing ; it defers to user goals and preferences. It offer signals, not answers. Sentient Design describes intelligent interfaces that are aware and adaptive���and frequently collaborative, multimodal, continuous & ambient, and deferential.

Note that ���making stuff��� is not included in this list���not explicitly, at least. Writing text, making images, or generating code might all be the means or even the desired outcomes of Sentient Design experiences���but they���re not defining characteristics.

The fundamentals of Sentient Design aren���t new. Some aspects of Sentient Design have been around for years or decades���long enough that they���ve become familiar design patterns. We encounter automated services hundreds of times per day that deliver bespoke experiences based on automated recommendation or prediction.

The commonality is that these services are aware and adaptive in presentation. Streaming music services are aware of our preferences and history, as well as what���s new and popular in the mainstream, and adapt content and presentation; the result is a playlist (or several!) that suits our tastes, or maybe just the moment. Similarly, predictive keyboards are aware of what you���re typing, along with the historical probability of the most likely next word or phrase; they adapt the keyboard in anticipation of what we���ll type next.

These kinds of just-in-time content presentation are familiar enough to seem commonplace, but they remain part of the Sentient Design toolkit alongside the fancy new stuff like generative AI and large language models (LLMs). As the underlying tools become more capable, awareness and adaptation grow more capable too. For example, AI-powered assistants can understand any question and come back with an appropriate (and sometimes even correct!) response. Talk about radically adaptive: you can craft a whole meandering conversation that is absolutely one of a kind. It���s an experience that totally bends to your want and need in the moment���indeed, it���s invented for that moment.

What���s also new is that Sentient Design experiences anticipate our needs based on models of what we have done in the past, individually or collectively. They propose paths forward, show us new possibilities, caution us about blind spots, or maybe just take care of simple tasks. These intelligent interfaces can also come to us instead of the reverse, shape-shifting into the form and interaction that is most convenient. In some cases, that may mean reformatting content or structure on a screen, or changing the UI metaphor entirely. More ambitiously, the AI-powered ghost in the machine may flit into a different machine entirely���shifting the point of interaction on the fly from screen to earbud to hand tool to appliance to car cockpit and back again.

New experiences, new perspective

This isn���t exactly ye old website anymore! While Sentient Design has an important place in traditional interface design, too, it introduces a very different experience to conceive and manage than the relatively static experiences of forms and dialog boxes that many digital designers have known so far.

In this new kind of experience, everything can potentially adapt on the fly. The content, the format, the mode of interaction, the point of contact, the tone and aesthetic, even the very goals of the system… All of these things can shift and adapt in a new kind of nimble collaboration with the people it serves. The tools and design patterns are here for us to use now.

It���s exciting and delicate and maybe even a little weird.

It is not as weird as full-blown consciousness. We���re not talking about machines with emotional feeling, moral consideration, or any of the hallmarks we might ascribe to a fully sentient being. The ���sentient��� in Sentient Design describes a combination that is far more modest but still powerful: awareness, interpretation, agency, and adaptation. Machine learning and AI can already enable all of these attributes in forms that range from simple to sophisticated.

This is a continuum. Think of it as a dial that you can turn: from simple utilities that add sparks of helpful intelligence to humble web forms to more capable companions like agents or assistants. Those experiences are comfortably attainable with Sentient Design and today���s technologies.

The opportunity is to begin thinking of machine-learning applications in the same way that you think of using JavaScript to layer interactivity into a webpage. Or how you use responsive design techniques to add flexibility and accessibility to an experience. Just like we���ve absorbed those techniques into our daily practice, we can do the same with Sentient Design. Machine learning features are just another new technique and applied technology, this time to add intelligence and awareness to our interfaces.

We can add that spark of intelligence anywhere it can be helpful. Apply Sentient Design���s awareness of context and intent, using things like recommendation, prediction, classification, clustering, and generation to elevate humble web forms or create entirely new kinds of experiences. What data do you have that can anticipate next steps and help people move more quickly through challenging or mundane tasks?

Grounded in solid UX principles

This is a lot���and it comes with new challenges and dangers. We invite trouble when we abdicate too much responsibility to intelligent systems, or when we misrepresent their statistical signals as facts or absolutes. It takes new perspective and new technique to realize the opportunities of AI while contending with its risks.

Instead of designing for success���for the happy path���we have to do more to anticipate failure and uncertainty. Sentient Design experiences allow AI and machine learning to mediate experiences in ways that are new and that sometimes take the designer out of the loop. We have to anticipate how and where the system is unreliable���where the system will be weird and where the human will be weird.

We invite trouble when we abdicate too much responsibility to intelligent systems.

Sentient Design introduces several principles for defensive design. How do we anticipate and manage the imperfect or frankly bizarre results that AI often delivers? How do we design the experience to set expectations and channel behavior in a way that best matches the system���s ability? And perhaps most important of all: how can Sentient Design experiences engage critical thought, or amplify human judgment and agency, instead of replacing them?

Sentient Design is about AI… but also not. While AI and machine learning are the enabling technologies, the goal is not to ���make AI products.��� Sentient Design lets us deliver meaningful human outcomes in ways that haven���t been possible until now. The intended outcomes themselves may not change���our creations should remain focused on human goals���but Sentient Design can collapse the effort it takes for people to realize those outcomes, and may even make some outcomes plausible, affordable, or attractive for the first time. Along the way, Sentient Design techniques can make experiences more valuable, more intuitive, and often more fun. That���s the real opportunity; the technology is only the means to the end.

How to learn more

We���re working hard to share the details of how all of this works in practice. Here are some resources to help you stay current.

Watch or listen. Check out Josh���s Sentient Design talk, which introduces lots of Sentient Design examples and design principles. The talk is available in video, audio, slides, and prose.

Read up. Our ever-growing series of Sentient Design articles and talks offer perspectives, examples, and design patterns. Sign up for our newsletter A Little Big Medium to keep up with all of it, including news about the book.

Workshops and Sentient Design sprints. Big Medium offers educational workshops about Sentient Design and the UX of AI. We also work with clients to conduct Sentient Design sprints���sessions for product designers to identify, frame, and validate applications of machine intelligence. Get in touch for details.

Product design engagements. Big Medium does lots of product design, and Sentient Design is a big part of our toolkit. If you���re considering AI-powered products or features, we can help. Get in touch.

A.I., Snake Oil, and Miracle-Cure Expectations

This essay is part of a series about Sentient Design, the already-here future of intelligent interfaces and AI-mediated experiences.

The book

Sentient Design by Josh Clark with Veronika Kindred will be published by Rosenfeld Media.

The talk

Watch Josh Clark's talk ���Sentient Design���

Need help?

If you’re working on strategy, design, or development of AI-powered products, we do that! Get in touch.

Turns out snake oil is actually good for you. The stuff is high in antioxidants, and it���s a potent anti-inflammatory. These modest health benefits do not mean that snake oil will cure cancer, regenerate lost limbs, or reverse aging. As always, be distrustful of folks hyping any solution as a cure-all���but don���t assume it’s useless, either. We might just need to come at it from a different angle.

AI has a snake oil problem. Generative AI has delivered an extraordinary leap in capability that is often jaw-dropping. But it���s sold and applied (by giant companies with vested interests) as something that can answer any question, do any job, or fill any role. When AI fails to live up to these outlandish promises, there���s a justified backlash���loss of trust, accusations of hucksterism, and wholesale dismissal of the technology.

Pushers of the AI miracle cure are ���promising God and delivering email prompts,��� Signal president Meredith Whittaker said last week. The high expectations of the soapbox spiels are a mismatch for the relatively humble capabilities that have been delivered so far. That���s too bad, because we���re missing the genuine utility of this latest crop of machine-learning tools by burying it in hype and poorly considered applications. What AI is reliably capable of is genuinely remarkable; yet our expectations are routinely jacked up beyond reason.

���Imagine having products THIS GOOD and still over-selling them,��� writes Drew Breunig in a plea for ���sober AI.��� ���The end result is we train users to dangerously trust whatever AI slop they���re presented or we train them to dismiss the whole field.���

This is not (only) a matter of marketing or hubris���it���s a design problem. The job of UX is to identify problems worth solving, pick the right technology (if any) to solve that problem, and then set expectations and channel behavior in ways that match the system’s capabilities. It requires a clear-eyed understanding of what the technology is good at, what it is not, and where it fits.

That’s always the job of design. But setting expectations and channeling behavior is more important than ever in cases like AI-mediated experiences where designers are not always directly in the loop. Let���s look at a couple of examples.

AI overviews are a good idea, poorly presented

When Google launched AI overviews last month, the feature got tripped up by embarrassing recommendations to add glue to your pizza or to eat rocks to improve your health. In addition to a few giggles, the gaffes contributed to a growing consensus that AI just isn���t good enough���and that Google miscalculated by shipping the feature.

Google is dead beyond comparison pic.twitter.com/EQIJhvPUoI

— PixelButts (@PixelButts) May 22, 2024

Here���s an unpopular take: AI overviews are a good idea and a worthwhile application for large language models (LLMs). But! Their current iteration also has a design problem: the presentation suggests confidence that the system is not designed for and that the underlying data does not support.

Here���s the good-idea part. There���s an LLM under the hood, but it���s not what���s providing the answer; LLMs are unreliable answer machines, so this is a good call. The LLM instead interprets the request, does the search, and then reads/synthesizes the top results. The good ol��� Google search engine still delivers the raw-material results, and the LLM merely rounds up what���s in those links. Like it says on the tin, it���s an ���overview������a layer on top of Google���s traditional ten blue links. The idea makes sense, and it���s well suited to what LLMs are good at���interpretation, synthesis, reformatting. The goal is to save you the effort of clicking through the ten blue links to tell you what���s inside. (The fact that this will throttle traffic to the sites that source the traffic is an important and separate problem.)

Alas, the way the AI overview is presented makes it seem like ���the answer,��� and that���s not what this search-results summary is. The promise does not agree with what���s delivered, and the mismatch delivers a broken experience. The just-the-facts result appears to prescribe glue for your pizza, when in fact, it���s giving you a book report of the ten blue links���one of which is the jokey Reddit thread that suggested preventing cheese slippage by adding glue to your pizza or by hammering nails into the crust. If there���s a data problem here, it���s with Google���s ten blue links. The AI overview is technically correct based on the job it was given: It delivers nothing substantively different from the ten links, and yet its presentation changes everything.

I wrote about this years ago in my essay Systems smart enough to know they���re not smart enough. Too often, we present machine-generated results with more confidence than the underlying system reports. We rush to present the results as one true answer when the data and results don���t support it. This is a kind of design dishonesty about the quality of the information���or at least an over-optimism in what we can confidently share with users.

The design challenge here is: what language or presentation could we use to suggest that the AI overview is only peeking into the top results for you? How can we be more transparent about sources? How and when should design or UX copy engage the user���s skepticism or critical thinking? How can AI systems handle humor, satire, and sarcasm in search results?

The takeaway: machine learning and AI are great at providing signals and suggestions, but unreliable (and sometimes flat-out awful) at providing facts and answers. Adjust your presentation accordingly. Acknowledging ambiguity and uncertainty is essential to designing the display of trusted systems. There���s real value to be provided here, but there���s hard design work yet to do.

A mind-bending demo dismissed

If incomplete or inexact results can still be useful with proper presentation, the same goes for incomplete process or artifacts. Rather than overpromising what partial solutions provide, let���s think smaller. Let���s see how machine-generated results can improve our work instead of replace it.

Last week, Eric Vyacheslav shared this demo on LinkedIn, writing ���GPT–4o is going to completely change the way we design. This is GPT–4o generating Figma designs based on PRD (product requirements document).���

~~

The comments are full of folks dismissing this demo out of hand: This isn���t real design. There���s no brainstorming. There���s no iteration. There���s no discussion. The result is generic and allows for no innovation.

Friends! Hang on a second. Let us start by acknowledging: this demo is freaking amazing. No details were provided about the quality of the PRD, or what tool was used to generate the results, and those things matter, of course. But going from requirements document to first-draft Figma file in 30 seconds is astonishing and, with the current crop of tools, entirely plausible. A year ago, our brains would have melted seeing this.

This is clearly not a replacement for the entire design process, but it���s not hard to imagine that this new capability���instant draft from a requirements doc���could somehow improve some stage of our process, or empower new audiences to share design ideas. Instead of dismissing the demo with ���no but��� why not respond with ���yes and���? How might a tool like this help our process or include a new audience?

The snake oil problem here is the assumption that this technology would replace the whole design process���click a button and done! Instead, it���s much more powerful to think smaller. What part of the design process could this it ease, even with a clumsy draft? What new conversations could it enable? How could it make design more inclusive?

The wonderful Scott Jenson swooped in to poke at the demo.

Sorry, I feel like such a wet blanket these days. This demo shows promise to be sure but it’s all the ‘little stuff’ that it conveniently doesn’t show:

PRDs are always inconsistent, incomplete, even flat out wrongWhat is the quality of the Figma output? Does it layer things well, create semantically meaningful groups, use auto layout, create components?If I make changes to the Figma file and the PRD changes, what happens?

None of these are ‘gotchas’ that kill the concept. They are just likely pain points that aren’t addressed by this demo, so I need to know more. By producing such impoverished output, this could actually place MORE burden on the designer as they have to clean up that mess to productize it. ���

Just like people freaked out when ChatGPT spoke like a pirate, they are equally freaking out when they see ANY output into Figma. It’s impressive, it’s a great start, it may eventually even work. But this is a complex problem and a demo like this isn’t nearly enough. I want us to be more nuanced in talking about problems like this.

Scott framed his response as being ���a wet blanket,��� but I see his comment as genuinely productive ���yes and��� thinking. He acknowledges that this demo is a promising first step���but it���s the beginning, not the end. It���s an opportunity to ask what���s next and what is this for. What are the barriers to using this? What needs to change to make it useful? Where does it sit in our process? What change do we hope to accomplish with it? What could go wrong? These are all excellent design and UX questions as we learn to use this new design material.

Reading between the lines of Scott���s response, it���s easy to see his fatigue about AI boosterism and the rush to say, ���see, AI solved it!��� He���s sick of the breathless enthusiasm and the snake oil pitch���but he also sees the undersold actual value of the medicine. Let���s slow down and see what this medicine can actually do and what it���s good for.

Putting snake oil to healthy use

There are options between swallowing snake oil promises and rejecting the medicine outright. As we adjust our expectations to what���s viable and reliable, here are some questions to ask:

What���s the problem to solve? Before rushing to apply AI to every little thing, let���s make sure we���re clear on the outcomes we���re trying to achieve.

Is it the right medicine for the problem? Is generative AI the best solution for what we���re trying to do? Would other, simpler machine-learning tools work better���or maybe no machine learning at all?

What do you take with the medicine? If AI won���t solve the whole problem, what other steps, process, communication, or technologies should be paired with it?

Would the medicine be more helpful for a different population? If an AI-powered solution doesn���t add much value for a certain group, would it be helpful for another one? Does it empower people with less domain experience? Does the Figma-building demo hint at an opportunity for product managers or other stakeholders to participate in design conversations in a new way? Instead of delivering design, what if we think of it as a way to validate our requirements?

Do you trust the doctor? Who���s pitching the medicine, and what do they have to gain from it?

Is the cure worse than the disease? What���s the cost or risk of using AI here? LLMs are expensive in a lot of ways (economically, socially, environmentally, organizationally); does the benefit outweigh the cost, and for whom?

Is there a warning label? If there are risks, what���s the best way to convey them? How and when do we engage the user���s productive skepticism and critical thinking?

Sentient Design is intentional design

Sentient Design is the already-here future of intelligent interfaces���experiences that seem almost self-aware in their response to user need in the moment. Veronika Kindred and I are writing a book about it. While Sentient Design describes the form of this new experience and what AI unlocks, it also describes a framework and philosophy for using it. As we make mindful interfaces, designers have to be mindful, too.

Sentient Design is about AI, but also not really. It���s about identifying worthwhile problems and seeing where this new crop of AI tools can enable meaningful solutions. There’s so much opportunity and good work to be done. Let���s prove what it can do before we rush to call it a revolution. Let���s not get high on our own supply.

Me, I feel the full mix of excitement, curiosity, creativity, skepticism, and anxiety about all of this. I���m letting myself feel all of those things as I explore these new tools, make new stuff, and see what they���re good and bad at.

There are two things I feel confident about. First, AI will continue to introduce us to new ways to screw things up, like any technology. And powerful interests will use AI to screw things up in ways they expect will increase their power or profit, like any technology. And we���re seeing those things happen. We can���t be pollyanna about the ways that these things can go sideways or be used for ill. People will use these things naively, cynically, carelessly, and often all three at once.

Second, I���m also confident that the anti-patterns we���re seeing aren���t the only way to use this stuff. There are a lot of good and meaningful things to create here. We���re still learning the grain of this design material and how it wants to be used. What new opportunities does it unlock, and what benefits can deliver���and at what cost? And is that a cost that is worth bearing?

Even as I hold my nose about some of its applications, I���m learning from what those things can teach us. The best response to failed experiments is not, ���What a dumb idea, there���s nothing here.��� For what it���s worth, I prefer: ���What can we learn from this? What���s next?���

I���m mighty excited about Sentient Design and AI-mediated experiences, even as (and maybe especially as) we learn from our mistakes. Let���s just, you know, make sure we actually do learn from them. That means distinguishing the snake oil from its pitch.

Is your organization trying to understand the role of AI in your mission and practice? We can help! Big Medium does design, development, and product strategy for AI-mediated experiences; we facilitate Sentient Design sprints; we teach AI workshops; and we offer executive briefings. Get in touch to learn more.

Kevin Coyle

Kevin Coyle is technical lead at Big Medium. He brings a blend of mad-scientist enthusiasm and enterprise-ready discipline���a potent mix as he helps our client companies metabolize emerging technology, including AI and machine learning.

In addition to AI, Kevin brings long experience in enterprise solution design, enterprise software, and test-driven development. He���s passionate about aesthetics, creativity, and innovation���and he loves doing all of those things with others. Kevin enjoys working with diverse and talented teams to make projects successful.

Kevin is founder and director of Drutek, a digital agency specializing in Drupal web development. He is an Acquia Certified Developer.

Kevin is also a prolific creator of new products and side projects, including Roast My Desk, Really Focussed, and an unpleasant AI assistant named Foul Susie.

Kevin lives in Manchester, England. When he���s not training our future robot overlords, Kevin is training his beagles���or tinkering with his narrowboat somewhere in the British canal network.

Stay in touchkevin@bigmedium.comLinkedIn

Veronika Kindred

Veronika Kindred is associate producer at Big Medium, where she defines and solves design problems alongside some of the world���s biggest companies.

Veronika is co-author with Josh Clark of Sentient Design: AI and the New Digital Experience, the forthcoming book from Rosenfeld Media.

Veronika graduated with high honors from New York University with a major in Politics and a minor in Data Science���a useful combination for navigating both organizational and technical design challenges. Her research projects have included building clean data sets from congressional hearings on climate change, studying the effects of mobile technology on African political engagement, and exploring how AI has impacted user experience.

Veronika also has a degree in photography from Fashion Institute of Technology. Her photographs have appeared in O, The Oprah Magazine and in UNWomen.org, the United Nations organization for women���s rights.

When she���s not wrangling Figma files and thorny research questions, Veronika wrangles (very) young dance students as a weekend ballet teacher in Brooklyn. In both dance and design, she enjoys bringing discipline, grace, and an air of lightness to her work.