Creative Selection: Inside Apple's Design Process During the Golden Age of Steve Jobs

by Ken Kocienda

What’s the best size for a home screen icon? Scott Herz, one of my Purple teammates, soon gave us the answer. He wrote an app and circulated it around the Purple team. There wasn’t much to it. The app launched showing a very large Start button. After tapping that button, the screen would go blank for a moment, then a box would appear somewhere on the display. The goal was to tap the box. After you tapped, whether you succeeded or failed, and after another momentary blank, another box would appear somewhere else. Only this next box would be a different size, maybe larger, maybe smaller. Tap the box. Tap the box. Tap the box.

After twenty or so boxes and taps, the “game” would end, and the app would show you your score: how many boxes you hit and how many you missed. Behind the scenes, the software tracked the sizes of the boxes and their location. Since it was a fun game to play … ahem … a serious test program to gather essential touchscreen usability data, the Herz tap app made a quick round of the Purple hallway. Within a few days, we had quite a bit of information about tap-target sizes and accuracy.

I refer you to United States Patent 7,479,949, sometimes called the ’949 Patent when dealing with Apple lawyers, or just the iPhone Patent when not. Its formal title is Touch Screen Device, Method, and Graphical User Interface for Determining Commands by Applying Heuristics.7 This document is Apple’s official statement on the novel software features and functions on the original iPhone, a 358-page patent that is dense with diagrams, embodiments, and claims. This filing aimed to provide an exhaustive rundown of the multitouch user interface, with sections delving into the nitty-gritty of numerous specific interactions.

However, it’s crucial to make the right call about whether to use an algorithm or a heuristic in a specific situation. This is why the Google experiment with forty-one shades of blue seems so foreign to me, accustomed as I am to the Apple approach. Google used an A/B test to make a color choice. It used a single predetermined value criterion and defined it like so: The best shade of blue is the one that people clicked most often in the test. This is an algorithm. At Apple, we never considered the notion of an algorithmically correct color. We used demos to pick colors and animation timings, and we put our faith in our sense of taste. When we went looking for the right “finger movement in accordance with a rule,” as mentioned in the excerpt I included from the ’949 Patent, we made a subjective call. We developed heuristics.

When we got an idea, we cobbled together a first cut on the algorithms and heuristics we would need to illustrate it. Then we pulled together the supporting resources—code, graphics, animations, sounds, icons, and more—to produce a demo. After we showed the demo and shared some feedback with each other, we made decisions about changes that might be an improvement. Many times this came down to tuning some heuristic, or modifying how an algorithm and heuristic combined. Whatever it was, the concrete and specific modifications we chose to make led to the actions items that justified making the next demo. Repeat, then repeat again. Doing this over and over again set our projects on the slow path to accumulating positive change. This is how we started with an idea and finished with software for a product.

After Steve died, the Apple software development culture started to change.