Recoding America: Why Government Is Failing in the Digital Age and How We Can Do Better

by Jennifer Pahlka

Prop 64 made hundreds of thousands of people statewide eligible to have their felony convictions expunged. But given the marathon of logistics, hardly anyone even tried to start the process. A year after the passage of the law, the San Francisco district attorney’s office had a whopping twenty-three petitions to seal marijuana-related records. Not even twenty-three expungements: twenty-three people who’d gotten as far as filing the initial paperwork.

Suddenly, lawmakers in many states were reminded that their state-run systems for administering unemployment benefits were sorely out of date and called for immediate modernization. But twenty-two states had already “modernized”—meaning they had moved from decades-old mainframe computers to the cloud—and they fared no better.9 Either the problem wasn’t the out-of-date tech or the modernization efforts had failed. The US Department of Labor is still trying to sort out today exactly what went wrong and what might be done to avoid these meltdowns in the future.

Haunted by the sense that government’s struggles to participate in the digital revolution spelled real trouble for an already weakened institution and threatened already declining faith in both government and democracy, I had founded an organization called Code for America, which enlisted technologists to work with local governments.

In the past, this all happened on paper. Stacks of paper with words and numbers and official seals got you things, like money or food stamps or even freedom. Now, though, those records are rarely magic without a digital element. Today, if someone tells you, “It says here you are a felon,” it’s usually a screen they’re looking at. Conversely, if the law changes and the act that made you a felon is no longer a crime, nothing is really different for you until the database is updated. The magic of law is now inextricably tied to the bits and bytes of computer code.

The famous slowness of bureaucracy is a key reason, but all too frequently, what now widens the gap between policy intentions and actual outcomes is the messy task of implementation through digital technology, and the ways government makes working with that technology uniquely complex.

For my part, I left DC having laid the foundation for a new unit within the White House called the United States Digital Service (USDS), which helps federal agencies make better use of technology.

But the reality of how and why government is failing to deliver on policy promises in the digital age is much more nuanced than these quick fixes suggest. Over the past twelve years, I’ve seen how more money, more technology, more outsourcing, and more oversight have played out, failing again and again. A deeper understanding of the dynamics is needed.

When all your time is spent answering questions and writing reports for other people inside government, it’s mighty hard to be focused on the people outside government you’re supposed to serve.

At times it almost seems that status in government is dependent on how distant one can be from the implementation of policy.

The digital revolution has disintermediated many sectors of our economy and society; you can now buy tickets from an airline instead of going through a travel agent, for instance, or get in touch directly with artists selling their wares on Etsy. But in government, countless bureaucratic processes and procedures—most notably, lengthy and burdensome procurement requirements—have had the opposite effect, putting ever more layers between the people creating the services and those who use them.

The temporal, organizational, structural, and cultural gaps between policy and tech teams, and between tech teams and the users of that tech, make it hard to try out strategies, learn what works, resolve ambiguities, and readjust. Instead of active collaboration and co-learning, implementing government policy through digital technology resembles a game of telephone, in which each party in sequence fumbles the translation a bit until, many stakeholders later, the message is mangled beyond recognition.

Soooo apt

Because digital is seen as a mere implementation detail, separate from the important work of creating policy, it is assumed that digital teams should simply follow orders from above and not exercise their own judgment.

The systems are designed instead to meet the needs of the bureaucracies that create them—they are risk-mitigation strategies for dozens of internal stakeholders. And they often fail even at that task; trying to be all things to all people, they’re unable to make any reasonable tradeoffs for the sake of usability.

Although government must adapt to an increasingly digital world, the heart of the adaptation isn’t mobile apps, cloud computing, or even artificial intelligence. It is a willingness to put the needs of government’s many users ahead of the needs of the bureaucracy, and to learn while doing.

What most people don’t appreciate, though, is just how difficult it can be to make computers do something as simple as counting the number of applications when there isn’t one system in question but many and those systems have accrued in layers over the decades. Tony Scott, a former US chief information officer, uses the metaphor of layers of paint. Because each successive leadership at an agency usually gets the budget or the mandate to deal only with the most pressing technology crises at hand, and because tech investments must always be pitched as adding some new capability to the system (rarely just renovating what already exists), each piece of the system gets built in different technology paradigms from different eras. But every new piece depends on everything that came before, so each successive layer is constrained by the limitations of the earlier technologies.

This is exactly the issue where i work at an airline

We think that the technology that runs our government has been designed to perform specific functions. In fact, it has merely accreted.

While we were trying to determine the right way to count backlogged claims, for example, we came across a work item called “Stop Payment Alert—Claim Review.” There were 730,023 open items of that type. It seemed alarming, but EDD staffers explained that this work item is merely an automatic system-generated notice and could be ignored—in fact, most of those claims were probably fine to pay. But there was a different alert called “Stop Payment Alert” (without the “Claim Review”) that did mean what it sounded like and required attention. But nobody had cleaned up the automated system notices, so these work items cluttered the queues and threw off the reporting when staff didn’t exclude them. We saw dozens of idiosyncrasies like this one.

In 2019, it began a limited pilot of one component of the new system at six locations, out of hundreds of its embassies and consulates around the world. As of late 2021, the bureau still had not expanded that pilot and the other components still had not launched. The program’s cost, originally estimated at $18 million, is now at least an order of magnitude higher. A report by the Office of Inspector General notes that it “was unable to determine the total cost of the ConsularOne modernization program with any precision. OIG’s best estimate is that, as of mid-June 2021, the cost for ConsularOne ranged between $200 million … and $600 million.”

Tell professor from cost benefit analysis! Why is estimation so difficult?

When we speak of “legacy systems” in government, it does not mean simply that they are old. It means that we are grappling with the legacy of decades of competing interests, power struggles, creative work-arounds, and make-dos that are opportune at the time but unmanageable in the long run.

Modernizing technology without rationalizing and simplifying the policy and processes it must support seldom works. Mostly, it results in much the same mess you had before, only now in the cloud. Many a tech modernization has gotten rid of the mainframes, the COBOL, and the green screens (or hidden them better) but has left the frontline workers just as confused and overburdened as before. Sometimes more so, because now they have to learn a new but still staggeringly complex system.

Nor is it just unemployment insurance systems that are doomed by overcomplexity. I once talked with a senior official in state government about a project one of her departments was about to bid out. This one had been in development for twelve years, and it was about to be released to vendors to bid with a $600 million price tag. When I told her I thought it would likely fail, she replied, “Do you think we don’t know that? The last seven IT projects in this state have all failed.”

Paula stayed on at the EDD amid the crisis, working long hours and withstanding enormous criticism, because she saw herself as a steward of a public program. Stewardship is a core value among civil servants, in part because of the frequent turnover in administrations, each of which brings its own priorities and approaches.1 What looks like resistance to change is also a ballast, stabilizing against what amounts to fads rolling through government every time there’s an election.

When systems or organizations don’t work the way you think they should, it is generally not because the people in them are stupid or evil. It is because they are operating according to structures and incentives that aren’t obvious from the outside.

In the case of VBMS, we’d heard that adjudicators could click on a file and then go start the coffee brewing and come back before the next page loaded.

Jfc

We were to find out the next day what Kevin meant. Some work had indeed been done to speed up the system, but his office had sent a memo defining latency as a delay of over two minutes. If you clicked on a button or link, waited for a minute and fifty-nine seconds, and the page appeared, you were not to report latency. You could easily start a pot of coffee in that amount of time, or already be drinking a fresh cup of tea. But officially, that wasn’t a problem; Kevin had defined it away.

For people stuck in waterfall frameworks, data is not a tool in their hands. It’s something other people use as a stick to beat them with.

If you got a digit wrong when you typed in your Social Security number, your claim was flagged. (This error was common because almost half the claimants applied using a mobile phone, but the UIOnline site worked very poorly on phone browsers, and you couldn’t really see what you were typing.) If your last name was hyphenated, had an apostrophe, or was more than twenty letters long, it was likely to be flagged, because some of the other databases didn’t accommodate those characters.

That number could be reduced if the EDD loosened the criteria that were flagging claims for manual processing, but Paula wouldn’t hear of relaxing what she thought of as fraud prevention practices. Those practices were not, in fact, preventing fraud. As proof, Marina’s team took claims that had been flagged for further identity verification during the previous quarter and looked at what the EDD had decided to do with them after its manual review. Out of 183,167 claims, only 804 were judged to be imposters. And even among those 804, many of them were likely to be not actually fraudulent. People whose claims are denied can appeal to the California Unemployment Insurance Appeals Board, and the board overturns the EDD’s identity verification decisions around half the time.5 So the real rate of fraud among the flagged applications that quarter was probably just a few hundred people out of almost 200,000. In other words, only about 0.2 percent of applicants who were flagged for identity verification turned out to be committing fraud.

THE GLOBAL POSITIONING System has to be one of the most successful investments the US has ever made, and not just because GPS-enabled map apps on mobile phones have saved the directionally challenged like me from perpetually getting lost. GPS also powers thousands of critical services that we take for granted today, like routing cell phone calls, sending electricity to our homes, and enabling us to get cash from an ATM. Even stock exchange trades now rely on GPS.1 It’s not much of an exaggeration to say we couldn’t function as a modern society without it.

In 2010, the US Air Force awarded the defense contractor Raytheon a $1.5 billion contract to develop the Next Generation GPS Operational Control System, known as OCX.

multicast user datagram protocol, or UDP.

UDP is one of the low-level protocols that make the internet work, and it’s built into almost every operating system in the world. It’s simple and stable; the technical paper that describes it is less than three pages long.

How does this happen? When there are big, visible delivery failures, like healthcare.gov or the unemployment insurance crisis, public servants are trapped between two distinct systems of accountability. In the first, politicians will hold the public servants accountable for outcomes: whether the website works to enroll people or whether benefits are actually getting to claimants. In this system there will be hearings.

In the second system of accountability, various parts of the administrative state—the agency itself, the inspector general, the Government Accountability Office—will hold these same public servants accountable to process.

Government’s obsession with requirements—voluminous, detailed requirements that can take so long to compile the software is obsolete before it’s even bid out—stems from a delusion that it’s possible to make a work plan so specific that it requires no further decision-making.

The first programmable, electronic, general-purpose digital computer, the ENIAC, was built to calculate artillery firing tables for the US Army and then immediately used to study the feasibility of the hydrogen bomb, kicking off a long history of connections between the military and computing advances.

The predecessor to COBOL, the computer language that still powers mainframes at California’s EDD and many other agencies, was developed by navy rear admiral Grace Hopper (who is also the source of the ever-useful phrase “it is easier to ask forgiveness than to get permission”). By the mid-1960s, the federal government purchased over 62 percent of the output of the entire US computer industry.

To him and many of his colleagues, it was this century’s version of rural electrification, a chance to connect the nearly one hundred million Americans who did not have broadband at home with all the opportunities that come with being online.

When I arrived in DC in 2013, I heard a lot of talk about “building a government fit for the twenty-first century,” as if we were not well into that century already. A big reason for the lag is that our government spent the first several decades of the digital revolution treating tech systems like steel—as a commodity to be bought, not a capability to be developed. Instead, the capability that government has developed well, perhaps too well, is that of procurement.

Policy people tend to see those who implement the policy decisions they make as being far below them in the pecking order, perhaps even at the bottom of it. OMB was the pinnacle of that pecking order. In other words, No, thank you, that’s lowbrow stuff. That’s not the kind of thing we do in powerful places in government.

This way of thinking has deep roots. The British civil service traditionally divided the work of government into two categories: the intellectual and the mechanical.6 Although Koskinen wouldn’t have used those terms, it would have been easy to mistake the field of digital technology in 1995 for just the latest evolution of mechanical work. It didn’t belong in OMB or anywhere else in the White House, where intellectuals were engaged in important, strategic work. It belonged wherever mechanicals bought things that carried out those strategies. So even as computers and the internet began reshaping society as we knew it, government leaders simply handed off anything digital to purchasing officers. They had more important things to think about. They did policy.

So misguided

The distance between tech leaders and real power speaks volumes about how little government culture values the work of implementation, technology included.

The EEOB is a massive edifice in florid French Second Empire style, located in the White House complex across a glorified driveway from the West Wing. It was originally built in the late 1800s for the Departments of State, War, and the Navy. By the Obama administration, it also housed several components of the Executive Office of the President, including many people in OMB.

was known at the time mostly for enterprise resource planning software—a category of back-office software you don’t need to understand, other than to recognize that asking CGI to make a website that practically a third of the entire country would use made about as much sense as asking the military to build Instagram.

There was a precedent for an office of tech nerds within central government. In 2011, the United Kingdom had established the Government Digital Service (GDS), whose director of digital had extraordinary authority for technology spending and strategy across ministerial departments and reported directly to the minister for the Cabinet Office.

As Mike Bracken, who ran the Government Digital Service in the UK, said in an interview a few weeks after the launch, “Government constructs its self-image in terms of size. It thinks of itself as huge. The harsh truth for governments all over the world is that many digital public services could be developed at a fraction of the size by very small teams.”

We named the unit the US Digital Service, in homage to the Government Digital Service in the UK. It would not work quite the way the GDS had been designed—legally, the White House can’t tell an agency which vendor to hire or fire or what tools to use, the way that the Cabinet Office in the UK could. But it would shake things up.

This kind of cultural tendency toward power sharing makes sense. It is akin to saying this project will have no king, no arbitrary authority who might act imperiously. But the result is bloat, and using a bloated service feels intrusive and onerous. It’s easy to start seeing government as overreaching if every interaction goes into needless detail and demands countless hours. Highly diffuse decision-making frameworks can make it very hard to build good digital services for the public. But they are rooted in laws that go back to long before the digital era.

And yet conducting user research has long been considered all but illegal within federal government.

Making user research impossible is particularly ironic because digital professionals typically use it to reduce the burden of their applications—it is the very practice that holds the key to more streamlined, easy-to-use services. Today, Americans spend 10.5 billion hours a year—about forty-two hours per adult—on paperwork just for the federal government.6 That doesn’t count time spent on forms from state and local government, which can add up quickly.