Recoding America: Why Government Is Failing in the Digital Age and How We Can Do Better

by Jennifer Pahlka

Having a felony on your record can make it very hard to get a job. Most employers won’t even consider you, and fields that require any kind of occupational licensing, from medical assistance to cosmetology, become off-limits. Former felons also can’t join the military.

it’s hard to rent an apartment or get a home loan. Veterans with felony convictions are denied certain retirement benefits. Students with drug felonies can’t deduct their tuition from their taxes, the way others can.

Whites and people of color use marijuana at approximately the same rates, but people of color went to jail for it nearly four times more often,

More money can be useful, of course, and is often necessary—but having big budgets from the start can be deadly, since they often require an entire megaproject to be planned up front, reducing the ability of the team to learn as it goes.

Spending more on blockbuster projects, pursuing the latest technologies, outsourcing more aggressively, and ramping up oversight doesn’t help. But it’s worse than that. These supposed solutions hurt, making it even harder to deliver on policy promises.

At times it almost seems that status in government is dependent on how distant one can be from the implementation of policy. There is a debilitating distance, too, between the people creating government systems and the people who use them. The digital revolution has disintermediated many sectors of our economy and society; you can now buy tickets from an airline instead of going through a travel agent, for instance, or get in touch directly with artists selling their wares on Etsy. But in government,

Joe Soss, a professor of political science at the University of Minnesota, found in the 1990s that participating in means-tested programs—benefits that you must prove that you are poor enough to qualify for—significantly reduces the chance you will vote.13 Applying for those programs can be a lot like applying for record expungement: hard to start, confusing, often insulting, and impossible to succeed at for many people. “Because clients interpret their experiences with welfare bureaucracies as evidence of how government works more generally, beliefs about the welfare agency and client involvement become the basis for broader political orientations,” Soss explains.

WE KNEW THE first thing we needed to do was to define and count the EDD’s backlog of unemployment claims.

Without a clear and consistent definition, each report was comparing apples to oranges, and we would never know how fast the backlog was shrinking—or growing, as the case turned out to be.

Around 2002, the EDD entered the internet era by making its paper application form for unemployment benefits available online through something called eApply4UI. This went away in the 2010s,

Why

Their state had spent ten years and over $500 million on a system to connect the courts with a common document management system—and then scrapped the entire effort.6

Why

every new person the EDD hired made it slower—not faster—to get a benefit check to an unemployed Californian.2 The backlog was growing unbounded not because of the bespoke and antiquated technology. It was growing because the policy and processes that govern unemployment insurance take seventeen years to learn.

equity policies are often interpreted to mean that the questions asked across the different channels must be exactly the same, which is why you see government websites that clearly could leverage such branching logic but don’t do so.

The problem is that the modernization projects all sought to “add functionality”—more layers of paint—or just to move to more modern infrastructure, particularly to the cloud. None of them targeted serving clients better or scaling to meet demand. If the people running the systems had set scale as a goal and truly analyzed their bottlenecks, they would have recognized the need to rationalize and simplify the accumulated layers of policy and process along with bringing in new technology. To be fair, none of them felt they had permission to.

You can have systems that do every possible thing policymakers can think of to ensure “program integrity” (in other words, making sure no one is getting a dollar more than they should) or you can have systems that scale. You can’t have both. And most of what policymakers do to ensure program integrity ends up costing far more in administration than the program saves on paying out benefits. Those costs include the failure of the system when the benefits are most needed.

The incumbent thesis was that if you wanted to manage something professionally, you structured it like a waterfall—or at least like a cartoon version of a waterfall, with several pools, each flowing into the next. There were separate, sequential stages each project had to go through: gathering requirements, design, implementation, verification, and maintenance. Different teams were generally responsible for each of the stages, and once a given stage was complete you didn’t go back. Once the requirements had been set, for instance, it was critical that the design and implementation faithfully fulfill those requirements.

the agile manifesto says at the top: “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

Clay Shirky once quipped that “waterfall amounts to a pledge by all parties not to learn anything while doing the actual work.”2

A SURE SIGN of a waterfall organization is how the people within it treat data. In an agile, empowered organization, data is a useful tool for adjusting course. The people in the organization not only have access to data and the ability to understand it but have the power to decide what to do based on it.

In a waterfall organization, on the other hand, data functions less like a compass that helps you steer and more like an after-the-fact evaluation, a grade you get that says how well or poorly you did on something that has already happened.

“burndown chart.”

To do how

I am much more likely to get my own Social Security number wrong than a sophisticated criminal enterprise is, especially if I’m two-fingering it into a wonky, hard-to-see web form on a tiny keyboard on my mobile phone.

The good news was that there were a variety of commercial offerings that would do just that. Our first recommendation as a task force was therefore very easy to make: procure and install an identity verification system as soon as humanly possible.

It was permission—permission to disrupt the waterfall.

each level of the waterfall can usually see only what happens just below and just above. Rarely does someone take a step back and trace a decision made at the highest levels of government all the way down to its impact on teams at ground level.

In 1983, however, the crew of Korean Air Lines Flight 007, flying from New York City to Seoul via Anchorage, Alaska, made an error in their navigational calculations and accidentally strayed into the prohibited airspace of the USSR. Mistaking the airliner for a spy plane, the Soviets shot it down, killing all 269 people aboard, including Larry McDonald, a US congressman from Georgia. Recognizing that GPS could have prevented this tragedy, President Ronald Reagan directed the Department of Defense to make GPS freely available for civilian use. As a common good, it became even more powerful than Reagan could have imagined.

the Department of Defense has an embarrassing history of paying contractors more for more lines of code—even though, as the UDP versus ESB situation shows, more code is seldom better and is almost always worse.

Figure 5–8 is not the last diagram to show an Enterprise Service Bus, nor is it the last mention of ESBs in the DoD Architecture Framework. Somehow, its authors have jumped from the very reasonable notions of broad value and open technology standards to specifying an Enterprise Service Bus. The bizarre setup Weaver was wrestling with was not a brilliant way for Raytheon to shake down the federal government for more money. It was a requirement set by the air force that the contractor was obliged to meet. It was friendly fire.

By any reasonable definition, the form didn’t work. But the way we build government technology is to specify the requirements and fulfill the requirements. That had happened. There had been no requirement to test the software outside the building and no requirement that the software actually work. So the team was stuck.

The Federal Information Security Management Act, or FISMA, provides a menu of some three hundred distinct “controls” that tech teams can choose from to secure government software and data from hackers. Competent developers should, in theory, create an informed, thoughtful security plan that chooses the controls most relevant to the circumstances and focus their efforts on implementing and testing those choices. But it’s the rare compliance officer who will take the risk of allowing anything less than all three hundred.

The problem is not that whoever wrote the contract for the VA health care form chose the wrong browser in their requirements, and it’s not that the CIO Council chose the wrong architecture when it specified ESBs. The problem is that it specified those things at all. Government’s obsession with requirements—voluminous, detailed requirements that can take so long to compile the software is obsolete before it’s even bid out—stems from a delusion that it’s possible to make a work plan so specific that it requires no further decision-making.

The first punch cards used in computing were the brainchild of Herman Hollerith, a former clerk at what is now the Census Bureau, who designed them explicitly to speed the tabulation of the 1890 census. (He succeeded: despite the country’s growing population, the counting was completed eighteen months faster than for the previous census.)

In 1921, for instance, concerned about high prices for steel amid war-driven demand, the US government embarked on an effort to make its own. It did eventually build a plant, but the steel it produced was half as good as what was commercially available and cost three times as much.6 Commodities like steel are best left to the market. It followed that the same would be true of automatic data processing. Both A-76 and the Brooks Act sought to make sure that the new tools of computing didn’t meet the fate of government steel.

For years, the agents used a computer program that directed them to choose one of three categories: “unaccompanied minor,” “individual adult,” or “adult with children.” Each case was assigned an “A number” (for “alien”); for an adult with children, the entire family shared the same A number. That’s how the process worked until, in 2018, the Trump administration ordered the Border Patrol to start separating detained children from their parents.

government procurement teams have a terrible habit of contracting for bespoke software when they could buy commercial products—partly because we tell these teams to collect every possible requirement they can think of, which encourages and even calcifies arcane practices within the departments they serve.

Though government should buy commodity products for commodity functions, when it’s not accounting or payroll but your agency’s mission, the technology needs to be your product. It can’t just be a project that was contracted for, developed, tested, and declared “done.” You need to own the code, and you need to be able to change it to meet your needs. This doesn’t mean that you can’t use contractors at all—in government, you will almost certainly use them. It means that you must have the core competencies to support a living, ever-adapting system.

One provision of the legislation they introduced charged the White House Office of Management and Budget with developing strategy for digital technology in the federal government. OMB is part of the executive branch; it oversees the performance of federal agencies, administers the federal budget, and reports to the president. It has real power and authority.

Highly diffuse decision-making frameworks can make it very hard to build good digital services for the public.

The law also required federal agencies to get permission from OIRA before collecting any information from the public. That’s why at the top of every passport application, tax return, and so on you’ll see something like “OMB No. 1545–0074.” The number tells you that OIRA, which is part of OMB, has reviewed the form and approved it. The intention of the Paperwork Reduction Act was to limit the administrative burden imposed on the American public by mandatory paperwork, such as filing taxes. But it has been interpreted to cover any time anyone in a federal agency wants to ask any member of the public any kind of question. And that interpretation has effectively functioned as a ban on user research. Getting OIRA approval typically takes six to nine months, includes several rounds of formally soliciting public comment through the government’s official daily journal, the Federal Register, and requires a researcher to submit in advance every single question that will be asked.

in the end they resolved most of the conflicts among the various definitions and got down to two definitions of a group that could take the place of the previous nine.

Legal precision causes confusion

In January 2020, before there were any known cases of COVID transmission on US shores, Seattle researchers who had been collecting nasal swabs for a flu study realized that they could use those swabs to find out whether the virus was circulating in their area. But federal and state officials repeatedly rejected their requests for permission to do so. The subjects hadn’t consented to any use of their samples other than for the flu study, so the officials were certainly technically correct. After weeks of trying, out of desperation, one of the researchers tested the swabs anyway. She found COVID present in swabs from people with no recent travel history, meaning that the virus had been circulating undetected in the United States for well over a month.

Sometimes it’s lawyers who do this, but other experts in niche subjects are just as likely to. Their identity—and the measure of their value—can become deeply tied to the specialized knowledge they possess. And that knowledge, which ought to serve the goals of the program, instead begins to serve its own, often conflicting goals.

Law scholar Larry Kramer, describing what he calls “popular constitutionalism,” explains that courts were never considered the final say on interpretation of the Constitution until the late twentieth century.

Liberals’ penchant for suing to enforce regulations contributes to the cramped professional thinking that loses sight of the big picture. As scholars have demonstrated, litigation changes agencies over time.6 To begin with, the more they are sued, the more legal staff they must invest in. But more importantly, courts tend to hold agencies accountable not for their outcomes but rather for their fidelity to procedures, compounding the accountability trap the bureaucracy is already stuck in.

The Supreme Court’s 2022 ruling in West Virginia v. EPA, for example, gutted the agency’s authority to do its job in ways that could have implications for many other federal agencies and could trickle down to states and municipalities.

What

however just the cause, every lawsuit over process and procedure has an undeniable impact not only on the culture of the agency it targets but on the zeitgeist of government. Each one is a cautionary tale for another agency or another jurisdiction. Each one makes government less tolerant of risk. Each one makes the bureaucracy more technically correct but entirely less helpful.

There’d been little thought given, for instance, to which questions were easy or hard to answer and when they might be asked. Do we really need the exact dates and locations of every out-of-country trip the applicant took in the past ten years at the very beginning of the process?

agencies have put forth visions of “prefilled applications” or “single sign-on services” that leverage the information an agency already has about its customers. But the number of agencies that have delivered on those visions is small, for the kinds of reasons I’ve already discussed, including lack of internal digital capacity, arcane and lengthy procurement processes, the many restrictive procedural requirements set forth in laws like the Computer Matching and Privacy Protection Act of 1988, and policy complexity that leads to thousands of formal requirements and doomed megaprojects.

Large health care systems aren’t naturally better at those tasks. They’re better because they can hire expensive legal counsel, pay for specialized training, and allocate dedicated staff to master the ins and outs of the program. They have money and resources. Small practices don’t, so they can’t make these huge investments to get the most out of the program. MACRA, like many other government programs, wanted to promote meritocracy—to have the rewards go to the most deserving, the providers who contribute the most to the overall health of our society. But because evaluating merit involved such a big administrative burden, the program itself tilted the playing field.

paperwork favors the powerful. And health care is far from the only field where this happens. For instance, scholars studying the impact of General Data Protection Regulation, an online privacy and security law covering the European Union, found that while large technology companies experienced a 4.6 percent drop in profits after it went into effect in 2018, small technology companies saw their profits drop by over 12 percent.9