More on this book
Community
Kindle Notes & Highlights
Read between
January 16 - January 23, 2024
If anything, all those failures stemmed from a lack of design. And all resulted in outcomes that no one wants—not the left, not the right, and not the average American who identifies with neither. The reality of occasional sabotage should not blind us to our self-inflicted wounds.
When all your time is spent answering questions and writing reports for other people inside government, it’s mighty hard to be focused on the people outside government you’re supposed to serve.
At times it almost seems that status in government is dependent on how distant one can be from the implementation of policy.
The temporal, organizational, structural, and cultural gaps between policy and tech teams, and between tech teams and the users of that tech, make it hard to try out strategies, learn what works, resolve ambiguities, and readjust. Instead of active collaboration and co-learning, implementing government policy through digital technology resembles a game of telephone, in which each party in sequence fumbles the translation a bit until, many stakeholders later, the message is mangled beyond recognition.
Marina Nitze, to lead the on-the-ground analysis. Marina’s government experience included helping clear one of the worst backlogs at the Department of Veterans Affairs; she had been appointed the chief technology officer of the VA when she was just twenty-eight years old.
Along with Marina, several of us had spent time at the US Department of Veterans Affairs, where the technology layers date back to the sixties and even earlier. The architecture diagram for just one benefit system at the VA was so complex that it was displayed on a wall twenty feet long and eight feet tall. Even at that size, many of the elements were printed in a font so small you had to be right up next to the wall to read them.
The program’s cost, originally estimated at $18 million, is now at least an order of magnitude higher. A report by the Office of Inspector General notes that it “was unable to determine the total cost of the ConsularOne modernization program with any precision. OIG’s best estimate is that, as of mid-June 2021, the cost for ConsularOne ranged between $200 million … and $600 million.”
It’s been around since the 1960s, when it stored data on magnetic tape. For years, its structure was akin more to a massive spreadsheet than to a modern database. (As one overview puts it, “accounts were stored sequentially so that information for taxpayer number 100,000,001, for example, could be accessed only by skipping through the first 100,000,000 accounts.”)
When we speak of “legacy systems” in government, it does not mean simply that they are old. It means that we are grappling with the legacy of decades of competing interests, power struggles, creative work-arounds, and make-dos that are opportune at the time but unmanageable in the long run.
The PowerPoint presentations commonly given to executives are sorely insufficient.
The training manual is eight hundred pages long. And even reading through it does not give you the skills you need to do the job, since much of the knowledge involves work-arounds that are passed along from employee to employee but not consistently known across teams or even written down. In
THE POLITICIANS WHO jeer when a government agency like the EDD says that it will need eleven years to modernize its systems do not understand the nature of the technology in question. They envision a system, not a bunch of tenuously connected layers that function by way of awkward work-arounds.
And like the tech, the policy is complex in part because it always accrues but is rarely reduced or reconciled. Like a hoarder, government rarely throws out the old to make room for the new.
To start, though the statutes are minimal, the regulations that have been written to guide their implementation are not.
Federalism also makes the job of the DOL, which has to regulate each state labor agency to ensure compliance with federal law, exceptionally difficult. And when crisis hits, Congress looks to the DOL to help all fifty-three systems deliver the necessary relief and economic stimulus. It’s a bit like trying to wrangle fifty-three IRSes.
In California, your benefits are supposed to be reduced by one-seventh for each day of a week you cannot work due to an illness or injury.8 (That’s just one rule among many hundreds; you start to see why the request for proposals for a new system to run the EDD detailed some thirty-six hundred unique requirements.)
More importantly, web forms can make things easier for users in ways that paper can’t, like hiding questions that are unnecessary for a particular user based on the answers they have already given. (If you don’t have dependents, for instance, you don’t need to see or answer questions about dependents.) But equity policies are often interpreted to mean that the questions asked across the different channels must be exactly the same, which is why you see government websites that clearly could leverage such branching logic but don’t do so.
As long as the training manual is eight hundred pages, as long as seventeen-year veterans of the department are still getting up to speed, making the tech better will be of only marginal help.
I thought that, too, when I first started working with government. But today, I’d bet on a legacy mainframe with a thirty-page manual over a modern system that takes decades to learn.
When a member of the California State Assembly asked her whether the EDD’s planned modernization would really take eleven years, for example, her answer was a dodge. “I can’t speak to that,” she began. “We follow the process set by the California Department of Technology.…” She was in a lot of hot water, with a lot of very powerful people angry at her, but she believed her ability to show that she was following established procedures was her best chance at avoiding consequences. That stance only further infuriated the assembly subcommittee. But she stuck to it in question after question.
Stewardship is a core value among civil servants, in part because of the frequent turnover in administrations, each of which brings its own priorities and approaches.1 What looks like resistance to change is also a ballast, stabilizing against what amounts to fads rolling through government every time there’s an election. I was always aware that I and the rest of the task force were around for what amounted to just a brief moment in Paula’s long career.
When systems or organizations don’t work the way you think they should, it is generally not because the people in them are stupid or evil. It is because they are operating according to structures and incentives that aren’t obvious from the outside.
Some work had indeed been done to speed up the system, but his office had sent a memo defining latency as a delay of over two minutes. If you clicked on a button or link, waited for a minute and fifty-nine seconds, and the page appeared, you were not to report latency. You could easily start a pot of coffee in that amount of time, or already be drinking a fresh cup of tea. But officially, that wasn’t a problem; Kevin had defined it away.
But there were whole categories of questions he would not answer. “That’s a question for the program people,” he said time and again.
Finally, a bit confused, I asked why was he so reticent to talk about what the system was designed to do. “I’ve spent my entire career training my team not to have an opinion on business requirements,” he told me. “If they ask us to build a concrete boat, we’ll build a concrete boat.”3 Why? I asked. “Because that way, when it goes wrong, it’s not our fault.”
It was a good test case for the digital services unit we were hoping to establish—it didn’t take over from the agency, it simply helped an agency team get unstuck. After just two weeks, my colleagues had helped the developers make enough changes to get VBMS heading in the right direction.
A SURE SIGN of a waterfall organization is how the people within it treat data. In an agile, empowered organization, data is a useful tool for adjusting course. The people in the organization not only have access to data and the ability to understand it but have the power to decide what to do based on it. If the compass says you’ve drifted off course, no one summons the inspector general or calls for a hearing. You just turn the wheel.
In a waterfall organization, on the other hand, data functions less like a compass that helps you steer and more like an after-the-fact evaluation, a grade you get that says how well or poorly you did on something that has already happened.
For people stuck in waterfall frameworks, data is not a tool in their hands. It’s something other people use as a stick to beat them with. After Marina discovered that the new employees were dramatically slowing down the EDD, she wanted Paula to work with her on what she called a “burndown chart.”
General Stanley McChrystal put it this way: “I tell people, ‘Don’t follow my orders. Follow the orders I would have given you if I were there and knew what you know.’”
Marina remembers the sudden shift in Paula’s thinking. “It was like she suddenly said, ‘Oh, wait a minute. I can greatly influence the day when we work this part of the backlog down to zero by moving people over to this area, and then when it gets to zero, I move them over there. And every time I play around with the spreadsheet and make the allocations better, the dates move forward a bit,’” Marina told me. “I think prior to that, Paula had thought she really had no control over it, except to try to make the numbers look as small as possible.”
I credit Marina and her team with superpowers—both the empathy to build trust with people like Carl and the analytical acumen to build the first accurate picture of what was going wrong. But it wasn’t IQ or EQ that the task force had and Paula’s team didn’t. And it wasn’t newer and sexier tools to replace legacy technology, since the infamous COBOL code chugged along just the same the whole time. It was permission—permission to disrupt the waterfall.
But Paula was the product of a system that values deference to the hierarchy and punishes risk taking. It had rewarded her with job security and successive promotions for thirty-seven years, just as it rewarded people like Kevin at the VA. State and federal civil service rules are a big part of that system, but they are simply the expression of a culture in which fidelity to flawed rules and practices is valued more than solving problems.
THE IEEE STANDARD Glossary of Software Engineering Terminology offers two definitions of requirement in the programming context. It can be something “needed by a user to solve a problem or achieve an objective” or something that “must be met or possessed by a system … to satisfy a contract, standard, specification, or other formally imposed document.” Government subscribes almost exclusively to the second definition. Requirements are the foundation of software development processes in government, and the source of many of its failures.
None of the systems that processed the applications were ready for ten times the volume. People all over the building were suddenly drowning in work, which was particularly bad because the VA was already facing fierce criticism for a myriad of other backlogs. Now there was going to be another one. Bureaucrats across the VA started calling up everyone in power: Revert to the old application form!
“I would create a PowerPoint that described what we were doing, and then somebody would say, ‘You don’t have an ESB on here.’ So I would just relabel one of the boxes ‘ESB’ and then go through the PowerPoint again.” No one ever asked Eric which software components were using the ESB, or even if there was anything installed on the machine at all. Today, Eric’s ESBs are all fictional.
However the document came together, the 434-page Federal Enterprise Architecture, released in 1999, requires that federal technology solutions have a “service-oriented architecture.” And one of the elements of a “service-oriented architecture” that it shows is an ESB.
ESBs became mandatory in practice within the Department of Defense through overzealous interpretations of law, policy, and guidance, combined with lack of technical understanding.
In government, culture eats policy. Even when legislators and policymakers try to give implementers the flexibility to exercise judgment, the words they write take on an entirely different meaning, and have an entirely different effect, as they descend through the hierarchy, becoming more rigid with every step. When rules rarely have their intended effect, more rules are not likely to improve outcomes.
Government’s obsession with requirements—voluminous, detailed requirements that can take so long to compile the software is obsolete before it’s even bid out—stems from a delusion that it’s possible to make a work plan so specific that it requires no further decision-making.
Circular A-76. It built on previous policies stating that the federal government “will not start or carry on any commercial activity to provide a service or product for its own use if such service or product can be procured from private enterprise.”4 The message was: if someone outside government can do your job, and especially if they can do it cheaper, we’re giving it to them. The memo is important because it formalized the distinction between functions that are “commercial” and those that are “inherently governmental,” a difference whose meaning has been debated ever since.
“That’s how this works,” he told me. “All of the staff—the core civil servants—they manage, but they don’t implement. One hundred percent of the implementation is contractors.”
Government procurement expert Mark Hopson calls our overreliance on outsourcing everything digital the Kodak Curse.
Policy people tend to see those who implement the policy decisions they make as being far below them in the pecking order, perhaps even at the bottom of it.
As Mike Bracken, who ran the Government Digital Service in the UK, said in an interview a few weeks after the launch, “Government constructs its self-image in terms of size. It thinks of itself as huge. The harsh truth for governments all over the world is that many digital public services could be developed at a fraction of the size by very small teams.”11
Why did the contractor do that? “I saw other lines had these slashes so I thought I should put them in too,” he explained. This supposedly technical staff member had never written a line of code. Just as government’s core competency has been in contracting with vendors, some vendors’ core competency has been in getting those contracts. They may have lots of people who do account management and project management, but they don’t always have the people who can make the websites work.
Code for America colleagues published a “teardown” of the website, over a hundred screenshots of it in action, with each page marked up to highlight the parts that confused and frustrated the people trying to use it.3 (To be fair, the teardown also highlighted elements that were helpful to users; there were just far fewer of them.) The teardown was a powerful critique.
The fear of having exercised too much power, and being criticized for it, is ever present for many public servants. The result is a compulsion to consult every imaginable stakeholder, except the ones who matter most: the people who will use the service.
Highly diffuse decision-making frameworks can make it very hard to build good digital services for the public.
conducting user research has long been considered all but illegal within federal government.
