Too Big to Know: Rethinking Knowledge Now That the Facts Aren't the Facts, Experts Are Everywhere, and the Smartest Person in the Room Is the Room

Filters no longer filter out. They filter forward, bringing their results to the front. What doesn’t make it through a filter is still visible and available in the background.

[T]here’s also way too much bad stuff. We can now see every idiotic idea put forward seriously and every serious idea treated idiotically. What we make of this is, of course, up to us, but it’s hard to avoid at least some level of despair as the traditional authorities lose their grip and before new tools and types of authority have fully settled in. The Internet may not be making me and you stupid, but it sure looks like it’s making a whole bunch of other people stupid.

The economics of knowledge make sense only if, after looking up the population of Pittsburgh in the almanac, people stop looking. If everyone were to say “Well, that may be a pretty good guess, but I can’t trust it,” and then hire their own census takers to recount the citizens of Pittsburgh, the cost of knowledge would be astronomical. Distrust is an expensive vice.

our information technologies are precisely the same as our communication technologies, so learning a fact can be precisely the same as publishing a fact to the world. The Internet’s abundant capacity has removed the old artificial constraints on publishing—including getting our content checked and verified. The new strategy of publishing everything we find out thus results in an immense cloud of data, free of theory, published before verified, and available to anyone with an Internet connection.

First, there was the Age of Classic Facts, represented by Darwin with a dissecting kit and by parliamentary blue books. These facts were relatively sparse, painstakingly discovered, and used to prove theories. Then, in the 1950s we entered the Age of Databased Facts, represented by punchcards stacked next to a mainframe computer. We thought we had a lot of information then, but it would have taken just under 2 billion cards to store what’s on a rather wimpy 200-gigabyte hard drive on a laptop—a stack about 300 miles high.42 So, of course the databases of the time had to strictly limit the amount of information they recorded: the employee’s name, date of birth, starting date, and Social Security number, but not hobbyist skills or countries lived in. The Age of Data still conformed to our ancient strategy for knowing the world by limiting what we know—a handful of fields, chosen and organized by a handful of people. Now, in the Age of the Net it makes sense to talk about networked facts. If classic facts and databased facts are both taken as fundamentally isolated units of knowledge, networked facts are assumed to be part of a network. Networked facts exist within a web of links that make them useful and understandable.

This makes our ordinary encounter with facts very different from what it used to be. We don’t see them marching single-file within the confines of an argument contained within a blue book, a scientific article, or a printed tome. We see them picked up, splatted against a wall, contradicted, torn apart, amplified, and mocked. We are witnessing a version of Newton’s Second Law: On the Net, every fact has an equal and opposite reaction. Those reactive facts may be dead wrong. Indeed, when facts truly contradict, at least one of them has to be wrong. But this continuous, multi-sided, linked contradiction of every fact changes the nature and role of facts for our culture. When Daniel Patrick Moynihan said, “Everyone is entitled to his own opinions, but not to his own facts,” what we heard was: Facts give us a way of settling our disagreements. But networked facts open out into a network of disagreement.

We see all too clearly how impotent facts are in the face of firmly held beliefs. We have access to more facts than ever before, so we can see more convincingly than ever before that facts are not doing the job we hired them for. [...] Our new medium of knowledge is shredding our old optimism that we could all agree on facts and, having done so, could all agree on conclusions.

Traditional knowledge is what you get when paper is its medium. There is nothing mystical about this. For example, if your medium doesn’t easily allow you to correct mistakes, knowledge will tend to be carefully vetted. If it’s expensive to publish, then you will create mechanisms that winnow out contenders. If you’re publishing on paper, you will create centralized locations where you amass books. The property of knowledge as a body of vetted works comes directly from the properties of paper. Traditional knowledge has been an accident of paper.

Indeed, on the Net, the measure of one’s strength as an expert often is not that you have the final word on some topic but that you have the first word. And from that first word—whether it’s on a blog post, a tweet, or a sheet of old-fashioned white paper—spin out a million gnats of difference, buzzing across the linked world, unsettled and unsettling

We thought that knowledge thrives in a lively “marketplace of ideas” because the constraints of paper-based knowledge kept most of the competing ideas outside our local market. Now that we can see just how diverse and divergent the ideas around us are—because Internet filters generally do not actually remove material, but only bring preferred material closer—we find ourselves tremendously confused about the value of this new diversity. [...] It seems we love diversity until we see what it actually looks like.

A diverse group of people who share a goal are likely to be more effective than a homogeneous group of people. Communities of knowers need walls around them. Those walls used to be like those of a fortress. These days, they tend to be usefully semipermeable. But they are walls nonetheless, and serve the good purpose that walls of every sort serve: permitting a group with enough in common to get something done by keeping out disruptive diversity.

Yet, it’s worth noting that it always seems to be “those other folks” who are being made stupid by the Net. Most of us feel, as we’re Googling around, that the Net is making us smarter—better informed (with more answers at our literal fingertips), better able to explore a topic, better able to find the points of view that explain and contextualize that which we don’t yet understand.

Birkerts is such a lovely writer that I enter a bibliophilic reverie. I am in a classic library—I personally envisage the Harvard Law Library where I work, an elegant epitome of the beauty of book culture—where I’m sitting in a leather-bound armchair reading a leather-bound book by one of those ancient writers I’ve always meant to read. Then I look at the actual book in my hand, a fifteen-year-old paperback of The Gutenberg Elegies. The top—the only part left exposed to the air in my bedroom—is dusty. When I open it, the dried glue crinkles and the pages begin to separate from the spine. I thumb through, afraid to spread it wider than the angle of a twig on an autumn branch. The outer margins of the pages look like they have been dipped in weak coffee. The book smells like an item from the past that was forgotten, abandoned. This is not what Birkerts means by books making the past present to us. This actual book’s past is present in its decrepitude. Instead of enjoying the frisson of connection to our culture’s continuous glory, I have to suppress a sneeze.

If amateur astronomers in the Philippines and Australia had not notified professionals about the two-second flash of light they independently observed when watching Jupiter, we might not have known that the flash was caused by a small comet or asteroid smashing into the giant planet.

But the contribution of amateurs becomes more substantial if we look not only at what individuals are doing but at what networks of amateurs are contributing. For example, Arfon Smith, the technical lead of Galaxy Zoo, told me about the discovery of “green peas.” It began as a joke in the discussion area of Galaxy Zoo about the green objects that showed up in some photos. After over a hundred posts on the topic, the amateurs at Galaxy Zoo realized that there was a type of astronomical object that the professionals had not noticed. “In mid-2008,” said Smith, “they put together a portfolio and delivered it to us, and insisted that we pay attention.” It turns out that “the green peas are important. We’re just beginning to understand how.” That insight, and its development, occurred within a network of amateurs; had it been only a single person’s observation, the importance of “green peas” would not have been noticed.

We have given up on the idea that there is a single, knowable organization of the universe, a Book of Nature that we’ll ever be able to read together or that will settle bar fights like the Guinness Book of Records. No, you organize your data one way, I’ll organize it another, namespaces and data model translators will let us benefit from each other’s research, and we’ll still be able to learn from one another’s research.

A Google search reports that Nature magazine has used the word “shitless” once in its over 140-year history.

If Wikipedia and Linux had to rely on centralized leadership, they would never have been built as rapidly or as well.