Edict from the Ministry of Correlations

[Tim]

Sometimes when you can't decide on the merits of a concept, person, or system it can be helpful to make a list of pros and cons. Here's my list for big data as an example.

Pros: I can look at GoodReads's stats page and see that I read 22,704 pages in 2017, my YMCA app can communicate with my fitbit app to record my workout schedule and all the calories I'm supposedly burning, Spotify can take my "likes" and compose a playlist of music recommendations and I will like most of them, and Google can track the spread of the flu better than the CDC based solely on an algorithm.

Cons: Alexa listening in on private conversations, 1984, Gattaca, HAL 2000, Irobot, Minority Report, Skynet.

As a field of study, big data falls under the category of machine learning. It's a subset of artificial intelligence. For most of history we have been forced to work with small data sets because our tools for capturing, storing, and analyzing were so limited. We trained our minds to seek causation and conducted experiments that relied on manipulating one variable at a time. Because of these limitations, we also became fixated on making sure that what we took the time to record was accurate.

We are now capturing more data than in all of recorded history. Storing it is easy and cheap. Analysis still takes some skill and talent but those field are growing as well. Data collected now has not only a primary use but also secondary and tertiary uses. With big data we look for correlations instead of causation and forgo explanation of the anomalies.

Where does this leave us? Well, Walmart knows that people facing a hurricane will buy all of the strawberry Poptarts they can carry and Target knows a woman is likely to be pregnant if she's buying unscented lotions. It's all about the ability to predict. Or to spot irregular patterns (in the case of credit card fraud).

Ric Edelman says the war for personal privacy has already been lost. Martin Lindstrom in Buyology showed how marketers are now using our own brain waves and senses against us. Behaviorism and Economics are joining forces to gain a deeper understanding of why we make certain choices. I don't buy the authors's arguments against causation, exactitude, and expertise. Smart people too often get tricked into entrenching themselves in these nature vs nurture type arguments when most of the rest of the world knows that both things have their place and can work together. There's no reason big data and efforts to understand causation can't work together. And I want the engineer who builds my car to be exact. I don't know what the future holds but we are at a point where it would be wise to be careful and intentional. Sci-Fi writers have been sounding the alarm on these subjects for some time. And while I do love my Amazon Prime membership, I'm not sure that it's worth living under the Ministry of Truth.