What do you do when every object is Internet connected?
Actually, there is a spoon.
Welcome to the future where everything collects data, not just your car, your phone and your Fuel band, but your toilet. Yeah, your toilet knows you're slightly dehydrated and you don't eat enough fiber, and it already called your nurse practitioner. Check your messages.
But what do we do when everything is connected?
So we're collecting all this data, and storage keeps getting cheaper. Heck, the biowizards at Harvard stored 700 terabytes on a single gram of DNA, so stop worrying about storage.[1] The problem is you.
You can't push a pumpkin through a garden hose.
What?
The three-pound engine at the top of your neck is a bottleneck. Everything around you is going to be spewing data, and you will have no idea what to do with it because you don't know how to process it all.
Hurricanes and hedge funds.
We analyze complex systems every day with the fastest computers and the best brains, but we don't understand them. So we collect more data and pile it deeper, but we can't process it all. We have more data, not better choices.
In "How Hurricane Forecasting Got So Good," Sarah Fecht wrote, "Hurricane forecasting begins with lots and lots of data. More data than weather modelers know what to do with, really." Yet when a hurricane is just two days from landfall, our predictions miss by an average of 100 miles. During Superstorm Sandy, we outdid ourselves and called it within 50 miles. Wow.[2]
Was our data on Sandy incomplete? No, our analysis was.
Buy. Sell. Jump.
Complex systems. The stock market is complex. Lotsa numbers, but computers are good at numbers and we have plenty of computers. Yet every investment group pays Stock Analysts to figure out what all that data means and distill it down into something they can understand and act on.
Hedge funds deal in billions of dollars. There are at least 30 different recognized hedge fund strategies, but many of them boil down to considering a single factor. All that data, and some hedge fund manager is looking at interest rates. Only interest rates.
In Running Money, hedge funder Andy Kessler wrote, "Because information is distributed in milliseconds, there is no time advantage anymore. You have to be ahead of news."[3] Someone has to sit down and analyze all this stuff, because the guy across the street has access to all the same data you do. So why do some hedge funds make money and others don't? Better analysis.
The Plans for the Death Star
We already collect more data than we know how to process. When all the objects in your life start chatting with you and each other, you'll be drowning in data. It's time to plan ahead.
The same computers that collect and store this stuff are not built to analyze it. There are numbers and letters and charts, but which ones are important and what do they tell us?
We need to build ourselves some new tools that can help us analyze this flood. Then we need to ask these tools the right questions so they can give us good advice. Because data is useless if we can't put it to work.
Sources:
[1] "Harvard cracks DNA storage, crams 700 terabytes of data into a single gram" by Sebastian Anthony for ExtremeTech. http://www.extremetech.com/extreme/13... [2] "How Hurricane Forecasting Got So Good" by Sarah Fecht for Popular Mechanics. http://www.popularmechanics.com/scien...
[3] Running Money by Andy Kessler. HarperCollins Publishers Inc. www.andykessler.com. www.harpercollins.com.
Welcome to the future where everything collects data, not just your car, your phone and your Fuel band, but your toilet. Yeah, your toilet knows you're slightly dehydrated and you don't eat enough fiber, and it already called your nurse practitioner. Check your messages.
But what do we do when everything is connected?
So we're collecting all this data, and storage keeps getting cheaper. Heck, the biowizards at Harvard stored 700 terabytes on a single gram of DNA, so stop worrying about storage.[1] The problem is you.
You can't push a pumpkin through a garden hose.
What?
The three-pound engine at the top of your neck is a bottleneck. Everything around you is going to be spewing data, and you will have no idea what to do with it because you don't know how to process it all.
Hurricanes and hedge funds.
We analyze complex systems every day with the fastest computers and the best brains, but we don't understand them. So we collect more data and pile it deeper, but we can't process it all. We have more data, not better choices.
In "How Hurricane Forecasting Got So Good," Sarah Fecht wrote, "Hurricane forecasting begins with lots and lots of data. More data than weather modelers know what to do with, really." Yet when a hurricane is just two days from landfall, our predictions miss by an average of 100 miles. During Superstorm Sandy, we outdid ourselves and called it within 50 miles. Wow.[2]
Was our data on Sandy incomplete? No, our analysis was.
Buy. Sell. Jump.
Complex systems. The stock market is complex. Lotsa numbers, but computers are good at numbers and we have plenty of computers. Yet every investment group pays Stock Analysts to figure out what all that data means and distill it down into something they can understand and act on.
Hedge funds deal in billions of dollars. There are at least 30 different recognized hedge fund strategies, but many of them boil down to considering a single factor. All that data, and some hedge fund manager is looking at interest rates. Only interest rates.
In Running Money, hedge funder Andy Kessler wrote, "Because information is distributed in milliseconds, there is no time advantage anymore. You have to be ahead of news."[3] Someone has to sit down and analyze all this stuff, because the guy across the street has access to all the same data you do. So why do some hedge funds make money and others don't? Better analysis.
The Plans for the Death Star
We already collect more data than we know how to process. When all the objects in your life start chatting with you and each other, you'll be drowning in data. It's time to plan ahead.
The same computers that collect and store this stuff are not built to analyze it. There are numbers and letters and charts, but which ones are important and what do they tell us?
We need to build ourselves some new tools that can help us analyze this flood. Then we need to ask these tools the right questions so they can give us good advice. Because data is useless if we can't put it to work.
Sources:
[1] "Harvard cracks DNA storage, crams 700 terabytes of data into a single gram" by Sebastian Anthony for ExtremeTech. http://www.extremetech.com/extreme/13... [2] "How Hurricane Forecasting Got So Good" by Sarah Fecht for Popular Mechanics. http://www.popularmechanics.com/scien...
[3] Running Money by Andy Kessler. HarperCollins Publishers Inc. www.andykessler.com. www.harpercollins.com.
No comments have been added yet.
