Dev Blog 3: Channeling Watson
The sheer amount of information that's being collected in one form or another in today's hyper-connected world is staggering, if not flat-out overwhelming. That's only going to become more and more the case in the years to come as that level of interconnectivity continues to increase and we embark upon a sensor revolution, with a host of advanced, miniature, and cheap sensors of all sorts or types being developed and introduced.
If wielded properly, that glut of information (often dubbed 'Big data') can prove terrifically powerful. Information is power, right? However, until that information is actually accessed and understood in a meaningful way, it is useless. Worse than useless, actually.
If an intelligence agency were simply collecting vast amounts of signals intelligence (SIGINT) but lacked the ability to efficiently sort and contextualize it, that agency would be rather toothless and serve no real use to its government. And a search engine that is simply hitting on keywords but not delivering results in a way its users find helpful stands no chance of succeeding.
With that in mind, it should come as no surprise that the NSA has an especially tight relationship with Cray, Inc. a leading force in supercomputer development. The NSA boasts multiple massive supercomputing centers and is rumored to boast a classified system that may just be the most powerful in the world.
Meanwhile, the likes of Google, Microsoft, and facebook are currently engaged in a brain race. They are battling each other to sign up the most talented and accomplished figures in the fields of deep learning and artificial intelligence as they look to gain a competitive advantage by best exploiting the oceans of data they are continually amassing.
A number of promising new machine learning techniques designed to do just that exist in various stages of development. IBM's Watson computer system is an especially showy example; A few years back Watson made headlines by proving so adept at filtering through databanks and presenting the relevant information it retrieved that it rather easily defeated the world's best Jeopardy players at their own game. Since its landmark victory, Watson has been continually refined as IBM attempts to prove its utility in a wide variety of real-world applications, including in the medical and business fields.
Conducting research for Engines of Extinction has occasionally left me envious of Watson's ability to generate results at near-instantaneous speeds. That said, collecting, organizing, and understanding mass amounts of information is something I've got a bit of a knack for… you know, at least for a being made of mere flesh and blood.
David Brown -- a journalist and writer I respect a great deal -- once paid me a compliment that I'm still quite fond of. Upon reading an advance copy of Beyond Neptune Spear, he said, "You're like a walking computer or something to synthesize all of this material into a cohesive work."
That struck a chord with me because I've always considered it a real strength of my work. It's a skill that's helped set me apart to some degree in the world of motorcycle racing journalism as I've consistently been able to recognize and communicate trends and patterns that others failed to notice.
That fact allowed me to not only succeed as a beat reporter and columnist covering the American scene --where I've practically lived in the paddock for well over a decade -- but also write stories that carried weight in the MotoGP and World Superbike paddocks -- where I was only an occasional visitor.
My first real forays into writing about special operations -- Shaping the World from the Shadows and the aforementioned Beyond Neptune Spear -- were based (almost) purely on open source information. As such, they more or less exclusively proved to be exercises in putting this talent to use.
Despite the units' existence as classified, sometimes denied assets, there was actually a ton of information floating around about Delta Force and SEAL Team Six, but much of it was widely scattered. Moving beyond the obvious sources, if one knew where and how to look, they could find a nugget of information to be gleaned in a single paragraph from this book, or maybe in two sentences from that report, and so on. Assembled together, they allowed me to piece together puzzles and deliver a somewhat more complete overview of Delta and ST6 than had previously been made available.
Modern American Snipers followed that example in some respects, but in a much larger, more complex manner. Of course, it was also rounded out with a wealth of exclusive information courtesy of the insight provided by real-deal special operators and other insiders.
But Engines of Extinction is quite simply the ultimate test of whatever particular skill I have in absorbing massive amounts of information and then repackaging it in a cohesive, coherent manner.
Certainly, it's much, much broader, including micro-and-macro-level looks at black operations, espionage, geopolitics, and bleeding-edge science and technology. Upping the difficulty factor is the fact that all those topics present a moving target, continually evolving and progressing.
To give you some idea of the effort that's gone into researching and building the world of EoE consider the following:
At one point I was working from multiple documents. I decided it would be best if I filtered through them to cut out the fat, get it down to a more manageable size, and organize into a single document.
That newly slim and trim 'manageable' document was still well over 1000 pages long.
Yeah.
Additionally, there's also a series bible that's approx. 130 pages. It's not much more than a straight-ahead collection of facts, organizational structures, technological explanations, and other basic notes to help me keep everything straight ; it barely even touches upon the series' plot.
Sound daunting? Well, yes, for me, it can be a bit daunting to tackle at times. Not overwhelming mind you, but daunting. It is big in many respects. Much bigger than anything I've attempted before.
But that burden is placed squarely on the author, not the reader. If I do my job properly, when you read EoE, you shouldn't feel thrown into the deep end, drowning in information… well, at least not unless you want to. Again, that speaks to the book's format, which I touched on a bit in the recent SOFREP Radio podcast and will elaborate upon in a future dev blog.
The fact is, there's a whole lot of cool/interesting/disturbing stuff out there in the real world right now with a whole lot more rapidly barreling down the pike.
Engines of Extinction is our attempt to pull all of this together and deliver it in a way that actually makes sense… and is awesome.
If wielded properly, that glut of information (often dubbed 'Big data') can prove terrifically powerful. Information is power, right? However, until that information is actually accessed and understood in a meaningful way, it is useless. Worse than useless, actually.
If an intelligence agency were simply collecting vast amounts of signals intelligence (SIGINT) but lacked the ability to efficiently sort and contextualize it, that agency would be rather toothless and serve no real use to its government. And a search engine that is simply hitting on keywords but not delivering results in a way its users find helpful stands no chance of succeeding.
With that in mind, it should come as no surprise that the NSA has an especially tight relationship with Cray, Inc. a leading force in supercomputer development. The NSA boasts multiple massive supercomputing centers and is rumored to boast a classified system that may just be the most powerful in the world.
Meanwhile, the likes of Google, Microsoft, and facebook are currently engaged in a brain race. They are battling each other to sign up the most talented and accomplished figures in the fields of deep learning and artificial intelligence as they look to gain a competitive advantage by best exploiting the oceans of data they are continually amassing.
A number of promising new machine learning techniques designed to do just that exist in various stages of development. IBM's Watson computer system is an especially showy example; A few years back Watson made headlines by proving so adept at filtering through databanks and presenting the relevant information it retrieved that it rather easily defeated the world's best Jeopardy players at their own game. Since its landmark victory, Watson has been continually refined as IBM attempts to prove its utility in a wide variety of real-world applications, including in the medical and business fields.
Conducting research for Engines of Extinction has occasionally left me envious of Watson's ability to generate results at near-instantaneous speeds. That said, collecting, organizing, and understanding mass amounts of information is something I've got a bit of a knack for… you know, at least for a being made of mere flesh and blood.
David Brown -- a journalist and writer I respect a great deal -- once paid me a compliment that I'm still quite fond of. Upon reading an advance copy of Beyond Neptune Spear, he said, "You're like a walking computer or something to synthesize all of this material into a cohesive work."
That struck a chord with me because I've always considered it a real strength of my work. It's a skill that's helped set me apart to some degree in the world of motorcycle racing journalism as I've consistently been able to recognize and communicate trends and patterns that others failed to notice.
That fact allowed me to not only succeed as a beat reporter and columnist covering the American scene --where I've practically lived in the paddock for well over a decade -- but also write stories that carried weight in the MotoGP and World Superbike paddocks -- where I was only an occasional visitor.
My first real forays into writing about special operations -- Shaping the World from the Shadows and the aforementioned Beyond Neptune Spear -- were based (almost) purely on open source information. As such, they more or less exclusively proved to be exercises in putting this talent to use.
Despite the units' existence as classified, sometimes denied assets, there was actually a ton of information floating around about Delta Force and SEAL Team Six, but much of it was widely scattered. Moving beyond the obvious sources, if one knew where and how to look, they could find a nugget of information to be gleaned in a single paragraph from this book, or maybe in two sentences from that report, and so on. Assembled together, they allowed me to piece together puzzles and deliver a somewhat more complete overview of Delta and ST6 than had previously been made available.
Modern American Snipers followed that example in some respects, but in a much larger, more complex manner. Of course, it was also rounded out with a wealth of exclusive information courtesy of the insight provided by real-deal special operators and other insiders.
But Engines of Extinction is quite simply the ultimate test of whatever particular skill I have in absorbing massive amounts of information and then repackaging it in a cohesive, coherent manner.
Certainly, it's much, much broader, including micro-and-macro-level looks at black operations, espionage, geopolitics, and bleeding-edge science and technology. Upping the difficulty factor is the fact that all those topics present a moving target, continually evolving and progressing.
To give you some idea of the effort that's gone into researching and building the world of EoE consider the following:
At one point I was working from multiple documents. I decided it would be best if I filtered through them to cut out the fat, get it down to a more manageable size, and organize into a single document.
That newly slim and trim 'manageable' document was still well over 1000 pages long.
Yeah.
Additionally, there's also a series bible that's approx. 130 pages. It's not much more than a straight-ahead collection of facts, organizational structures, technological explanations, and other basic notes to help me keep everything straight ; it barely even touches upon the series' plot.
Sound daunting? Well, yes, for me, it can be a bit daunting to tackle at times. Not overwhelming mind you, but daunting. It is big in many respects. Much bigger than anything I've attempted before.
But that burden is placed squarely on the author, not the reader. If I do my job properly, when you read EoE, you shouldn't feel thrown into the deep end, drowning in information… well, at least not unless you want to. Again, that speaks to the book's format, which I touched on a bit in the recent SOFREP Radio podcast and will elaborate upon in a future dev blog.
The fact is, there's a whole lot of cool/interesting/disturbing stuff out there in the real world right now with a whole lot more rapidly barreling down the pike.
Engines of Extinction is our attempt to pull all of this together and deliver it in a way that actually makes sense… and is awesome.
No comments have been added yet.
