Daniel Miessler's Blog, page 80

The Difference Between Data, Information, and Intelligence

The terms intelligence, information, and data are thrown around pretty loosely in most tech circles, and this inevitably leads to people confusing and/or conflating them. What follows is a simple explanation of how the related terms are different from each other, and how they work together.

Let’s get into it.

Sand, pebble, boulder.

Data are raw, individual, and unarguable facts. Examples might include the temperature at 13:37PM on the roof of San Francisco City Hall, or the amount of light coming off the Sun according to a particular solar telescope.

Information is the combination of data into a form that can answer an everyday question. Examples might include whether a country has a higher than average GDP, or whether a particular plant is native to California.

Intelligence is the combination of information into a form that tells a story and inform decisions. Examples might include a narrative stating that Iran seems poised to invade Iraq, and that now is the time to move key resources. Or one that says a politician in Northern California seems ready to vote against Cannabis legislation, so it might not be a good time to enter that business.

Intelligence is clutch when decision-makers are facing uncertainty, which is usually always.

If you’re wrong about either current state or the rules of the game, predictions become precarious.

Intelligence is decision-support. It’s a tool for making intelligent predictions about the future—based on solid understanding of the present—in order to take a course of action that improves outcomes.

Information is used to piece together the necessary understanding of the world to create that narrative, and data is what provided the inputs for the information.

A real-world example of all three

Let us say that there is a war zone in Canada (I know, far-fetched already), and some troops are on patrol in a city center looking for the resistance’s most notorious warlord. Canadian military leaders are eager for intelligence that can help find him.

Imagine many more examples of the same kind of data.

Data: A man crossed the street at Montegue and DeGaul at 19:07PM looking suspicious.
Data: That same man left a nearby building carrying a briefcase chained to his hand.
Data: An anonymous caller called in and said the warlord is staying within 3 blocks.
Information: Encrypted radio traffic has increased by 37% compared to 2 hours ago.
Information: A number of locals sympathetic to the rebels have started leaving the area.
Intelligence: Since this warlord likes to use suitcase bombs, and this type of pattern has been seen before in previous attacks, another attack is likely to occur in the next few hours.
Intelligence: Therefore, police should be notified and we should get all of our VIPs out of the area immediately.

The data need to be harvested and assembled into information, but even then you still (generally) need a human analyst to go from information to intelligence.

It’s also important to note that intelligence can be wrong. Just because the pattern happened before doesn’t mean it’ll repeat. And vice versa.

In this case we have a simple terrorist attack that follows a pattern, but the problem gets a lot harder with events that have less of a pattern and that happen less frequently.

Summary


Data are individual observations.
Information is a useful collection of data.
Intelligence combines information to form a predictive narrative that enables better decision-making.
All three levels can have errors, and problems compound as they move up the stack.


Notes


Image from Recorded Future, which is a really cool (but expensive) product by the way.

—

Become a direct supporter of my content for less than a latte a month ($50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.

Unsupervised Learning: No. 188 (Member Edition)

This is a Member-only episode. Members get the newsletter every week, and have access to the Member Portal with all existing Member content.

Non-members get every other episode.

Sign in

or…

—

Become a direct supporter of my content for less than a latte a month ($50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.

Humans Are Genebots

Somewhere in my 20’s or 30’s I became enamored with the idea of prep schools and elite colleges. I’d never gone to either of those, so it wasn’t nostalgia. And I didn’t really know about them growing up, so it’s not as if I was bitter about missing a tangible opportunity.

I didn’t even know I was supposed to go to college, let alone a prestigious one.

The feeling was more of a slow-building envy for something that I imagined to be extremely pleasurable.

I think the catalyst may have actually been Harry Potter. I loved everything about the process he went through. He gets selected into an elite school that only certain people can go to. He shows up and there are uniforms, there are special magic classes, and the dorms are split up by age. Then there’s the sorting hat, and the Quidditch team, and the special courses like Defense Against the Dark Arts.

It hit all the right “arrival” notes on an instrument within me that I didn’t know existed.

The military is full of nested tiers of eliteness in various areas.

Now that I think about it, I didn’t have the prep school experience, but I had something similar through the military. In the Army I was in the 101st Airborne, and I was lucky enough to get Airborne and Air Assault qualified right out of basic training, and most newish people thought I was something special because of the two winged badges on my uniform.

But there are many levels above just the 101st. There are Rangers (which I passed all the excruciating mental and physical tests for but couldn’t proceed in due to an earlier injury), then there’s Special Forces, then there’s Delta. And it’s the same in the Navy and Marines and Air Force—they each have their elite, elite of the elite, and then elite within that.

Evolution gives you little rest before prodding once again

Something about that has always appealed to me. As I think about it, it reminds me of Abercrombie and Fitch marketing, when I used to walk through the mall when I was in the military. The models there often had the prep school look. The northeastern, yachting, rowing, “I got into that elite school” look about them, which is why they call them models, and why they call it marketing.

They’re selling a better version of yourself, a version that you wish you were.

It reminds me of how they told me in elementary to fear junior high, and junior high told me the same about high school, and so on through college and the real world.

And this doesn’t stop with prep schools or the military. Oh no. It just keeps going. As soon as you think you’ve hit some sort of plateau, where you can be happy with your accomplishments, that’s when you learn about the tier above that you’re not yet a part of.

This is quite strong in the tech industry. Oh, you’re in tech? Cool, are you a developer? What language do you use? Do you work at one of the top 5 tech companies? Do you have stock? Did you start a company? How much money did you raise? Do you have an exit yet?

It’s endless.

If you look at any industry, or any social structure, you can see instantiations of this. Hollywood has the awards, and the ratings, and the dinners, and short lists of top directors and actors.

I have a new goal of getting something published in the New York Times. Anything really.

Writers can be respected for getting published, but where did you get published? Salon? Oh, that’s nice. I was in WSJ. WSJ? That’s nice. I was in the Economist. It’s very exclusive. That’s cute: I have a column with the NYT. And on and on it goes.

The Pyramid of Capitalism

Of course there are multiple tiers of authors as well. It’s one thing to have a book on Amazon, and it’s another to be top in category. But the real question is whether you’ve been on the New York Times’ best sellers list. Oh, you were there? For how long? How many times? For how many books?

Oh, you have a podcast? Where are you on the charts? How many subscribers? How much do you make from sponsors? Oh, you’re a painter? How many showings have you done, at what locations? Who showed up at the afterparty?

These are all instances of evolution playing its only hand, which is prodding you towards higher and higher tiers of greatness, only to remind you immediately—if you do in fact make it—that there’s another level just beyond. Our genes can only high-five us for a brief moment, after which point your current level of success becomes the new baseline, and it’s time to start hustling once again.

Social media so addicting because it’s emulating this loop.

This is why prep schools, and uniforms, and special badges and challenge coins are so coveted: they’re the visible accouterments of a status plateau that others can look up to, even if you are already searching for the next ladder.

I know it sounds like I’m assigning agency to evolution, but I’m not.

We are exoskeletons for genes fighting to propagate. All those little squirts of happiness are part of the gamification system that genes—using evolution—created to inspire us to be their unknowing champions.

Some see this as reductionist—and perhaps it is—but I don’t see it as a cause for nihilism.

I know that beauty and friendship and love are—in some sense—illusory constructs designed to inspire our ambition, which in turn helps our genes to win.

But I don’t care.

I’d characterize this as Absurdist because it’s simultaneously accepting bleak truth and embracing the enjoyment of life.

These exoskeletons are all we have for now. They’re our whole universe. You could tell me that love is a chemical, or that a flower is a configuration of atoms. Or that these things only exist at a very specific size scale. Go down to the atomic level, or up to the planet level, and flowers have little meaning.

But so what? That happens to be the exact level of the genebot. And it’s pretty glorious.

The genebot level is where love and flowers and poetry are the most spectacular things in the universe.

Yes, it’s an illusion created to serve another life form, but it’s still pretty remarkable. And compared to being nothing, I’ll happily take it.

At least until I find something better.

—

Become a direct supporter of my content for less than a latte a month ($50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.

Unpacking the Evolution-Granted Bliss of Prep Schools and Elite Institutions

Somewhere in my 20’s or 30’s I became enamored with the idea of prep schools and elite colleges. I’d never gone to either of those, so it wasn’t nostalgia. And I didn’t really know about them growing up, so it’s not as if I was bitter about missing a tangible opportunity.

I didn’t even know I was supposed to go to college, let alone a prestigious one.

The feeling was more of a slow-building envy for something that I imagined to be extremely pleasurable.

I think the catalyst may have actually been Harry Potter. I loved everything about the process. He gets selected into an elite school, that only certain people can go to. He shows up and there are uniforms, and there are classes, with older kids and younger kids.

Then there’s the sorting hat, and the Quidditch team, and the special classes, and the interesting professors, and the various cliques within the students.

The military is full of nested tiers of eliteness in various areas.

Now that I think about it, I didn’t have the prep school experience, but I had something similar through the military. In the Army I was in the 101st Airborne, and I was lucky enough to get Airborne and Air Assault qualified right out of basic training, and most newish people thought I was something special because of the two winged badges on my uniform.

But there are many levels above just the 101st. There are Rangers (which I passed all the excruciating mental and physical tests for but couldn’t proceed in due to an earlier injury), then there’s Special Forces, then there’s Delta. And it’s the same in the Navy and Marines and Air Force—they each have their elite, elite of the elite, and then elite within that.

Evolution gives you little rest before prodding once again

Something about that has always appealed to me. As I think about it, it reminds me of Abercrombie and Fitch marketing, when I used to walk through the mall when I was in the military. The models there often had the prep school look. The northeastern, yachting, rowing, “I got into that elite school” look about them, which is why they call them models, and why they call it marketing.

They’re selling a better version of yourself, a version that you wish you were.

It reminds me of how they told me in elementary to fear junior high, and junior high told me the same about high school, and so on through college and the real world.

And this doesn’t stop with prep schools or the military. Oh no. It just keeps going. As soon as you think you’ve hit some sort of plateau, where you can be happy with your accomplishments, that’s when you learn about the tier that you’re not yet a part of.

This is quite strong in the tech industry. Oh, you’re in tech? Cool, are you a developer? What language do you use? Do you work at one of the top 5 tech companies? Do you have stock? Did you start a company? How much money did you raise? Do you have an exit yet? It’s endless.

If you look at any industry, or any social structure, you can see signs of this. Hollywood has the awards, and the ratings, and the dinners, and the social groups of top directors and actors.

I have a new goal of getting something published in the New York Times. Anything really.

Writers can be respected for getting published, but where did you get published? Salon? Oh, that’s nice. I was in WSJ. WSJ? That’s nice. I was in the Economist. It’s very exclusive. That’s cute: I have a column with the NYT. And on and on it goes.

The Pyramid of Capitalism

And there are infinite tiers of authors as well. It’s one thing to have a book on Amazon, and it’s another to be top in category. But the real question is whether you’ve been on the New York Times’ best sellers list. Oh, you were there? For how long? How many times? For how many books?

You see the same patterns everywhere. You have a podcast? How are you on the charts? How many subscribers? How much do you make from sponsors? You’re a painter? How many showings have you done, at what locations? What parties do you get invited to?

These are all instances of evolution playing its only hand, which is prodding you towards eliteness, only to remind you immediately—if you do in fact make it—that there’s another level just beyond.

It seems evolution can only high-five us for a brief moment, after which point your current level of success becomes the new baseline, and it’s time to start hustling.

This is why prep schools, and uniforms, and special badges are so coveted. They’re the visible accouterments of a status plateau that others can look up to, even if you are already searching for the next ladder.

—

Become a direct supporter of my content for less than a latte a month ($50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.

Unsupervised Learning: No. 187

Get the podcast for this episode.

.errordiv { padding:10px; margin:10px; border: 1px solid #555555;color: #000000;background-color: #f8f8f8; width:500px; }#advanced_iframe {visibility:visible;opacity:1;}#ai-layer-div-advanced_iframe p {height:100%;margin:0;padding:0}

—

Become a direct supporter of my content for less than a latte a month ($50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.

Time Speeds Up When You’re Wasting It

Perhaps the title is a tiny bit strong, but I don’t think by much.

I read something recently where a memory expert was talking about why time seems to speed up as we age, while it seemed to move infinitely slow when we were kids.

He said it was because new and interesting experiences are like temporal milestones, and the more you have the slower time seems to be moving.

Intuitively, that checks out. If you spend 10 years of your life eating the same cereal, watching the same show on TV, and going to bed at the same time, it seems like you’re brain wouldn’t have anything to mark time on. You could have been doing that for 10 days or 10 years.

But as children, everything is new. Everything is novel. Everything is interesting.

Every year in school is a new type of magic. Every new friend. Every girl or boy you get a crush on. Your first music. Favorite foods. Becoming somewhat independent. Adolescence. It’s all non-stop novelty, with no time to rest.

Perhaps living in the present slows time, while living in the past or future speeds it up.

And at that speed of experience, you can’t possibly imagine even a few months in the future. Might as well be 30 years, it’s so far off.

But when you’re 55 and just going through the motions, you blink, look at the calendar, and another decade has passed.

I can’t remember what the solution was, and perhaps it’s not known for certain anyway. But it was some combination of the following:

Do new things all the time.
Always be learning.
Focus on the pure joy of whatever experience you’re having; don’t focus on the past or the future.

That #3 is key, and it could be a major part of the formula for slowing time.

As kids we’re not just having more experiences, but we’re fully immersed in them as well. We’re not thinking about college and jobs, and we’re not thinking about mistakes we made when we were 2 years younger. We’re fully dedicated to the moment.

So whether it’s the constant newness or the mindfulness, some combination thereof is producing this effect of slowing time. And slowing time is the same as living longer.

Someone who lives like a child until they’re 80 might live 200 years in experience-time, while someone who locks into a routine or obsesses over the future and past might live to 90 but only live for 30 of them.

I think the lesson is: the more you experience life, the more life you will experience.

—

Become a direct supporter of my content for less than a latte a month ($50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.

Unsupervised Learning: No. 186 (Member Edition)

This is a Member-only episode. Members get the newsletter every week, and have access to the Member Portal with all existing Member content.

Non-members get every other episode.

Sign in

or…

—

Become a direct supporter of my content for less than a latte a month ($50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.

Book Summary: Fall

3/10

My One-Sentence Summary
Content Extraction
Takeaways

My book summaries are designed as captures for what I’ve read, and aren’t necessarily great standalone resources for those who have not read the book.

Their purpose is to ensure that I capture what I learn from any given text, so as to avoid realizing years later that I have no idea what it was about or how I benefited from it.

My One-Sentence Summary

This book is like someone describing a dream to you, in vivid detail, for around 30 hours. The only respite from that purgatory comes in the form of copious asides that are engineered to make you believe the author is smart, which we already knew.

Content Extraction

I thought some parts of the description of the apocalyptic America were interesting
I thought some of the tech in the virtual world was interesting

Takeaways

If you are a total fan of Stephenson and you love anything he writes, you might like this book.
Everyone else probably won’t.
I doubt I’ll be reading any more Stephenson.

You can find my other book summaries here.

—

Become a direct supporter of my content for less than a latte a month ($50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.

The Dangers of Abruptly Destroying Meaning Structures

I recently watched an older debate between Sam Harris and Jordan Peterson in which I saw Sam admonishing Jordan for creating an opening for foolish beliefs. He took a poll of who was religious in the audience, and then all but blamed Jordan for their credulity.

That’s when it hit me that while Jordan might have enabled the religious, Sam actually enabled Jordan.

How? The same way George Bush enabled Iran to take over Iraq by removing Saddam.

Sam’s ideology is brilliant at stripping someone to the bone, but it unfortunately leaves people wishing they had a jacket.

The New Atheists created a vacuum of belief and meaning among their followers, and their sadness can be felt in questions posed to Sam in AMAs and QAs. I’ve seen multiple questions like the following from his fans, delivered in almost shameful and apologetic tones:

Sam actually does answer this, but he does it in book form (Waking Up) as opposed to offering guides and methodologies like Jordan.

Thank you for freeing me, but…um…what do I believe now?

It’s a question Sam’s never given a good answer to, and one that Jordan Peterson has become a religious figure by answering well.

Peterson is like Iran taking advantage of the vacuum left in Iraq. People need structure, and he’s providing it.

I’m not implying here that Peterson is malicious or cynical in what he’s offering.

Peterson is Iran sweeping into a post-Saddam Iraq. He’s offering structure to those who desperately need it to explain their struggles, just as Iraqis have been craving a strong hand to govern them.

The lesson here is not that Sam should not have dispelled the bad magic of supernatural belief. Neither is it that Jordan is being opportunistic. I think he’s as surprised as anyone that people are so eager to hear him speak.

The lesson is that most humans need structure, and Sam made a mistake in assuming that more people were like him (or me) in being able to erect an existential scaffolding of our own making. Most people can’t do that. And even for those who might be able to, it’s likely to require a process.

Iraqis are just as capable of self-government as anyone else on Earth. Just as former Christians are plenty capable of secular and existential meaning creation. But it takes time.

He also thought that well-constructed conversations could solve all disagreements.

Summary


Harris was too optimistic about the ability for people to construct their own meaning once false structures had been removed.
That miscalculation left people 1) feeling empty, and 2) feeling weak and ashamed that they did need something.
Peterson’s message suddenly started resonating because it was precisely the structure that could fill the void left by the New Atheists.
Sam (and others) are now frustrated that Peterson acolytes might have just swapped one religion for another, and/or left the door open to other fallacious beliefs.



I think Sam is one of the top living intellectuals in the world, by the way. I have massive respect for him. This is not an attack on what he did, but rather an explanation of what happened afterwards.
I disagree significantly with Jordan Peterson on major topics, and particularly his flirtation with what I consider to be numerology, but I do think he is fundamentally a good person who is trying to help people. And I think he’s tapped into something powerful in the lack of meaning in the lives of young men.

—

Become a direct supporter of my content for less than a latte a month ($50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.

Unsupervised Learning: No. 185

.errordiv { padding:10px; margin:10px; border: 1px solid #555555;color: #000000;background-color: #f8f8f8; width:500px; }#advanced_iframe {visibility:visible;opacity:1;}#ai-layer-div-advanced_iframe p {height:100%;margin:0;padding:0}

—

Become a direct supporter of my content for less than a latte a month ($50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.