Prawdziwa głębia OSINT. Odkryj wartość danych Open Source Intelligence

by Rae L. Baker, Piotr Rakowski (Translator)

OSINT (ang. open source intelligence) polega na pozyskiwaniu danych wywiadowczych z ogólnodostępnych źródeł. Jest to zestaw praktycznych umiejętności, które przydadzą się nie tylko analitykom - metody białego wywiadu okazują się pomocne na wielu ścieżkach kariery, a także w życiu codziennym. Łatwo się przekonasz, że OSINT pozwala uzyskać niezwykle cenne informacje, a przy tym jest satysfakcjonującym i ciekawym zajęciem!

Dzięki tej książce nauczysz się gromadzić publicznie dostępne informacje, korzystać z wiedzy o cyklu życia wrażliwych danych i przekształcać je w informacje wywiadowcze przydatne dla zespołów zajmujących się bezpieczeństwem. Opanujesz proces gromadzenia i analizy danych, poznasz również strategie, które należy wdrożyć podczas poszukiwania informacji z publicznie dostępnych źródeł. Ugruntujesz wiedzę na temat bezpieczeństwa operacyjnego i uświadomisz sobie, w jaki sposób niektórzy używają publicznie dostępnych danych do nielegalnych celów. Książkę tę szczególnie docenią inżynierowie społeczni i specjaliści do spraw bezpieczeństwa, a także kadra kierownicza.

Najciekawsze zagadnienia:

strategie stosowania urządzeń IoT do gromadzenia danych wywiadowczych
pozyskiwanie danych przy użyciu publicznie dostępnych informacji transportowych
techniki poprawy bezpieczeństwa operacyjnego
zagrożenia związane z ogólnodostępnymi danymi
metody gromadzenia danych wywiadowczych stosowane przez najlepsze zespoły do spraw bezpieczeństwa
Dostrzegaj to, co pozostaje niewidoczne dla innych!

Genres: Nonfiction, Computer Science, Research

504 pages, ebook

First published May 9, 2023

Reviews

[Amy · Rating 5 out of 5]

As an OSINT instructor and practitioner, this is the best primer I’ve read thus far on the subject. It’s the book I wish I had when I started working for the government over 15 years ago. Despite having a firm foundation of knowing how to search and how to stay safe online because of my background as a librarian, there are nuances to OSINT, especially working in LE or the IC, and this is the guide I would give those starting out now. It’s what I would recommend those who want a refresher in investigative methods, and OPSEC, and it’s what I’ll recommend to my managers who want something new refer to when they dig back into casework. Rae has thought of everything - from kicking off with critical thinking, bias, and ethics - which were mainstays in our Bibliographic Instruction days, to the HOW and WHY we do this, not just WHERE to go. This is the book the OSINT community needs now, and they’ll continue to reach for time and again.

[C · Rating 4 out of 5]

Useful guide to a wide variety of OSINT techniques. It covers the intelligence cycle and OPSEC, and OSINT related to people, social media, businesses and organizations, transportation, critical infrastructure and industrial controls, finance, cryptocurrency, and NFTs. The book includes many tools, but Baker says she deliberately limited their number, to focus on methodology over tools. Her site lists OSINT tools and resources.

The book includes several examples that feature the political right in a negative light, but I didn't notice the political left referenced negatively. I think it would've been better to avoid political examples, or at least treat the political spectrum more evenly.

Notes
Open Source Intelligence
OSINT: production of intel from passively collected data that's available for public access without use of secret clearance or system intrusion; may also include data behind paywall

Jobs & fields that involve OSINT
• Journalism
• Intelligence
• Government
• Armed forces
• Business
• Genealogy
• Education (training)
• Private investigation
• Security assessments

David T. Moore's interpretation of Paul and Elder's Critical Thinking model
1. Requirements: Define scope of data collection
2. Key Questions: Define key questions intel should answer
3. Considerations: What evidence should we see? What effects would evidence have?
4. Inferences: Determine evidence being inferred, biases involved
5. Assumptions: Determine what's being assumed about evidence, key questions
6. Concepts: Determine reliability of evidence, outcome of collection method
7. Implications and Consequences: Define potential outcomes given correct or incorrect conclusions for key questions
8. Points of View: Define other points of view on situation

The Intelligence Cycle
Hunchly, Vortimo: collection and documentation

Documentation
• Always assume notes will be shared
• Include screenshots with captions
• Record URLs and sources you capture
• Use defanged or disabled links
• Include tables to organize selectors
• Document processes and pivoting steps
• Record dates and times
• Articulate what you did and how

Obsidian: note‐taking, knowledge base software

Reports
• Include both data and visuals, because people differ in preference
• Focus on "so what?" so reader understands value, takeaways
• Ensure all readers understand message, same key points
• Use active, persuasive, analytical tone
• Include figure numbers for images, table numbers for tables
• Cite all URLs, references

Inverted Pyramid for reports
1. Beginning: executive summary
2. Middle: what you found and how; remediation suggestions
3. End: least important info (index, appendix)

Report layout
1. Title, date
2. Executive summary: bottom‐line‐up‐front (BLUF) statement
3. Body/analysis: describe investigative work, answer intel questions
4. Summary: restate executive summary, give condensed version of analysis and findings
5. Recommendations
6. Appendix

The Adversarial Mindset
First, develop idea of types of attacks that hackers can perform, what type of data they may be interested in. Then, try to understand attacker's motivates, capabilities.

Questions to analyze attacker means, motives, opportunities
• What tools could the attacker have access to?
• What times might they be active?
• What could be important to them? Why?
• What resources might they have?
• Could we detect any patterns in attack?

Operational Security
Threat modeling
• Persona non grata (PnG): develop personas for potential attackers, identifying their possible goals, methods, abilities
• Security or "baseball" cards: develop personas for potential attackers behind sophisticated or extraordinary attacks. Create deck of 42 cards in 4 threat categories (Human Impact, Adversary Motivations, Adversary Resources, Adversary Methods). Shuffle cards to create combinations, develop OPSEC strategy.
• Attack trees: create tree diagrams for each attack goal, break down each step in process, develop OPSEC countermeasures.

OPSEC Tips
• Block microphone, camera
• Block tracking cookies
• Be aware of data shared by browser
• Be aware of browser and IP fingerprinting
• Keep software updated
• Adjust time and location of social media posts to appear to be elsewhere
• Use VPN when accessing public Wi‐Fi
• Delete unused accounts
• Use MFA

Using VPN can prevent ISP from detecting Tor traffic (some ISPs block Tor).

I2P can be difficult to properly install and configure within browser. There's no guaranteed privacy when browsing indexed sites on surface web. Mandated login reduces privacy.

Mobile emulators (BlueStacks, Genymotion) are free alternative to burner phones.

Sock puppets (research accounts)
• Design accounts to blend in
• Mature accounts before needing them, by performing normal activities during normal hours (for persona location)
• Generate fake info (fakepersongenerator.com can help)
• Generate fake photo (thispersondoesnotexist.com can help) and adjust (flip, crop, change color, move eyes off center)
• When required to give phone number for account, use prepaid SIM card and cheap phone (VoIP numbers usually rejected)
• Automatically post with IFTTT

Sock puppet details
• First and last name
• Nickname
• Username
• Physical appearance (weight, height, tattoos)
• Date of birth
• Place of birth
• Location (country, state, city)
• Phone number(s)
• Education
• Employment
• Dependents (children, elderly)
• Work history
• Political affiliation
• Relationship status
• Pets
• Hobbies
• Relatives

OSINT Touchpoints
Searching
• Use search engine(s) most relevant to subject you're researching (e.g., Yandex for Russian subjects).
• Use multiple search engines.
• Set VPN to location of subject.
• When searching foreign subjects, try English and native spelling.

Industrial intel Google dorks
• "CompanyName" and "Siemens"
• "Partners" site: companysite.com
• intitle:index of "aws/credentials"

Subject Intelligence
Put part of subject's email address through username tools (e.g., WhatsMyName.app, Sherlock).

Google plots each user's reviews on Google Maps, revealing pattern of life. Search Gmail address with Epieos to find subject's reviews.

EmailRep.io shows user accounts associated with email address.

Intelx.io provides access to breach data.

Public records
• Free, public resources (search engine queries, state/local websites, publication databases)
• Paid tools (LexisNexis, Thomson Reuters, etc.)
• Go to building where records are kept

PACER gives access to federal records (court documents, case files).

Find state resources with Google dorks (e.g., "New Jersey" and "Criminal Records" or "California" and "Divorce Records"). Use only .gov sites.

Each state's Secretary of State site contains government records which may lead to names of shareholders, directors, others.

Search entity's address to see if other businesses were registered under it. If you know names of officers, look into their connections with other entities, which could reveal motivations.

Voting records can reveal subject's address by year, other people who may live at address, employment, political connections that might influence subject's actions.

Local government records include misdemeanor crimes, traffic violations, ordinance violations.

Property records can give insight into subject's addresses, wealth, family. Access through county tax assessor websites, or Zillow and Trulia.

Airbnb and Vrbo can reveal pattern of life, photos, building layouts, renter's personal details.

Know privacy laws of country you're in and subject is in.

BRB Publications provides free access to listing of datasets from government agencies. Public Accountability Project provides curated public information about people and organizations.

Social Media Analysis
Track data points with OneNote and i2 link analysis chart.

Correlate accounts
• Accounts of subject's friends or follows that share same username
• Accounts of subject's friends or follows that share same profile photo
• Accounts that consistently react to subject's images, posts

The above indicate subjects follow and interact with themselves. On Instagram, scroll to bottom of friends and followers list and use Instant Data Scraper browser extension to create CSV file.

Visualize connections between individuals with association matrix or link analysis chart (use i2, Maltego, mind maps, etc.).

Check posts for indications of stress, family problems, addiction, etc.; code words (gangs, trafficking, exploitation) and emoji indicating those codes.

TGStat.com offers catalogs of Telegram channels for several non-US countries.

Misinformation is not knowingly deceptive. Disinformation is deliberately deceptive. Malinformation is based on reality, but purposefully harmful.

Bot Sentinel rates Twitter users on bot-like behavior.

Social media network analysis: Neo4j Community Edition, Gephi, Maltego Community Edition

Error‐level analysis (ELA) of images: Forensically, FotoForensics

Spot deep fakes
• Facial distortions (smooth skin, strange shadowing)
• Unnatural lighting on glasses
• Strange edges around facial hair, moles
• Mismatched coloring of face and lips
• Abnormal blinking
• Lack of detail within teeth
• Glitching when person turns side to front

Password knocking: requesting new password, which sometimes reveals partial email address; can alert owner to investigation

Business and Organizational Intelligence
Data points
• Corporate/business structure disclosures (parent, subsidiary, holding companies)
• Contract disclosures
• Financial records, annual reports
• Affiliations, relationship disclosures
• Procurement, supply chain disclosures
• Proprietary technology disclosures
• Business discretions, lawsuits
• Sanctions, illegal activity
• Public disclosures
• Published material disclosures

OpenCorporates: company info pulled from government data

Paid sources (D&B Hoovers, Dow Jones Risk and Compliance, World Compliance Lexis Nexis) may have access to more comprehensive data than free sources.

EDGAR: company quarterly and annual statements

Annual reports include a lot of company info, as do Forms 10‐K, 10‐Q, 8‐K.

SBIR.gov: see research projects of US universities, through small business loan and seed fund applications

Good Jobs First Violation Tracker: find regulatory violations, misconduct

USASpending.gov, SAM.gov: federal government spending data

Power mapping: understanding who holds power in an org or community, their motives

LittleSis: data about how money and power dictate policies, contributions, contracts

Find illegal shell companies
• Company's registration supports anonymity (e.g., registered in tax haven)
• Company has meaningless or vague name
• Owners can't be found online
• No real way to contact company

Find sanctions
• UN Security Council
• Office of Foreign Assets Control
• Paris MOU on Port State Control
• Financial Action Task Force (FATF)

Nonprofit docs
• Form 990 (find at ProPublica, Candid)
• https://apps.irs.gov/app/eos
• Federal Audit Clearinghouse

robots.txt may reveal hidden pages.

Google dork to find publicly exposed files: site:domain.com ext:doc | ext:docx | ext:odt | ext:rtf | ext:sxw | ext:psw | ext:ppt | ext:pptx | ext:pps | ext:csv

https://github.com/ElevenPaths/FOCA: find hidden data within website

dnslytics.com: visualize info about domains, IPs, providers, relationships between them

Transportation Intelligence
Sentinel Hub's EO Browser combines satellite imagery in one database.

soar.earth and Google Earth aggregate public and private satellite imagery.

marinetraffic.com, vesselfinder.com: marine tracking

OpenRailwayMap: railway details

Geops: live train tracker

TravelTime: see how far subject could've traveled in time frame

Plane tracking: planespotters.net, #planespotters, FlightRadar24, ADSBExchange, FlightAware

Find webcams: Google dorks such as "airport name" and "webcam" or Windy.com

Airportia: search flights

utopiax.org: track airborne cargo (requires Airwaybill [AWB] number)

carnet.ai: recognizes car make and model

platesmania.com: license plate photos

faxvin.com: car history, license plates

worldlicenseplates.com: current and historical license plates

VIN Decoder: https://vpic.nhtsa.dot.gov/decoder

truckingdatabase.com: truck and bus company data

Critical Infrastructure and Industrial Intelligence
Kamerka: ICS, IoT device search

Opensignal: market insights for communication networks

macaddress.io: MAC address info

OpenCellID.org, cellmapper.net: cell tower info

Financial Intelligence
binlist.io, binlist.net: credit or debit card info

Prepaid cards may indicate money laundering.

Organized crime Wikipedia entry: criminal organization and activity info

National Gang Center's Information Exchange: gang info

Cryptocurrency
dark.fail: onion links for markets, forums, sites

https://hunch.ly/darkweb-osint/: weekly Dark Web market reporting, including onion links

Start with subject of interest
• Can we tie subject to cryptocurrency wallet?
• Are there relevant cases tied to subject?
• Does subject's name appear in NFT search results?
• Does subject's name appear in wallet search results?
• Does subject disclose wallet addresses (collecting donations, posting on forum, etc.)?
• Can you tie transactions to subject?
• Does subject have accounts on multiple blockchains?
• Does subject own NFTs?
• Are usernames, passwords, or other identifiers attached to subject?
• Do wallet addresses show up in search engines?
• What are dates of transfers?
• Does subject have vanity address?

etherscan.io: wallet search

opensea.io: NFT search

Start with wallets of interest
• Can we tie wallet addresses to a person?
• Are there relevant cases to reference for ideas?
• Do addresses have history?
• Do addresses appear on Dark Web?
• Do addresses appear in wallet search results?
• Do wallet accounts disclose an identity?
• Do associated transactions lead to an identity?
• Do addresses appear in search engines?
• Do addresses appear in sanctions search?
• Are addresses tied to illicit sales ads?
• Are usernames, passwords, or other identifiers attached to addresses or wallet?
• What are dates of transfers?
• Do fund recipients reveal their identity?
• Do wallets appear on other blockchains?
• If address is used on Dark Web, is it tied to any online accounts?

Start with transaction of interest
• Can we tie transaction to a subject?
• Are there relevant cases to reference for ideas?
• Does address have history?
• Do you have wallet address tied to transaction through wallet search?
• Are there other transactions?
• Do any wallet accounts disclose an identity?
• Do any transactions lead to an identity?
• Is address tied to other blockchains?
• Which addresses do transactions go to?
• Is there a pattern to payment amounts?
• Can addresses be found in search engines?
• Are usernames, passwords, or other identifiers attached to addresses or wallet?
• What are dates of transfers?
• Do addresses appear on Dark Web?
• Do addresses appear on websites with identifying info?
• If address is used on Dark Web, is it tied to any online accounts?

Non‐fungible Tokens
https://www.nyckel.com/nft-finder/: search text or image to find similar NFTs

app.ens.domains: search .eth address to see associations

Look for metadata in NFT files, often saved in IPFS. Inspect smart contract under Details.

[Vincent S. · Rating 4 out of 5]

This was such a great primer for OSINT and I think it covered every conceivable area. This also made it extremely dense, and at points a tough read. I highly recommend it.

[Teoman · Rating 5 out of 5]

The best OSINT book in my opinion

[Jonathan Cote · Rating 5 out of 5]

Deep Dive does an excellent job at providing the reader with foundational-to-intermediate knowledge of OSINT strategies, techniques, and tooling. It is methodically written and can serve as an exceptional OSINT reference book. BUT, that does not mean it is a dry read. To the contrary, it contains many interesting references and stories. Rae’s investigation of a puppy scam was truly amazing work and an excellent example of the power of OSINT. She also uses great examples of viewing images with an investigator’s eye, the use of security ‘baseball’ cards for threat modeling, and how predicting future crimes like in the 2002 movie, Minority Report, is not as far-fetched as it sounds.

This book is simply loaded with great learnings like Google dorking, pattern of life analysis, image geolocation, data aggregator usage, web crawlers, mind map diagrams to better understand pivoting in an investigation and more. Particularly valuable is operational security of covering one’s tracks, and this goes far beyond using a VPN or Tor.

Rae stresses that intangible skills, such as critical thinking and having a curious mindset, are far more important than technical mastery of various tools. Rae focuses on teaching the methodologies for the various touch points over that of technical skills. She still of course references many technical tips and tools, including a super valuable web page of her favorite resources, but if you don’t understand the why or how to adopt the proper approach, then your technical abilities may be of little value.

I definitely recommend Deep Dive as your first place to go to learn about the world of OSINT.