Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification

by Jonathan A. Zdziarski

Join author John Zdziarski for a look inside the brilliant minds that have conceived clever new ways to fight spam in all its nefarious forms. This landmark title describes, in-depth, how statistical filtering is being used by next-generation spam filters to identify and filter unwanted messages, how spam filtering works and how language classification and machine learning combine to produce remarkably accurate spam filters. After reading Ending Spam , you'll have a complete understanding of the mathematical approaches used by today's spam filters as well as decoding, tokenization, various algorithms (including Bayesian analysis and Markovian discrimination) and the benefits of using open-source solutions to end spam. Zdziarski interviewed creators of many of the best spam filters and has included their insights in this revealing examination of the anti-spam crusade. If you're a programmer designing a new spam filter, a network admin implementing a spam-filtering solution, or just someone who's curious about how spam filters work and the tactics spammers use to evade them, Ending Spam will serve as an informative analysis of the war against spammers. TOCIntroduction PART An Introduction to Spam FilteringChapter 1: The History of SpamChapter 2: Historical Approaches to Fighting SpamChapter 3: Language Classification ConceptsChapter 4: Statistical Filtering Fundamentals PART Fundamentals of Statistical FilteringChapter 5: Uncombobulating MessagesChapter 6: The Building Blocks of SpamChapter 7: The Low-Down Dirty Tricks of SpammersChapter 8: Data Storage for a Zillion RecordsChapter 9: Scaling in Large Environments PART Advanced Concepts of Statistical FilteringChapter 10: Testing TheoryChapter 11: Concept Advanced TokenizationChapter 12: Fifth-Order Markovian DiscriminationChapter 13: Intelligent Feature Set ReductionChapter 14: Collaborative Algorithms Shining Examples of Filtering Index

312 pages, Paperback

First published July 1, 2005

Reviews

[Nick Black · Rating 2 out of 5]

I work with Zdziarski on McAfee's research team, and have no qualms calling out this book as a cash-in pile of shit (admittedly, his recent book on Android or the iPhone or something like that is supposed to be pretty good) -- not that there's anything wrong with being the author of such, but I wouldn't want to be the buyer (again). I remember it being largely a farrago of existing, widely-known ideas (Graham's A Plan for Spam, etc), some confused and inconclusive material on machine learning (Markovian discrimination, n-gram automata, etc) and winnowing hashes, and filler material about setting up various systems with his DSPAM software (the equivalent of including hard-copy manual pages in your bottom-market UNIX reference).

[James Luedke · Rating 4 out of 5]

Well written covers advanced concepts of statistical filtering in depth. A good read for software developers or sys admins who deal with spam.

[David · Rating 3 out of 5]

On the off chance that you need to roll some spam classification technology yourself, and you need a really gentle introduction in book form, then have at it.