Text Processing in Java

by Mitzi Morris

This book teaches you how to master the subtle art of multilingual text processing and prevent text data corruption. It provides an introduction to natural language processing using Lucene and Solr. It gives you tools and techniques to manage large collections of
text data, whether they come from news feeds, databases, or legacy documents. Each chapter contains executable programs that can also be used for text data forensics. Topics • Unicode code points • Character encodings from ASCII and Big5 to UTF-8 and UTF-32LE • Character normalization using International Components for Unicode (ICU) • Java I/O, including working directly with zip, gzip, and tar files • Regular expressions in Java • Transporting text data via HTTP • Parsing and generating XML, HTML, and JSON • Using Lucene 4 for natural language search and text classification • Search, spelling correction, and clustering with Solr 4 Other books on text processing presuppose much of the material covered in this book.
They gloss over the details of transforming text from one format to another and assume perfect input data. The messy reality of raw text will have you reaching for this book again and again.

328 pages, Paperback

First published December 1, 2013