Clean Data - Data Science Strategies for Tackling Dirty Data

by Megan Squire

Key FeaturesGrow your data science expertise by filling your toolbox with proven strategies for a wide variety of cleaning challengesFamiliarize yourself with the crucial data cleaning processes, and share your own clean data sets with othersComplete real-world projects using data from Twitter and Stack OverflowBook DescriptionIs much of your time spent doing tedious tasks such as cleaning dirty data, accounting for lost data, and preparing data to be used by others? If so, then having the right tools makes a critical difference, and will be a great investment as you grow your data science expertise.

The book starts by highlighting the importance of data cleaning in data science, and will show you how to reap rewards from reforming your cleaning process. Next, you will cement your knowledge of the basic concepts that the rest of the book relies file formats, data types, and character encodings. You will also learn how to extract and clean data stored in RDBMS, web files, and PDF documents, through practical examples.

At the end of the book, you will be given a chance to tackle a couple of real-world projects.

What you will learnUnderstand the role of data cleaning in the overall data science processLearn the basics of file formats, data types, and character encodings to clean data properlyMaster critical features of the spreadsheet and text editor for organizing and manipulating dataConvert data from one common format to another, including JSON, CSV, and some special-purpose formatsImplement three different strategies for parsing and cleaning data found in HTML files on the WebReveal the mysteries of PDF documents and learn how to pull out just the data you wantDevelop a range of solutions for detecting and cleaning bad data stored in an RDBMSCreate your own clean data sets that can be packaged, licensed, and shared with othersUse the tools from this book to complete two real-world projects using data from Twitter and Stack OverflowAbout the AuthorMegan Squire is a professor of computing sciences at Elon University. She has been collecting and cleaning dirty data for two decades. She is also the leader of FLOSSmole.org, a research project to collect data and analyze it in order to learn how free, libre, and open source software is made.

Table of ContentsWhy Do You Need Clean Data?Fundamentals – Formats, Types, and EncodingsWorkhorses of Clean Data – Spreadsheets and Text EditorsSpeaking the Lingua Franca – Data ConversionsCollecting and Cleaning Data from the WebCleaning Data in Pdf FilesRDBMS Cleaning TechniquesBest Practices for Sharing Your Clean DataStack Overflow ProjectTwitter Project

274 pages, Kindle Edition

First published May 1, 2015

Reviews

[Daniel Noventa · Rating 3 out of 5]

Decent enough as an overview of cleaning data. Lacks a lot of practical information and doesn't focus on a single language. Would've been nice for consistency. I would have preferred a more generic brushing over topics like using excel and text search, but for a complete beginner that may be useful.

[Claudio Moreno · Rating 3 out of 5]

This book is okay, I was looking for a book that came with a framework that helped me think about cleaning and preparing data in a consistent and repeatable way. Instead I got the beginnings of a framework and some observations and tips that are bread and butter for anyone that works with data for even a short time. Having said that, this book did give me the basics for making my own framework and did give me some insights into the way to make the initial treatment of data more generic

[Johnny · Rating 4 out of 5]

The introduction in Clean Data is well done and explains the need to work along scientific rules. Only when one rigorously documents all the actions one does with data one can reproduce the outcome later on. The process described by Megan Squire sound well thought through and should help with the more complex work described in the book.

The next chapters are all about extracting data and transforming the input into the required output. That part is on a low technical level and talks about character encodings and file formats. All things you need to work with data, but the high level introduction in chapter one is now abandoned until you reach the big projects in chapter 9 and 10.

The big projects where unfortunately not the strong parts of the book. Where all the parts should have come together is now another mix from the high level data science process with low level inner workings of Twitter and StackOverflow. It’s hard to keep the levels apart when there is so much going on. If Twitter and StackOverflow would have been introduced in a chapter on their own, then the last two chapters could focus on the data cleaning part. As it is now, you may need to do that work for yourself, what harms the idea of the book.

However, this book is the best I read from Packt Publishing in a long time. If you are interested in working with data from different sources, you still should buy the book. The technical part is done very well and will help you a lot.

[Jose Manuel · Rating 1 out of 5]

La autora establece la limpieza de datos previa a cualquier paso de analítica como el más importante proceso a tener en cuenta a la hora de presentar resultados con la calidad adecuada, en el primer capítulo.

El resto de los capítulos se dedica a desmontar lo que ha dicho en el primero.
Insustancial, con pocas aportaciones de valor, sin entrar en profundidad en nada. Un par de proyectos de data scraping, cuatro formateos de fechas, y a correr.

Desde luego este no es un libro que debas leer si llevas tiempo en esto de las bases de datos porque a buen seguro has tenido que hacer cosas muchisimo más complejas.

[Richard Pavlovsky · Rating 3 out of 5]

A decent book, some good strategies. Learned a few tools and approaches. Not revolutionary for me.