Semi-Supervised Learning and Domain Adaptation in Natural Language Processing

by Anders Søgaard

This book introduces basic supervised learning algorithms applicable to natural language processing (NLP) and shows how the performance of these algorithms can often be improved by exploiting the marginal distribution of large amounts of unlabeled data. One reason for that is data sparsity, i.e., the limited amounts of data we have available in NLP. However, in most real-world NLP applications our labeled data is also heavily biased. This book introduces extensions of supervised learning algorithms to cope with data sparsity and different kinds of sampling bias. This book is intended to be both readable by first-year students and interesting to the expert audience. My intention was to introduce what is necessary to appreciate the major challenges we face in contemporary NLP related to data sparsity and sampling bias, without wasting too much time on details about supervised learning algorithms or particular NLP applications. I use text classification, part-of-speech tagging, and dependency parsing as running examples, and limit myself to a small set of cardinal learning algorithms. I have worried less about theoretical guarantees ("this algorithm never does too badly") than about useful rules of thumb ("in this case this algorithm may perform really well"). In NLP, data is so noisy, biased, and non-stationary that few theoretical guarantees can be established and we are typically left with our gut feelings and a catalogue of crazy ideas. I hope this book will provide its readers with both. Throughout the book we include snippets of Python code and empirical evaluations, when relevant.

Genres: Nonfiction

103 pages, Kindle Edition

First published June 1, 2013

About the author

Anders Søgaard (f. 1981, Odense) er forfatter og professor ved Datalogisk Institut på Københavns Universitet. Han er uddannet fra Forfatterskolen i 2002 og har udgivet seks skønlitterære bøger.

I 2019 udkom Landskabsdigternes klub, om hvad der sker, når vi forsøger at skubbe grænserne for, hvad der er os menneskeligt muligt – i livet og i sproget. I 2023 udkom Iliaden, der forholder sig til Homers Iliaden, som både en omskrivning, en fantasi, en landskabspoesi, en rejsefortælling, en kritik af oldtidsforskning og et forsøg på at læse sig ind på fiktive personer.

I 2024 udkom Maskinerne kommer indefra - Forstå kunstig intelligens, før den forstår dig. I den populærvidenskabelige bog, der fik fem hjerter af Politikens anmelder, skriver Søgaard om det moderne menneskes møde med kunstig intelligens, og om hvordan teknologien bag den opstod.

I Pakkeliste til fremtiden, der udkom i 2025, giver Søgaard både filosofiske værktøjer og råd til rejsen ind i en fremtid domineret af tech-industrien. Bogen er både kulturkritik, historisk analyse og en filosofisk "prepper-manual" til det mulige samfund, der venter lige om hjørnet.

Anders Søgaard betragtes som en af Danmarks førende eksperter i maskinlæring og kunstig intelligens.

Reviews

[Zhijing Jin · Rating 4 out of 5]

Finished reading. I use it to survey traditional NLP works on (1) semi-supervised learning, and (2) domain adaptation. Most papers covered in the book are on syntax parsing and part-of-speech tagging.

(Takeaway 1) The spirit behind many methods can still be seen in recent papers, e.g., in 2010-2020.
(Takeaway 2) In most works mentioned in the paper, there is not necessarily guarantee that semi-supervised learning will bring improvements. See the potential reasons listed in Chapelle et al. (2006), and Schölkopf et al. (2010).

[Nancy · Rating 4 out of 5]

Good light overview. Has Python/SkLearn examples to help with explanation.