Property-Based Testing in a Screencast Editor

by Oskar Wickström

This is a short book on using property-based testing (PBT) within Komposition, a screencast editor. Get it on Leanpub: https://leanpub.com/property-based-te...

First, this book introduces the system under test, Komposition. Next, it goes into PBT and highlights some challenges in testing properties of an application like Komposition. Finally, the main part of the book is compromised of case studies. Each case study covers increasingly complex components and how they are tested. It includes this author's personal reflections, what bugs the tests have found, and what still remains to be improved.

This is not a tutorial on PBT, but rather a collection of motivating examples. That said, you should be able to follow along without prior knowledge of PBT.

For example, it explains how using PBT helped find and fix bugs in the specification and implementation of Komposition's video classifier. Those were bugs that would be very hard to find using example-based tests or using a static type system.

This book is based on articles published on https://wickstrom.tech. While the content is mostly the same, there are few changes bringing it up-to-date. Also, if you've already enjoyed the articles, you might want support my work by purchasing this book. Finally, you might enjoy a nicely typeset PDF, or an EPUB book, over a web page.

64 pages, ebook

First published January 1, 2019

Reviews

[Panashe M. · Rating 3 out of 5]

A good case study on using PBT. I particularly liked the idea of generating outputs and then using them to generate inputs, that hadn't occurred to me to do.

I would have liked a discussion of the drawbacks of PBT, and more examples.

[Sandy Maguire · Rating 4 out of 5]

It's mostly a rehash of his blog on the same topic, but the information is good, the topic is interesting, and as far as I can tell, nobody's ever managed to do gui-based property testing before. It's worth the read!

[Alpha]

I'm not really sure who this book is for, but I thought it was a good skim. There's good food for thought on using PBT on a real application. The code samples were somewhat obtuse for me, as I'm hardly fluent in Haskell. The scenarios make sense, though there is some amount of assumption that the codebase is already quite testable, which is probably fair.

[Bugzmanov · Rating 4 out of 5]

Good

[pluton · Rating 4 out of 5]

The title is somewhat misleading, at least for me: a "screencast editor" implies a GUI program, so property-based testing of a GUI program would be very interesting. But no, this short book (or a long how-to) is about testing several key business logic functions in the core with Hedgehog; nothing about the UI, and I think all of them are pure and don't have side effects, so this is a more mundane testing area. Of course, the book is still helpful because it reminds us of the need to separate layers of a program (some of which are testable) and that PBT is a useful approach to testing.