Software Testing with Generative AI

by Mark Winteringham

Speed up your testing and deliver exceptional product quality with the power of AI tools.

The more you test, the more you learn about your software. Software Testing with Generative AI shows you how you can expand, automate, and enhance your testing with Large Language Model (LLM)-based AI. Your team will soon be delivering higher quality tests, all in less time.

In Software Testing with Generative AI you’ll learn how

• Spot opportunities to improve test quality with AI
• Construct test automation with the support of AI tools
• Formulate new ideas during exploratory testing using AI tools
• Use AI tools to aid the design process of new features
• Improve the testability of a context with the help of AI tools
• Maximize your output with prompt engineering
• Create custom LLMs for your business’s specific needs

Software Testing with Generative AI is full of hype-free advice for supporting your software testing with AI. In it, you’ll find strategies from bestselling author Mark Winteringham to generate synthetic testing data, implement automation, and even augment and improve your test design with AI.

Foreword by Nicola Martin.

About the technology

There’s a simple rule in software the more you test, the more you learn. And as any testing pro will tell you, good testing takes time. By integrating large language models (LLMs) and generative AI into your process, you can dramatically automate and enhance testing, improve quality and coverage, and deliver more meaningful results.

About the book

Software Testing with Generative AI shows you how AI can elevate every aspect of testing—automation, test data management, test scripting, exploratory testing, and more! Learn how to use AI coding tools like Copilot to guide test-driven development, get relevant feedback about your applications from ChatGPT, and use the OpenAI API to integrate AI into your data generation. You’ll soon have higher-quality testing that takes up less of your time.

What's inside

• Improve test quality and coverage
• AI-powered test automation
• Build agents that act as testing assistants

About the reader

For developers, testers, and quality engineers.

About the author

Mark Winteringham is an experienced software tester who teaches many aspects of software testing. He is the author of Testing Web APIs.

The technical editor on this book was Robert Walsh.

Table of Contents

Part 1
1 Enhancing testing with large language models
2 Large language models and prompt engineering
3 Artificial intelligence, automation, and testing
Part 2
4 AI-assisted testing for developers
5 Test planning with AI support
6 Rapid data creation using AI
7 Accelerating and improving UI automation using AI
8 Assisting exploratory testing with artificial intelligence
9 AI agents as testing assistants
Part 3
10 Introducing customized LLMs
11 Contextualizing prompts with retrieval-augmented generation
12 Fine-tuni

Genres: Artificial Intelligence

304 pages, Paperback

Published December 10, 2024

Reviews

[Daniil Maslov · Rating 2 out of 5]

I had higher expectations for this book, given my familiarity with LLMs and AI. While it's a solid introduction to AI and LLMs for beginners, it falls short for those with more experience in software testing with AI. I was hoping for more advanced content, such as creative prompt ideas or innovative ways to leverage GPT models in testing. Unfortunately, many sections only cover basic concepts, such as what a prompt is, which may be useful for newcomers but offer little value for experienced testers. Additionally, the appendix focuses on configuring tools like ChatGPT and GitHub Copilot, which essentially serves as a UI guide and account setup instructions—areas I found overly simplistic and lacking depth.

[David Cabrero · Rating 4 out of 5]

Although I believe that the expectations generated around generative AI far exceed its capabilities, it would also be negligent to ignore them since they provide great value in many aspects of software development. And that is precisely what the author offers us in this book, in his own words: success is both knowing when to use AI and knowing how to use it.
This book follows a pragmatic approach, showing through real examples how to build prompts that help us in the various tasks that make up software testing.

[Sumit Bhattacharyya · Rating 5 out of 5]

"Software Testing with Generative AI" by Mark Winteringham is a practical guide for software testers exploring the use of AI tools to improve their processes. The book tackles the common challenge of creating realistic test data and shows how Generative AI (GenAI) can provide effective solutions.

Winteringham explains how GenAI can save time by automating the creation of test data and test cases, helping teams focus on critical tasks like improving coverage and identifying potential risks. The book offers clear, actionable steps for integrating AI into testing workflows, from tool selection to implementation, with examples to help readers get started.

It also doesn’t shy away from addressing important topics like privacy and the ethical considerations of using AI in testing. These insights are valuable for ensuring AI is applied responsibly.

[Johnny · Rating 4 out of 5]

The book explains in great details how you must formulate the prompts to get what you want from the AI tool. While there is much more to the book than the prompts, it was the part I found the most helpful one.

However, please do not let the AI generate code comments that explains what it does – that is the job of the code. If you do not understand what is going on, let the AI explain it without creating that kind of duplication in the code itself.