Agile Testing: A Practical Guide for Testers and Agile Teams

by Lisa Crispin

Agile teams are highly cross-functional, with programmers, testers, and others working side by side throughout each iteration so that quality can be baked into products through techniques such as acceptance-test driven development, a heavy emphasis on automated testing, and whole-team thinking.

Each agile team member is focused on delivering a high-quality product that provides business value.

Agile testers work to ensure that their team delivers the quality their customers need. We use the terms “programmer” and “tester” for convenience.

Testers don’t sit and wait for work; they get up and look for ways to contribute throughout the development cycle and beyond.

When tests demonstrating minimum functionality are complete, the team can consider the story finished.

We define an agile tester this way: a professional tester who embraces change, collaborates well with both technical and business people, and understands the concept of using tests to document requirements and drive development.

They’re willing to learn what customers do so that they can better understand the customers’ software requirements.

“Without the attitude, the skill is nothing.”

Testers tend to see the big picture. They look at the application more from a user or customer point of view, which means they’re generally customer-focused.

Testers might have special expertise and experience in testing, but a good agile tester isn’t afraid to jump into a design discussion with suggestions that will help testability or create a more elegant solution.

One of the agile tester’s most important contributions is helping the product owner or customer articulate requirements for each story in the form of examples and tests.

If we let new features creep in, we risk delivering nothing on time. If we get too caught up with edge cases and miss core functionality on the happy path, we won’t provide the value the business needs.

The worst-case scenario is that only the core functionality gets released. That’s better than delivering nothing or something that works only halfway.

Some of us worked in software organizations where we, as testers and quality assurance staff, were asked to set quality standards. We believe this is backwards, because it’s up to the customer team to decide what level of quality they want to pay for.

Simple doesn’t mean easy. For testers, it means testing “just enough” with the lightest-weight tools and techniques we can find that will do the job. Tools can be as simple as a spreadsheet or a checklist.

Automated testing is one key to the solution. One thing we know for sure: No agile team will succeed doing only manual testing. We need robust automation in order to deliver business value in a time frame that makes it valuable.

Testing knowledge is one component of any team’s ability to deliver value.

Unfortunately, customers aren’t generally good at articulating their requirements.

Traditional testing activities, such as logging bugs, keeping track of metrics, and writing test plans, might not seem like a good fit in an agile project.

independent thinking. For example, if a company

If overtime is required for short periods, the whole team should be working extra hours. If it’s the last day of the sprint and some stories aren’t tested, the whole team should stay late to finish the testing, not just the testers.

implement agile processes. Find the areas of the most

Rather than keeping the testers separate as an independent team to test the application after coding, think about the team as a community of testers.

Many organizations fail to recognize that testers also need training on pair testing, working with incomplete and changing requirements, automation, and all of the other new skills that are required.

Testers need to be full-fledged members of the development team, and testing tasks need to be given the same attention as other tasks.

when they’re empowered to identify and solve their own problems.

If you’re a manager, resist the temptation to impose all your good ideas on the team.

Every team member has equal value to the team. If testers or any other team members feel left out or less valued, the whole-team approach is doomed.

If major bugs made it to production, nobody should blame the testers. Instead, the whole team should analyze what happened and start taking steps to prevent a recurrence.

The development team needs to keep the business needs in mind. Set goals that serve the business, increase profitability, and make the customers happier.

Teams need to self-organize, identify and find solutions to their own problems, and look for ways to improve. They can’t wait for someone to tell them what to do.

Defects in a DTS become a queue or a mini product backlog. According to lean principles, this inventory of defects is a waste. As a team, we should be thinking of ways to reduce this waste.

The more information that is contained in a document, the less likely it is that someone is going to read it all.

The Agile Testing Quadrants matrix helps testers ensure that they have considered all of the different types of tests that are needed in order to deliver value.

Deliver value

For example, agile development starts with customer tests, which tell the team what to code.

Acceptance criteria

The timing of the various types of tests depends on the risks of each project, the customers’ goals for the product, whether the team is working with legacy code or on a greenfield project, and when resources are available to do the testing.

Which tests to choose which not

The testing done in Quadrants 1 and 2 are more requirements specification and design aids than what we typically think of as testing.

Q1 &2

Unit tests verify functionality of a small subset of the system, such as an object or method. Component tests verify the behavior of a larger part of the system, such as a group of classes that provide some service

We refer to these tests as programmer tests, developer-facing tests, or technology-facing tests. They enable the programmers to measure what Kent Beck has called the internal quality of their code

Purpose of Q1 tests

A major purpose of Quadrant 1 tests is test-driven development (TDD) or test-driven design. The process of writing tests first helps programmers design their code well.

Unit and component tests are automated and written in the same programming language as the application.

Programmer tests are normally part of an automated process that runs with every code check-in, giving the team instant, continual feedback about their internal quality.

The tests in Quadrant 2 also support the work of the development team, but at a higher level. These business-facing tests, also called customer-facing tests and customer tests, define external quality and the features that the customers want.

It’s possible this quadrant could duplicate some of the tests that were done at the unit level; however, the Quadrant 2 tests are oriented toward illustrating and confirming desired system behavior at a higher level.

Q2

these automated tests run directly on the business logic in the production code without having to go through a presentation layer. Still, some automated tests must verify the user interfaces and any APIs that client applications might use. All of these tests should be run as part of an automated continuous integration, build, and test process.

The tests in this group are tests that help support the team to get the product built right but are not automated.

Some people use the term “acceptance tests” to describe Quadrant 2 tests, but we believe that acceptance tests encompass a broader range of tests that include Quadrants 3 and 4. Acceptance tests verify that all aspects of the system, including qualities such as usability and performance, meet customer requirements.

Acceptance criteria is not in Q3… its a broader term

The quick feedback provided by Quadrants 1 and 2 automated tests, which run with every code change or addition, form the foundation of an agile team.

These tests first guide development of functionality, and when automated, then provide a safety net to prevent refactoring and the introduction of new code from causing unexpected results.

you know how hard it can be to know exactly what you want until you see it. Even if you’re confident about how the feature should work, it can be hard to describe it

Customer dont know what he needs