Guidance Cards - Test

From School of CS Student Wiki
Jump to navigation Jump to search

Part of Red-Green-Go! See FizzBuzz Example

Test Card 1

Requires Review/Re-wording

When you have a choice of next simplest test, it doesn't matter which you choose as long as you complete all the tests in one feature group before starting on the next.
Test 1.jpg

When you have a choice of next simplest test, it doesn't matter which you choose as long as you complete all the tests in one feature group before starting on the next.

"In theory, there's no difference between theory and practice; but in practice there is." (attributed to various authors)

In theory, you can tackle the test cases for a piece of functionality in any order when doing TDD. In practice, however, some orders will make your life much easier than others. Here's why.

When we apply the practice of TDD, we first write very simple code that makes the tests we have written so far pass, as simply and directly as possible. As we write more tests, and more simple code of this kind, we start to see patterns within the production code. These patterns point the way to more general implementations, that will be able to cope not just with the input values provided by the tests cases given, but with any input of that kind given in future. So, in the coding step, we write very simple code, knowing that we are writing code that is tied too closely to the current set of test cases. In subsequent refactoring steps, when we have made several tests pass, we spot the familiar patterns and convert them into their more elegant general forms.

Here's where our choice of the order in which to write test cases comes in. If we tackle the test cases in a "good" order, then the patterns we are looking for will be easy to see. If we choose a "bad" order, then the patterns will be hard to spot, and we run the risk of generalising them incorrectly and maybe prematurely.

So what makes for a "good" test order? This is still an open question, to an extent. We know that starting with the simplest tests is a good starting point. Beyond that, one useful rule of thumb is encapsulated on the test guidance card that led you here: once you've embarked on working on a set of test cases for a feature, finish all the test cases that describe that feature, before starting on the next one. If you ignore this, and work with test cases from several features interleaved, you will have partial patterns for the different features overlaid on top of one another, and will make it difficult for you to work out where you can safely refactor and how.

Can You Give an Example?

These groups of functionality can be seen clearly in the Roman Numerals example. If we choose to work on conversion of 1, 2, 3, 4, 5 and 6 as the first few test cases, in that order, we'll be mixing tests from a number of features. In this sequence, we mix up test cases for numbers that are represented by single Roman digits, test cases for numbers that are represented by adding the value of smaller numbers to larger numbers, and test cases for numbers that are represented by subtracting the value of smaller numbers from larger numbers. Since all these difference kinds of number need differently handling in software, we may not be able to see the patterns clearly when all mixed up in this way.

For an even simpler example, see the FizzBuzz scenario. This program must handle the following types of number:

  • Numbers that are not multiples of three or five.
  • Numbers that are multiples of three but not of five.
  • Numbers that are multiples of five but not of three.
  • Numbers that are multiples of both five and three.

The test cases for the first three of these categories look very simple, so we can't use the "choose the simplest test" maxim to help us out here. So, we can choose any of them to start with, but, to make the patterns easy to see and refactor, we want to finish all the test cases needed for each one, before we start on the next.

But I'm Stuck!

All this means that, if you have tested yourself into a dead-end with your current TDD project, and can't work out how to move forward, you may have made unhelpful test case choices prior to reaching this point. You may need to go back and look again at what different test case decisions you might have made. "Go back", here, of course, means deleting code or reverting back to an older version from the code repository. (And don't just comment the code out - really delete it. You need it properly out of the way if you want to make the patterns you are looking for shout out at you more clearly. Don't worry! It is not gone forever. You can always dig it out of your source code repository, again, should you want to resurrect it.) | QR Code

Test Card 2

Martin: Requires Review/Re-wording
"Choose the simplest 'Happy Path' case for your first test"

Test 2.jpg

The first test is an important one in TDD, because in writing it, and then making it pass, we make our initial design decisions about the code we are going to write. This maxim advises you to choose a test that is both very simple and a 'happy path' test. Let's look at each of these in turn.

Choose the Simplest Test

We want to start with a very simple test, because we want each red-green-green cycle to make some meaningful progress, and we can only hope to achieve that quickly on our first TDD cycle if we start with the simplest, smallest piece of behaviour we can think of.

Let's look at a few examples of 'simplest' test cases, to help get the idea across.

  • If writing code to sort a list of names alphabetically, we would start with a test that sorts an empty list of names, and expects to get back an empty list.
  • If writing code to calculate the amount of tax payable on goods purchased, we would start with a test that calculates the tax for a type of product that is taxed at 0%, and expects to get back £0.00 tax payable.
  • If writing software to translate simple English sentences into Cyrillic, we would start with a test that translates the sentence 'No.', and expects to get back the sentence 'не'.

Get the idea? The test cases in these examples are not artificially simple. They all describe real cases that the final software must support, cases that could actually happen when the software is in use. So, working on implementing them helps us make meaningful progress towards our final goal. But, they are also all very simple. We don't have to set up complicated fixture code, or think hard about what the expected outcome should be, so the tests are quick to write. And it shouldn't take much code to make these simple tests pass.

Choose a 'Happy Path' test

The second characteristic we want of our first test is that it should be what is called a 'happy path' test. A 'happy path' test is one that describes a situation where the goal of the client for the code is achieved, in a simple and direct way. Nothing goes wrong, nothing is missing, nothing unexpected happens that needs to be handled before the goal can be reached.

By contrast, a 'sad path' test describes a situation where something has gone wrong in some way. We might be able to correct it and achieve the goal, by doing some error handling or additional processing, or the goal might not be achieved at all.

For example, if we are writing tests for an ATM, a happy path test would be one where the customer requests to withdraw an amount of money, valid account and identification details are given, and there are sufficient funds in the account to accommodate the withdrawal. The correct money is dispensed to the customer, and the balance of the account updated. The customer's goal (of getting the required amount of money) is reached.

It's easy to think of sad path tests for this same scenario. The customer might request to withdraw more funds than are present in the account. Or, the funds might be there, but an incorrect PIN is given. In this latter case, the goal of dispensing the money might be eventually reached, after the customer is prompted to re-enter their PIN.

Hopefully, these examples go some way towards explaining why its a good idea to start with a happy path test when doing TDD. We want the simplest test, so of course we want to avoid sad path test cases that involve lots of error handling. Some sad path tests can themselves be very simple and might therefore be candidates for our first test. But, the happy path test cases describe the 'normal' operation of the code, and therefore give us a better foundation for building quickly towards delivering some real value for our customer. Consider the sad path test for the English-to-Bulgarian translator that says that attempts to translate the string 'xxx' should result in an exception being thrown. This is a very simple test, that can be written easily and made to pass with little effort. But passing it takes us around the edges of the translation problem, whereas the happy path test for translating 'No.' takes us a step right towards the heart of it. | QR Code

Test Card 3

Requires Review/Re-wording
For the next test case, write the simplest test you can think of that will fail against the current production code.

Test 3.jpg

Recall that our aim when using test-driven development is to break the development process down into very fine-grained steps. We want to grow the functionality that is implemented by the production code in increments that are so small that we stand a good chance of getting them more or less right first time. Adding a sequence of these tiny increments together creates a more complex artefact, that we might have struggled to implement correctly in one go.

When we write a new test case in the "red" step of the red-green-green cycle, that test is a description of the increment in functionality that we plan to make next. It sets both the direction and the scale of the change we plan to make to the production code in this cycle.

The test should fail because, if it passes, that means the production code already offers the functionality the test describes, and no change is needed. We are effectively asking for a zero-sized increment.

The test should be the simplest we can think of because then we are stating our intention to make the smallest meaningful increment of functionality we can think of. And when we keep the increment sizes small, we help to control the intellectual complexity of the task.

So TDD Makes Programming Easy?

This does not mean, of course, that programming complex code becomes "easy" with TDD. We still have to do all the "hard thinking" that we would need to do when programming in a conventional way. And, since we are building a high-coverage test suite at the same time, it might be said that we have more hard thinking to do in TDD, than in a typical development effort where all the brainwork goes into the production code and very little (or zero) effort goes into the tests.

What it does mean is that we have less rework to do, and less risk. In TDD, we are never far from an executable program that we can demonstrate (or even deploy), even if it does not yet implement all the required functionality. Since we keep our tests green, and the test coverage high, when a test does fail we know that the problem is (at least) related to the few lines we have changed since we last ran the tests. And, if we've written our tests well, then the failure message coming from the broken test should give us lots of information about where the problem itself lies. Furthermore, while we have to think hard about our tests, and the order in which we work on them, with TDD it is rare to have to embark on hour-long debugging sessions, trying to track down some obscure bug.

I thought my new test would fail but...

Sometimes, we write a new test, feeling confident that it will fail, but it actually passes. What do we do in this case? We can't write any new production code until we have a failing test. So the next major step will be to write another test case, that (hopefully) actually will fail.

But what about the test case we've just written? Should we delete it, or keep it?

The answer is: it depends. If, on reflection, you see that the new test case really just describes another example of the kind of situation that your other tests cover, then it is adding no value, and should probably be deleted. If the system you are building is small, then the odd redundant test hanging around probably isn't going to do much harm. But, if the system is industrial-scale, with 10,000s or 100,000s of unit tests, then we really can't afford to have much redundancy in our unit test suites. These suites need to run blindingly fast, if we're going to run them regularly while performing TDD. We have to ruthlessly cut out any wasted effort/redundant test cases, to achieve that.

It can sometimes be worthwhile to keep a test case that passes, if it genuinely seems to cover a case that is not described by the other test cases. This can happen when we have been over-eager in our refactoring, and have made our production code more general than is necessary for the set of test cases written so far. | QR Code

Test Card 4

Requires Review/Re-wording

Write test cases at the boundaries of the different types of behaviour that you need to support.
Test 4.jpg

We can reuse lots of conventional testing wisdom in TDD, especially in terms of guidelines about which tests it is useful to write and which not. Boundary Value Analysis is a long-established technique for designing test cases, and it can be directly applied in TDD.

In Boundary Value Analysis, we make sure that we write test cases that use values at the key boundary points of the domain that a particular input parameter can accept. For example, a program that works out a person's birthdate given their current age and the current date takes two input parameters (the age, of type Integer, and the current date, of type Date). We can split the set of Integer's into ranges, which have different characteristics when used to represent an age for this program. All negative numbers describe an invalid age, for example, while values over 130 are higher than the age of the oldest known living human, and so are also invalid.

Boundary Value Analysis says we should write tests that make use of values at the boundaries of these ranges. In this case, the boundaries for the "age" parameter are: 0, -1 and 130, 131. If we were working on code that expected only to be asked to calculate the age for adults (defined, perhaps, as being aged 18 or over) then we would have another boundary at 18, and would want to write tests with age 17 and 18.

This technique was originally proposed to help us catch logical errors in our programs, on the grounds that programmers are more likely to make errors at the places where the behaviour of the program changes. A very common type of error for example is the "off-by-one" error, which causes a program to get the boundary point for some behaviour wrong by +1 or -1. A typical example is in the termination condition of a loop, where (for instance) a termination condition of "i < z" has been written, when "i =< z" was intended. A good coverage of tests at these boundary points can help to identify these off-by-one errors, plus other mistakes at the key boundaries.

In TDD, we hope to avoid making such errors by fitting our code tightly to the tests in the first place. But Boundary Value Analysis is still very useful to us, as it tells us the places where different behaviours are needed, and therefore where there is the scope to write tests that might not be handled by the current production code (and therefore might fail).

Test Card 5

Requires Review/Re-wording

Write test cases for sad path cases, too.
Test 5.jpg

In traditional software testing wisdom, test cases can be broadly classified into two types: happy path tests and sad path tests.

Happy path tests describe cases where everything goes well, and the goal of the software is met through a direct and simple route. For example, a happy path test for software that computes the final mark of an undergraduate student in some course unit is one where marks for coursework, lab work and the exam are all available, where the student has passed each component of the course unit, and no mark scaling has had to be applied to the class overall. In this case, all the information needed to compute the mark is available, and the most simple and direct calculation can be made.

A sad path test, on the other hand, is one where something goes wrong. Maybe incorrect information is supplied (the student specified is not enrolled on the course unit), or some key element is missing (one of the lecturers hasn't yet completed the exam marking). Maybe some aspect of the situation is out of the ordinary (the student has failed, but will receive a basic pass mark, due to having been ill for a large part of the course unit). In sad path cases, the goal may or may not be reached, depending on whether the problem elements of the case can be handled or not.

On another of our test guidance cards, we recommend that you focus on happy path tests as your starting point for development of a new code unit using TDD. However, it is not sufficient only to write test cases for the happy path cases. We expect good quality code to deal gracefully with unexpected and invalid cases, too. And, in TDD, that means we must write test cases to cover these requirements.

So, if you have exhausted all your ideas for happy path cases, take some time to consider the sad path behaviours that you want your code to support, and write tests for those.


  • gravatar Mbasssme [userPHRhYmxlIGNsYXNzPSJ0d3BvcHVwIj48dHI+PHRkIGNsYXNzPSJ0d3BvcHVwLWVudHJ5dGl0bGUiPkdyb3Vwczo8L3RkPjx0ZD51c2VyPGJyIC8+PC90ZD48L3RyPjwvdGFibGU+] ·
  • gravatar Mbax9mb5 [userPHRhYmxlIGNsYXNzPSJ0d3BvcHVwIj48dHI+PHRkIGNsYXNzPSJ0d3BvcHVwLWVudHJ5dGl0bGUiPkdyb3Vwczo8L3RkPjx0ZD51c2VyPGJyIC8+PC90ZD48L3RyPjwvdGFibGU+]