Key vocabulary

Test plans and error types

Why test at all?

Testing exists to answer one specific question: does the program actually do what it was supposed to do, as identified back in SDD2's analysis stage (its functional requirements — inputs, processes, and outputs)? A program that runs without crashing is not the same thing as a program that is correct — testing is how the gap between "runs" and "correct" gets closed. A comprehensive test plan, as required at Higher, must show that every functional requirement has genuinely been checked, not just that the program was run once and seemed to work.

Building a test plan

A test plan is a table. Each row is one test case, and typically includes: a description of what is being tested (tied to a specific functional requirement), the test data used, the expected result (worked out by hand, before running the program), the actual result (what the program genuinely produced when run), and a pass/fail outcome. A test plan with only one or two rows rarely counts as comprehensive — every distinct functional requirement needs at least one test case, and most requirements need several, covering different kinds of test data.

Three kinds of test data

National 5 already introduced three categories of test data, and Higher continues to expect all three in a comprehensive test plan: normal data (a typical, valid input the program should clearly handle correctly), extreme data (a valid input right at the edge of what's acceptable — a boundary value), and erroneous data (an invalid input that should be rejected or handled safely, not silently accepted as if it were valid). Testing only with normal data is not comprehensive — it's exactly the extreme and erroneous cases that most often expose a hidden logic error.

Three kinds of error

When a test case fails, it helps to know which of three distinct kinds of error caused it. A syntax error breaks the language's own grammar rules — a missing colon, an unmatched bracket — and is caught before the program ever starts running at all, usually flagged immediately by the editor or interpreter. An execution error (also called a runtime error) happens once the program is already running, causing it to crash partway through — dividing by zero, or accessing an array index that doesn't exist. A logic error is the most dangerous of the three precisely because it produces no error message whatsoever: the program runs to completion successfully, but the result it produces is simply wrong, because the underlying logic (a comparison, a calculation, an initial value) was mistaken.

Worked examples

def isEven(number)
    return number % 2 == 0

def calculateAverage(total, count):
    return total / count

result = calculateAverage(100, 0)

def findMinimumBuggy(array):
    smallest = 0
    for i in range(len(array)):
        if array[i] < smallest:
            smallest = array[i]
    return smallest

stockLevels = [30, 45, 20, 15, 60, 18]
result = findMinimumBuggy(stockLevels)

Task Set A — Core questions

Task Set B — Extension