Software testing techniques – use examples, p. 1

Testing techniques allow processing requirements (testing conditions) into test cases. They help define the “flows” and test data, and can be divided into black box, white box and experience-based techniques. In this part of the article we’ll introduce them all, and then focus solely on the black box testing, providing valid, practical examples.White box and experience-based techniques will be discussed in the second part of the series.

Black box testing

Black box techniques are based on specification. They focus on the test subject’s input and output data without referring to a module’s or a system’s inner structure. And that’s where the name “black box” comes from – we can’t see the structure of a tested module/system, and the knowledge is derived from documentation which describes functionalities as well as nonfunctional attributes. 

Tests based on these techniques concentrate on launching a program in conditions that are maximally similar to the natural ones (actual system handling). One of black box advantages is no need to acquire skills such as analyzing and reading code.

We can distinguish:

• equivalence partitioning
• boundary value analysis
• cause-effect decision table testing
• state transition testing
• use case based testing

Pic. 1  Black box testing [own study]

White box testing

White box techniques are based on the test subject’s structure. As opposed to the black box techniques, they focus on the structure and the processing within the test subject. 

They can be used regardless of other techniques (e.g. during unit tests), but they work best as a black box testing supplementation.  After creating a test set based on specification, we analyze which structural elements of the code were not covered with these tests and the test suite is completed with cases that cover the missing fragments. The basis of creating tests in the structure-based technique is code. These techniques – as the name suggests – focus on the software’s structure and not on its functionality. That’s why they are not a sufficient testing method – they don’t use specification to design tests. 

We can distinguish:

• code statement testing and coverage
• decision testing and coverage

Pic. 2  White box testing [own study]

Experience-based testing techniques

They allow using the experience of developers, testers and users to design, implement and perform tests. These techniques are commonly used along with black box and white box ones. 

• bugs guessing
• explorational testing
• checklist-based testing

Black box – techniques and examples

Equivalence partitioning 

It’s a set of data for which the system works enduringly. The technique allows partitioning data into groups (called equivalence or abstract classes) so that each group consists of elements which need to be processed in the same manner. These classes are designated for both correct and incorrect values. 

One of the fundamental rules of testing states that it’s not feasible to perform a thorough testing. Thanks to the equivalence partitioning technique, we can reduce a potentially unlimited set of test data to a finite number of cases with an equal number of equivalence classes. 

Boundary Value Analysis (BVA)

Just as in the case of equivalence partitioning technique, data is divided into equivalence classes, but the difference is that the elements chosen for tests lie on the edges of these classes.  It’s an extension of the equivalence partitioning technique and can be used only in case of ordered classes with numerical or sequence data. Boundary values of the equivalence class are its minimal and maximal value (or the first and the last one). 

Analysing and setting the boundary values is performed to design a test which is aimed directly at the edges of the interval (condition) as they’re usually most error-prone. To choose the boundary values for tests, we need to select predetermined equivalence classes, and then establish the values located in their boundaries: right before and right after them. 

Example no. 1

We need to test a basket in an online store by setting the equivalence classes and boundary values.

Here’s a presentation of setting the equivalence classes:

Pic. 3 Equivalence classes and boundary values [own study]

While purchasing 10 pieces of a certain commodity the price stays the same. However, if a client buys more than 10 pieces (e.g. 11), he’s granted with a free delivery. Purchasing at least 50 pieces (from the 50th piece), the customer gets a 10% discount. There’s also a limitation in which a client can’t order more than 99 pieces. 

The designated intervals of the equivalence classes according to the dependence described above will look like this:

0 pieces – no purchase,

<1,11) pieces – regular price,

<11,50) pieces – free delivery,

<50,99) pieces – free delivery and a 10% discount,

>99 pieces – it’s not possible to order the commodity.

A closed parenthesis “<” means that the value is included in a certain collection, and an open parenthesis “(“ means that the value doesn’t belong to the collection. 

As a result, we have five equivalence classes for tests, their boundary values being: 0, 1, 10, 11, 49, 50, 99, 100.

Equivalence partitioning and boundary value analysis techniques work well while partitioning the collection of possible values for classes and considering them separately. However, their main weakness is that they don’t include combinations of varied input values. 

Cause-effect decision table testing

Decision tables are useful while testing complex business logic rules which need to be implemented in the system. Many programs can be described with rule-based systems: if the X and Y conditions are satisfied, we need to perform the Z action. 

Working out the decision tables, a tester identifies the conditions (usually input data) and the resulting actions (usually output data) of the system. They create array lines in which the conditions are located above and the actions below. 

Example no. 2 

Let’s get back to the example of testing an online store. Now, the decision table will be used to present possible combinations connected with granting a customer a discount for subscribing to the mailing database (action). Information about the promotion terms are displayed on the store’s website. 

Steps to proceed:

Step 1. Establishing a subject and elements of tests. In this case, there’s one test subject, and it’s an online store. 

Step 2. Derivation of test conditions. Test conditions include conditions and actions. In this example, we can distinguish 4 conditions and 1 action. They add up to five test conditions with the following form:

C1: A customer’s subscription to the mailing database;

C2: A customer gets a discount for subscribing to the mailing database;

C3: The discount is active (it hasn’t been used before);

C4: The products in the basket meet the promotion terms

and one action:

C5: (A1) The 10% discount is charged.

Step 3. Derivation of the coverage elements. The decision table allows defining the coverage elements. Table no. 1 describes the decision rules of the discussed case.

Table 1. A decision table for an online store

where *= condition irrelevant

Step 4. Defining test cases.

Final tables have been transformed into a table of test cases, presented in Table 2.

Table 2. Test cases

An example for opening a TC1 test case

An example for opening a TC2 test case

State transition testing

The techniques described above focus on a “static” aspect of testing. After receiving the input data, the output analysis was performed. State transition testing, on the other hand, allows testing the dynamics of the program performance. It’s represented by the concept of state. The result of the action is transitioning from state to state. The same action can result in two or more transitions from the same state.

Pic. 4  State transition diagram. General model of states and transitions between them [own study]

A general model of states and transitions between them is presented in the Pic. 4. The system in state 1 transits into state 2 after the action. This transition is presented with an arrow which goes from state 1 to state 2 as the action happens. 

The hypothesis of an error states that errors occur as a result of an incorrect implementation of a software’s actions sequence (e.g. error in the business logic). 

State transition tests can be designed to provide a coverage of a typical states sequence, testing all the states or all the transitions, certain transitions sequence or incorrect transitions. 

Example no. 3

State transition testing technique will be presented with an example of an application for booking a parking space in a company’s car park. To book a space, we need to choose a location, and then, after clicking “Book”, enter a correct name and password in adequate text fields of the form approving the reservation.

The assumption was that the application allows only two attempts of entering an incorrect password. During the third try, the system automatically closes the app.

Pic. 5 Diagrams of the state transition in the application for booking a parking space [own study]

The state diagrams help determine correct and incorrect state transitions for testing. 

Table 3. State Table Transitions

Use-case based testing

Tests can be derived from use cases. They describe interactions with the elements of software, verifying requirements for functionalities. Use case determines an agreement between the system’s users considering its behaviour. Every participant/actor has a particular goal to achieve in a given test case. A test case has to determine a so-called main scenario (i.e. a typical sequence of actions leading to a happy ending of interactions between the actors) and so-called alternative scenarios describing interaction in case of problems. 

Example no. 4

The technique will be presented based on an example of registering for a company training for employees. The use case will look as follows:

The name of the use case: Registering for a training.

Description: An employee who has an account on the platform can be registered for a chosen training. 

Actors: An employee.

Initial conditions: An employee has to be registered in the system. 

Minimal guarantee: An employee is informed that he or she has passed the registration process for training. 

Guarantee of success: An employee is registered for the training and has all the necessary materials. 

Main scenario: 

1. An employee chooses a training and registers for it.
2. An employee gets information about the finished process of registration and is provided with the necessary materials.

Alternative scenario:

(SA1) An employee who is not logged in cannot register for the training.  

(SA2) An employee is in the middle of registering for the training, loses a web connection and is informed about an interrupted process of registration.

Table 4. Table of test cases for the main and alternative scenario of the use case of Registering for a training. 

In test cases, the tests refer to scenarios (requirements), so the coverage will always be 100%. 

The last example of black box technique finishes the first part of the article. In the next one, we’ll describe other software testing techniques, such as white box testing and testing based on experience.