ISTQB-CTFL Test Analysis and Design

Test Analysis and Design Detailed Explanation

4.1 Test Basis

What is Test Basis?

The test basis refers to the set of documents, specifications, or knowledge that provide the foundation for creating test cases. It serves as the source of information for identifying test conditions.

Examples of Test Basis

1. Requirements Documents

   • These documents define what the system must do.
   • Example:
     A requirement states, “The system must allow users to log in with a valid username and password.”
     • Test Condition: Verify login with valid credentials.
     • Test Condition: Verify error handling for invalid credentials.

2. Design Specifications

   • These documents describe how the system works from a technical perspective.
   • Example:
     A design specifies that an email field must allow up to 50 characters and validate format.
     • Test Condition: Verify email input accepts 50 characters.
     • Test Condition: Verify error for invalid email format.

3. User Stories (Agile Projects)

   • In Agile projects, user stories describe functionalities from the user’s perspective.
   • Example:
     “As a user, I want to reset my password so that I can regain access to my account.”
     • Test Condition: Verify password reset link works.
     • Test Condition: Verify a confirmation email is sent.

4. Risk Analysis Reports

   • Risk analysis identifies high-risk areas of the system. These areas are prioritized for testing.
   • Example:
     If the payment gateway is a high-risk area, test conditions focus on scenarios like:
     • Successful payments.
     • Payments with invalid credit cards.
     • Network failure during payment.

Why is Test Basis Important?

• Ensures tests are aligned with project requirements and user expectations.
• Helps testers create effective and thorough test cases.
• Allows traceability between test cases and requirements (important for identifying gaps).

4.2 Test Conditions

What is a Test Condition?

A test condition is an item, feature, or event that can be verified by one or more test cases. It represents what you need to test.

Characteristics of Test Conditions

• Derived from the test basis (e.g., requirements, design).
• Should be specific, clear, and measurable.
• Each test condition can result in multiple test cases.

Examples of Test Conditions

1. Specific System Inputs

   • For a login feature:
     • Condition: Input valid username and password.
     • Condition: Input an invalid password.

2. Expected System Responses

   • For a search function:
     • Condition: Enter a valid product name → System displays the product list.
     • Condition: Enter a product name not in the catalog → System shows “No results found.”

3. Boundary Values

   • For a text field that accepts input from 1 to 100 characters:
     • Condition: Input 1 character.
     • Condition: Input 100 characters.
     • Condition: Input 0 characters (invalid).
     • Condition: Input 101 characters (invalid).

Steps to Identify Test Conditions

1. Analyze the Test Basis:
   • Review requirements, designs, or user stories.
2. Break Down Requirements:
   • Split complex requirements into smaller, testable conditions.
3. Prioritize Conditions:
   • Identify high-priority test conditions based on risk.
4. Document Test Conditions:
   • List test conditions in a clear and organized manner.

Example of Test Conditions Based on Requirements

Requirement	Test Condition
“The system must allow users to log in with a valid username and password.”	Verify login with a valid username and password.
	Verify error message for an invalid password.
	Verify error message when both fields are empty.
“The email field must allow up to 50 characters.”	Verify email input accepts exactly 50 characters.
	Verify error message for input over 50 characters.

4.3 Test Case Design Techniques

What is a Test Case?

A test case is a set of inputs, actions, and expected results that verify whether a specific test condition is met.

Test Case Design Techniques

Test case design techniques are categorized into three main types:

1. Black-Box Techniques (Specification-Based Testing)
2. White-Box Techniques (Structure-Based Testing)
3. Experience-Based Techniques

1. Black-Box Techniques

Definition

• Black-box techniques focus on inputs, outputs, and system behavior without considering the internal code structure.
• These techniques are based on requirements or functional specifications.

Key Black-Box Techniques

Technique	Description	Example
Equivalence Partitioning	Divide inputs into groups where each group behaves similarly. Test one value per group.	Input range 1–100 → Test 50 (valid), -1 (invalid).
Boundary Value Analysis	Test at the edges of input ranges (minimum, maximum, just outside the range).	Input range 1–100 → Test 0, 1, 100, 101.
Decision Table Testing	Represent combinations of inputs and corresponding outputs in a table format.	Loan approval system: Income >50K, Credit Score >700 → Approved.
State Transition Testing	Verify changes in system states based on inputs or events.	Verify account lock after 3 failed login attempts.
Use Case Testing	Derive test cases based on user workflows or scenarios.	Simulate placing an order in an e-commerce app.

Detailed Examples of Black-Box Techniques

Equivalence Partitioning

• Scenario: A system accepts input between 1 and 100.
• Partitions:
  • Valid: 1–100 → Test one value, e.g., 50.
  • Invalid: Below range → Test 0.
  • Invalid: Above range → Test 101.

Boundary Value Analysis

• Scenario: Input range is 1–100.
• Boundary Values:
  • Minimum boundary: Test 1.
  • Just outside minimum: Test 0.
  • Maximum boundary: Test 100.
  • Just outside maximum: Test 101.

Decision Table Testing

Example: Loan approval rules:

Income	Credit Score	Loan Approved?
>50,000	>700	Yes
≤50,000	>700	No
>50,000	≤700	No

2. White-Box Techniques (Structure-Based Testing)

What is White-Box Testing?

• White-box testing focuses on testing the internal logic, structure, and code of the system.
• Testers need to have knowledge of the code or logic being tested.
• These techniques ensure the code is well-structured, efficient, and fully exercised.

Key White-Box Techniques

Technique	Description	Example
Statement Coverage	Ensures that every statement in the code is executed at least once.	Run all lines of code in a simple “if-else” structure.
Branch Coverage	Ensures that every decision branch (e.g., true/false in “if” statements) is executed.	Test both true and false outcomes of an “if-else.”
Path Coverage	Ensures that all possible paths through the code are executed.	Test all routes in a decision tree.

1. Statement Coverage

Definition

• Statement coverage ensures every line of code is executed at least once during testing.

Why it’s important

• Helps identify lines of code that are never executed.
• Ensures basic functionality of all parts of the code.

Example of Statement Coverage

Consider the following code:

def check_even(number):
    if number % 2 == 0:
        print("Even")
    print("Done")

Test Cases for Statement Coverage:

1. Input: number = 2
   • Output: “Even” → “Done.”
   • Covered Lines: All lines are executed.
2. Input: number = 3
   • Output: “Done.”
   • Covered Lines: Line 1, line 4.

Observation: Both test cases combined ensure every line of code is executed at least once.

2. Branch Coverage

Definition

• Branch coverage ensures that every decision branch (e.g., the true and false conditions in an “if-else” statement) is executed.

Why it’s important

• Ensures all outcomes of decision points are tested.

Example of Branch Coverage

Consider the following code:

def check_number(number):
    if number > 0:
        print("Positive")
    else:
        print("Non-Positive")

Test Cases for Branch Coverage:

1. Input: number = 5
   • Output: “Positive” → Tests the true branch.
2. Input: number = -1
   • Output: “Non-Positive” → Tests the false branch.

Observation: Both the true and false branches are executed, achieving full branch coverage.

3. Path Coverage

Definition

• Path coverage ensures that all possible paths through the code are tested.
• A path refers to a sequence of statements that can be executed during a program’s flow.

Why it’s important

• Ensures that every combination of conditions and branches is tested.
• Provides the most thorough coverage.

Example of Path Coverage

Consider the following code:

def check_grade(score):
    if score >= 90:
        print("Grade A")
    elif score >= 80:
        print("Grade B")
    else:
        print("Grade C")

Possible Paths:

1. Path 1: score = 95 → Condition score >= 90 is true → “Grade A.”
2. Path 2: score = 85 → Condition score >= 90 is false, score >= 80 is true → “Grade B.”
3. Path 3: score = 70 → Condition score >= 90 is false, score >= 80 is false → “Grade C.”

Test Cases for Path Coverage:

1. Input: 95 → Path 1.
2. Input: 85 → Path 2.
3. Input: 70 → Path 3.

Observation: All possible paths are executed, achieving full path coverage.

Summary of White-Box Techniques

Technique	Focus	Purpose
Statement Coverage	Execute all statements at least once.	Detect unused lines of code.
Branch Coverage	Execute all decision branches.	Test both true and false outcomes.
Path Coverage	Execute all possible code paths.	Test all combinations of branches.

3. Experience-Based Techniques

Definition

Experience-based techniques rely on the knowledge, intuition, and experience of testers to identify defects. These techniques are useful when formal documentation is unavailable or incomplete.

Key Experience-Based Techniques

Technique	Description
Error Guessing	Testers “guess” where defects might occur based on experience.
Exploratory Testing	Testers simultaneously design and execute tests to explore the system.

1. Error Guessing

What is it?

• Error guessing is a technique where testers use their experience and intuition to guess where defects are likely to occur.

How it works

• Testers focus on areas of the software where defects are commonly found, such as:
  • Boundary conditions (e.g., entering 0 in a field expecting a positive number).
  • Invalid inputs.
  • Complex business logic.

Example of Error Guessing

Imagine testing a registration form:

• Input empty fields → Check for validation errors.
• Input invalid email format → Check for appropriate error messages.
• Input special characters in the “Name” field → Verify if the system rejects them.

2. Exploratory Testing

What is it?

• Exploratory testing is a hands-on testing approach where testers design and execute tests simultaneously while exploring the system.

Key Characteristics:

• No predefined test cases are required.
• Testers use creativity and intuition to identify defects.
• Focuses on learning about the system and uncovering unexpected issues.

When to Use Exploratory Testing

• When there is limited documentation.
• For complex systems where formal test cases may miss issues.
• To supplement formal testing techniques.

Example of Exploratory Testing

Suppose you are testing a new mobile banking app:

• Explore the “Account Transfer” feature by:
  • Sending money using different account types.
  • Entering invalid amounts (e.g., -100, 0).
  • Testing with a poor internet connection.
• Observe the app’s behavior and document defects as you find them.

4.4 Test Data

What is Test Data?

Test data refers to the input values or data sets used during test case execution to verify whether the system behaves as expected.

Why is Test Data Important?

• Test data helps confirm that the software behaves correctly for both valid and invalid inputs.
• It ensures that edge cases, unusual conditions, and real-world scenarios are tested effectively.
• Quality test data makes testing more efficient and comprehensive.

Types of Test Data

1. Valid Data

   • Definition: Input values that are within the expected or acceptable range.
   • Purpose: To verify the system processes correct inputs properly.
   • Example:
     For a login form requiring a username and password:
     • Valid username: user123
     • Valid password: Password123

2. Invalid Data

   • Definition: Input values that are outside the acceptable range or format.
   • Purpose: To test how the system handles incorrect inputs (e.g., error handling, validation).
   • Example:
     For a numeric age field (1–100):
     • Invalid values: -5, 0, 101, abc.

3. Boundary Data

   • Definition: Input values at the edges of valid ranges, as well as just outside those ranges.
   • Purpose: To test boundary conditions where errors are likely to occur.
   • Example:
     For an input range of 1–100:
     • Boundary values: 1, 100 (valid), 0, 101 (invalid).

4. Null or Empty Data

   • Definition: Input fields with no data or null values.
   • Purpose: To test how the system handles empty inputs or missing values.
   • Example:
     • Empty email field → Expected response: “Email cannot be blank.”
     • Null username → Expected behavior: Error message.

5. Special Characters

   • Definition: Inputs that contain special characters (e.g., !@#$%, <script>).
   • Purpose: To test for vulnerabilities such as injection attacks or improper input handling.
   • Example:
     • Entering !@#$% in a name field → Should return an error.

Examples of Test Data in Different Scenarios

Scenario	Valid Data	Invalid Data	Boundary Data	Null/Empty Data
Login Form	user123 / Pass123	user123 / wrongpass	N/A	Empty username field
Age Input (1–100)	25	-1, 101, abc	1, 100	Empty age field
Email Field	[email protected]	[email protected], invalid	Long email (50 chars)	Empty email
Password Strength Check	Pass@123	123, password	N/A	Empty password

How to Prepare Test Data

1. Understand Requirements: Analyze the system requirements and identify the data inputs to test.
2. Classify Input Types: Identify valid, invalid, boundary, and special cases.
3. Create Test Data Sets: Design combinations of input data to test all conditions.
4. Use Tools for Test Data Generation:
   • Tools like Mockaroo, TestDataGenerator, and databases can generate large volumes of test data automatically.

Example of Test Data for a Registration Form

Imagine you are testing a user registration form with fields: Name, Age, and Email.

Field	Test Data Type	Test Data Example	Expected Outcome
Name	Valid	John Doe	Registration successful.
Name	Invalid	!@#123	Error: “Invalid characters.”
Age	Valid	25	Registration successful.
Age	Invalid	-5, abc	Error: “Age must be a number.”
Age	Boundary	1, 100	Registration successful.
Age	Boundary Invalid	0, 101	Error: “Age out of range.”
Email	Valid	[email protected]	Registration successful.
Email	Invalid	[email protected], testmail	Error: “Invalid email format.”
Email	Null/Empty	Empty field	Error: “Email cannot be blank.”

4.5 Traceability

What is Traceability?

Traceability ensures that every requirement is linked to one or more test cases. This process guarantees that:

1. All requirements are tested.
2. There are no gaps in test coverage.

Requirements Traceability Matrix (RTM)

The Requirements Traceability Matrix (RTM) is a document that maps requirements to test cases. It helps verify that all requirements are adequately covered by testing.

Structure of an RTM

Requirement ID	Requirement Description	Test Case ID	Test Case Description	Test Result
R1	Login with valid credentials	TC_001	Verify login with valid username/password	Passed
R2	Handle invalid login attempts	TC_002	Verify error message for invalid login	Failed
R3	Password reset functionality	TC_003	Verify reset password link works	Passed

Benefits of Traceability

1. Ensures Full Coverage: Ensures that no requirement is missed during testing.
2. Identifies Gaps: Highlights untested or partially tested requirements.
3. Improves Test Planning: Helps prioritize testing efforts based on critical requirements.
4. Supports Change Management: When a requirement changes, the RTM helps identify which test cases need to be updated.
5. Facilitates Reporting: Provides a clear, structured way to show test progress and coverage to stakeholders.

Example of How RTM is Used

Imagine a project with the following requirements:

1. R1: Login must work with valid credentials.
2. R2: The system must display an error for invalid login attempts.
3. R3: The system must allow password reset.

In the RTM:

• R1 maps to Test Cases: TC_001, TC_002.
• R2 maps to Test Case: TC_003.
• R3 maps to Test Case: TC_004.

By reviewing the RTM, the testing team can ensure that all requirements have corresponding test cases and have been tested.

Summary of Traceability

Aspect	Description
What is RTM?	A matrix linking requirements to test cases.
Purpose	Ensure full test coverage of requirements.
Benefits	Identifies gaps, supports change management, and improves reporting.

Summary of Test Data and Traceability

Concept	Definition	Example
Test Data	Input values used during test execution.	Valid, invalid, boundary, null data.
Traceability	Mapping requirements to test cases to ensure full coverage.	Requirements Traceability Matrix (RTM).

Test Analysis and Design (Additional Content)

1. Comparison Table: Three Types of Test Design Techniques

This table helps learners quickly compare and memorize the key differences between Black-box, White-box, and Experience-based techniques.

Technique Type	Key Focus	Requires Code Knowledge?	Example Techniques
Black-box	System’s external behavior (inputs, outputs, responses)	No	Boundary Value Analysis, Equivalence Partitioning, Decision Table
White-box	System’s internal logic and control flow	Yes	Statement Coverage, Branch Coverage, Path Coverage
Experience-based	Tester’s intuition, experience, and creativity	No	Error Guessing, Exploratory Testing

Exam Tip: Questions often ask which techniques require code knowledge or are suited for well-documented vs. undocumented systems.

2. Reinforcing the Importance of RTM in Exams

What is RTM?

RTM stands for Requirements Traceability Matrix. It maps each requirement to one or more test cases, ensuring complete coverage.

Requirement ID	Requirement	Test Case ID(s)
R1	Users must log in with valid credentials.	TC_001, TC_002
R2	Password reset must send an email.	TC_003

Exam Focus: RTM for Change Impact Analysis

Why is RTM crucial in change scenarios?

When a requirement changes, you must quickly identify all affected test cases. RTM enables this by maintaining traceability between:

• Requirements ↔ Test Cases
• Requirements ↔ Design Documents
• Test Cases ↔ Defects

Typical Exam Question:
Q: Which document helps assess the impact of requirement changes on testing activities?
A: Requirements Traceability Matrix (RTM)

3. Summary Box – You Should Know

At the end of the chapter or subchapter, include a quick recap box for review:

You Should Know – Test Analysis and Design

• Test basis includes requirements, design docs, and user stories.
• Test conditions are derived from the test basis.
• Three types of test design techniques:
  • Black-box (no code needed)
  • White-box (code knowledge required)
  • Experience-based (intuition-driven)
• RTM ensures test coverage and supports impact analysis when requirements change.