ISTQB-CTFL Fundamentals of Testing

Fundamentals of Testing Detailed Explanation

1.1 What is Testing?

Definition

• Testing is the process of evaluating a system or its components to determine:
  • If it meets the specified requirements (e.g., user expectations, functional needs).
  • If there are any defects (problems) that need to be fixed.

Objective of Testing

Testing is not just about finding defects; it serves several purposes:

1. Ensuring quality – verifying the software works correctly.
2. Building reliability – confirming the system can perform without failures.
3. Delivering performance – ensuring it runs efficiently under expected conditions.

Example Analogy:
Imagine building a car. Testing ensures the car:

• Starts without issues (quality).
• Runs smoothly for many kilometers (reliability).
• Achieves good speed and fuel efficiency (performance).

Key Points of Testing

1. Verification and Validation:

   • Verification answers the question: “Are we building the product right?” (Does the software meet its design?).
   • Validation answers the question: “Are we building the right product?” (Does the software meet user needs?).

2. Static and Dynamic Testing:

   • Static Testing: Checking work products (e.g., documents, code) without running the program.
     • Example: Code reviews, analyzing documentation.
   • Dynamic Testing: Executing the software to observe its behavior.
     • Example: Running test cases to check the output of a program.

3. Early Testing:

   • The earlier testing starts in the software development lifecycle (SDLC), the cheaper it is to fix defects.

Example:
Suppose a requirement document has an error.

• Detecting it during the design phase = low cost (just fix the document).
• Detecting it in production = high cost (fixing the live system, support costs, customer dissatisfaction).

1.2 Why is Testing Necessary?

Testing is critical to software development for several reasons. Let’s examine five key reasons with real-world examples.

1. Avoid Failures

• Prevent critical failures in real-life scenarios that could cause financial loss or safety issues.

Example:

• In 1996, the Ariane 5 rocket exploded seconds after launch due to a software bug. The cost? $370 million!
• In healthcare systems, a bug in medical equipment software can put lives at risk.

Key Insight: Testing prevents such failures by identifying issues before deployment.

2. Improved Quality

• Testing ensures the product works as intended and delivers a positive user experience.

Example:
Imagine an e-commerce website:

• Without proper testing, users might experience crashes when adding items to the cart.
• This leads to customer frustration, bad reviews, and lost sales.

3. Compliance

• Testing ensures software meets legal and regulatory requirements in industries like finance, healthcare, and aviation.

Example:

• In banking systems, a bug that miscalculates interest rates can lead to non-compliance with financial regulations.

4. Cost of Defects

• Early detection of defects reduces the cost of fixing them.
• Defects found in later stages (like production) are far more expensive to resolve.

Example:

• Finding a typo in a requirements document costs little to fix.
• Fixing a defect in a live app requires development time, testing, and possibly compensating affected customers.

Stage of Detection	Cost of Fixing Defect
Requirements Phase	Low (document changes).
Coding Phase	Moderate (recode, test).
Production/Release	High (customers affected, major rework).

5. Stakeholder Confidence

• Testing builds trust with stakeholders (customers, users, project sponsors).
• Demonstrates the product is reliable and high quality.

Example:

• A software company tests its product thoroughly and shares a test summary report with clients.
• The clients gain confidence that the product has been well-tested and meets their needs.

1.3 Testing Objectives

The main goals of testing include:

1. Finding Defects:

   • Identify errors, bugs, or issues in the software.
   • Testing helps uncover problems developers may have missed.

2. Gaining Confidence:

   • Provide assurance that the software works as expected.
   • Testing builds confidence in the product’s quality and reliability.

3. Providing Information for Decisions:

   • Test results help stakeholders make decisions, such as:
     • “Is the software ready for release?”
     • “Does the system meet user needs?”

4. Preventing Future Defects:

   • Testing provides feedback to developers to improve their work and avoid recurring issues.

5. Ensuring Compliance:

   • Ensure the product meets requirements (e.g., functional, security, and performance).

Misconception About Testing

• Testing does not prove a product is defect-free.
• Instead, testing:
  • Reduces the likelihood of defects.
  • Provides information about the software’s current quality.

Analogy:
Testing is like checking a ship for leaks.

• You may not find every single leak, but you gain confidence that the ship is seaworthy.

1.4 The Seven Testing Principles

The seven testing principles are essential guidelines for understanding the nature of testing. Here’s an explanation of each principle with practical examples:

Principle	Explanation	Example
1. Testing shows defects	Testing can identify defects, but it cannot prove the software is free of defects.	Testing 100 scenarios doesn’t guarantee there are no hidden bugs.
2. Exhaustive testing is impossible	It’s impossible to test every possible input, condition, and combination. Focus on the most important tests.	A login page can have countless input combinations; focus on key tests (e.g., valid, invalid).
3. Early testing	Start testing as early as possible (e.g., requirements phase) to reduce costs and efforts.	Detecting a missing requirement early is cheaper than fixing it after coding.
4. Defect clustering	A small number of modules usually contain most of the defects (80/20 rule – Pareto Principle).	In a complex app, the checkout process might contain most defects.
5. Pesticide paradox	Repeatedly running the same tests won’t find new defects. Tests need to be updated regularly.	Adding new test cases uncovers different issues after a software update.
6. Testing is context-dependent	Testing varies based on the software’s context (e.g., safety-critical systems require rigorous testing).	A banking app requires stricter testing than a personal blogging site.
7. Absence-of-errors fallacy	Fixing defects doesn’t mean the software meets user needs. Testing should focus on requirements and usability.	A perfectly functioning app that’s difficult to use still fails user expectations.

1.5 The Fundamental Test Process

Testing is not a random activity. It follows a structured process to ensure tests are effective and reliable. The Fundamental Test Process consists of five main activities:

1. Test Planning and Control

Test Planning

Test planning is the first step in the testing process. It defines the scope, objectives, and strategy of testing.

Key Activities in Test Planning:

1. Define Scope: What will be tested (e.g., features, modules) and what will not be tested.
2. Set Objectives:
   • Identify the goals of testing (e.g., finding defects, ensuring performance).
   • Example: “Test that the login page works with valid and invalid inputs.”
3. Define Strategy/Approach:
   • Decide on testing techniques (e.g., black-box, white-box).
   • Choose tools (e.g., automation tools like Selenium).
4. Allocate Resources:
   • Assign roles (e.g., test manager, testers).
   • Ensure hardware/software requirements are available.
5. Create a Schedule:
   • Plan testing phases, execution timelines, and deadlines.
6. Identify Risks:
   • Identify potential challenges and how to address them (e.g., lack of resources, unclear requirements).

Test Control

Test control involves monitoring and managing testing activities throughout the process.

Key Activities in Test Control:

1. Track testing progress: Compare actual progress with planned progress.
2. Analyze test metrics: Examples include defect counts, test coverage, and test case execution rate.
3. Adjust plans: If issues arise (e.g., delays), update the test plan or priorities.

Example:
If you planned to execute 100 test cases by the end of the week but only completed 70, test control helps decide whether to add more testers or reduce the scope.

2. Test Analysis and Design

This phase answers the question: “What do we need to test, and how will we test it?”

Key Activities:

1. Identify Test Conditions:

   • Analyze requirements, design, or risks to find what needs to be tested.
   • Example: For a login page, test conditions could include:
     • Valid username and password.
     • Invalid password.
     • Empty username field.

2. Design Test Cases:

   • Develop detailed test cases based on identified test conditions.
   • Each test case includes:
     • Input: What data to enter (e.g., "username: user123").
     • Expected Output: The correct response (e.g., “Login successful”).
     • Steps: Instructions to execute the test.

3. Design Test Data:

   • Create the data needed for testing.
   • Example: For a payment system, test data could include valid and invalid credit card numbers.

4. Set Up the Test Environment:

   • Prepare tools, systems, and configurations to execute tests.
   • Example: Install the software on a test server and set up required databases.

5. Coverage Criteria:

   • Define how thoroughly you need to test the software.
   • Examples:
     • Requirements coverage: Ensure all requirements are tested.
     • Code coverage: Ensure all parts of the code are executed.

3. Test Implementation and Execution

This phase is where tests are prepared and executed.

Test Implementation

1. Prepare test procedures: Write step-by-step instructions to execute the test cases.
2. Prepare test scripts: For automated tests, scripts are written using tools like Selenium or UFT.
3. Arrange test data: Ensure the test data (valid and invalid inputs) is ready.

Test Execution

1. Run Test Cases: Execute the test cases either manually or using tools.
2. Log Test Results:
   • Compare actual results with the expected results.
   • Example: If a login button doesn’t work as expected, note it as a defect.
3. Log Defects:
   • Document defects in a defect management tool like JIRA or Bugzilla.
   • Include details:
     • Steps to reproduce.
     • Severity (e.g., critical, major, minor).
     • Environment (e.g., Windows 10, Chrome browser).

Example:

Test Case	Input	Expected Result	Actual Result	Status
Login with valid details	Username: user123, Password: Pass123	"Login successful"	"Login successful"	Passed
Login with invalid details	Username: user123, Password: wrong	"Login failed"	"Login failed"	Passed
Login with empty fields	Empty inputs	"Fields cannot be empty"	"Unexpected error"	Failed

4. Evaluating Exit Criteria and Reporting

This phase determines when to stop testing and prepares the final report for stakeholders.

Exit Criteria

Exit criteria are the conditions that must be met to end testing. Examples include:

• Test coverage: 90% of requirements are tested.
• Defect resolution: All critical and major defects are fixed.
• Test case execution: 100% of planned test cases are executed.

Test Reporting

• Create a Test Summary Report to share with stakeholders.
• Key contents of the report:
  • Test objectives and scope.
  • Summary of executed tests.
  • Number of defects found and fixed.
  • Remaining risks and recommendations.

Example: "Testing is complete with 95% requirement coverage. 50 defects were identified; all critical defects have been resolved."

5. Test Closure Activities

The final phase ensures all test activities are formally completed.

Key Activities:

1. Finalizing Deliverables:

   • Ensure test plans, cases, scripts, and defect logs are complete and archived.

2. Analyzing Lessons Learned:

   • Identify what went well and what could be improved for future projects.

3. Test Environment Cleanup:

   • Shut down test environments and tools.

4. Archiving Test Artifacts:

   • Store test cases, reports, and defect logs for future reference.

1.6 The Difference Between Testing and Debugging

Testing and debugging are closely related but not the same.

Aspect	Testing	Debugging
Objective	To find defects in the system.	To identify and fix the root cause of defects.
Performed by	Testers (independent team).	Developers (often the same who wrote the code).
When	Performed during all phases of development.	Performed after a defect is detected in testing.
Outcome	Defects are reported.	Defects are fixed.

Example:

• A tester finds that clicking the “Submit” button does nothing.
• A developer investigates, identifies a missing function in the code, and fixes it.

1.7 The Psychology of Testing

1. Understanding the Tester’s Mindset

Testing requires a unique and critical mindset that is different from development.

1. Objective and Critical Thinking:

   • Testers must approach testing objectively and with the intent to find defects.
   • They should question assumptions about how the software works and identify areas where it could fail.

2. Focus on Defects:

   • A tester’s role is to look for areas where the system does not perform as expected.
   • Example: While a developer is proud the login feature works, the tester focuses on cases like invalid inputs, empty fields, or invalid password combinations.

3. Detail-Oriented:

   • Testers must pay attention to small details that may affect the software, such as:
     • Incorrect error messages.
     • Slight performance delays.
     • UI misalignment or formatting issues.

4. No Assumptions:

   • Testers should avoid assuming the software works as intended and instead verify each part of the system.

Analogy:
Imagine a food critic reviewing a new dish.

• A chef focuses on preparing the dish perfectly.
• The critic evaluates its taste, presentation, and flaws (like too much salt or an undercooked portion).
• Similarly, testers evaluate the system critically to uncover hidden defects.

2. The Developer’s Mindset

The mindset of a developer is different because their goal is to build and deliver a functioning product.

1. Building Functionality:

   • Developers focus on implementing features and ensuring the software works as per requirements.
   • They often assume their code is correct unless proven otherwise.

2. Bias Toward Creation:

   • Developers are often emotionally attached to their code because they’ve spent time building it.
   • This bias can make it difficult for developers to find defects in their own work.

3. Limited Testing Scope:

   • Developers typically test only the happy path or main scenarios:
     • Example: Testing that a login feature works with valid inputs.

Difference in Focus:

• A developer focuses on “making the software work.”
• A tester focuses on “finding where the software does not work.”

3. The Importance of Independent Testing

Independent testing refers to testing performed by individuals who are not directly involved in building the software. This reduces bias and improves the chances of finding defects.

Levels of Independence

Level	Who Performs Testing	Level of Independence
Low Independence	Developers test their own code.	Minimal
Medium Independence	Developers from the same team test the code.	Moderate
High Independence	Dedicated testers or QA teams test the code.	High
Very High Independence	Testing is performed by external testers.	Very High

Why Independence Matters

1. Avoid Bias:

   • Developers may unconsciously overlook defects in their own code due to familiarity.
   • Independent testers approach the system with a fresh perspective.

2. Focus on Quality:

   • Dedicated testers have the skills, mindset, and focus needed to find defects.

3. Improved Coverage:

   • Independent testers explore edge cases, negative scenarios, and less obvious issues.

4. Objective Feedback:

   • Independent testers provide unbiased feedback, helping improve product quality.

Example of Independence:

• Low Independence: A developer writes a feature and tests it. They miss edge cases like invalid inputs.
• High Independence: A tester who was not involved in coding tests the feature thoroughly and identifies issues like performance delays or incorrect error messages.

4. Overcoming the Conflict Between Developers and Testers

A common challenge in software development is the conflict between developers and testers.

• Why Conflict Happens:

  • Developers may feel testers are “criticizing” their work.
  • Testers may feel developers are not fixing defects quickly.

• How to Overcome Conflict:

  1. Promote a team mindset: Developers and testers share the goal of delivering quality software.
  2. Foster clear communication: Use defect management tools (e.g., Jira) to log, prioritize, and track defects.
  3. Encourage collaboration: Developers and testers should work together during test planning, reviews, and defect resolution.

5. Understanding the Human Element in Testing

Testing is influenced by human psychology. Here’s how psychology plays a role in testing:

1. Cognitive Bias:

   • People tend to overlook their own mistakes (self-bias).
   • Independent testers help mitigate this by reviewing the software objectively.

2. Negative Perception of Testing:

   • Developers may view testing as “breaking their work.”
   • Testing should instead be viewed as a positive process to improve software quality.

3. Testers as Quality Advocates:

   • Testers are not trying to “find faults” but to ensure the software meets expectations and is defect-free for users.

4. Continuous Improvement:

   • Testing provides valuable feedback to help developers improve coding practices.

Analogy:
Think of testers as safety inspectors in a factory:

• They don’t blame workers for problems.
• Instead, they ensure products are safe, reliable, and ready for customers.

6. Summary of Psychology of Testing

Aspect	Developer	Tester
Mindset	Focuses on making software work.	Focuses on finding where it doesn’t work.
Bias	Tends to overlook own mistakes.	Looks critically at the software.
Testing Approach	Tests the “happy path.”	Tests both positive and negative paths.
Role in Quality	Builds the software.	Ensures quality by identifying defects.

Final Notes on Fundamentals of Testing

In this explanation, we covered:

1. What is Testing?: Understanding the process, objectives, and key points.
2. Why is Testing Necessary?: Importance of testing to prevent failures, ensure quality, and reduce costs.
3. Testing Objectives: Finding defects, gaining confidence, and supporting decisions.
4. The Seven Testing Principles: Guidelines to ensure effective and efficient testing.
5. The Fundamental Test Process: Structured phases of planning, design, execution, and closure.
6. The Difference Between Testing and Debugging: Who performs each and their goals.
7. The Psychology of Testing: How mindset and independence influence testing outcomes.

By mastering these concepts, you’ll build a strong foundation for software testing and be ready to move on to advanced topics like testing techniques and test management.

Fundamentals of Testing (Additional Content)

Chapter Summary – What You Should Know

Topic	Key Understanding
Why Testing is Needed	Testing helps find defects, ensure quality, and build confidence in the software.
What Testing Is	It’s a process to verify that software meets requirements and works as expected.
Seven Testing Principles	These are fundamental truths (e.g., early testing, defect clustering, absence-of-errors fallacy) that guide effective testing.
Test Process	Consists of structured activities: Planning → Monitoring and Control → Analysis → Design → Implementation → Execution → Completion.
Psychology of Testing	Testers need independence and objectivity to effectively find issues, and developers/testers must collaborate with mutual respect.

Mind Map: Overview of Fundamentals of Testing

Main Node: Fundamentals of Testing

├── 1. Why Testing is Necessary
│   ├── Detect Defects
│   ├── Improve Quality
│   └── Meet Stakeholder Expectations
│
├── 2. What is Testing?
│   ├── Verification vs Validation
│   └── Testing Objectives
│
├── 3. Seven Testing Principles
│   ├── 1. Testing shows presence of defects
│   ├── 2. Exhaustive testing is impossible
│   ├── 3. Early testing saves time and money
│   ├── 4. Defects cluster together
│   ├── 5. Pesticide paradox
│   ├── 6. Testing is context-dependent
│   └── 7. Absence-of-errors fallacy
│
├── 4. Test Process
│   ├── Test Planning
│   ├── Test Monitoring and Control
│   ├── Test Analysis
│   ├── Test Design
│   ├── Test Implementation
│   ├── Test Execution
│   └── Test Completion
│
└── 5. Psychology of Testing
    ├── Independence in testing
    ├── Tester mindset vs Developer mindset
    └── Team collaboration