Test Levels, Test Types, and Test Approaches in CTFL

The ISTQB-CTFL (Certified Tester Foundation Level) curriculum details 5 levels of testing, 4 test types, and 3 testing approaches.

The five levels of testing are as follows: Component Testing, Integration Testing, System Testing, User Acceptance Testing, and Maintenance Testing.  They are missing a needed sixth level, Smoke Testing. Perhaps one day they will add it.  Here is a description of all of them.

Smoke Testing: A quicks run through of all of the screens and reports within a system to ensure the developers gave you everything.  This ensures you don't get 2 days into testing a large app only to discover a ".dll not found" error due to the developers not including everything in the install package. Smoke testing should take now longer than 20 minutes or so.  Two added benefits once an organization has smoke test in place is that the implementation team can use the smoke test to test a new install, and if a production system goes down and is brought back online a smoke test and ensure all of it was brought back online.  But, I digress....

Component Testing: Checking each component of a system. There are 5 types of comports in software: 1. Screens, 2. Reports, 3. Interfaces, 4. Logic Modules, and 5. Data Structures.  Typically, component testing in software is checking each screen, and each report. The interfaces are tested at the next level.  While screens and reports are tested using requirements-based "black-box" testing, logic modules are best tested using "white-box" tools like logic trees and logic tables.

Integration Testing: There are two types of integration testing.  Internal integration testing is testing module to module (screen to report, for example).  External integration testing is testing system to system (your app to Paypal interface, for example).

System Testing: The best way to system test an app is to create a set of "typical day in the life of a user" flows representing different ways a user would use an app.  Providing all of the previous test levels worked reasonable well, system testing is a way to ensure everything is working together.

User Acceptance Testing: Once the system has been tested enough that the release success criteria has been met, the system should be shown to a real end-user for a final approval.  This should not be a surprise to the end-user.  In a healthy environment, the end-user should have been working closely with the development team as the software was created. Since the product should match requirements, any changes at this point would require updated requirements, and likely a new work order.

Maintenance Testing: This type of testing is used after an OS updated, or database change, or some other type of refactoring has occurred.

 

The Four Test Types found on the CTFL Exam are:

Functional testing (Black-box Testing): Requirements-based testing. When you cannot see the code, but all you have is a set of requirements, or user-stories. This is traditionally what people think of when they envision testing software.

Non-functional testing: Non-functional requirements are "implied" requirements which apply to the entire software app.  For example, security requirements, or HIPAA requirements, or PCI requirements.

Structural Testing (White-box Testing):This is used for component testing logic modules, where the developer share the code with you, and together you create flowcharts, logic trees, and logic tables.

Change Testing: This testing happens after a bug is found and the software is sent back to the developer, and the developer fixes the bug and returns the updated app to be tested again.

 

The Three Testing Approaches are:

Black-box Testing: For testing components with requirements, such as screens and reports. 

White-box Testing: for testing components where you can see the code, like logic modules.

Ad-hoc Testing: for testing an app when no requiems are present.  This can be due to time constraints, or perhaps you were just given an app and have to learn about it to create an approach and set of test plans to test it adequately.

 

 

What do you learn from a CTFL Certification?

I've been instructing students on the ISTQB-CTFL Certification curriculum for a number of years now.  I like to tell students the CTFL program is really a mini project-management training program.

Somebody hands you a software app to test, what do you do now?  You decompose the App in to smaller portions which make sense to you, constructing a Work Breakdown Structure.  Then you identify items to test within the WBS, organizing them by Test Plans. They you prioritize all test plans into a Test Execution Schedule so and your test team will know in what order to perform the tests.

You'll keep track of test coverage, and results, reporting on progress as you work through the app. Here's where it's important to communicate with your developers thoroughly so they can fix bugs you find and get the updated app install back to you for confirmation testing.

finally, when your testing matches the success criteria, you'll report of the final results of your findings and wrap up the testing project, adding lessons learned, estimates vs. actual, and other information into a test historical repository.

All of these activities are the foundation for project management.