ISTQB syllabus to aid engineers and AI professionals in upgrading AI testing methodologies

The global AI software market size is expected to experience rapid growth, expanding from $87 billion in 2021 to an estimated $1.597 trillion in 2030, according to a report by Precedence Research released on April 19, 2022. And with the advancement of AI technologies, AI models will also start to be used in more applications, some where the predictions made by the AI are critical to the system and users of those systems. As a result, AI systems will require more advanced testing to increase reliability, a crucial factor in their success or failure.

However, the AI software market is still in its infancy and systems are often developed using open-source algorithms with uncertain quality. Even tech giants like Google, Amazon, and Meta face issues such as data bias and ethical concerns.

As a result, a new field of AI testing is emerging. The demand for engineers with AI expertise has increased, leading companies to recruit AI experts and software testing engineers to bring in the necessary skills and knowledge.

Professional testing methods for AI professionals are being evaluated by industry experts, and the International Software Testing Qualification Board(ISTQB)’s CT-AI syllabus  is an example of this progress. The syllabus introduces various AI-based systems and explains how machine learning, a crucial aspect of AI, works. It highlights the challenges of testing AI systems, which are more complex than traditional systems due to their self-learning, autonomy, and probabilistic and non-deterministic models.

Test data plays a vital role in the performance testing of AI systems, and it’s important to consider preparing data for testing and verifying the entire system, including the AI component. Testing techniques such as Pairwise Testing, Back-to-Back Testing, A/B Testing, and Metamorphic Testing should be selectively applied to accommodate the characteristics of AI systems.

The ISTQB, with more than 160 member countries worldwide, provides certification programs for test engineers. In response to the rapidly growing AI market, the committee has quickly developed practical guidelines for AI testing.

“Many companies are developing AI systems but struggle to define performance indicators and measure performance,” said Won-il Kwon, CEO of STA Testing Consulting Inc., which operates the Korean branches of the global AI testing platform ‘AITest.AI’ and ISTQB. “They need guidance, and software testers and companies should start preparing for AI testing through professional learning and training. ISTQB AI Testing, which provides international best practices for AI software testing, could be a great start. We expect the continued development of AI testing technology to further boost the growth of the AI market.”