The new ISTQB CT-AI 2.0

Added: 2026-06-03

[00:00:01]AI systems are different. They learn, they evolve, they behave probabilistically. Testing them requires a new level of expertise. The ISTQB certified tester. AI testing certification focuses exclusively on validating AI based systems from data quality and model behavior to bias, robustness, and life cycle risks.

[00:00:25]Structured around the machine learning life cycle, CTAI equips professionals to assess trust, reliability, and safety in AI solutions. As AI adoption accelerates, quality assurance must evolve with it. For those ready to test beyond traditional software, AI testing begins here. Discover the ISTQBI certification.

[00:01:20]Greetings from my TQV good morning, good afternoon and good evening wherever you are in the world. We have the immense pleasure of having Dr. Claudia Dusa Ziger uh again with us after the ISTQB product uh certification portfolio. We had a great discussion over there about different certifications on ISTQB. Now it's time to tackle the AI segment of the certification. So we'll start with um the CTI certified tester artificial intelligence. So uh Claudia, you are the president of ISTQB. You're also the product owner of this uh uh product of ISTQB of this certification of ISTQB. So yes, I'll give you the stage for you to enlight us on this AI testing. The floor is yours.

[00:02:13]>> Thank you. Love welcome also from my side. If you have been following the panel discussion, I'm now very happy to give you more details and show you how actually ISDQB and AI in general got together and we will end the presentation with the version 2.0 which is the new one, the thrilling one. Let me just briefly discuss who I am. So I am the current ISDQB president. The reason why I'm here now talking about AI is because I'm actually the product owner for the certified tester AI testing syllabus and I'm immensely proud of that. And besides being the product owner, I'm also working on a national level. So I'm member of the German testing board. I'm also working in standardizations and my paid work is actually head of consulting for a testing company in Germany for imbuss.

[00:03:02]But let's get into the interesting topics. So just a reminder what is ISDQB all about? It is about qualifications for software testers. It's a leading scheme in this area with more than 1 million certificates and the the base idea is to provide testers with terminology but with knowledge knowledge about testing in general and in particular also for new topics for new technologies coming up and AI definitely is one of them. AI is reshaping the entire software engineering industry and we definitely definitely strongly need to look at AI and that's actually what we did and of course we also update our existing sylli. So we started looking into AI already a long time ago. Here you can see our portfolio as it is on the website right now and we will be looking at the AI testing syllabus. So this is what I meant before. We started looking into AI testing already a long time ago.

[00:04:07]Definitely before the advent of chat GPT which happened years later back in 2019 AI was already a big topic and several groups within ISDQB within the ISDQB ecosystem already had worked on syllabi on testing and AI. So we had a group in India, we had a group in China, Korea and a third group sitting in Europe.

[00:04:32]They came forward with their syllabi and then we said it doesn't make any sense to have three of them. So the decision was made to merge them into one ISDQBI testing syllabus and that's what we did.

[00:04:45]We started the work in uh January 2020 and we finally released the new syllabus in October 2021. So again way before chat GPT what was the content there? It was an introduction to AI and in particular to machine learning. So that was one-third or perhaps a little more than onethird of the syllabus. It is the main focus is on testing of AI based system. How do you go along to do that?

[00:05:14]And a little part was on testing with AI. How does AI support you when you want to do testing? What you see on the screen now is actually the setup of the syllabus. The little uh red boxes are the chapter names and then you can see the subchapters. So moving from the left to the right, you can see that we really start with a definition of AI, what is a strong AI, what is a super AI, what is an AI technology, what are pre-trained models. So we really start from the basics covering the specific quality characteristics for AI based systems like autonomy like adaptability and stuff like that. We go through machine learning we look at the workflow we look at the data that you need. We look at metrics quality criteria in that sense like precision like accuracy based all on the confusion matrix. We look at neural networks and then starting with chapter six we start to look into the question how do you test that? So we look at different test levels we look at new techniques like metamorphic testing like the the application of pairwise testing or AB testing and then in the last chapter in the 10th chapter we look at the test environments that you need and then in the last chapter we have been looking into using AI for testing. Now looking at that chapter that really triggered some thought especially after chat GPT was released back in 2021 generative AI at large language models were not yet a hot topic and just looking at the outline of the syllabus you can see they're not even covered there with the publication of chat GPT with the immense evolution of all these models whether it's Gemini whether it's croc whether it's deepseek This area has attracted a lot of attention and within ISTDP it became clear that we need to develop a syllabus that is specifically dealing with generative AI and in that sense also using generative AI for testing and that was the point in time when we decided we need to come up with something new and that something new is really our new version the second version of the AI testing syllabus. So already back two two and a half years we started into looking what is up to date right now what should we take out what should we add in in order to be really upto-date and relevant for the testers to really use the syllabus for their professional growth and this is what we did so we looked at the existing version one we decided to reduce the introduction into AI I we decided to take the part on using AI for testing out because now we have a separate syllabus on that. Taking that out, we got a little bit of space and we added in generative AI. What is that? We added in large language models. What is that?

[00:08:24]How can you use them? And not how can you use them, how how do they work? We added in some more testing techniques like red teaming. How can you influence large language models in order to give you different results?

[00:08:39]In total, we slimlined all the chapters and we did an editorial facelift so that they read nicer.

[00:08:49]At the end, we we came to this result.

[00:08:52]So, right now, we only have seven chapters. You see that they're way slimmer. So, the introduction to AI is shorter. We still look at the quality characteristics because I think it's still very important to understand what these are. And then we look into machine learning. So basically machine learning is still the focus. If you look at testing AI based systems, you can see that we now have a new subsection on testing generative AI and large language models. And when we do the introduction to AI, large language models and generative AI are already covered there.

[00:09:29]So we're starting from the beginning at explaining what it is and then here in the fourth chapter we talk about testing these models. Of course we're also looking at the technique in comparison to the previous syllabus. They just moved down one further level but they're all still there. So we're still talking about AB testing. We're still talking about metamorphic testing and other techniques pair-wise testing. And then at the end we added or we defined more precisely three additional test levels in comparison to the four test levels that you already know from the foundation level. So when you are trying to test an AI based systems we look at the data what kind of input data testing you need to do in order to guarantee that the quality is high. Then we look at the model testing once the algorithm is ready and it is trained. How can you test the model? Here again we will look at some of the metrics that we had and the last test level is the machine development machine learning development testing where you look at aspects like whether the pipeline is working correctly whether the conversion of the model to the final target is working correctly and stuff like that.

[00:10:44]Altogether the content was shrunk and instead of a 4-day training we now have a three-day training.

[00:10:52]One more detail I want to point out since I think it is important for AI testing is that in this syllabus and it's not the only one but I think it is important here we have a lot of hands-on exercises or we have hands-on objectives which are then connected to hands-on exercises.

[00:11:10]What does that mean? You are doing exercises where you really work with the software system. So it's not a pen and a paper um exercise but a real work with existing language models with existing data and stuff like that where you really need to apply it. And you can see here we have eight hands-on exercises within our syllabus where you create your own machine learning model where you actually look how data influences that goes into overfitting and underfitting. you do an implementation of a person and you're actually the last three are the application of LLMs for example on boundary value analysis or to perform other testing input data testing for machine learning data sets and actually apply metamorphic testing. So these hands-on exercises give you really a practical grip on the content that is within the syllabus.

[00:12:06]If I would want to sum it up and put it as at a glance at this again, this is a picture that was generated by AI, you can see the chapters on AI on the left hand side with the key points in there with the timing in there. You can see the chapters that are directly connected or mapped to testing. On the right hand side again with the key points and the timing. In the middle you have the the general understanding what you will learn and also the business outcomes that are connected to the syllabus. So this gives you an overall picture of what the syllabus what the content of the syllabus is what the syllabus is giving you if you really want to take it and and move forward with it.

[00:12:54]I think it is a very valuable syllabus if you have a system that has an AI component and you need to figure out how to test it. What would your strategy be?

[00:13:04]Where do you have to look? What do you do with the data? How do you make sure that you don't have a bias that you get rid of um a concept drift or a data drift and all stuff like that that is covered in the syllabus.

[00:13:19]In total, let me just read through my points. AI is not going to go away. AI has been around for a long time and AI components are integrated in more and more systems. So, I think we really need to talk about that technology. We need to know how it works. We need to improve our knowhow and our skills and we still as humans need to do that. So, there is no way around. We need to tackle the topic and know what AI can do and what it can't do and how we would reasonably test it. ISDQP has two syllabi that are focused on AI. The one that I was presenting you is actually the ISDQP certified tester AI testing where you learn how to test an AI based component.

[00:14:12]We do have the second one which is the other side of the coin which is the ISDQB certified tester testing with generative AI and this is the syllabus that explains to you in detail how you can use language models how you can use generative AI in order to support you with software testing. So it's not about testing an AI based component but about using it in order to improve your testing in order to become more efficient. So we do have two syllabi.

[00:14:44]have a look at them and benefit from the expertise which is gathered there because experts from around the world came together to develop these two syllabi and it really I really think it is worth it spending the time going through the material getting exposed getting an introduction to AI and then really using that technology with the generative AI syllabus to improve your testing but with the AI testing syllabus definitely ely to be aware to be ready that we can test systems that have an AI component or AI based component that we can finally trust these system to work properly and I think that is the main intention the main incentive why we testers are doing stuff like that so yes go for it that would be my recommendation that is the end of my presentation I know that presenting syllabi is a little boring but But in this case I really do believe that we have an excellent product which is viable which is really really important that we understand it as testers and which is going to change the way how software is developed. So yes we need to look at AI and that actually leads me back or I want to get back give back to the studio. So love are there any questions for me?

[00:16:09]Uh I can see some question flowing in but one important message that you you said is that AI is not going away. We as a tester will have to adopt it. We'll have to um live around it and use it as a tool to make sure that we are making the maximum leverage or we are leveraging the maximum on them as a tool. So uh yes let's uh go through those questions. Um the first one, what is the difference between the AI testing syllabus and the Gen AI syllabus? You've you've uh introduce the genai already.

[00:16:46]>> Well, >> yeah. The main difference >> the the difference I tried to explain but perhaps not good enough. So these two syllabi are like two sides of a coin. If you are testing and you're not testing AI based system, let's say a conventional system and if you want to improve your testing process, then you should go for the testing with generative AI because then perhaps you are faster, you're moving uh you you can test more, you have a bigger coverage.

[00:17:15]So that would be the syllabus that you should look at if you want to improve your testing. On the other side, if you have a product that now has an AI component because some smart people figured that your product needs an AI component now and they implemented it and you as the quality person or the test manager or tester want to ensure that it is working properly, then you need to know how to test the AI based component, the entire system, how you do the integration test and so on and so on. And this is the point where you should have a look at the AI testing syllabus because the AI testing syllabus definitely tells you how to test an AI component. Whether you're using AI to support you there is a completely different question but the focus is the the product itself which has an AI component now and how to test that that is the AI testing syllabus.

[00:18:08]>> That's the essence of the difference.

[00:18:10]Yes. All right. Thank you Claudia. We'll take another question.

[00:18:15]Do I need to be an AI expert to understand the AI testing syllabus?

[00:18:20]>> No. No. And I think that is the beauty of the syllabus and sometimes also a point that is criticized quite a lot. If you go to the version 10 of the syllabus, you have a very nice long introduction into AI. So if you are not aware of how AI is working, what it is in general, when it was started to be developed, go to the syllabus and you get a nice introduction. You don't need to be an expert in order to understand the concepts of the AI testing syllabus.

[00:18:52]In the version 20, you still don't need to be an expert. The introduction is a little shorter. We slimmed it down a bit, but still you get an introduction.

[00:19:03]You will develop an understanding what AI is and then you can look at the different levels and techniques to apply to an AI system. But you do not you do not need to be an expert before it is really meant as an entry level for everybody who's exposed to this technology for the first time.

[00:19:25]>> All right, understood.

[00:19:27]Uh we'll take a third question. Ah it's about the update. AI is moving quite fast. Uh so are you planning to update the AI testing syllabus?

[00:19:37]>> Yet this is a little a tricky question.

[00:19:40]Um in ISDQB we introduced a general rule that every five years you need to review a syllabus and just state whether it's still up to date, whether it needs to be completely uh over um redone or whether it's still okay. Uh we are also going to do that for the AI testing syllabus.

[00:20:01]Perhaps we're also already doing that earlier. That really depends on the speed which with which the technology is evolving. If you look at the generative AI there the update will be way faster because the large language models are developing way faster. I think for the AI testing we will not keep up to that level but I would expect that perhaps in two three years we might have a new version. So yes, we will definitely as soon as the technology changes that much that we feel that it is necessary, we will do an update like we did from version one to version two because in version one we didn't even talk about generative AI and LLMs because back in 2019 2020 2021 it was not the topic yet.

[00:20:50]So perhaps in the next version we will talk about something that will be developed now or in the next couple of days or years. We'll see.

[00:20:59]>> We'll see. We don't know what's coming up.

[00:21:01]>> No, we don't know.

[00:21:02]>> So, we'll see.

[00:21:04]>> All right. Uh let's take a last question then. Is there a prerequisite for the AI testing exam?

[00:21:13]Yes. Um the AI testing exam is or the AI testing syllabus is a specialist module and for all the specialists like for all the other syllabi within ISTQP the prerequisite is the foundation level to take the exam. If you want to take the training you can always take the training but if you want to do the exam you need to have the foundation level certificate in order to be uh allowed to take the AI testing exam. So yes, there is a prerequisite for taking the exam, but I personally think it is a very reasonable one because you should know the basics of software testing before venturing into AI. That's true.

[00:21:52]That's true. So to our audience, if ever you want to learn more about this uh uh certification from ISTQB, please visit the website.org where you will have on the portfolio AI testing. You click on it, then you can click on learn more. You will have the syllabus over there. You will have the sample paper and also the answers to those questions in the sample paper so you could prepare yourself and test your knowledge and then maybe to have the hands on. I think that's that's the value of of having a training as well is to have the hands-on.

[00:22:23]>> Yes.

[00:22:23]>> Right.

[00:22:24]>> Definitely.

[00:22:24]>> Uh so then if you want to have the the practical side of u of of the certification then to take up uh a training. Um yeah those are the questions that we we had Claudia. So um thank you thank you for sharing uh uh all these donates with us. Thank you for giving a sneak peek at this uh brand new syllabus um 2.0 city AAI version 2.0. Um and if uh anyone in our audience is interesting with the is interested deeply interested with the AI topic we have in few minutes um Dr. Abas Ahmed will be joining us to talk about generative AI. So we will get the other side of the coin of the AI uh syllabus from my TV. So Cla thank you again.

[00:23:14]>> Yes, >> my pleasure. My pleasure. You take care.

[00:23:19]>> Thank you. And to our audience please um as usual please follow us on Facebook, LinkedIn and Tik Tok and you will be up to date with what's going on with ISTV.

[00:23:29]Thank you.

[00:23:29]>> Thank you.

[00:23:47]AI systems are different. They learn, they evolve, they behave probabilistically. Testing them requires a new level of expertise. The ISTQB certified tester AI testing certification focuses exclusively on validating AI based systems from data quality and model behavior to bias, robustness and life cycle risks. Structured around the machine learning life cycle, CTI equips professionals to assess trust, reliability and safety in AI solutions.

[00:24:20]As AI adoption accelerates, quality assurance must evolve with it. For those ready to test beyond traditional software, AI testing begins here.

[00:24:31]Discover the ISTQBI certification.