Pass the ISTQB AI Testing CT-AI Questions and answers with ValidTests

The syllabus states:

“A/B testing can be used to test updates to an AI-based system where there are agreed acceptance criteria, such as ML functional performance metrics, as described in Chapter 5. A/B testing is used to compare the updated variant with the previous variant.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.4, page 68 of 99)

Detecting biases in the ML pipeline involves various tests to ensure fairness and accuracy throughout the ML process.

Testing the distribution shift in the training data for inappropriate bias (A): This involves checking if there is any shift in the data distribution that could lead to bias in the model. It is an important test but not the most direct method for detecting biases.

Test the model during model evaluation for data bias (B): This is a critical stage where the model is evaluated to detect any biases in the data it was trained on. It directly addresses potential data biases in the model.

Testing the data pipeline for any sources for algorithmic bias (C): This test is crucial as it helps identify biases that may originate from the data processing and transformation stages within the pipeline. Detecting sources of algorithmic bias ensures that the model does not inherit biases from these processes.

Check the input test data for potential sample bias (D): While this is an important step, it focuses more on the input data and less on the overall data pipeline.

Hence, the most likely useful test to help detect different kinds of biases in the ML pipeline isB. Test the model during model evaluation for data bias.

Clustering is a type of machine learning technique used to group similar data points into clusters. It is a key concept in unsupervised learning, where the algorithm tries to find patterns or groupings in data without prior knowledge of output classes. Let's analyze each option:

A. Clustering is classification of a continuous quantity.

This is incorrect. Classification typically involves discrete categories, whereas clustering involves grouping similar data points. Classification of continuous quantities is generally referred to as regression.

B. Clustering is supervised learning.

This is incorrect. Clustering is an unsupervised learning technique because it does not rely on labeled data.

C. Clustering is done without prior knowledge of output classes.

This is correct. In clustering, the algorithm groups data points into clusters without any prior knowledge of the classes. It discovers the inherent structure in the data.

D. Clustering requires you to know the classes.

This is incorrect. Clustering does not require prior knowledge of classes. Instead, it aims to identify and form the classes or groups based on the data itself.

Therefore, the correct answer isCbecause clustering is an unsupervised learning technique done without prior knowledge of output classes​​.

 AI-Based Autonomous Functions:An AI-based autonomous system is one that can respond to its environment without human intervention. The other options either involve human decisions or do not use AI at all.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Sections on Autonomy and Testing Autonomous AI-Based Systems​​​​.

Decision trees are a foundational algorithm used in supervised machine learning. The syllabus describes:

"A decision tree is a tree-like ML model whose nodes represent decisions and whose branches represent possible outcomes."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.4)

The ISTQB CT-AI syllabus explains in Section1.6 – Machine Learning Approachesthatsupervised learningis appropriate when labeled data exists and the goal is to predict an output based on known historical examples. Predicting a customer’spurchase probabilityis aclassificationtask when the output corresponds to discrete categories such as“likely to purchase”vs.“not likely to purchase.”The syllabus gives similar examples in describing classification as the process of assigning instances to predefined classes based on learned patterns in labeled data. Because the retail company wants to determine whether a customer will make a purchase based on marketing actions, classification is the most appropriate choice .

Option A (regression) predicts continuous numeric values and is less suitable because the task centers oncategorical likelihood, not estimating exact monetary values. Option C (unsupervised learning) is used when labels are not available—here, the company has labeled purchase histories. Option D (reinforcement learning) requires an interactive environment with reward-driven behavior, which is not applicable to this scenario.

Thus,supervised learning (classification)is the most suitable approach according to the syllabus.

 Type of Testing for Stock-Price Prediction Models:Concept drift refers to the change in the statistical properties of the target variable over time. Severe weather storms causing massive losses in coffee production and affecting stock prices would require testing for concept drift to ensure that the model adapts to new patterns in data over time.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 7.6 Testing for Concept Drift, which explains the need to test for concept drift in models that might be affected by changing external factors​​.

Section1.6.3 – Reinforcement Learningof the ISTQB CT-AI syllabus states that reinforcement learning (RL) is based on anagent interacting with an environment, performing actions, and receivingrewards or penalties. The core concept is thereward function, which guides the agent’s learning process. The syllabus emphasizes that training in RL isdriven by rewards, and the agent aims to maximize cumulative reward over time. Therefore, OptionCdirectly reflects the correct description: the agent learns by being rewarded for successful actions .

Option A is incorrect because RL doesnotuse labeled data; that applies to supervised learning. Option B contradicts the syllabus definition: RL fundamentallyrequiresinteraction with the environment. Option D is incorrect because the reward function isdefined by humans, not learned by the agent; the agent learns apolicy, not the reward function itself.

Thus, OptionCis the only statement consistent with RL as defined in the syllabus.

According to the syllabus, pre-trained models often inherit biases and limitations from the data and processes used in their original training, which may not align with the new use case. Specifically, the syllabus states:

"When using a pre-trained model, the training data and process cannot be fully controlled or known by the user of the model. As a result, the model can inherit biases or inaccuracies that were part of its original development and training process."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 1.8.3)

The syllabus explains:

"Where the internal structure of an AI-based system is too complex for humans to understand, the system can only be tested as a black box. Even when the internal structure is visible, this provides no additional useful information to help with testing."

This confirms that black-box testing is chosen because the tester does not need to understand the system’s internal structure.

(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.5, page 61 of 99)