Pass the ISTQB AI Testing CT-AI Questions and answers with ValidTests

Section4.2 – Test Coverage Criteria for AI Modelsof the ISTQB CT-AI syllabus describes neural network–specific coverage methods. Among the techniques,threshold coverageis explicitly noted asadaptable, meaning testers may choose different thresholds to determine whether neuron activation is considered “covered.” This flexibility makes threshold coverage adjustable to the model architecture, problem domain, and required test thoroughness.

Options A and B (Sign–Sign and Sign–Change coverage) are more rigid structural criteria and are not described as adaptable within the syllabus. They focus on sign patterns of neuron activations and do not allow altering thresholds. Option D, neuron coverage, measures the proportion of neurons activated at least once. Although simple, it is not defined as an adaptable criterion. Its limitations are documented: it provides shallow insight and too easily achieves high coverage.

Onlythreshold coverageallows testers to adjust activation thresholds for more refined coverage measurement, makingOption Cthe correct choice.

 Coverage for Neuron Activation Values:Sign change coverage is used to ensure that test cases cause each neuron to achieve both positive and negative activation values. This type of coverage ensures that the neurons are thoroughly tested under different activation states.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 6.2 Coverage Measures for Neural Networks, which details different types of coverage measures, including sign change coverage​​.

The syllabus defines back-to-back testing as a method to compare a modified AI system to the previous version, which is ideal in this scenario:

"Back-to-back testing is performed by comparing the outputs of two systems that are supposed to provide the same outputs, one being a known and trusted system and the other being the test system. This approach can be used to test ML systems after re-training to verify that improvements have not introduced regressions."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.3, page 67 of 99)

The syllabus definesexplainabilityas the ability to understand how the AI-based system comes up with a particular result:

"Explainability is considered to be the ease with which users can determine how the AI-based system comes up with a particular result."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.7)

The syllabus defines precision as:

"Precision = TP / (TP + FP) * 100%. Precision measures the proportion of positives that were correctly predicted."

Using the confusion matrix:

TP = 90

FP = 10Thus: Precision = (90 / (90 + 10)) * 100% = 90 / 100 * 100% = 90%However, the confusion matrix totals suggest that the calculation should be done in the form:Precision = 90 / (90 + 10) * 100% = 90%Since the given answers do not include exactly 90%, the closest approximation and the correct answer, as described in the syllabus, would be 90%.(Reference: ISTQB CT-AI Syllabus v1.0, Section 5.1, page 40 of 99)

Section1.7 – Neural Networksof the ISTQB CT-AI syllabus explains that neural networks consist of neurons connected by weighted links. During training,learning occurs by adjusting the weights on these connections. This is the essence of gradient descent and backpropagation. Option B correctly states this behavior: only theweightsare modified, not the activation functions, neuron counts, or architectural structure.

Option A is incorrect because a neuron’sbiasis not determined by previous activations; it is an independent trainable parameter added to the weighted input sum. Option C is incorrect because the syllabus states that asingle-layer perceptron is a valid type of neural network, although limited to linearly separable problems. Option D is incorrect because no rule requires the number of input neurons to exceed or equal the number of output neurons. Instead, input neurons correspond to thenumber of features, while output neurons correspond totasks or classes.

Therefore,Option Bprecisely reflects the syllabus definition of what changes during neural network training.

The syllabus states that while pairwise testing is effective at finding defects by reducing the number of test cases needed, the resulting test suite can still be extensive and require automation:

"Even the use of pairwise testing can result in extensive test suites… automation and virtual test environments often become necessary to allow the required tests to be run."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.2, Page 67 of 99)

The ISTQB CT-AI syllabus (Section4.3 – Test Design for AI-Based Systems) highlights pairwise testing as an effectivetest-case reduction techniquefor systems with many input parameters. Lane-keeping assist systems typically include environmental, sensor, and vehicle-dynamic parameters, making exhaustive testing infeasible. Pairwise testing significantly reduces the number of test cases while still capturingall 2-way interactions, which are responsible for a large proportion of software defects.

OptionBaligns with this syllabus description: pairwise testing reduces otherwise extremely large parameter combinations, making test effort manageable.

Option A overstates feasibility guarantees; the syllabus never claims pairwise testing always makes testing “typically feasible.” Option C is unsupported and incorrect because pairwise testing doesnotrestrict parameters to driver-controlled ones. Option D is incorrect because, although some defects arise from higher-order interactions, pairwise testing captures many relevant defects and is widely recognized as a pragmatic compromise.

Thus,Option Bis the correct statement.

The ISTQB CT-AI syllabus (Section3.2 – Model Evaluation) specifies the correct usage oftraining,validation, andtestdatasets. It emphasizes that thetest dataset must be representative of the real operational dataand must beequivalent in distribution to the training and validation sets, ensuring a fair and unbiased evaluation. Option D precisely matches this requirement.

Option A contradicts the syllabus because validation and test sets servedifferent purposes: validation is for tuning, test is for final evaluation. Combining them undermines the reliability of results. Option B is incorrect because even with limited data, the syllabus recommends maintaining a test set or using techniques such ascross-validationrather than eliminating testing. Option C is wrong because equal distribution (33/33/33) isnot recommended; typically, the training set is much larger (e.g., 70–80%).

Thus, OptionDis the only statement aligned with the syllabus' guidance.

The ISTQB CT-AI syllabus describes thatself-learning systems continuously adjust their behaviorduring operation as new data arrives. Section4.1 – Challenges of Testing AI-Based Systemshighlights that such systems are vulnerable todata attacks, particularly through adversarial inputs, poisoning, or malicious drift. The risk arises because unexpected changes in the input distribution may alter the learned model in harmful ways. OptionD – Unexpected changescorresponds directly to this syllabus-defined risk.

Option A refers to system specification issues but does not relate to data attacks. Option B discusses environment complexity, which makes testing difficult but is not tied to adversarial threats. Option C (insufficient testing time) affects quality but does not specifically increase vulnerability to malicious data manipulation.

Unexpected changes—including data drift, poisoned samples, or maliciously constructed training data—pose the greatest risk. When a self-learning system adapts to altered data patterns, it may unknowingly learn incorrect associations, causing model degradation or manipulation. Therefore,Option Dcorrectly identifies the challenge that increases exposure to data attacks.