Summer Special Limited Time 65% Discount Offer - Ends in 0d 00h 00m 00s - Coupon code: myex65

Home > ISTQB > ISTQB AI Testing > CT-AI

CT-AI ISTQBCertified Tester AI Testing Exam Question and Answers

Question # 4

There is a growing backlog of unresolved defects for your project. You know the developers have an ML model that they have created which has learned which developers work on which type of software and the speed with which they resolve issues. How could you use this model to help reduce the backlog and implement more efficient defect resolution?

A.

Use it to prioritize defects automatically based on the time expected for the fix to be made, the speed of the fix, and the likelihood of regressions

B.

Use it to assign defects to the best developer to resolve the problem and to load balance the defect assignments among the developers

C.

Use it to determine the root cause of each defect and develop a process improvement plan that can be implemented to remove the most common root causes

D.

Use it to review the code and determine where more defects are likely to occur so that testing can be targeted to those areas

Full Access
Question # 5

"Splendid Healthcare" has started developing a cancer detection system based on ML. The type of cancer they plan on detecting has 2% prevalence rate in the population of a particular geography. It is required that the model performs well for both normal and cancer patients.

Which ONE of the following combinations requires MAXIMIZATION?

SELECT ONE OPTION

A.

Maximize precision and accuracy

B.

Maximize accuracy and recall

C.

Maximize recall and precision

D.

Maximize specificity number of classes

Full Access
Question # 6

In a certain coffee producing region of Colombia, there have been some severe weather storms, resulting in massive losses in production. This caused a massive drop in stock price of coffee.

Which ONE of the following types of testing SHOULD be performed for a machine learning model for stock-price prediction to detect influence of such phenomenon as above on price of coffee stock.

SELECT ONE OPTION

A.

Testing for accuracy

B.

Testing for bias

C.

Testing for concept drift

D.

Testing for security

Full Access
Question # 7

Which ONE of the following options describes a scenario of A/B testing the LEAST?

SELECT ONE OPTION

A.

A comparison of two different websites for the same company to observe from a user acceptance perspective.

B.

A comparison of two different offers in a recommendation system to decide on the more effective offer for same users.

C.

A comparison of the performance of an ML system on two different input datasets.

D.

A comparison of the performance of two different ML implementations on the same input data.

Full Access
Question # 8

Which of the following aspects is a challenge when handling test data for an AI-based system?

A.

Personal data or confidential data

B.

Output data or intermediate data

C.

Video frame speed or aspect ratio

D.

Data frameworks or machine learning frameworks

Full Access
Question # 9

Arihant Meditation is a startup using Al to aid people in deeper and better meditation based on analysis of various factors such as time and duration of the meditation, pulse and blood pressure, EEG patters etc. among others. Their model accuracy and other functional performance parameters have not yet reached their desired level.

Which ONE of the following factors is NOT a factor affecting the ML functional performance?

SELECT ONE OPTION

A.

The data pipeline

B.

The quality of the labeling

C.

Biased data

D.

The number of classes

Full Access
Question # 10

"AllerEgo" is a product that uses sell-learning to predict the behavior of a pilot under combat situation for a variety of terrains and enemy aircraft formations. Post training the model was exposed to the real-

world data and the model was found to be behaving poorly. A lot of data quality tests had been performed on the data to bring it into a shape fit for training and testing.

Which ONE of the following options is least likely to describes the possible reason for the fall in the performance, especially when considering the self-learning nature of the Al system?

SELECT ONE OPTION

    The difficulty of defining criteria for improvement before the model can be accepted.

    The fast pace of change did not allow sufficient time for testing.

    The unknown nature and insufficient specification of the operating environment might have caused the poor performance.

A.

There was an algorithmic bias in the Al system.

Full Access
Question # 11

A tourist calls an airline to book a ticket and is connected with an automated system which is able to recognize speech, understand requests related to purchasing a ticket, and provide relevant travel options. When the tourist asks about the expected weather at the destination or potential impacts on operations because of the tight labor market, the only response from the automated system is, "I don’t understand your question."

This AI system should be categorized as?

A.

General AI

B.

Narrow AI

C.

Super AI

D.

Conventional AI

Full Access
Question # 12

“BioSearch” is creating an Al model used for predicting cancer occurrence via examining X-Ray images. The accuracy of the model in isolation has been found to be good. However, the users of the model started complaining of the poor quality of results, especially inability to detect real cancer cases, when put to practice in the diagnosis lab, leading to stopping of the usage of the model.

A testing expert was called in to find the deficiencies in the test planning which led to the above scenario.

Which ONE of the following options would you expect to MOST likely be the reason to be discovered by the test expert?

SELECT ONE OPTION

A.

A lack of similarity between the training and testing data.

B.

The input data has not been tested for quality prior to use for testing.

C.

A lack of focus on choosing the right functional-performance metrics.

D.

A lack of focus on non-functional requirements testing.

Full Access
Question # 13

An airline has created an ML model to project fuel requirements for future flights. The model imports weather data such as wind speeds and temperatures, calculates flight routes based on historical routings from air traffic control, and estimates loads from average passenger and baggage weights. The model performed within an acceptable standard for the airline throughout the summer but as winter set in, the load weights became less accurate. After some exploratory data analysis, it became apparent that luggage weights were higher in the winter than in summer.

Which of the following statements BEST describes the problem and how it could have been prevented?

A.

The model suffers from drift and therefore should be regularly tested to ensure that any occurrences of drift are detected soon enough for the problem to be mitigated

B.

The model suffers from drift and therefore the performance standard should be eased until a new model with more transparency can be developed

C.

The model suffers from corruption and therefore should be reloaded into the computer system being used, preferably with a method of version control to prevent further changes

D.

The model suffers from a lack of transparency and therefore should be regularly tested to ensure that any progressive errors are detected soon enough for the problem to be mitigated

Full Access
Question # 14

Consider a machine learning model where the model is attempting to predict if a patient is at risk for stroke. The model collects information on each patient regarding their blood pressure, red blood cell count, smoking status, history of heart disease, cholesterol level, and demographics. Then, using a decision tree the model predicts whether or not the associated patient is likely to have a stroke in the near future. Once the model is created using a training dataset, it is used to predict a stroke in 80 additional patients. The table below shows a confusion matrix on whether or not the model made a correct or incorrect prediction.

The testers have calculated what they believe to be an appropriate functional performance metric for the model. They calculated a value of 0.6667.

Which metric did the testers calculate?

A.

F1-score

B.

Precision

C.

Recall

D.

Accuracy

Full Access
Question # 15

An engine manufacturing facility wants to apply machine learning to detect faulty bolts. Which of the following would result in bias in the model?

A.

Selecting training data purposely excluding specific faulty conditions

B.

Selecting training data by purposely including all known faulty conditions

C.

Selecting testing data from a different dataset than the training dataset

D.

Selecting testing data from a boat manufacturer's bolt longevity data

Full Access
Question # 16

Which of the following is one of the reasons for data mislabelling?

A.

Lack of domain knowledge

B.

Expert knowledge

C.

Interoperability error

D.

Small datasets

Full Access
Question # 17

Which ONE of the following approaches to labelling requires the least time and effort?

SELECT ONE OPTION

A.

Outsourced

B.

Pre-labeled dataset

C.

Internal

D.

Al-Assisted

Full Access
Question # 18

A team of software testers is attempting to create an AI algorithm to assist in software testing. This particular team has gone through over 40 iterations of testing and cannot afford to spend as much time as it takes to run the full regression test suite. They are hoping to have the algorithm reduce the amount of testing required, thus reducing the time needed for each testing cycle.

How can an AI-based tool be expected to assist in this reduction?

A.

By using a clustering method to quantify the relationships between test cases and then assigning each test case to a category

B.

By performing optimization of the data from past iterations to see where the most common defects occurred and select the corresponding test cases

C.

By performing Bayesian analysis to estimate the types of human interactions that are expected to be seen in the system and then selecting those test cases

D.

By using A/B testing to compare the last update with the newest change and compare metrics between the two

Full Access
Question # 19

Which ONE of the following combinations of Training, Validation, Testing data is used during the process of learning/creating the model?

SELECT ONE OPTION

A.

Training data - validation data - test data

B.

Training data - validation data

C.

Training data • test data

D.

Validation data - test data

Full Access
Question # 20

Which ONE of the following describes a situation of back-to-back testing the LEAST?

SELECT ONE OPTION

A.

Comparison of the results of a current neural network model ML model implemented in platform A (for example Pytorch) with a similar neural network model ML model implemented in platform B (for example Tensorflow), for the same data.

B.

Comparison of the results of a home-grown neural network model ML model with results in a neural network model implemented in a standard implementation (for example Pytorch) for same data

C.

Comparison of the results of a neural network ML model with a current decision tree ML model for the same data.

D.

Comparison of the results of the current neural network ML model on the current data set with a slightly modified data set.

Full Access
Question # 21

Which ONE of the following characteristics is the least likely to cause safety related issues for an Al system?

SELECT ONE OPTION

A.

Non-determinism

B.

Robustness

C.

High complexity

D.

Self-learning

Full Access
Question # 22

A system was developed for screening the X-rays of patients for potential malignancy detection (skin cancer). A workflow system has been developed to screen multiple cancers by using several individually trained ML models chained together in the workflow.

Testing the pipeline could involve multiple kind of tests (I - III):

I.Pairwise testing of combinations

II.Testing each individual model for accuracy

III.A/B testing of different sequences of models

Which ONE of the following options contains the kinds of tests that would be MOST APPROPRIATE to include in the strategy for optimal detection?

SELECT ONE OPTION

A.

Only III

B.

I and II

C.

I and III

D.

Only II

Full Access
Question # 23

Written requirements are given in text documents, which ONE of the following options is the BEST way to generate test cases from these requirements?

SELECT ONE OPTION

A.

Natural language processing on textual requirements

B.

Analyzing source code for generating test cases

C.

Machine learning on logs of execution

D.

GUI analysis by computer vision

Full Access
Question # 24

Which ONE of the following types of coverage SHOULD be used if test cases need to cause each neuron to achieve both positive and negative activation values?

SELECT ONE OPTION

A.

Value coverage

B.

Threshold coverage

C.

Sign change coverage

D.

Neuron coverage

Full Access