High overall accuracy can mask poor performance on minority classes, which is critical in medical applications. The correct approach is to first properly diagnose the issue using class-specific metrics. Solutions like data augmentation or adjusting the loss function to penalize errors on the rare class more heavily are standard, practical steps to address class imbalance without scrapping the entire model.

This is a classic case of domain shift. Language, context, and sentiment indicators in movie reviews (e.g., ‘thrilling’, ‘plot twist’) are fundamentally different from those in appliance reviews (e.g., ‘durable’, ‘easy to clean’). A model trained on one domain often fails to generalize to another without fine-tuning or retraining on target domain data.

This describes sensitivity analysis or robustness testing. The tester is probing how small, deliberate changes in input affect the output. This is crucial for understanding model stability, identifying fragile decision boundaries, and uncovering potential unfair biases where a tiny change in one feature (like postcode) unjustly flips a decision.

RMSE squares errors before averaging, making it much more sensitive to large errors than MAE. A substantially higher RMSE compared to MAE is a clear indicator that the model’s prediction errors include several large outliers (e.g., it’s very wrong on some expensive or unusual properties). This prompts investigation into those specific cases.

In the context of AI quality (often aligned with ISO/IEC 25010), ‘response latency’ directly measures how long the system takes to perform its function—generating a response. This falls under the characteristic of Efficiency, and more precisely, its sub-characteristic ‘Time Behaviour’. While important for user experience, it is distinct from correctness (functionality) or fairness.

White-box testing (or clear-box testing) implies full visibility into the internal workings of the system under test. Having access to the model’s architecture, weights, and the ability to inspect intermediate layer outputs provides this full transparency, allowing for tests based on the internal structure and logic of the AI model itself.

Recall (True Positive Rate) measures the proportion of actual frauds that the model correctly identifies. The business requirement (‘catch as many fraudulent cases as possible’) directly translates to maximizing Recall. This often comes at the cost of lower Precision (more false positives, i.e., legitimate claims flagged for review), which the business has stated is an acceptable trade-off.

This is a classic example of Metamorphic Testing (MT). Since the ‘correct’ output is often unknown for complex AI systems, MT verifies that the system’s outputs satisfy certain expected properties (metamorphic relations) when inputs are transformed. Here, the relation is that the order of historical data should not drastically change core recommendations, testing for robustness to irrelevant permutations.

Bias testing is paramount for a high-stakes, human-facing system like facial recognition. Historical and recent evidence shows such systems can have significantly higher error rates for certain demographic groups, leading to unfair denial of service. This poses severe ethical, legal, and reputational risks. While load testing (B) and performance (D) are important, fairness is the most critical non-functional concern here.

High overall accuracy can mask poor performance on minority classes, which is critical in medical applications. The correct approach is to first properly diagnose the issue using class-specific metrics. Solutions like data augmentation or adjusting the loss function to penalize errors on the rare class more heavily are standard, practical steps to address class imbalance without scrapping the entire model.0