AI-300 Operationalizing Machine Learning and Generative AI Solutions (beta) Question and Answers

In Azure ML command jobs, MLflow automatically creates a run context, so explicit start_run is optional. The training script must call mlflow.log_param to record each hyperparameter for the run. If the script calls this for each hyperparameter, the requirement for capturing parameters per run is met. The script must also call mlflow.log_metric to record numeric metrics such as accuracy or loss, optionally indexed by training step. Metrics and parameters logged inside an Azure ML command job are automatically associated with the correct experiment and run — no manual specification of experiment or run ID is required. If all three elements (run context, parameter logging, metric logging) are present in the script, all experiment tracking statements are True. If any element is absent, the corresponding statement is False and the tracking requirement is not met.

Microsoft Learn Reference Topic: Log metrics, parameters, and artifacts during Azure Machine Learning training runs with MLflow

Imagine a hospital analytics firm with data scientists who kick off dozens of training jobs throughout the week. During peak hours, five jobs compete for the same GPU cluster and fail or queue for hours. On quiet nights, that cluster sits completely idle, burning money. Managed compute targets with autoscaling solve both problems: the cluster scales out automatically when multiple jobs arrive simultaneously and scales back to zero when idle. Option A (single shared cluster) is exactly the resource-contention problem Fabrikam already has. Option B (fixed-size cluster) wastes money during off-peak hours. Option C (dedicated per-experiment clusters) eliminates contention but is prohibitively expensive for a cost-conscious healthcare firm. Autoscaling managed compute is the cloud-native answer to variable workload demand.

Microsoft Learn Reference Topic: Azure Machine Learning compute targets – Configure autoscale for compute clusters

To ensure runtime dependencies are consistent and reusable, first create a conda.yaml or requirements.txt file that lists all Python packages and system libraries required by your training code — this file is the single source of truth for your runtime. Next, create an Environment object using the Azure ML Python SDK v2 with a name and reference to the conda.yaml file, specifying the base Docker image. Then register the Environment by calling ml_client.environments.create_or_update, which publishes it to the workspace registry with an auto-incremented version. Finally, reference the registered environment by name and version in all pipeline job steps. Azure ML will build or retrieve the cached Docker image and use it as the execution container. This approach means updating dependencies only requires modifying the conda.yaml and registering a new version — training code remains unchanged.

Microsoft Learn Reference Topic: Create and manage Azure Machine Learning environments – Reusable curated environments

During experimentation, a data science team produces trained model artifacts they want to share with the MLOps team for deployment, with the governance team for compliance, and with the QA team for testing. Registering the model in the Azure ML model registry gives each of these teams access to the versioned, immutable artifact through a single reference. Option B (Component) packages a reusable pipeline step, not the model output itself. Option C (Environment) captures the Python runtime, not the trained artifact. Option D (Pipeline) defines the orchestration workflow, not the resulting model. In Azure ML, the model registry is the governance store for trained model artifacts, recording who created it, when, from what data and code, and which metrics it achieved — making it the asset to register for reuse and governance across projects.

Microsoft Learn Reference Topic: Register and manage models in Azure Machine Learning – Model registry

For deployment failures when containers do not start, review deployment logs in Azure ML Studio or use the CLI get-logs command to surface dependency installation errors, missing files, or scoring script import failures. For authentication errors such as 401 or 403 responses, check that client applications are using valid authentication keys or bearer tokens, as mismatched or expired credentials cause auth failures. For performance degradation and slow responses, review compute instance scaling configuration, since a surge in traffic saturates a fixed number of replicas causing queuing and high latency without autoscaling. For internal server errors such as HTTP 500, these typically originate in the scoring script, so inspect the init and run functions for unhandled exceptions and verify the environment has all required packages installed.

Microsoft Learn Reference Topic: Troubleshoot online endpoint deployment failures – Azure Machine Learning

Different query types require fundamentally different retrieval algorithms. For queries that include exact policy identifiers, keyword or BM25 search is the correct choice because BM25 scores documents based on term frequency and inverse document frequency — an exact match on a specific policy identifier such as POL-2024-HR-042 scores very highly regardless of semantic context. This is the right approach when semantic similarity is low but exact term matching is critical. For natural-language questions where keywords may not be an exact match, semantic or vector search is the correct choice because vector embeddings capture meaning rather than exact tokens, finding relevant documents even when the user ' s vocabulary differs from the document ' s terminology. Azure AI Search supports both modes through its hybrid search capability, and the correct configuration maps each query type to its optimal retrieval algorithm.

Microsoft Learn Reference Topic: Configure hybrid search in Azure AI Search – BM25 keyword search vs. semantic vector search

In Microsoft Foundry ' s built-in evaluation framework, each evaluator measures a specific dimension of language quality. Fluency evaluates the grammatical correctness, sentence structure, and overall linguistic smoothness of generated text. A high fluency score means the response reads naturally, with proper grammar, punctuation, and sentence construction — regardless of factual accuracy or relevance to the topic. Coherence (option A) measures whether ideas flow logically from one sentence to the next — it is about logical structure, not grammar. Textual Similarity (option B) measures how closely the generated text matches a reference text using metrics like ROUGE or BLEU — it is a comparison metric, not a standalone quality dimension. Groundedness (option C) measures whether claims are supported by the provided context — it is a factual fidelity metric. For grammatical correctness specifically, Fluency is the correct evaluator.

Microsoft Learn Reference Topic: Built-in AI evaluators in Microsoft Foundry – Fluency, Coherence, Groundedness, and Similarity

MLflow is the industry-standard open-source platform for experiment tracking, and Azure Machine Learning has first-class native integration with it. When you use MLflow within an Azure ML job, parameters, metrics, and artifacts are automatically logged to the run history of the AML workspace, making every run reproducible and comparable. Option A (AML job output logs) only captures console output and lacks structured parameter and metric logging. Option C (Application Insights logs) is designed for application-level telemetry, not ML experiment metadata. Option D (Azure Monitor alerts) is a reactive notification tool, not a tracking system. MLflow ' s autologging capability means that for common frameworks such as scikit-learn, XGBoost, and PyTorch, parameters and metrics are captured without a single line of custom code, directly answering Fabrikam ' s requirement for consistent experiment tracking.

Microsoft Learn Reference Topic: Track ML experiments with MLflow – Azure Machine Learning MLflow integration

In Azure Machine Learning, a workspace is the top-level resource that acts as a centralized hub for every ML activity. It stores experiments, pipelines, datasets, models, environments, and compute targets all in one place. Before you can register assets or deploy endpoints, the workspace itself must exist and be shared across the team. Fabrikam ' s core challenge is inconsistent experiment tracking, ad-hoc versioning, and no standardized deployment path — all symptoms of the absence of a single, governed workspace. Option A (registering assets) is a subsequent action only possible after the workspace exists. Option C (online endpoint) is a deployment concern, not an asset-management foundation. Option D (Foundry project) is appropriate for generative AI workloads, not for the traditional ML pipeline standardization described here. The workspace is the prerequisite that makes every other governance action possible.

Microsoft Learn Reference Topic: Azure Machine Learning workspaces – Microsoft Learn: Manage Azure Machine Learning workspaces