Databricks-Machine-Learning-Professional Databricks Certified Machine Learning Professional Question and Answers

The Feature Store Client fs provides a method called create_table that can be used to create and populate a Feature Store table from a Spark DataFrame. The create_table method takes the following parameters:

• name: The name of the feature table to create. It must include the database name, such as ‘recommender_system.new_table’.
• primary_keys: The name or list of names of the columns that uniquely identify each row in the feature table. For example, ‘customer_id’ or [‘customer_id’, ‘date’].
• df: The Spark DataFrame that contains the data to populate the feature table. It must have the same schema as the feature table, and include the primary key columns.
• description: An optional string that describes the feature table.
• partition_columns: An optional list of column names to partition the feature table by. For example, [‘date’, ‘region’].
• online_store: An optional boolean flag that indicates whether to publish the feature table to an online store. The default value is False.
• online_store_options: An optional dictionary that specifies the configuration options for the online store, such as table name, database name, and write mode. For example, {‘table_name’: ‘new_table_online’, ‘database_name’: ‘recommender_system’, ‘write_mode’: ‘overwrite’}.

The create_table method creates a Delta table with the specified name, primary keys, partition columns, and description, and writes the data from the DataFrame to the table. If online_store is True, it also publishes the feature table to an online store with the specified options. The create_table method returns a FeatureTable object that represents the feature table12

The other code blocks are incorrect because:

• B. The create_table method requires the df parameter to populate the feature table with data.
• C. The write method of the DataFrame does not have a mode called ‘fs’ or a path parameter. To write a DataFrame to a Delta table, the mode should be one of ‘append’, ‘overwrite’, ‘ignore’, or ‘error’, and the format should be ‘delta’. To write a DataFrame to a Feature Store table, the create_table or write_table method of the Feature Store Client should be used.
• D. The create_table method does not have a function parameter. To use a Python function to compute the features, the function should be called first and the output DataFrame should be passed to the df parameter.
• E. The write method of the DataFrame does not have a mode called ‘feature’ or a path parameter. To write a DataFrame to a Feature Store table, the create_table or write_table method of the Feature Store Client should be used.

References:

• Feature Store Python API reference - Databricks
• Work with features in Workspace Feature Store - Databricks

Z-Ordering is an optimization technique that can speed up the query by colocating similar records while considering values in multiple columns. Z-Ordering is a way of organizing data in storage based on the values of one or more columns. Z-Ordering maps multidimensional data to one dimension while preserving locality of the data points. This means that rows with similar values for the specified columns are stored close together in the same set of files. This improves the performance of queries that filter on those columns, as they can skip over irrelevant files or data blocks. Z-Ordering also enhances data skipping and caching, as it reduces the number of distinct values per file for the chosen columns1. The other options are incorrect because:

• Option B: Bin-packing is an optimization technique that compacts small files into larger ones, but does not colocate similar records based on multiple columns. Bin-packing can improve the performance of queries by reducing the number of files that need to be read, but it does not affect the data layout within the files2.
• Option C: Writing as a Parquet file is not an optimization technique, but a file format choice. Parquet is a columnar storage format that supports efficient compression and encoding schemes. Parquet can improve the performance of queries by reducing the storage footprint and the amount of data transferred, but it does not colocate similar records based on multiple columns3.
• Option D: Data skipping is an optimization technique that skips over files or data blocks that do not match the query predicates, but does not colocate similar records based on multiple columns. Data skipping can improve the performance of queries by avoiding unnecessary data scans, but it depends on the data layout and the metadata collected for each file4.
• Option E: Tuning the file size is an optimization technique that adjusts the size of the data files to a target value, but does not colocate similar records based on multiple columns. Tuning the file size can improve the performance of queries by balancing the trade-off between parallelism and overhead, but it does not affectthe data layout within the files5. References: Z-Ordering (multi-dimensional clustering), Compaction (bin-packing), Parquet, Data skipping, Tuning file sizes

 In a continuous integration, continuous deployment (CI/CD) process for machine learning pipelines, the arrival of a new model version in the MLflow Model Registry is a common event that triggers the execution of automated testing. This is because the MLflow Model Registry is a centralized repository for managing the full lifecycle of MLflow models. It provides model lineage, model versioning, stage transitions, and annotations. When a new model version is registered, it can be automatically tested for quality, performance, and compliance using various tools and frameworks, such as MLflow Projects, MLflow Models, MLflow Tracking, and MLflow Model Serving. The automated testing can help ensure that the new model version meets the desired criteria and standards before it is deployed to production or other stages. The automated testing can also help detect and prevent any errors, bugs, or drifts that may affect the model functionality or accuracy. References:

• MLflow Model Registry Documentation, p. 1-2
• MLOps: Continuous delivery and automation pipelines in machine learning, Continuous integration and delivery (CI/CD) for ML, p. 5-6
• Machine Learning Engineering with MLflow, Testing and Validating ML Models, p. 1-2
• CI/CD for Machine Learning: What it is & Benefits in 2023 - AIMultiple, CI/CD for Machine Learning, p. 1-2

The correct line of code is “MODEL_VERSION_TRANSITIONED_STAGE”, which is the name of the event that occurs when a model version transitions to any stage in the MLflow Model Registry. This event can be used to trigger a webhook that will execute a Databricks Job based on the model version and the stage it transitions to. The other options are either invalid event names or too specific for the task.

References: MLflow Model Registry Webhooks, MLflow Model Registry Events

Real-time deployment is a paradigm that can centrally compute predictions for a single record with exceedingly fast results. Real-time deployment involves exposing a model as a REST API endpoint that can receive requests and return predictions in milliseconds. This paradigm is suitable for scenarios where low latency and high throughput are required, such as fraud detection, recommendation systems, or chatbots. The other deployment paradigms are not as fast or as centralized as real-time deployment. Streaming deployment involves processing data streams in micro-batches, which can introduce some delay. Batch deployment involves processing large batches of data at scheduled intervals, which can take hours or days. Edge or on-device deployment involves deploying models to edge devices, such as mobile phones or IoT sensors, which can reduce the network overhead but also limit the computational power and model size. https://docs.databricks.com/en/machine-learning/ml-tutorials.html

The DESCRIBE HISTORY command can be used to view the commit history of a Delta table, including the schema changes, operations, and timestamps. This command can help identify when a column was dropped from the feature set and by which operation. The other commands are either invalid or do not provide the required information. References:

• Delta Lake - View Commit History
• Databricks Certified Machine Learning Professional Exam Guide - Section 1: Experimentation - Data Management

Feature drift is a change in the distribution of the input data over time, which can affect the performance and accuracy of the model. Monitoring feature drift is important to ensure that the model is still valid and reliable for the current data. One simple and low-cost method of monitoring numeric feature drift is to track the summary statistics trends of the input features, such as mean, standard deviation, minimum, maximum, etc. These statistics can be computed easily and efficiently, and can provide a quick overview of the changes in the data distribution. If the summary statistics deviate significantly from the baseline values, it may indicate a feature drift. However, summary statistics trends may not capture all the nuances of the data distribution, such as outliers, multimodality, skewness, etc. Therefore, other methods, such as statistical tests or visualizations, may be needed to complement the summary statistics trends and provide a more comprehensive analysis of the feature drift123 References:

• Monitoring Feature Drift - Databricks
• Monitor feature attribution skew and drift | Vertex AI | Google Cloud
• Monitoring Model Drift - 6 Different Methodologies - Qualdo

To drop a column from a Delta table, you need to enable column mapping mode for the table and use the ALTER TABLE statement with the DROP COLUMN clause. Alternatively, you can use the spark.read.format(“delta”).table(path) method to read the Delta table as a DataFrame and then use the drop method to drop the column. This will return a new DataFrame with the column removed. You can then overwrite the originalDelta table with the new DataFrame using the write.format(“delta”).mode(“overwrite”) method. The other options are incorrect because:

• Option A: spark.read.format(“delta”).load(path) will not read the Delta table as a table, but as a directory of files. This will not preserve the schema and metadata of the table, and will not allow you to overwrite the table with the new DataFrame.
• Option C: Delta tables can be modified using the Delta Lake APIs or SQL commands. Delta Lake supports schema evolution and column mapping to enable dropping columns from Delta tables.
• Option D: spark.read.table(path) will not work for Delta tables, as it expects a table name, not a path. To read a Delta table by its path, you need to use the spark.read.format(“delta”).load(path) or spark.read.format(“delta”).table(path) methods.
• Option E: spark.sql(“SELECT * EXCEPT star_rating FROM path”) will not drop the column from the Delta table, but only return a new DataFrame with the column excluded. This will not affect the original Delta table, and will not allow you to overwrite the table with the new DataFrame. References: Drop columns, Delta Lake Quickstart, Delta Lake column mapping

To log feature importance data from a CSV file as an artifact of an MLflow run, the correct method is mlflow.log_artifact, which takes the local file path and the artifact path as arguments.The mlflow.log_model method is used to log a model with a specific flavor, such as pyfunc or spark. The mlflow.log_data method does not exist. The mlflow.log_model_and_data method also does not exist, and it is not necessary to log both the model and the data in this case. References:

• MLflow Tracking API
• MLflow Models

To create nested runs in MLflow, you need to specify the nested=True argument in the parent run, and omit it in the child runs. This will set a special tag (mlflow.utils.mlflow_tags.MLFLOW_PARENT_RUN_ID) in the child runs that indicates the parent run ID. This way, the UI will show the child runs under the parent run in a collapsible hierarchy. The indentation of the code blocks is not relevant, as long as the child runs are created within the context of the parent run. The run name parameter is also not relevant, as it does not affect the nesting of the runs. References: Understanding Parent and Child Runs in MLflow, Nested runs using MLflowClient, creating-child-runs-in-mlflow

To replace all data in a feature table with newly computed data, you can use the fs.write_table method with the mode argument set to “overwrite”. This will overwrite the existing data in the feature table with the data in the provided DataFrame. The other options are incorrect because:

• Option A: fs.create_table will create a new feature table, not replace the data in an existing one. If the feature table already exists, it will raise an exception.
• Option B: fs.write_table without specifying the mode argument will use the default mode of “merge”, which will merge the new data with the existing data, not replace it.
• Option C: fs.write table is not a valid method name, as it is missing an underscore. Also, the mode argument of “merge” will not replace the data, but merge it with the existing data.
• Option E: fs.create_table with the mode argument of “merge” is not a valid combination, as the mode argument is only applicable for fs.write_table. Also, the mode argument of “merge” will not replace the data, but merge it with the existing data. References: Databricks FeatureStoreClient, Work with features in Workspace Feature Store

Concept drift is a change in the relationship between input and output data over time, which can affect the performance and accuracy of the model. To monitor concept drift, one needs to compare the predicted values from the model with the observed (actual) label values from the data source. A common way to do this is to compute an evaluation metric, such as accuracy, precision, recall, F1-score, etc., using the observed and predicted values. This metric can indicate how well the model is performing on the current data, and how it differs from the baseline performance on the training data. If the metric shows a significant decline over time, it may suggest that concept drift has occurred and the model needs to be updated123 References:

• Monitoring Concept Drift - Databricks
• Data drift, concept drift, and how to monitor for them - Mona Labs
• 8 Concept Drift Detection Methods To Use With ML Models - Aporia

MLflow Model Serving is a feature that allows users to host MLflow models as REST endpoints on Databricks1. To query a model version that is in the Production stage of the MLflow Model Registry, the user can use the following model URI format: https:// /model-serving///invocations2. In this case, the model name is recommender and the stage is Production, so the correct model URI is https:// /model-serving/recommender/Production/invocations. The other options are incorrect because they either use the wrong prefix (model instead of model-serving), the wrong stage name (stage-production instead of Production), or require the version number of the model version in Production, which is not necessary for querying the model by stage. References: Announcing MLflow Model Serving on Databricks, [MLflow Model Serving], [MLflow Model Registry]

Containers are a type of software packaging that can assist in real-time deployments by packaging software with its own application, tools, and libraries. Containers are isolated and portable environments that run on a shared operating system, but have their ownfile system, dependencies, configuration, and resources. Containers can help developers and operators to build, test, and deploy software faster, easier, and more reliably, as they ensure that the software runs the same way in any environment, whether it is a developer’s laptop, a test server, or a production cluster. Containers can also help to achieve scalability, high availability, and load balancing, as they can be easily created, destroyed, replicated, and orchestrated using tools such as Kubernetes, Docker Swarm, or Amazon ECS. Containers can also integrate with other tools and services, such as MLflow, Databricks, AWS CodeDeploy, and Google Cloud Deployment Manager, to enable end-to-end machine learning pipelines with tracking, reproducibility, and governance1234 References:

• How to Package Your Application’s Infrastructure with Docker
• 21 open source DevOps tools (and what they do)
• Microservices [What is it & Top Benefits]
• 9 Best Software Deployment Tools for 2023 (Paid & Free)

The Databricks REST APIs allow users to programmatically access Databricks features and services, such as clusters, jobs, workspaces, libraries, secrets, and more. The Jobs API enables users to create, edit, delete, run, and list jobs, as well as manage job runs. Users can use the Jobs API to programmatically create a new Databricks job whose schedule depends on the result of some automated tests in a machine learning pipeline. For example, users can use the create endpoint to create a new job with a cron_schedule parameter that specifies the cron expression for the job schedule. Users can also use the run_now endpoint to run an existing job with a run_name parameter that specifies the name of the run. Users can use the runs/get endpoint to get the run result state of a job run, and use conditional logic to trigger another job run based on the result. Users can also use the runs/list endpoint to list all the runs of a job, and use the runs/cancel endpoint to cancel a job run if needed. References:

• Databricks REST API — Databricks Documentation, p. 1
• Jobs API — Databricks Documentation, p. 1-2
• Create and run Databricks Jobs | Databricks on AWS, Create and run jobs using the CLI, API, or notebooks, p. 1