Databricks-Certified-Data-Engineer-Associate Databricks Certified Data Engineer Associate Exam Question and Answers

Pipeline A has a driver-side failure because collect() transfers every row to the driver process. Databricks states that collect() should be used only when the result is expected to be small. For a 100 GB dataset, the preferred correction is to remove the full collection and use distributed processing, aggregation, sampling, or limit; increasing driver capacity alone is not a durable solution. Pipeline B is an executor-side problem during a shuffle-intensive join. Increasing spark.sql.shuffle.partitions can divide the shuffled data into smaller task partitions, while workers with more memory per core can provide additional executor memory. Increasing the number of executors does not solve Pipeline A’s driver bottleneck, and increasing driver memory does not address Pipeline B’s executor failures. Therefore, option D correctly distinguishes the affected Spark components and appropriate remedies.

Lakeflow Jobs provides task-level Run if conditions such as All succeeded, All done, At least one failed, and All failed. No single Run if condition directly represents “A succeeded and B failed” when Task C depends directly on both tasks. The required outcome can be modeled with two guarded dependency branches: one gateway proceeds only after Task A succeeds, and another proceeds only after Task B fails. Task C then uses All succeeded against those two successful gateway tasks. A negative dependency is not a supported dependency type. Task values pass arbitrary application values and do not automatically represent task outcomes. Notebook widgets are task parameters rather than dependency-state conditions. Therefore, the two-chain configuration described in option B correctly represents the required conjunction.

================

The best practice is to use " Complete " as output mode instead of " append " when working with aggregated tables. Since gold layer is work final aggregated tables, the only option with output mode as complete is option E.

Option A is the supported migration path for bringing existing workspace resources under Declarative Automation Bundle management. The bundle generate command creates bundle configuration for supported existing resources, including jobs, pipelines, and dashboards, and can download referenced artifacts where applicable. The bundle deployment bind command then associates the bundle resource key with the existing workspace resource so that future deployments update that resource rather than creating a duplicate. The completed bundle project can be committed to Git and promoted through CI/CD. Exporting REST API JSON does not create a complete bundle workflow, manually recreating every resource is unnecessary, and bundle init creates a project from a template but does not automatically discover existing resources. Therefore, option A provides the intended generation and binding process.

================

Option D is the correct answer because it enables the Serverless feature for the SQL endpoint, which allows the endpoint to automatically scale up and down based on the query load. This way, the endpoint can handle more concurrent queries and reduce the time it takes to return results. The Serverless feature also reduces the cold start time of the endpoint, which is the time it takes to start the cluster when a query is submitted to a non-running endpoint. The Serverless feature is available for both AWS and Azure Databricks platforms.

 Databricks SQL Serverless, Serverless SQL endpoints, New Performance Improvements in Databricks SQL

The Continuous Pipeline Mode for Delta Live Tables allows the pipeline to run continuously and process data as it arrives. This mode is suitable for streaming ingest and CDC workloads that require low-latency updates. The Development mode for Delta Live Tables allows the pipeline to run on a dedicated cluster that is not shared with other pipelines. This mode is useful for testing and debugging the pipeline logic before deploying it to production. Therefore, the correct answer is B, because the pipeline will run continuously on a dedicated cluster until it is manually stopped, and the compute resources will be released only after the pipeline is shut down. References: Databricks Documentation - Configure pipeline settings for Delta Live Tables, Databricks Documentation - Continuous vs. triggered pipeline execution, Databricks Documentation - Development vs. production mode.

 A data lakehouse is a new paradigm that can be used to simplify and unify siloed data architectures that are specialized for specific use cases. A data lakehouse combines the best of both data lakes and data warehouses, providing a single platform that supports diverse data types, open standards, low-cost storage, high-performance queries, ACID transactions, schema enforcement, and governance. A data lakehouse enables data engineers to build reliable and scalable data pipelines that can serve various downstream applications and users, such as data science, machine learning, analytics, and reporting. A data lakehouse leverages the power of Delta Lake, a storage layer that brings reliability and performance to data lakes. References: What is a data lakehouse?, Delta Lake, Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics

Databricks Asset Bundles use YAML configuration, and Databricks documentation specifies that bundle configuration is expressed in a root databricks.yml file. The resources mapping is used to define supported objects such as jobs , and job definitions include keys like name and tasks. Tasks in a job are defined under the tasks key, and a notebook task is represented with notebook_task and notebook_path. Option A is the only answer written in the correct general YAML bundle structure and matches the documented style of defining bundle resources. Options B and C use JSON- and Python-style syntax rather than bundle YAML. Option D uses Terraform/HCL-style syntax, which is not the Databricks Asset Bundle format. While a full production bundle often includes more top-level settings such as bundle, targets, or workspace, the question asks which format is valid, and only option A matches the documented Databricks Asset Bundles configuration model.

=========

he approach that can be used to identify the owner of new_table is to review the Owner field in the table’s page in Data Explorer. Data Explorer is a web-based interface that allows users to browse, create, and manage data objects such as tables, views, and functions in Databricks1. The table’s page in Data Explorer provides various information about the table, such as its schema, partitions, statistics, history, and permissions2. The Owner field shows the name and email address of the user who created or owns the table3. The data engineer can use this information to contact the table owner and request for permission to access the table.

The other options are not correct or reliable for identifying the owner of new_table. Reviewing the Permissions tab in the table’s page in Data Explorer can show the users and groups who have access to the table, but not necessarily the owner4. Reviewing the Owner field in the table’s page in the cloud storage solution can be misleading, as the owner of the data files may not be the same as the owner of the table5. There is a way to identify the owner of the table, as explained above, so option E is false.

1: Data Explorer | Databricks on AWS

2: Table details | Databricks on AWS

3: Set owner when creating a view in databricks sql - Databricks - 9978

4: Table access control | Databricks on AWS

5: External tables | Databricks on AWS

A left anti-join returns rows from the left-side relation for which no matching row exists in the right-side relation. By placing customers on the left and orders on the right and joining by customer_id, the result contains precisely the customers who have no corresponding order. A left anti-join also returns only columns from the left relation, which satisfies the requirement to exclude all order-related fields. A left outer join could produce the desired result only with an additional NULL filter and correct table order. An inner join returns customers who have placed orders, which is the opposite requirement. A full outer join can introduce order-only rows and unnecessary columns. Therefore, the left anti-join described in option C is the most exact and efficient expression for this requirement.

================

A table is a data entity that is stored in a physical location and can be accessed by other data engineers in other sessions. A table can be created from one or more tables using the CREATE TABLE or CREATE TABLE AS SELECT commands. A table can also be registered from an existing DataFrame using the spark.catalog.createTable method. A table can be queried using SQL or DataFrame APIs. A table can also be updated, deleted, or appended using the MERGE INTO command or the DeltaTable API. References:

Create a table

Create a table from a query result

Register a table from a DataFrame

[Query a table]

[Update, delete, or merge into a table]

In the Medallion Architecture promoted by Databricks, the Bronze layer represents the raw ingestion layer where data is stored in its original form with minimal transformation. The primary goal is to ensure data fidelity and traceability , not to apply business logic or heavy transformations. Typical operations in the Bronze layer include schema inference (or enforcement), ingestion of raw data, and the addition of metadata columns such as ingestion timestamp, load date/time, and process identifiers. These metadata fields help track lineage, enable debugging, and support incremental processing downstream. Extensive data cleansing (A), aggregations (B), and business transformations (C) are intentionally deferred to the Silver and Gold layers , where data is refined and structured for analytics. By keeping Bronze data close to the source and append-only, organizations maintain a reliable audit trail and can reprocess data if needed. Therefore, adding operational metadata like load timestamps and process IDs is the correct characteristic of Bronze layer transformations.

=========

Lakeflow Connect managed connectors are designed to simplify ingestion from supported enterprise applications and databases. They provide managed authentication, incremental processing, schema handling, and integration with Unity Catalog. Salesforce is supported as a managed application source, while SQL Server ingestion can accommodate database connectivity patterns, including privately connected or on-premises environments when the required networking is configured. Auto Loader is the appropriate service for incrementally discovering and processing new JSON files arriving in ADLS. It supports schema inference and evolution and can be used within Lakeflow Spark Declarative Pipelines. Option B would require substantial custom API and CDC code. Option D incorrectly treats cloud files as the same connector use case as Salesforce and SQL Server. Therefore, the combination in option C minimizes maintenance while matching each source with its appropriate ingestion mechanism.

================

Delta Sharing is an open protocol developed by Databricks that enables secure, real-time sharing of data across organizations without requiring recipients to have a Databricks account. It is specifically designed for scenarios where external partners need read-only access to data stored in Delta format. With Delta Sharing, data providers create a share and grant access to recipients via a secure mechanism (such as a profile file or URL-based access), ensuring that the data cannot be modified by the consumer. This approach eliminates the need for data duplication or manual file transfers, maintaining data freshness and governance. Option B is incorrect because Unity Catalog governs internal access within an organization, not external sharing without accounts. Option C is inefficient and breaks data lineage and freshness. Option D violates security best practices by granting excessive permissions. Delta Sharing ensures fine- grained access control, auditability, and secure external collaboration , making it the correct solution for this requirement.

=========

To minimize the total running time of the SQL endpoint used in the refresh schedule of a dashboard in Databricks, the most effective approach is to utilize the Auto Stop feature. This feature allows the SQL endpoint to automatically stop after a period of inactivity, ensuring that it only runs when necessary, such as during the dashboard refresh or when actively queried. This minimizes resource usage and associated costs by ensuring the SQL endpoint is not running idle outside of these operations.

Databricks Repos is a visual Git client and API in Databricks that supports common Git operations such as commit, pull, push, branch management, and visual comparison of diffs when committing1. However, merge is not supported in the Git dialog2. You need to use the Repos UI or your Git provider to merge branches3. Merge is a way to combine the commit history from one branch into another branch1. During a merge, a merge conflict is encountered when Git cannot automatically combine code from one branch into another. Merge conflicts require manual resolution before a merge can be completed1. References: 4: Run Git operations on Databricks Repos4, 1: CI/CD techniques with Git and Databricks Repos1, 3: Collaborate in Repos3, 2: Databricks Repos - What it is and how we can use it2.

Databricks Repos is a visual Git client and API in Databricks that supports common Git operations such as commit, pull, push, merge, and branch management. However, to clone a remote Git repository to a Databricks repo, you must use the Databricks UI or API. You cannot clone a Git repo using the CLI through a cluster’s web terminal, as the files won’t display in the Databricks UI1. References: 1: Run Git operations on Databricks Repos | Databricks on AWS2

Delta Lake provides schema enforcement on write, preventing incompatible data from being silently added to a table. When schema evolution is explicitly enabled through the appropriate write or ingestion configuration, new optional columns can be added without dropping and recreating the existing table. Delta Lake also maintains a transaction log containing committed table versions, enabling table-history inspection and time-travel queries for debugging or auditing, subject to retention settings. A plain Parquet table does not provide Delta’s transactional history or equivalent write enforcement. Recreating a Delta table whenever columns change is unnecessary and can disrupt its history. Auto Loader can detect evolving source schemas, but an external table with schema inference alone does not provide the complete set of required Delta capabilities. Therefore, option D is correct.

================

In a Databricks customer ' s cloud account, the primary elements stored include:

Data: This is the central type of content stored in the customer ' s cloud account. Data might include various datasets, tables, and files that are used and managed through Databricks platforms.

Notebooks: These are also stored within a customer ' s cloud account. Notebooks include scripts, notes, and other information necessary for data analysis and processing tasks.

Cluster management metadata is indeed managed through the cloud, but it ' s primarily handled by Databricks rather than stored directly in the customer ' s account. The Databricks web application itself is not stored within the customer ' s cloud account; rather, it ' s a service provided by Databricks.

Option A uses the supported ALTER TABLE ... SET MANAGED conversion for a Unity Catalog external Delta table. Databricks documents that this conversion retains the table name, permissions, dependent objects, settings, and table history while minimizing downtime for readers and writers. After conversion, Databricks manages the table’s storage lifecycle, satisfying the requirement for platform-controlled cleanup when the managed table is dropped. Dropping the original table with PURGE destroys the object instead of migrating it. SET LOCATION changes the storage path referenced by an external table but does not convert the table into a managed table. Creating a replacement table with CREATE TABLE AS SELECT introduces a new object and requires manual migration of permissions, dependencies, and workloads. Therefore, ALTER TABLE catalog.schema.sales SET MANAGED is the correct action.

================

JSON data is a text-based format that represents data as a collection of name-value pairs. By default, when Auto Loader infers the schema of JSON data, it treats all columns as strings. This is because JSON data can have varying data types for the same column across different files or records, and Auto Loader does not attempt to reconcile these differences. For example, a column named “age” may have integer values in some files, but string values in others. To avoid data loss or errors, Auto Loader infers the column as a string type. However, Auto Loader also provides an option to infer more precise column types based on the sample data. This option is called cloudFiles.inferColumnTypes and it can be set to true or false. When set to true, Auto Loader tries to infer the exact data types of the columns, such as integers, floats, booleans, or nested structures. When set to false, Auto Loader infers all columns as strings. The default value of this option is false. References: Configure schema inference and evolution in Auto Loader, Schema inference with auto loader (non-DLT and DLT), Using and Abusing Auto Loader’s Inferred Schema, Explicit path to data or a defined schema required for Auto loader.

COPY INTO is idempotent by default and tracks files that have already been loaded into the target table. A corrected file uploaded under the same name and path can therefore be treated as previously processed and skipped. The reliable pattern is to publish the correction under a new unique filename so that ingestion detects it as a new file. The pipeline can then use a business key with MERGE, UPDATE, or equivalent transformation logic to apply the corrected values to the target Delta table. Overwriting or recreating the complete table risks unnecessary data loss, increases processing cost, and disrupts downstream consumers. Reloading every historical source file is also inefficient and can introduce duplicates. Therefore, option B supports incremental ingestion while applying corrections safely and efficiently.

================

 To set up the new task to run a new notebook prior to the original task in a single-task Job, the data engineer can use the following approach: In the existing Job, create a new task that corresponds to the new notebook that needs to be run. Set up the new task with the appropriate configuration, specifying the notebook to be executed and any necessary parameters or dependencies. Once the new task is created, designate it as a dependency of the original task in the Job configuration. This ensures that the new task is executed before the original task.

Auto Loader in Databricks utilizes Spark Structured Streaming for processing data incrementally. This allows Auto Loader to efficiently ingest streaming or batch data at scale and to recognize new data as it arrives in cloud storage. Spark Structured Streaming provides the underlying engine that supports various incremental data loading capabilities like schema inference and file notification mode, which are crucial for the dynamic nature of data lakes.

A job compute cluster triggered by a scheduled workflow is appropriate for predictable nightly ETL. Job compute is designed for non-interactive workloads and is created for the job run, then terminated automatically when the run finishes. It can be configured with autoscaling so that worker capacity adjusts to the workload, while the organization pays for compute only during execution rather than maintaining an always-running cluster. An all-purpose cluster is optimized for interactive development and can incur idle costs. A single-node cluster cannot scale across workers and is unsuitable for large-volume processing. A serverless SQL warehouse is intended for SQL workloads, whereas the scenario describes a general Delta Lake ETL pipeline. Consequently, option D provides the required scheduling, scaling, reliable isolation, and cost efficiency.

================

In Databricks, when creating a table, you can add a comment to columns or the entire table to provide more information about the data it contains. In this case, since it’s organization policy to indicate that the new table includes personally identifiable information (PII), option D is correct. The line of code would be added after defining the table structure and before closing with a semicolon. References: Data Engineer Associate Exam Guide, CREATE TABLE USING (Databricks SQL)

https://www.w3schools.com/python/python_functions.asp

https://www.geeksforgeeks.org/python-functions/

Delta Lake tables use the Parquet format as their underlying storage format. Delta Lake enhances Parquet by adding a transaction log that keeps track of all the operations performed on the table. This allows features like ACID transactions, scalable metadata handling, and schema enforcement, making it an ideal choice for big data processing and management in environments like Databricks.

Option C explicitly selects every required leaf field by using dot notation for nested structs. The resulting DataFrame retains event_id and event_ts while projecting device.id, device.model, device.location.latitude, and device.location.longitude as output columns. Option B expands only the immediate fields of device; the location value remains a nested struct, so the data is not fully flattened. Option A creates separate columns for the device identifier and model but retains the complete location struct instead of extracting latitude and longitude. Option D places bracket-like expressions inside column-name strings, which do not represent the required nested struct paths. Spark DataFrames support selecting nested fields through qualified column paths, making option C the direct transformation for creating analytics-friendly root-level fields.

================

Databricks Asset Bundles are a Databricks CLI feature that lets teams define Databricks projects “as code” using YAML configuration files . In a bundle, metadata is expressed in YAML (for example, a required databricks.yml at the root), and the configuration can declare resources (such as Databricks Jobs/workflows and other supported assets), artifacts , and deployment targets . After defining these YAML manifests, teams use the Databricks CLI to validate, deploy, and run the bundle into a Databricks workspace, which matches the scenario described (YAML + CLI deployment). This approach standardizes how workflows and related resources are packaged and promoted across environments (dev/test/prod) while keeping definitions version-controlled and reproducible. GitHub is only a hosting/version-control platform (not the YAML deployment feature itself), Terraform is a separate IaC tool, and “DataOps” is a methodology rather than a specific Databricks feature. ( Databricks Documentation )

With schema evolution enabled, Auto Loader detects source fields that are not present in the current schema and evolves the destination schema to include them. The new userAgent field is therefore added to the Delta table as a column, and values from newly processed records are written to it. Existing rows cannot be retroactively populated because the earlier source records did not contain that field, so they receive NULL for the newly added column. The column is consequently nullable unless a later transformation explicitly enforces another constraint. Option A describes behavior without suitable schema evolution. Option B incorrectly claims that historical rows receive a default value and that the column becomes non-nullable. Option D contradicts schema-evolution behavior. Therefore, option C correctly describes how the new JSON field is handled.

A Gold table is a table that contains highly refined and aggregated data that powers analytics, machine learning, and production applications. It represents data that has been transformed into knowledge, rather than just information. A Gold table is typically the final output of a medallion lakehouse architecture, where data flows from Bronze to Silver to Gold tables, with each layer improving the structure and quality of data. A job that queries aggregated data designed to feed into a dashboard is an example of a data workload that will utilize a Gold table as its source, as it requires data that is ready for consumption and analysis. The other options are either data workloads that will use a Bronze or Silver table as their source, or data workloads that will produce a Gold table as their output. References: Databricks Documentation - What is the medallion lakehouse architecture?, Databricks Documentation - What is a Medallion Architecture?, K21Academy - Delta Lake Architecture & Azure Databricks Workspace.

DataFrame.dropna() removes rows containing NULL or NaN values. Its default how= " any " behavior removes a row when any column included in the specified subset is NULL. By passing [ " customer_email " , " customer_phone " ] as the subset, the operation removes a record when either required field is missing, while ignoring NULL values in unrelated columns. Option B can produce the same filtering result, but it uses two chained where() calls and therefore does not meet the single-method-call requirement. Option C uses how= " all " and would remove a row only when all evaluated columns are NULL. Option D contains one filter() operation but requires multiple column-function calls. Consequently, option A is the direct PySpark method designed for this requirement.

================

The SQL keyword that can be used to convert a table from a long format to a wide format is PIVOT. The PIVOT clause is used to rotate the rows of a table into columns of a new table1. The PIVOT clause can aggregate the values of a column based on the distinct values of another column, and use those values as the column names of the new table1. The PIVOT clause can be useful for transforming data from a long format, where each row represents an observation with multiple attributes, to a wide format, where each row represents an observation with a single attribute and multiple values2. For example, the PIVOT clause can be used to convert a table that contains the sales of different products by different regions into a table that contains the sales of each product by each region as separate columns1.

The other options are not suitable for converting a table from a long format to a wide format. CONVERT is a function that can be used to change the data type of an expression3. WHERE is a clause that can be used to filter the rows of a table based on a condition4. TRANSFORM is a keyword that can be used to apply a user-defined function to a group of rows in a table5. SUM is a function that can be used to calculate the total of a numeric column.

1: PIVOT | Databricks on AWS

2: Reshaping Data - Long vs Wide Format | Databricks on AWS

3: CONVERT | Databricks on AWS

4: WHERE | Databricks on AWS

5: TRANSFORM | Databricks on AWS

[SUM | Databricks on AWS]

The reason why all of the data files and metadata files were deleted from the file system after dropping the table is that the table was managed. A managed table is a table that is created and managed by Spark SQL. It stores both the data and the metadata in the default location specified by the spark.sql.warehouse.dir configuration property. When a managed table is dropped, both the data and the metadata are deleted from the file system.

Option B is not correct, as the size of the table’s data does not affect the behavior of dropping the table. Whether the table’s data is smaller or larger than 10 GB, the data files and metadata files will be deleted if the table is managed, and will be preserved if the table is external.

Option C is not correct, for the same reason as option B.

Option D is not correct, as an external table is a table that is created and managed by the user. It stores the data in a user-specified location, and only stores the metadata in the Spark SQL catalog. When an external table is dropped, only the metadata is deleted from the catalog, but the data files are preserved in the file system.

Option E is not correct, as a table must have a location to store the data. If the location is not specified by the user, it will use the default location for managed tables. Therefore, a table without a location is a managed table, and dropping it will delete both the data and the metadata.

Managing Tables

[Databricks Data Engineer Professional Exam Guide]

In Python, the == operator is used to compare the values of two variables, while the = operator is used to assign a value to a variable. Therefore, option A and E are incorrect, as they use the = operator for comparison. Option B and C are also incorrect, as they compare the review_period variable to a string value  " True " , which is different from the boolean value True. Option D is the correct answer, as it uses the == operator to compare the day_of_week variable to the integer value 1, and the and operator to check if both conditions are true. If both conditions are true, then the final block of the Python program will be executed. References: [Python Operators], [Python If … Else]

 The VACUUM command is used to remove files that are no longer referenced by a Delta table and are older than the retention threshold1. The default retention period is 7 days2, but it can be changed by setting the delta.logRetentionDuration and delta.deletedFileRetentionDuration configurations3. If the VACUUM command was run on the table with a retention period shorter than 3 days, then the data files that were needed to restore the table to a 3-day-old version would have been deleted. The other commands do not delete data files from the table. The TIME TRAVEL command is used to query a historical version of the table4. The DELETE HISTORY command is not a valid command in Delta Lake. The OPTIMIZE command is used to improve the performance of the table by compacting small files into larger ones5. The HISTORY command is used to retrieve information about the operations performed on the table. References: 1: VACUUM | Databricks on AWS 2: Work with Delta Lake table history | Databricks on AWS 3: [Delta Lake configuration | Databricks on AWS]  4: Work with Delta Lake table history - Azure Databricks 5: [OPTIMIZE | Databricks on AWS] : [HISTORY | Databricks on AWS]

The query performing a streaming hop from raw data to a Bronze table is identified by using the Spark streaming read capability and then writing to a Bronze table. Let ' s analyze the options:

Option A: Utilizes .writeStream but performs a complete aggregation which is more characteristic of a roll-up into a summarized table rather than a hop into a Bronze table.

Option B: Also uses .writeStream but calculates an average, which again does not typically represent the raw to Bronze transformation, which usually involves minimal transformations.

Option C: This uses a basic .write with .mode( " append " ) which is not a streaming operation, and hence not suitable for real-time streaming data transformation to a Bronze table.

Option D: It employs spark.readStream.load() to ingest raw data as a stream and then writes it out with .writeStream, which is a typical pattern for streaming data into a Bronze table where raw data is captured in real-time and minimal transformation is applied. This approach aligns with the concept of a Bronze table in a modern data architecture, where raw data is ingested continuously and stored in a more accessible format.

 The keyword that can be used to compact the small files associated with a Delta table is OPTIMIZE. The OPTIMIZE command performs file compaction on a Delta table by rewriting a set of small files into a set of larger files1. This can improve the performance of queries that scan the table by reducing the number of files that need to be read and the amount of metadata that needs to be processed1. The OPTIMIZE command can also optionally sort the data within each file by a given set of columns, which can further improve the query performance by enabling data skipping and predicate pushdown1. The OPTIMIZE command can be applied to the whole table or to a specific partition of the table1.

The other keywords are not suitable for compacting the small files associated with a Delta table. REDUCE is a keyword used in the SQL syntax for aggregating data using a user-defined function2. COMPACTION is not a valid keyword in SQL or Python. REPARTITION is a keyword used in the Python syntax for changing the number of partitions of a DataFrame or an RDD3. VACUUM is a keyword used to remove files that are no longer referenced by a Delta table and are older than a retention threshold4.

1: OPTIMIZE | Databricks on AWS

2: REDUCE | Databricks on AWS

3: repartition | Databricks on AWS

4: VACUUM | Databricks on AWS

To include a property indicating that a table contains personally identifiable information (PII), the TBLPROPERTIES keyword is used in SQL to add metadata to a table. The correct syntax to define a table property for PII is as follows:

CREATE TABLE customersInFrance

USING DELTA

TBLPROPERTIES ( ' PII ' = ' true ' )

AS

SELECT id,

firstName,

lastName

FROM customerLocations

WHERE country = ' FRANCE ' ;

The TBLPROPERTIES ( ' PII ' = ' true ' ) line correctly sets a table property that tags the table as containing personally identifiable information. This is in accordance with organizational policies for handling sensitive information.

In Unity Catalog, an external table is defined by referencing data that already exists in cloud storage, and Databricks documentation states that this is done by using a LOCATION clause that points to the storage path. Databricks also notes that the external table’s path should be contained within an existing external location that has been configured with the appropriate storage credential and access permissions. This makes option A the correct choice. Option B is incorrect because simply calling something “unmanaged” without specifying a location does not create a proper Unity Catalog external table. Option C is also incorrect because Databricks documentation explicitly requires a storage location for Unity Catalog external tables. Option D is wrong because managed tables are intended for Databricks-managed storage and lifecycle handling, not for referencing existing external files. So, for ADLS-based data that must remain in place while still being queryable through Databricks, the right pattern is to configure the external location and then create the table with a CREATE TABLE ... LOCATION statement.

=========

spark.sql.autoBroadcastJoinThreshold controls the estimated table-size limit under which Spark can automatically select a broadcast join. Increasing it to a value above the lookup table’s estimated size allows Spark to copy that small relation to each executor. The large fact table can then be processed locally without being fully shuffled across the cluster, often reducing network transfer and join duration. The threshold should remain within executor-memory capacity because broadcasting an excessively large relation can cause memory pressure. spark.default.parallelism controls default task parallelism, while spark.sql.shuffle.partitions controls the number of partitions used by shuffle operations. spark.executor.memory changes executor capacity but does not instruct the optimizer to select a broadcast join. Therefore, option A is the relevant configuration.

================

The correct command to grant the necessary permission on the entire database to the new team is to use the GRANT USAGE command. The GRANT USAGE command grants the principal the ability to access the securable object, such as a database, schema, or table. In this case, the securable object is the database customers, and the principal is the group team. By granting usage on the database, the team will be able to see what tables already exist in the database. Option E is the only option that uses the correct syntax and the correct privilege type for this scenario. Option A uses the wrong privilege type (VIEW) and the wrong securable object (CATALOG). Option B uses the wrong privilege type (CREATE), which would allow the team to create new tables in the database, but not necessarily see the existing ones. Option C uses the wrong securable object (CATALOG) and the wrong principal (customers). Option D uses the wrong securable object (team) and the wrong principal (customers). References: GRANT, Privilege types, Securable objects, Principals

Structured Streaming provides native Kafka integration and processes incoming records in repeated micro-batches, making it suitable for the required sub-minute clickstream latency. For the customer data, an incremental Lakeflow Connect ingestion pattern tracks source progress and processes only new or changed records during subsequent executions. This avoids repeatedly transferring and processing unchanged Snowflake rows. Option B performs a complete extraction every four hours, so the later MERGE does not eliminate unnecessary source reading and network transfer. Option C relies on a time filter that may miss late changes, updates without a suitable timestamp, or records affected by clock boundaries. Option D explicitly performs full refreshes and therefore violates the requirement to prevent reprocessing unchanged customer data. Consequently, option A is the only approach that satisfies both the streaming-latency and incremental-ingestion requirements.

================

The --target prod option explicitly selects the prod target from the bundle configuration. Databricks then resolves ${var.dataset_catalog} using the target-level value catalog_prod, which takes precedence over the top-level default of catalog_dev. This allows the same dashboard resource definition to be promoted across environments without editing the resource itself. The --var option can override an individual variable, but it does not automatically select the production target. The --profile option selects an authentication profile from .databrickscfg; it does not select a bundle target or automatically apply the target’s variables. bundle execute is not the deployment command. Consequently, option D correctly combines production-target selection with the environment-specific variable override already defined in the YAML configuration.

================

Option A enables predictive optimization, which is designed to eliminate manual maintenance scheduling for Unity Catalog managed tables. Databricks evaluates table activity and automatically runs operations such as OPTIMIZE, VACUUM, and ANALYZE when beneficial, using serverless jobs compute. Predictive optimization can be enabled at multiple hierarchy levels, allowing tables to inherit the applicable setting. Auto Loader schema evolution handles changes in incoming file schemas but does not schedule table optimization. Liquid clustering improves data organization and query pruning, but it does not independently provide automated stale-file removal. Optimized writes and auto compaction reduce the creation of small files during write operations, but they do not replace adaptive VACUUM scheduling and the broader maintenance capabilities of predictive optimization. Therefore, option A directly satisfies the requirement for platform-managed table maintenance.

================

Reading a Unity Catalog table requires privileges at all relevant levels of its three-part namespace. The group needs USE CATALOG on corp_marketing, USE SCHEMA on campaigns, and SELECT on email_stats. These privileges allow users to resolve the catalog and schema and read the table without granting permission to create or modify objects. Option A omits USE CATALOG, so table access is incomplete. Option B unnecessarily grants CREATE SCHEMA, violating the requirement that the group must not create objects. Option D omits USE CATALOG and grants MODIFY, which would permit data changes. Unity Catalog follows a hierarchical permissions model, but possession of SELECT alone is insufficient without the required usage privileges on the parent containers. Therefore, the least-privilege sequence in option C is correct.

================