Databricks-Generative-AI-Engineer-Associate Databricks Certified Generative AI Engineer Associate Question and Answers

Problem Context : After switching to a model with a shorter context length, the error message indicating that the prompt token count has exceeded the limit suggests that the input to the model is too large.

Explanation of Options :

Option A: Use a smaller embedding model to generate – This wouldn ' t necessarily address the issue of prompt size exceeding the model’s token limit.

Option B: Reduce the maximum output tokens of the new model – This option affects the output length, not the size of the input being too large.

Option C: Decrease the chunk size of embedded documents – This would help reduce the size of each document chunk fed into the model, ensuring that the input remains within the model ' s context length limitations.

Option D: Reduce the number of records retrieved from the vector database – By retrieving fewer records, the total input size to the model can be managed more effectively, keeping it within the allowable token limits.

Option E: Retrain the response generating model using ALiBi – Retraining the model is contrary to the stipulation not to change the response generating model.

Options C and D are the most effective solutions to manage the model’s shorter context length without changing the model itself, by adjusting the input size both in terms of individual document size and total documents retrieved.

In this scenario, the chatbot must avoid answering political questions and instead provide a standard message for such inquiries. Implementing a Safety Guardrail is the appropriate solution for this:

What is a Safety Guardrail? Safety guardrails are mechanisms implemented in Generative AI systems to ensure the model behaves within specific bounds. In this case, it ensures the chatbot does not answer politically sensitive or irrelevant questions, which aligns with the business rules.

Preventing Responses to Political Questions : The Safety Guardrail is programmed to detect specific types of inquiries (like political questions) and prevent the model from generating responses outside its intended domain. When such queries are detected, the guardrail intervenes and provides a pre-defined response: “Sorry, I cannot answer that. I am a chatbot that can only answer questions around insurance.”

How It Works in Practice : The LLM system can include a classification layer or trigger rules based on specific keywords related to politics. When such terms are detected, the Safety Guardrail blocks the normal generation flow and responds with the fixed message.

Why Other Options Are Less Suitable :

B (Security Guardrail) : This is more focused on protecting the system from security vulnerabilities or data breaches, not controlling the conversational focus.

C (Contextual Guardrail) : While context guardrails can limit responses based on context, safety guardrails are specifically about ensuring the chatbot stays within a safe conversational scope.

D (Compliance Guardrail) : Compliance guardrails are often related to legal and regulatory adherence, which is not directly relevant here.

Therefore, a Safety Guardrail is the right framework to ensure the chatbot only answers insurance-related queries and avoids political discussions.

Problem Context : The engineer needs a cost-effective deployment strategy for an LLM application with relatively low request volume.

Explanation of Options :

Option A : Switching to external models may not provide the required control or integration necessary for specific application needs.

Option B : Using a pay-per-token model is cost-effective, especially for applications with variable or low request volumes, as it aligns costs directly with usage.

Option C : Changing to a model with fewer parameters could reduce costs, but might also impact the performance and capabilities of the application.

Option D : Manually throttling requests is a less efficient and potentially error-prone strategy for managing costs.

Option B is ideal, offering flexibility and cost control, aligning expenses directly with the application ' s usage patterns.

ï‚· Problem Context : The goal is to monitor the serving endpoint for incoming requests and outgoing responses in a provisioned throughput model serving endpoint within a Retrieval-Augmented Generation (RAG) application . The current approach involves using a microservice to log requests and responses to a remote server, but the Generative AI Engineer is looking for a more streamlined solution within Databricks.

ï‚· Explanation of Options :

Option A: Vector Search : This feature is used to perform similarity searches within vector databases. It doesn’t provide functionality for logging or monitoring requests and responses in a serving endpoint, so it’s not applicable here.

Option B: Lakeview : Lakeview is not a feature relevant to monitoring or logging request-response cycles for serving endpoints. It might be more related to viewing data in Databricks Lakehouse but doesn’t fulfill the specific monitoring requirement.

Option C: DBSQL : Databricks SQL (DBSQL) is used for running SQL queries on data stored in Databricks, primarily for analytics purposes. It doesn’t provide the direct functionality needed to monitor requests and responses in real-time for an inference endpoint.

Option D: Inference Tables : This is the correct answer. Inference Tables in Databricks are designed to store the results and metadata of inference runs. This allows the system to log incoming requests and outgoing responses directly within Databricks, making it an ideal choice for monitoring the behavior of a provisioned serving endpoint. Inference Tables can be queried and analyzed, enabling easier monitoring and debugging compared to a custom microservice.

Thus, Inference Tables are the optimal feature for monitoring request and response logs within the Databricks infrastructure for a model serving endpoint.

ï‚· Context : Deploying an application that uses an MLflow Pyfunc model involves managing sensitive information such as secrets and credentials securely.

ï‚· Explanation of Options :

Option A: Use spark.conf.set() : While this method can pass configurations within Spark jobs, using it for secrets is not recommended because it may expose them in logs or Spark UI.

Option B: Pass variables using the Databricks Feature Store API : The Feature Store API is designed for managing features for machine learning, not for handling secrets or credentials.

Option C: Add credentials using environment variables : This is a common practice for managing credentials in a secure manner, as environment variables can be accessed securely by applications without exposing them in the codebase.

Option D: Pass the secrets in plain text : This is highly insecure and not recommended, as it exposes sensitive information directly in the code.

Therefore, Option C is the best method for securely passing secrets and credentials to an application, protecting them from exposure.

The task is to extend a cinema chatbot to provide movie showtime information using a RAG application, leveraging user location and a continuously updated Delta table, with minimal effort and high performance. Let’s evaluate the options.

Option A: Create a Feature Serving Endpoint from a FeatureSpec that references an online store synced from the Delta table. Query the Feature Serving Endpoint as part of the agent logic / tool implementation

Databricks Feature Serving provides low-latency access to real-time data from Delta tables via an online store. Syncing the Delta table to a Feature Serving Endpoint allows the chatbot to query showtimes efficiently, integrating seamlessly into the RAG agent’s tool logic. This leverages Databricks’ native infrastructure, minimizing effort and ensuring performance.

Databricks Reference : " Feature Serving Endpoints provide real-time access to Delta table data with low latency, ideal for production systems " ( " Databricks Feature Engineering Guide, " 2023).

Option B: Query the Delta table directly via a SQL query constructed from the user ' s input using a text-to-SQL LLM in the agent logic / tool

Using a text-to-SQL LLM to generate queries adds complexity (e.g., ensuring accurate SQL generation) and latency (LLM inference + SQL execution). While feasible, it’s less performant and requires more effort than a pre-built serving solution.

Databricks Reference : " Direct SQL queries are flexible but may introduce overhead in real-time applications " ( " Building LLM Applications with Databricks " ).

Option C: Write the Delta table contents to a text column, then embed those texts using an embedding model and store these in the vector index. Look up the information based on the embedding as part of the agent logic / tool implementation

Converting structured Delta table data (e.g., showtimes) into text, embedding it, and using vector search is inefficient for structured lookups. It’s effort-intensive (preprocessing, embedding) and less precise than direct queries, undermining performance.

Databricks Reference : " Vector search excels for unstructured data, not structured tabular lookups " ( " Databricks Vector Search Documentation " ).

Option D: Set up a task in Databricks Workflows to write the information in the Delta table periodically to an external database such as MySQL and query the information from there as part of the agent logic / tool implementation

Exporting to an external database (e.g., MySQL) adds setup effort (workflow, external DB management) and latency (periodic updates vs. real-time). It’s less performant and more complex than using Databricks’ native tools.

Databricks Reference : " Avoid external systems when Delta tables provide real-time data natively " ( " Databricks Workflows Guide " ).

Conclusion : Option A minimizes effort by using Databricks Feature Serving for real-time, low-latency access to the Delta table, ensuring high performance in a production-ready RAG chatbot.

Building a basic LLM-enabled chat application with conversational capabilities, knowledge retrieval, and contextual memory requires specific components that work together to process queries, maintain context, and retrieve relevant information. Databricks’ Generative AI Engineer documentation outlines key components for such systems, particularly in the context of frameworks like LangChain or Databricks’ MosaicML integrations. Let’s evaluate the required components:

Understanding the Requirements :

Conversational capabilities : The app must generate natural, coherent responses.

Knowledge retrieval : It must access external or domain-specific knowledge.

Contextual memory : It must remember prior interactions in the conversation.

Databricks Reference: " A typical LLM chat application includes a memory component to track conversation history and a retrieval mechanism to incorporate external knowledge " ( " Databricks Generative AI Cookbook, " 2023).

Evaluating the Options :

A. (Q) : This appears incomplete or unclear (possibly a typo). Without further context, it’s not a valid component.

B. Vector Stores : These store embeddings of documents or knowledge bases, enabling semantic search and retrieval of relevant information for the LLM. This is critical for knowledge retrieval in a chat application.

Databricks Reference : " Vector stores, such as those integrated with Databricks’ Lakehouse, enable efficient retrieval of contextual data for LLMs " ( " Building LLM Applications with Databricks " ).

C. Conversation Buffer Memory : This component stores the conversation history, allowing the LLM to maintain context across multiple turns. It’s essential for contextual memory.

Databricks Reference : " Conversation Buffer Memory tracks prior user inputs and LLM outputs, ensuring context-aware responses " ( " Generative AI Engineer Guide " ).

D. External tools : These (e.g., APIs or calculators) enhance functionality but aren’t required for a basic chat app with the specified capabilities.

E. Chat loaders : These might refer to data loaders for chat logs, but they’re not a core chain component for conversational functionality or memory.

F. React Components : These relate to front-end UI development, not the LLM chain’s backend functionality.

Selecting the Two Required Components :

For knowledge retrieval , Vector Stores (B) are necessary to fetch relevant external data, a cornerstone of Databricks’ RAG-based chat systems.

For contextual memory , Conversation Buffer Memory (C) is required to maintain conversation history, ensuring coherent and context-aware responses.

While an LLM itself is implied as the core generator, the question asks for chain components beyond the model, making B and C the minimal yet sufficient pair for a basic application.

Conclusion : The two required chain components are B. Vector Stores and C. Conversation Buffer Memory , as they directly address knowledge retrieval and contextual memory, respectively, aligning with Databricks’ documented best practices for LLM-enabled chat applications.

Separate Genie Spaces provide domain-specific access to structured business datasets through Databricks-native conversational querying. The Sales specialist calls the Sales Space, and the HR specialist calls the HR Space. Each Space should contain the appropriate datasets and business context, with Unity Catalog permissions enforcing access to its underlying resources. Separating Spaces makes domain boundaries explicit and avoids relying only on an LLM’s routing instructions. Knowledge Assistants primarily support document-based knowledge retrieval, making Genie the better fit for structured business questions. A shared Space with distinct permissions can enforce security, but separate Spaces better express the requested domain separation. The video’s phrase “HR Analyst for stock questions” appears inconsistent; “staff questions” is an inferred typing correction. Databricks documentation

Option D is the best-supported answer for the endpoint behavior described. An input guardrail configured to block detected PII prevents the offending request from proceeding normally and returns a blocking response. Inference tables record request and response payloads, so the response represents the guardrail outcome rather than a hypothetical answer the model might otherwise have produced. Blocking a request is distinct from disabling its audit logging, and a blocked response is not inherently an empty response. This answer assumes the phone number is detected and the record is successfully logged. Databricks documents best-effort logging and version-dependent gateway behavior; newer guardrail implementations may return a structured blocking message rather than the same HTTP error format used by older endpoints. Guardrail documentation , inference-table documentation

When prioritizing cost and latency over quality in a Large Language Model (LLM)-based application, it is crucial to select a configuration that minimizes both computational resources and latency while still providing reasonable performance. Here ' s why D is the best choice:

Context length : The context length of 512 tokens aligns with the chunk size used for the documents (maximum of 512 tokens per chunk). This is sufficient for capturing the needed information and generating responses without unnecessary overhead.

Smallest model size : The model with a size of 0.13GB is significantly smaller than the other options. This small footprint ensures faster inference times and lower memory usage, which directly reduces both latency and cost.

Embedding dimension : While the embedding dimension of 384 is smaller than the other options, it is still adequate for tasks where cost and speed are more important than precision and depth of understanding.

This setup achieves the desired balance between cost-efficiency and reasonable performance in a latency-sensitive, cost-conscious application.

ï‚· Context : The Generative AI Engineer is setting up a vector store using Databricks ' VectorSearchClient. This is typically done to enable fast and efficient retrieval of vectorized data for tasks like similarity searches.

ï‚· Explanation of Options :

Option A: vsc.get_index() : This function would be used to retrieve an existing index, not create one, so it would not be the logical next step immediately after creating an endpoint.

Option B: vsc.create_delta_sync_index() : After setting up a vector store endpoint, creating an index is necessary to start populating and organizing the data. The create_delta_sync_index() function specifically creates an index that synchronizes with a Delta table, allowing automatic updates as the data changes. This is likely the most appropriate choice if the engineer plans to use dynamic data that is updated over time.

Option C: vsc.create_direct_access_index() : This function would create an index that directly accesses the data without synchronization. While also a valid approach, it ' s less likely to be the next logical step if the default setup (typically accommodating changes) is intended.

Option D: vsc.similarity_search() : This function would be used to perform searches on an existing index; however, an index needs to be created and populated with data before any search can be conducted.

Given the typical workflow in setting up a vector store, the next step after creating an endpoint is to establish an index, particularly one that synchronizes with ongoing data updates, hence Option B .

In this case, the Generative AI Engineer is developing an application to generate personalized birthday poems, but there’s a need to safeguard against malicious user inputs . The best solution is to implement a safety filter (option A) to detect harmful or inappropriate inputs.

Safety Filter Implementation : Safety filters are essential for screening user input and preventing inappropriate content from being processed by the LLM. These filters can scan inputs for harmful language, offensive terms, or malicious content and intervene before the prompt is passed to the LLM.

Graceful Handling of Harmful Inputs : Once the safety filter detects harmful content, the system can provide a message to the user, such as " I ' m unable to assist with this request, " instead of processing or responding to malicious input. This protects the system from generating harmful content and ensures a controlled interaction environment.

Why Other Options Are Less Suitable :

B (Reduce Interaction Time) : Reducing the interaction time won’t prevent malicious inputs from being entered.

C (Continue the Conversation) : While it’s possible to acknowledge malicious input, it is not safe to continue the conversation with harmful content. This could lead to legal or reputational risks.

D (Increase Compute Power) : Adding more compute doesn’t address the issue of harmful content and would only speed up processing without resolving safety concerns.

Therefore, implementing a safety filter that blocks harmful inputs is the most effective technique for safeguarding the application.

MLflow supports custom scorers for application-specific evaluation requirements that built-in metrics do not address. The engineer can define a scorer that examines the generated answer and, when necessary, the complete execution trace. It can verify references against an approved refund-rule list and check whether escalation statements violate the policy. The scorer returns structured feedback, such as a Boolean result, numerical score, or Feedback object, and is supplied to mlflow.genai.evaluate(). Deterministic checks are appropriate when repeatability is essential; semantic checks may require a carefully specified judge. Latency and token counts measure operational behavior rather than policy compliance. Reviewing traces manually does not create an automated metric, and the generating model’s self-assessment is not an independent compliance evaluation. Databricks documentation

Problem Context : The engineer needs an efficient way to store chunks of unstructured documents to facilitate easy retrieval and search. The current dataframe consists of document filenames and associated text chunks.

Explanation of Options :

Option A : Splitting into train and test sets is more relevant for model training scenarios and not directly applicable to storage for retrieval in a Vector Search index.

Option B : Flattening the dataframe such that each row contains a single chunk with a unique identifier is the most performant for storage and retrieval. This structure aligns well with how data is indexed and queried in vector search applications, making it easier to retrieve specific chunks efficiently.

Option C : Creating a unique identifier for each document only does not address the need to access individual chunks efficiently, which is critical in a Vector Search application.

Option D : Storing each chunk as an independent JSON file creates unnecessary overhead and complexity in managing and querying large volumes of files.

Option B is the most efficient and practical approach, allowing for streamlined indexing and retrieval processes in a Delta table environment, fitting the requirements of a Vector Search index.

The task is to set up a Databricks Vector Search index for news articles, supporting queries like “monster truck news around January 5th, 1992,” with minimal effort. The index must filter by topic and a 10-day date range. Let’s evaluate the options.

Option A: Split articles by 10-day blocks and return the block closest to the query

Pre-splitting articles into 10-day blocks requires significant preprocessing and index management (e.g., one index per block). It’s effort-intensive and inflexible for dynamic date ranges.

Databricks Reference : " Static partitioning increases setup complexity; metadata filtering is preferred " ( " Databricks Vector Search Documentation " ).

Option B: Include metadata columns for article date and topic to support metadata filtering

Adding date and topic as metadata in the Vector Search index allows dynamic filtering (e.g., date ± 5 days, topic = “monster truck”) at query time. This leverages Databricks’ built-in metadata filtering, minimizing setup effort.

Databricks Reference : " Vector Search supports metadata filtering on columns like date or category for precise retrieval with minimal preprocessing " ( " Vector Search Guide, " 2023).

Option C: Pass the query directly to the vector search index and return the best articles

Passing the full query (e.g., “Tell me about monster truck news around January 5th, 1992”) to Vector Search relies solely on embeddings, ignoring structured filtering for date and topic. This risks inaccurate results without explicit range logic.

Databricks Reference : " Pure vector similarity may not handle temporal or categorical constraints effectively " ( " Building LLM Applications with Databricks " ).

Option D: Create separate indexes by topic and add a classifier model to appropriately pick the best index

Separate indexes per topic plus a classifier model adds significant complexity (index creation, model training, maintenance), far exceeding “least effort.” It’s overkill for this use case.

Databricks Reference : " Multiple indexes increase overhead; single-index with metadata is simpler " ( " Databricks Vector Search Documentation " ).

Conclusion : Option B is the simplest and most effective solution, using metadata filtering in a single Vector Search index to handle date ranges and topics, aligning with Databricks’ emphasis on efficient, low-effort setups.

A scheduled SQL query using ai_query() is a straightforward way to process text already stored in a Delta table. The query can send each post to a supported model with instructions to return a summary and sentiment, then persist the results for internal reporting. A pay-per-token service avoids dedicating serving capacity to a workload containing only a few hundred daily records. Calling an external batch API introduces additional integration and result-handling work, while downloading and hosting a model adds infrastructure responsibilities. Provisioned throughput is generally more appropriate when sustained demand justifies dedicated capacity. B is the best fit among these choices; actual costs still depend on model selection, token volume, SQL compute, and the applicable inference pricing. Databricks documentation

Problem Context : The engineer is developing an LLM-powered live sports commentary platform that needs to provide real-time updates and analyses based on the latest game scores. The critical requirement here is the capability to access and integrate real-time data efficiently with the platform for immediate analysis and reporting.

Explanation of Options :

Option A: DatabricksIQ : While DatabricksIQ offers integration and data processing capabilities, it is more aligned with data analytics rather than real-time feature serving, which is crucial for immediate updates necessary in a live sports commentary context.

Option B: Foundation Model APIs : These APIs facilitate interactions with pre-trained models and could be part of the solution, but on their own, they do not provide mechanisms to access real-time game scores.

Option C: Feature Serving : This is the correct answer as feature serving specifically refers to the real-time provision of data (features) to models for prediction. This would be essential for an LLM that generates analyses based on live game data, ensuring that the commentary is current and based on the latest events in the sport.

Option D: AutoML : This tool automates the process of applying machine learning models to real-world problems, but it does not directly provide real-time data access, which is a critical requirement for the platform.

Thus, Option C (Feature Serving) is the most suitable tool for the platform as it directly supports the real-time data needs of an LLM-powered sports commentary system, ensuring that the analyses and updates are based on the latest available information.

In the context of developing a chatbot for a company ' s internal HelpDesk Call Center, the key is to select data sources that provide the most contextual and detailed information about the issues being addressed. This includes identifying the root cause and suggesting resolutions. The two most appropriate sources from the list are:

Call Detail (Option D):

Contents: This Delta table includes a snapshot of all call details updated hourly, featuring essential fields like root_cause and resolution.

Relevance: The inclusion of root_cause and resolution fields makes this source particularly valuable, as it directly contains the information necessary to understand and resolve the issues discussed in the calls. Even if some records are incomplete, the data provided is crucial for a chatbot aimed at speeding up resolution identification.

Transcript Volume (Option E):

Contents: This Unity Catalog Volume contains recordings in .wav format and text transcripts in .txt files.

Relevance: The text transcripts of call recordings can provide in-depth context that the chatbot can analyze to understand the nuances of each issue. The chatbot can use natural language processing techniques to extract themes, identify problems, and suggest resolutions based on previous similar interactions documented in the transcripts.

Why Other Options Are Less Suitable:

A (Call Cust History): While it provides insights into customer interactions with the HelpDesk, it focuses more on the usage metrics rather than the content of the calls or the issues discussed.

B (Maintenance Schedule): This data is useful for understanding when services may not be available but does not contribute directly to resolving user issues or identifying root causes.

C (Call Rep History): Though it offers data on call durations and start times, which could help in assessing performance, it lacks direct information on the issues being resolved.

Therefore, Call Detail and Transcript Volume are the most relevant data sources for a chatbot designed to assist with identifying and resolving issues in a HelpDesk Call Center setting, as they provide direct and contextual information related to customer issues.

Option D best matches the small catalog, infrequent updates, and maintenance requirements. Databricks-managed embeddings remove the need to build and maintain a separate embedding-generation pipeline. A Delta Sync index connects the searchable content to its source Delta table. Triggered synchronization updates the index when explicitly invoked, making it suitable when inventory changes are occasional. Continuous synchronization maintains a running streaming pipeline and therefore introduces additional synchronization costs that this scenario does not justify. Self-managed embeddings add engineering responsibilities without a stated benefit. Storage-optimized endpoints primarily address much larger vector collections. Triggered synchronization reduces synchronization overhead; it does not eliminate the separate costs of serving the vector index or computing embeddings. Databricks documentation

Problem Context : The goal is to deploy a trained LLM on Databricks in the simplest and most integrated manner.

Explanation of Options :

Option A : This method involves unnecessary steps like logging the model as a pickle object, which is not the most efficient path in a Databricks environment.

Option B : Logging the model with MLflow during training and then using MLflow’s API to register and start serving the model is straightforward and leverages Databricks’ built-in functionalities for seamless model deployment.

Option C : Building and running a Docker container is a complex and less integrated approach within the Databricks ecosystem.

Option D : Using Flask and Gunicorn is a more manual approach and less integrated compared to the native capabilities of Databricks and MLflow.

Option B provides the most straightforward and efficient process, utilizing Databricks’ ecosystem to its full advantage for deploying models.

The task involves building a question-answering chatbot for an automotive company using a database of vehicle-related documents. The chatbot must efficiently process customer inquiries and provide accurate responses. Let’s evaluate each component to determine which is not useful, per Databricks Generative AI Engineer principles.

Option A: Response-generating LLM

An LLM is essential for generating natural language responses to customer queries based on retrieved information. This is a core component of any chatbot.

Databricks Reference : " The response-generating LLM processes retrieved context to produce coherent answers " ( " Building LLM Applications with Databricks, " 2023).

Option B: Invite users to submit long, rather than concise, questions

Encouraging long questions is a user interaction design choice, not a technical component of the chatbot’s architecture. Moreover, long, verbose questions can complicate intent detection and retrieval, reducing efficiency and accuracy—counter to best practices for chatbot design. Concise questions are typically preferred for clarity and performance.

Databricks Reference : While not explicitly stated, Databricks’ " Generative AI Cookbook " emphasizes efficient query processing, implying that simpler, focused inputs improve LLM performance. Inviting long questions doesn’t align with this.

Option C: Vector database

A vector database stores embeddings of the vehicle documents, enabling fast retrieval of relevant information via semantic search. This is critical for a question-answering system with a large document corpus.

Databricks Reference : " Vector databases enable scalable retrieval of context from large datasets " ( " Databricks Generative AI Engineer Guide " ).

Option D: Embedding model

An embedding model converts text (documents and queries) into vector representations for similarity search. It’s a foundational component for retrieval-augmented generation (RAG) in chatbots.

Databricks Reference : " Embedding models transform text into vectors, facilitating efficient matching of queries to documents " ( " Building LLM-Powered Applications " ).

Conclusion : Option B is not a useful component in building the chatbot. It’s a user-facing suggestion rather than a technical building block, and it could even degrade performance by introducing unnecessary complexity. Options A, C, and D are all integral to a Databricks-aligned chatbot architecture.

When deploying an agent via the Mosaic AI Agent Framework (which leverages Databricks Model Serving), operational metrics are captured automatically by default. These include system-level telemetry such as the number of requests per second (volume), the time taken for the model to respond (latency), and the rate of 4xx/5xx HTTP errors. These are essential for monitoring Service Level Agreements (SLAs). However, Quality metrics (B), such as correctness, groundedness, or adherence to custom guidelines, cannot be determined " automatically " by the serving infrastructure because they require either human feedback or an LLM-as-a-judge evaluation (using Databricks Agent Evaluation). While Databricks makes it easy to generate quality metrics using the mlflow.evaluate API or the inference table, they are not " default operational metrics " that appear without additional evaluation configuration.

The AI Gateway usage table is designed to capture operational consumption information for SQL-based reporting. Its records include identifiers for the workspace, endpoint, and request, together with token-usage information and timestamps. The engineer can aggregate consumption by day and endpoint, then join those results to the company’s cost-center mapping. Request counting should respect the table’s request and invocation identifiers because a single logical request can involve multiple inference calls. Served-entity metadata describes deployed entities rather than providing the primary consumption records. Inference tables emphasize request and response payloads, while MLflow runs organize experiments and evaluations. Token consumption supports cost allocation, but calculating actual monetary charges may additionally require applicable pricing or billing records. Databricks documentation

To optimize a chunking strategy for a Retrieval-Augmented Generation (RAG) application, the Generative AI Engineer needs a structured approach to evaluating the chunking strategy, ensuring that the chosen configuration retrieves the most relevant information and leads to accurate and coherent LLM responses. Here ' s why C and E are the correct strategies:

Strategy C: Evaluation Metrics (Recall, NDCG)

Define an evaluation metric : Common evaluation metrics such as recall, precision, or NDCG (Normalized Discounted Cumulative Gain) measure how well the retrieved chunks match the user ' s query and the expected response.

Recall measures the proportion of relevant information retrieved.

NDCG is often used when you want to account for both the relevance of retrieved chunks and the ranking or order in which they are retrieved.

Experiment with chunking strategies : Adjusting chunking strategies based on text structure (e.g., splitting by paragraph, chapter, or a fixed number of tokens) allows the engineer to experiment with various ways of slicing the text. Some chunks may better align with the user ' s query than others.

Evaluate performance : By using recall or NDCG, the engineer can methodically test various chunking strategies to identify which one yields the highest performance. This ensures that the chunking method provides the most relevant information when embedding and retrieving data from the vector store.

Strategy E: LLM-as-a-Judge Metric

Use the LLM as an evaluator : After retrieving chunks, the LLM can be used to evaluate the quality of answers based on the chunks provided. This could be framed as a " judge " function, where the LLM compares how well a given chunk answers previous user queries.

Optimize based on the LLM ' s judgment : By having the LLM assess previous answers and rate their relevance and accuracy, the engineer can collect feedback on how well different chunking configurations perform in real-world scenarios.

This metric could be a qualitative judgment on how closely the retrieved information matches the user ' s intent.

Tune chunking parameters : Based on the LLM ' s judgment, the engineer can adjust the chunk size or structure to better align with the LLM ' s responses, optimizing retrieval for future queries.

By combining these two approaches, the engineer ensures that the chunking strategy is systematically evaluated using both quantitative (recall/NDCG) and qualitative (LLM judgment) methods. This balanced optimization process results in improved retrieval relevance and, consequently, better response generation by the LLM.