Databricks Generative AI Engineer Associate Practice Exam Questions and Answers – Part 4/6

Answer: C,E. C. Select an appropriate evaluation metric (e.g., recall or NDCG) and experiment with variations in the chunking strategy, such as splitting by paragraphs or chapters, to identify the best-performing approach. || E. Develop a metric where the LLM acts as a judge, scoring how well previous questions are answered by the retrieved chunks. Adjust the chunking parameters based on the results of this metric.

Options C and E are correct because systematically optimizing chunking requires defining measurable retrieval metrics or using an LLM-as-a-judge to evaluate ground truth. Simply asking the model to guess token counts or changing unrelated pipeline components does not measure actual retrieval performance. Expect questions on evaluating chunking variations.

Answer: B. B. Ingest documents from a source → Index the documents and store them in Vector Search → Develop and test the RAG chain with representative queries → Retrieve relevant documents and generate responses using the LLM → Evaluate the RAG application → Deploy the application using Model Serving

Option B is correct because building a RAG application on Databricks follows a specific workflow of ingesting data, indexing it, testing the chain, evaluating performance, and finally deploying. You must always evaluate the retrieval and generation quality before pushing the application to production. Look for the standard sequence.

Answer: C. C. Feature Serving

Option C is correct because Databricks Feature Serving provides low-latency, real-time access to structured features like live sports scores. Foundation Model APIs generate text, but they cannot fetch live external data independently. Rely on Feature Serving to bridge real-time data streams with deployed models.

Answer: D. D. Inference Tables

Option D is correct because Databricks Inference Tables automatically capture incoming requests and outgoing responses for Model Serving endpoints. This native feature eliminates the operational overhead of building custom logging microservices. Rely on Inference Tables for built-in production monitoring and auditing.

Answer: D. D. Break the HR documentation into chunks and store them in a vector database. Use the employee's question to retrieve the most relevant chunks, and use the LLM to generate a response based on the retrieved documentation.

Option D is correct because breaking documentation into chunks and storing them in a vector database enables targeted, cost-effective retrieval before generation. Averaging embeddings or summarizing entire documents loses the granular context required for accurate answers. Stick to standard retrieval-augmented generation patterns for document tasks.

Answer: B. B. Create a dataset to separately test the retrieval and generation components of the system. Utilize MLflow's built-in evaluation metrics for this assessment.

Evaluating retrieval and generation components separately using MLflow metrics pinpoints specific failure areas in a RAG pipeline. Option A fails because cosine similarity alone cannot assess generation fluency or correctness, making a holistic, component-level approach necessary for optimization.

Answer: B. B. Log the model using MLflow during the training phase, register the model directly in Unity Catalog via the MLflow API, and start a serving endpoint.

Logging the model with MLflow, registering it in Unity Catalog, and starting a serving endpoint is the most seamless deployment path on Databricks. Option A fails because pickling models introduces security risks and bypasses the native governance provided by MLflow model tracking.

Answer: B. B. Deploy the model using pay-per-token throughput, as it provides cost guarantees.

Pay-per-token throughput provides cost effectiveness for applications with unpredictable or low traffic volumes by aligning expenses directly with usage. Provisioned throughput guarantees capacity but wastes resources for low-volume applications, making it an inefficient strategy here.

Answer: B. B. Utilize a neutralizer to standardize the tone and style of the source documents.

Utilizing a neutralizer strips stylistic variations from source documents, directly reducing the linguistic richness needed to generate haikus. The other options actively steer the model through direct instructions, few-shot examples, or fine-tuning.

Answer: D. D. Create an agent equipped with tools for SQL querying of Delta tables and web searching, and provide the retrieved data to an LLM for response generation.

Creating an agent with SQL and web search tools enables the dynamic retrieval of real-time stock prices and current news. Option C fails because pre-storing news in a vector database leads to stale information, whereas agents fetch live data during execution.

Answer: B. B. Use LangChain's Agent Framework with distinct tools exposed to agents for parsing, interpreting, and suggesting

Using an agent framework with distinct tools supports asynchronous execution and multi-stage reasoning for complex workflows. Option A fails because chaining tasks in a single prompt hides intermediate steps and prevents dynamic function exposure.

Answer: B. B. Evaluate responses using an LLM-as-a-judge setup that scores correctness, completeness, and alignment with legal standards

Using an LLM-as-a-judge setup to score responses provides an automated, repeatable way to catch misrepresentations before production. Option D fails because temperature only controls variation; deterministic settings do not inherently fix hallucinations.

Answer: B. B. Rewrite the prompt as a metaprompt that instructs the LLM to retain all technical terms and produce a structured summary with sections like "Background", "Methods", and "Findings"

Rewriting the prompt as a detailed metaprompt directly forces the model to preserve terminology and adhere to a specific structure. Option D fails because relying on zero-shot prompting with appended examples lacks the strict constraints needed for formatting.

Answer: D. D. Summarization

Summarization is the correct NLP task because it specifically condenses longer text into a shorter form while preserving core information. The other options are incorrect because text classification assigns labels, and a sentencizer merely splits text without summarizing.