AI Engineer Interview Questions (with real STAR answers)

Retrieval-augmented generation finds relevant documents at query time and adds them to the prompt so the model answers from them. Use RAG when the model needs facts that change or are private, and when you need citations. Use fine-tuning to change style, format, or behavior on a narrow task - it is a poor way to add facts.

An embedding is a vector of numbers that represents the meaning of a text, so similar texts have vectors close to each other. Similarity is usually measured with cosine similarity or dot product. Use the same embedding model for documents and queries.

Chunks must be small enough to be specific and large enough to hold a complete idea - a few hundred tokens is a common start. Split on natural boundaries such as headings and paragraphs, add some overlap, keep metadata like the source and title, and pick the final size by testing retrieval on real questions.

A store for embeddings that finds the nearest vectors to a query quickly, using approximate nearest neighbor indexes such as HNSW. Examples are pgvector, Chroma, Qdrant, and Pinecone. Most also support metadata filters, and many support hybrid search that combines vector and keyword matching.

First retrieve many candidates quickly with vector search (a bi-encoder), then score each query and document pair with a more accurate but slower model (a cross-encoder) and keep the top few. This puts the best passages at the top of the context.

The model predicts likely text, not checked facts, so it can produce confident wrong answers. Reduce it by grounding answers in retrieved context, telling the model to say when it does not know, asking for citations, using a lower temperature for factual tasks, and checking answers with evaluations.

Both control how random the output is. Temperature scales the probabilities - low values (near 0) give focused, repeatable output, higher values give more varied output. top_p keeps only the most likely tokens whose probabilities add up to p. Usually you change one of them, not both.

Tokens are the pieces of text the model reads and writes - roughly parts of words. The context window is the maximum number of tokens the model can handle in one request, input and output together. Cost and latency grow with tokens, so keep prompts and retrieved context focused.

Build a golden set of real inputs with expected outputs or grading rules. Score with exact checks where possible (valid JSON, correct label), retrieval metrics such as recall@k for RAG, and an LLM-as-judge for open answers like faithfulness and relevance. Run the evaluation on every prompt or model change and track scores over time.

Prompt injection is input - from a user or from a retrieved document - that tries to override the system instructions. There is no complete fix, so limit the damage: separate trusted instructions from untrusted content, give tools the least privilege, validate model output before acting on it, and require human approval for risky actions.

You describe tools to the model with names and JSON schemas. The model responds with a tool name and arguments instead of text. Your code validates the arguments, runs the tool, and sends the result back, and the model uses it to continue. An agent repeats this loop until the task is done, with a limit on steps.

Use the provider's structured output or JSON mode when available, give a clear schema and an example in the prompt, then parse and validate the result (for example with Pydantic). Strip markdown code fences if present, and retry with the validation error when parsing fails.

Overfitting is when a model learns the training data too closely, including noise, so it scores well on training data and poorly on new data. Detect it with a separate validation set. Prevent it with more data, regularization, simpler models, early stopping, and cross-validation.

Precision is the share of predicted positives that are really positive. Recall is the share of real positives the model found. Raising one often lowers the other. Use precision when false alarms are costly, recall when missing a case is costly, and F1 when you need a balance of both.

Measure where the time goes - retrieval, the model call, or your own code. Stream tokens so users see output right away, cache repeated requests, shorten the prompt and context, use a smaller or faster model for simple requests, and run independent steps such as retrieval calls in parallel.

Expect behavioral questions (tell me about a time you...), technical questions on LLM APIs, RAG, embeddings, evaluation, and core machine learning, and scenario questions such as reducing latency or improving answer quality. Many companies also give a build task, such as a small RAG app. The questions on this page cover all of these.

Use STAR: Situation, Task, Action, Result. Keep it concrete - a specific project, what you owned, what you did, and the result. The answers above are all structured this way.