Don't Ask Candidates Whether They Use AI. Ask How They Judge It

The interviewer asks, "How comfortable are you with AI?"

The candidate answers smoothly. They use several popular tools, build custom prompts, automate research, and describe themselves as an early adopter. Five minutes later, the interviewer has learned a great deal about the candidate's vocabulary and almost nothing about whether they can be trusted with AI-assisted work.

This question is appearing in more interviews for a sensible reason. LinkedIn reported on October 27 that the share of job postings requiring AI literacy had risen roughly 65 percent over the previous 12 months. The demand extends beyond technical roles into marketing, design, project management, administration, and writing. Yet only 1.2 percent of LinkedIn members list an AI skill, which suggests employers and candidates are still learning what the category means. LinkedIn's analysis is based on activity within its own network, not the entire labor market, but the direction is hard to miss.

The hiring problem is that "AI literacy" can mean at least four different things: knowing which tools exist, operating one fluently, redesigning a workflow, or judging where an AI output deserves trust. The last two are usually more valuable at work, and much harder to infer from a list of products.

A better interview does not ask candidates to perform enthusiasm or skepticism. It asks them to reconstruct one real decision: what they gave to AI, what they kept for themselves, how they checked the result, and what changed afterward.

Begin with a piece of work, not an opinion

Ask for a recent example in which AI materially affected a work product or decision. "Materially" matters. Grammar correction and calendar suggestions are legitimate uses, but they reveal little about judgment when the role will involve analysis, customer communication, hiring, planning, or other consequential work.

Use a neutral opening:

Tell me about a task where AI changed how you approached the work. What was the outcome you needed, and which part did you choose to give the system?

Candidates should be allowed to say that they have not used AI in a professional setting. Give them a comparable scenario instead of treating access as aptitude. A customer support candidate might review an AI-drafted reply built from a short case file. A project manager might decide which parts of a status synthesis to automate. The goal is to observe reasoning, not purchasing history.

Do not score the first answer. It will often be polished because candidates reasonably prepare for obvious questions. Follow the story through five checkpoints: choice, boundary, inspection, repair, and ownership.

1. Choice: Why was AI appropriate here?

Strong candidates can explain the job they wanted AI to perform. They do not stop at "it was faster."

Ask:

• What made this part of the task suitable for AI?
• Which part did you deliberately keep human?
• What would have made you avoid AI altogether?

Listen for a match between the tool's role and the work. Drafting several low-stakes options is different from deciding which customer deserves a refund. Summarizing a known document set is different from establishing whether the set is complete.

There is no universally correct division. A candidate may sensibly automate more in a reversible internal draft than in a legal, financial, medical, personnel, or customer commitment. What matters is whether consequence, uncertainty, and reversibility shaped the choice.

2. Boundary: What could the system actually know?

People who work well with AI understand that an answer inherits the limits of its context.

Ask the candidate to name the inputs, missing information, and access boundary. If they used meeting notes to draft a project update, did the system also have the current plan, the latest customer message, and the decision log? If not, how was that absence handled?

A useful answer sounds like this: "I gave it the approved research notes and the current brief, but not the raw customer recordings. I asked it to group repeated problems, then checked whether the grouping had erased minority cases."

A weaker answer treats fluent output as evidence of complete context. Watch for phrases such as "it knew our project" when the candidate cannot explain what sources were available or current.

3. Inspection: What did you check, exactly?

"I reviewed it" is the AI-era equivalent of "detail-oriented." It sounds reassuring while hiding the method.

Ask for the inspection path:

• Which claim or section did you check first, and why?
• What did you compare against a source?
• Did you sample the output or inspect all of it?
• Which error would have been easiest to miss?

The answer should fit the output. A candidate reviewing a synthesized customer report might trace quotations, confirm sample coverage, and look for categories built from a single interview. Someone reviewing generated code might test failure paths and examine data handling. Someone reviewing a forecast might challenge assumptions and recalculate a few pivotal figures.

This is not merely personal caution. NIST's Generative AI Profile recommends documenting fact-checking techniques, verifying sources and citations during ongoing monitoring, and identifying human oversight roles. The guidance is written for organizational risk management, but it makes a useful interview principle: trustworthy AI use leaves an inspectable route from output back to evidence. NIST AI 600-1

4. Repair: Tell me about the correction

The most revealing part of the story is often the moment the output failed.

Ask: "What did the AI get wrong, flatten, or overlook, and what did you do next?"

A candidate who says the result was perfect may have chosen a trivial example, inspected it lightly, or feel pressure to present AI use as magic. Make correction safe to discuss. The interview should reward detection and recovery, not a spotless story.

Listen for more than editing. Did the candidate change the prompt, narrow the source set, replace the tool, ask a domain expert, or abandon the AI route? Did they correct only that output, or improve the workflow so the same error was less likely to return?

Also ask who was affected before the error was found. A repaired internal draft is one thing. A mistaken customer promise that was sent and quietly edited later is another. Good judgment includes knowing when a correction needs disclosure.

5. Ownership: Who stood behind the result?

Finish with consequence: "Who made the final call, and what responsibility did you personally accept?"

Good answers keep authority legible. The candidate may have produced the analysis while a manager approved the spend, or drafted a policy while legal owned the interpretation. They can distinguish doing the work from holding the decision right.

They should also be able to name the outcome. Did the work save review time, improve coverage, surface an overlooked risk, or simply create more output? An impressive workflow that nobody used is not automatically a success.

Microsoft's 2026 Work Trend Index offers a useful caution here. In its survey of 20,000 AI-using knowledge workers, respondents classified as "Frontier Professionals" were much more likely than others to say their managers set quality standards for AI work. Microsoft reports an association, not proof that standards caused better results. Still, it reinforces the hiring lesson: individual skill is easier to use well when a team can describe what good work looks like. Microsoft's report

Do not hire for private cleverness and then place the person in an organization with no review standard, source boundary, or escalation path.

Score judgment without rewarding theater

After the interview, score the five checkpoints separately. Use a simple three-level scale:

Thin: The answer stays general, tool-centered, or impossible to inspect.

Credible: The candidate names a real choice, relevant boundary, concrete check, correction, and human owner.

Strong: The candidate also adapts the workflow after learning, calibrates review to consequence, and can explain a tradeoff without pretending uncertainty disappeared.

Record evidence before adjectives. "Checked all customer quotations against six transcripts and restored two minority objections" is useful. "AI savvy" is not.

Keep the evaluation role-specific. A communications hire does not need to explain model architecture. A software engineer may. A junior candidate may have fewer workplace examples but show excellent reasoning in the scenario. Avoid rewarding expensive tool access, confident jargon, or a willingness to automate everything.

The best signal is not enthusiasm for AI. It is calibrated responsibility: the candidate can use assistance without surrendering the judgment the role requires.

That makes the interview itself a demanding listening task. The interviewer must follow a story, notice which checkpoint is missing, ask a precise follow-up, and preserve evidence for the debrief. Caspi supports that work with live recap, suggested questions, contextual chat, proactive flags from connected tools, post-call action items, and persistent meeting memory. Used carefully, a meeting copilot can help the interviewer keep the conversation human while making the candidate's actual reasoning easier to examine after the call.