CCAO-F exam prep The seven blueprint domains
Domain 2 · Output Evaluation and Validation
Twenty-one percent, the heaviest of the seven, and the first of the guide's three published sample items comes from it — which is unlikely to be an accident
· Part 2 · The seven blueprint domains · 3 min read
Weight: 21% of the exam. The heaviest of the seven.
If you read one domain, read this one. The first of the guide's three published sample items comes from here too, which is unlikely to be a coincidence.
The objectives, as the guide states them
Six of them — more than any other domain.
"Evaluate Claude-generated outputs for accuracy and completeness"
"Identify hallucinations, inconsistencies, and biases in responses"
"Apply fact-checking and validation techniques"
"Determine when human review or additional verification is required"
"Edit, adapt, refine, and compare outputs for the intended audience"
"Organize and curate information and select appropriate output formats (artifacts, inline, structured data)"
Why it is the heaviest
Because it is what separates someone who can use AI from someone who merely does. Section 3 describes the candidate as a person who can
"critically evaluate AI-generated content"
and Section 4 lists among the recommended experience
"A practical understanding of AI limitations, including hallucinations, context constraints, and data sensitivity"
Those three phrases map onto three domains exactly — hallucinations is this one, context constraints is domain 3, data sensitivity is domain 6. Which happen to be three of the four heaviest domains on the exam.
The official sample item
Section 8 gives three samples, with the caveat that
"They are not drawn from the live item bank."
The first belongs to this domain. In summary: Claude produces a confident summary of a new regulation citing a specific subsection number, and the question is what to do before sending it to the compliance team.
The answer is verify the cited subsection against the official text first, and the rationale the guide gives is
"Language models can fabricate specific-looking details such as citation numbers, a hallucination. Validating factual claims, especially citations bound for a compliance audience, against an authoritative source is the diligence step required. Self-reported confidence (A, C) is not a reliable accuracy signal"
That last sentence is the lesson that carries across the whole exam — asking the model how sure it is does not count as checking. Options that have the model assess itself are wrong in items of this shape.
Output format lives here too
The final objective is about choosing the shape of the output, and it names three.
artifacts a separate piece of work you can keep editing
inline answered in the conversation
structured data something with a shape you can use downstreamThis is the only objective in the domain that tests feature knowledge. The rest test judgment, and Section 7 sends you to the documentation for exactly these features.
"Review official Anthropic documentation and help articles for Claude features such as Projects, Artifacts, Memory, Skills, and Code Execution"
When it lies
"If the model says it is confident, it is probably right." The guide rejects this outright in the rationale for its own first sample. Self-reported confidence is not an accuracy signal.
"A hallucination means a wholly invented answer." The guide's own example is a fabricated subsection number inside content that is otherwise correct, which is far harder to catch.
"Checking the work is the reviewer's job." The fourth objective makes deciding when a human review is required your skill, not theirs.
References
Official documents
- CCAO-F Exam Guide Section 6 domain 2, Section 8 samples and rationale, Sections 3 and 4 on the candidate profile
Elsewhere in this course
- Reading the items works through all three published samples