On this page
Cloudflare AI Search is a managed search service for application and agent data. Start an evaluation with a small owned corpus, explicit source metadata and questions whose answers you can check. A useful result should point to the right document and respect the caller's access scope. Fluent generated text is not enough when a user needs a reliable answer from private records.
The current AI Search documentation describes indexing, hybrid retrieval, metadata filtering and agent-facing capabilities. Use that documentation for the available product contract. Treat this guide's corpus and evaluation examples as a proposed workflow, not a report of a tested integration.
Decide what belongs in the searchable corpus
Choose material you own or are authorized to use. A product-help corpus might contain published support articles, versioned feature documentation and approved troubleshooting notes. A private employee corpus has different access requirements and should not be mixed into the same public result stream by convenience.
For every document, preserve an identifier, title, source URL, owner, version, language and access scope. Record the date on which it was last checked. These fields help your application select current material and explain where an answer came from.
Avoid indexing drafts as though they were approved policy. A proposed refund rule and the current refund rule may contain similar language while implying different actions. Preserve approval state as metadata and define which states are eligible for each search experience. The retrieval setup should reflect the business distinction rather than asking the model to guess it.
Keep retrieval and answering separate
A search response identifies potentially relevant material. An answer layer interprets that material for the user's question. Evaluate these steps separately so you can tell whether a failure came from missing retrieval, wrong document scope or unsupported synthesis.
Cloudflare's AI Search announcement describes changes to source management and public endpoints. Public convenience features deserve a deliberate access review before they are used with private material. Confirm the actual configuration of your instance and application, rather than assuming that the word “private” in a project name protects its data.
| Evaluation layer | Question to ask | Evidence to inspect |
|---|---|---|
| Corpus | Is the correct approved document indexed? | Source register and version |
| Retrieval | Does the query find the relevant passage? | Returned document references |
| Scope | Is the result allowed for this caller? | Metadata filters and authorization checks |
| Answer | Does the statement follow from the source? | Claim beside its supporting passage |
| Maintenance | Does a changed document affect results? | Updated index and repeated query |
A result can pass one layer and fail another. For example, a relevant passage may come from a retired version. Preserve that observation and repair the version-selection rule. Adding a more persuasive answer prompt does not resolve an obsolete source selection.
Create an evaluation set before tuning
Write representative questions with expected source references. Include a direct factual question, a question requiring more than one document, an ambiguous question and one that the corpus cannot answer. Use synthetic private records for access checks so a failed test does not reveal real confidential material.
For a product-help example, the direct question might ask which export formats are supported in the approved documentation. The multi-document question might ask how an export feature interacts with a plan limit. The unsupported question might request a future release date that no approved source provides.
The expected behavior for the unsupported question is an explicit evidence gap, not a fabricated roadmap. For an ambiguous question, the system can request the product version or describe the uncertainty. Keep those outcomes in the test set rather than counting every confident sentence as success.
Do not tune only on one familiar question. A workflow can retrieve that passage reliably and still fail on a synonym, a changed version or a different access scope. Expand the evaluation with real user wording as it becomes available, while keeping the original cases stable enough to compare revisions.
Test access at the application boundary
Search filters are useful, but inspect the complete access path. The caller's identity, eligible document set, result references and linked document access should agree. A result snippet that hides a private title can still expose information if the underlying URL opens a restricted file to an unauthorized user.
Use a public reader and a private reviewer in the test. Give them intentionally different document scopes. Ask the same question under both identities and inspect the returned references. Then follow the source links as each reader. Verify that private material does not appear through summaries, previews or direct links outside its approved scope.
Test a revoked permission as well. Define how your system updates eligibility when a document moves or a user loses access. Retain the observed timing and remaining uncertainty. A permission check performed at initial indexing may not describe the current caller's authority later.
Plan document changes and failed ingestion
A retrieval system needs a visible maintenance path. When a document changes, record which version is indexed and repeat its known evaluation question. When ingestion fails, show that failure to the owner rather than quietly serving an old answer as current.
Keep deletion and retirement explicit. Removing a page from navigation does not necessarily remove its indexed content. Define how the service and application retire a document and verify that the old reference no longer appears where it should be absent. Preserve historical records in an appropriate archive if your workflow needs them, but keep their retired status visible.
Prototype a source-cited search interface with Atoms
Atoms can help you explore a search interface and document register with synthetic results. A UI prototype is separate from configuring and validating an AI Search instance. Keep the sample result path honest while deciding what users need to inspect.
Build a source-cited document search app with synthetic help articles.
Each document has ID, title, source URL, version, language, owner,
approval state and access scope. Show results with a source link,
relevant excerpt and version rather than an unsupported confident answer.
Include direct, multi-document, ambiguous and unanswered demo questions.
Keep public and private reviewer scopes distinct.
Show ingestion failures and retired-source states in an admin register.
Do not claim live Cloudflare connectivity until an integration is tested.
List the provider setup, authentication and permission checks still needed.
Read the client portal guide for the surrounding application access model. A search experience becomes useful when the reader can inspect the right evidence and the system reliably preserves its scope.
Use the sample brief to start a document search interface in Atoms.
Conclusion
Evaluate Cloudflare AI Search with a small approved corpus, checkable questions and explicit access rules. Inspect retrieval, answer support and source-link access separately. Prototype the reviewable search interface in Atoms, then validate the real provider connection and document lifecycle before using private records.
Create a first version with synthetic data and review the results.
FAQs
Q1: Is indexing a document enough to make answers accurate?
Indexing makes material available for retrieval, but an accurate answer also needs the correct document version, relevant passages and a synthesis that stays within the source evidence. Evaluate those steps separately. A search result can be relevant while referring to retired policy, and an answer can overstate a passage it retrieved correctly. Keep source identifiers and versions beside the result, then test questions with known references. Include unanswered and ambiguous cases so the workflow can expose evidence gaps rather than interpreting confidence as correctness.
Q2: How should I test search over private documents?
Use synthetic records and at least two identities with different access scopes. Ask comparable questions, inspect snippets and source references, and follow the linked documents as each user. Verify that the application enforces scope at retrieval and at document access, including after a permission is revoked. Record the actual configuration and observations. A project name or a private-looking interface does not establish that the search endpoint is protected. Confirm the complete request path before indexing material that would be harmful to expose.
Q3: Can I build the interface before choosing the retrieval service?
Yes. A synthetic prototype can help you define result cards, source links, version labels, unanswered states and review controls. Those interface decisions often remain useful across provider choices. Keep the demo clearly separate from live retrieval, and preserve the data contract you expect from an eventual integration. Before launch, test the chosen service with the real corpus, identity and maintenance rules. The prototype clarifies what should happen; it does not prove that indexing, permission enforcement or source-cited answering already works in production.
