On this page
PerplexityBot and Perplexity-User are different request identities described by Perplexity. The first supports discovery for search results; the second supports a fetch initiated by a user's action. When checking your website, identify the requester, inspect the applicable access controls and verify the response at the intended URL. Access is a prerequisite for reading a page, not a guarantee that the page will be cited.
The official crawler documentation is the starting point for current user-agent and IP information. Use its published endpoints and distinctions rather than an old third-party list of crawler names.
Identify which request you are investigating
A scheduled crawler request and a user-requested fetch can have different access behavior. Perplexity's documentation says PerplexityBot surfaces and links websites in search results, while Perplexity-User visits pages in support of user actions. It also explains the different treatment of robots rules.
That distinction matters when you interpret logs. A request carrying one identity does not prove that the other has visited your site. A human asking a question that mentions your page does not prove that a scheduled crawler has indexed it. Preserve the observed user agent, request path, time and response status.
A copied user-agent string is not sufficient identity verification. Any ordinary client can send a header that claims to be a named bot. Perplexity publishes IP ranges for its requesters and recommends using user-agent and address information together for firewall configuration. Keep those source lists current and verify your actual network path before relying on a rule.
Read your own access controls in order
Start with the URL that should be readable. Fetch the canonical page and confirm that it returns the intended content. A successful home page request says little about a blog route that is missing, redirected incorrectly or dependent on a logged-in session.
Next, read robots.txt and the relevant firewall configuration. Confirm whether a rule applies to the requester you are investigating. Keep a deliberate private-page restriction intact; a publishing audit should not make account records or preview environments public merely to improve access for a crawler.
This is an illustrative robots rule for public paths. It is not a recommendation to replace an existing policy without reviewing its other directives:
User-agent: PerplexityBot
Allow: /blog/
Disallow: /account/
Robots directives and firewall rules operate at different layers. Allowing a path in robots.txt does not remove a firewall challenge. A firewall allow rule does not make a page useful or correct. Check the response the intended requester receives, and retain the evidence for each layer separately.
Keep an access observation register
Use a small register to explain what you checked. It helps distinguish an actual page problem from a missing observation.
| Access observation | What to record | What it establishes |
|---|---|---|
| Canonical page request | URL, status and content marker | The current response at that route |
| Robots policy | Applicable requester and paths | The published crawl directive |
| Firewall evaluation | Matched rule and action | How that request was handled |
| Request identity | User agent and verified address source | Evidence about the requester |
| Server log entry | Request time, route and status | An observed fetch of that route |
| Answer observation | Question, service and cited URL | A particular citation result |
Do not fill an empty log entry with a guessed visit. “No matching request observed in this log window” is a useful statement when its time range and coverage are known. It is different from “the service never crawls this page.” Log retention, sampling and proxy layers can limit what you see.
Likewise, a successful fetch is not evidence of a citation. An answer service may read several sources and choose a different one for a particular question. Track that selection separately so a technical access check does not become an unsupported marketing result.
Verify the content the requester can read
Open the rendered page without relying on an account-specific browser session. Confirm that the central answer, title and supporting links are present. Check a representative source link and any essential downloadable artifact. A visible heading with an empty article body does not fulfill the reader's question.
If your publishing system uses caching, compare the stored content with the public route. An updated CMS record can coexist with an older cached page. The access audit should name which version the public request actually returned. Keep the publication time and factual review date accurate rather than changing them simply to make the page look recent.
For product pages, make the answer concrete. Name the applicable plan, input condition or published specification where that matters. Avoid replacing vague quality claims with equally vague “verified” labels. The reader should be able to follow the source and understand exactly what was established.
Review unexpected requests without guessing motives
When you see a request with an unfamiliar identity or a mismatch between the header and address evidence, preserve the observation and inspect the applicable controls. Do not classify it as a trusted requester solely because the name resembles a familiar bot. Also avoid inferring a provider's entire behavior from a single ambiguous request.
Separate the question you can answer from the one you cannot. You may be able to establish that a request reached a route and received a challenge. You may not be able to establish whether the requester later obtained the content through another route. Report the observed path accurately and use the provider's current documentation for the declared behavior.
Build a source-page audit register with Atoms
Atoms' AI app builder can help you prototype a page-access register for your team. Start with manually entered observations and safe synthetic log rows. Connecting live logs, changing firewall rules or running external measurements are separate integrations and actions.
Build a website source-page audit app with synthetic records.
Store canonical URL, observed date, status, content version, requester,
robots observation, firewall observation and source evidence link.
Keep unknown requester identity and missing logs explicitly unknown.
Add a separate citation-observation table with exact question and service.
Do not infer citations from crawl access or successful HTTP responses.
Show unresolved access issues by page owner and retain review history.
Provide CSV export; do not change any live network or publishing settings.
The agent-ready website guide addresses the related problem of making content and actions understandable. Use both checks when needed: the page must be accessible, and its answer must remain clear and well supported.
Use the sample brief to start a crawl access audit board in Atoms.
Conclusion
Check Perplexity access using the correct requester, current identity sources and the actual response at your page. Keep crawling, readable content and citation observations separate. Create an audit register in Atoms when your team needs a reliable place to maintain those observations and their evidence.
Create a first version with synthetic data and review the results.
FAQs
Q1: Does allowing PerplexityBot guarantee that my page will be cited?
No. An access rule helps a requester reach content, while citation selection depends on the service's behavior for a particular question. Verify the page response and useful answer first, then record actual answer observations separately. Preserve the exact question, service and observation date when checking citations. A successful fetch establishes that a request could read the route you checked. It does not establish indexing, answer inclusion or a stable citation rate. Treat each of those as a separate outcome that needs its own evidence.
Q2: Can I verify a crawler using only its user-agent header?
The header is a claim made by the client and can be copied. Use the provider's current identity-verification guidance, including its published IP information, alongside the observed user agent. Verify what address your logging and firewall layers actually use, especially when a proxy is involved. Keep the source and check time for the range data. If the evidence is incomplete, leave identity unresolved. A familiar name is useful for finding a request in logs, but it is not by itself a trustworthy basis for granting privileged access.
Q3: Should I make every route public for AI crawlers?
No. Apply a crawl and access policy that fits each route's purpose. Published articles can be readable while account areas, private records and previews remain restricted. Inspect existing controls before changing them, and avoid a broad allow rule that unintentionally exposes other paths. The audit should identify a concrete public-content access problem and the layer responsible. After an authorized change, verify the intended route and relevant restricted routes. Better access to a public answer does not require removing the application's privacy and permission boundaries.
