Methodology

Last updated: 21 April 2026

This page describes how AEOps collects, processes, and publishes data on AI engine citation behaviour. It is intended to be readable by practitioners without an engineering background. If you spot an error, email hello@aeops.io.

What we measure

AEOps tracks what three AI engines — ChatGPT, Perplexity, and Claude — recommend when answering realistic buyer-intent queries across two B2B sectors:

  • B2B SaaS — CRM, marketing automation, analytics, customer success, AEO tooling.
  • Digital marketing agencies — SEO, PPC, content, social, affiliate, AEO tooling.

Two sectors are currently active; two further sectors (legal services, financial services) are deferred and their prompts are held in the database as inactive so the category structure is preserved for future reactivation.

The engines

Each query runs against all three engines in parallel. We log the exact model identifier and the exact date of the run with every record.

  • ChatGPT — gpt-4o-search-preview via the OpenAI Chat Completions API.
  • Perplexity — sonar via the Perplexity Chat Completions API with search_context_size: low.
  • Claude — claude-sonnet-4-6 via the Anthropic Messages API with the web_search tool enabled (beta header web-search-2025-03-05).

Gemini is deferred. When it is added, it will be labelled clearly and the date of addition will be recorded.

Engines change their underlying search and retrieval behaviour over time. We do not control the model vendors' indexing, ranking, or retrieval. All findings are conditional on the engine's behaviour on the day of the run.

The prompts

We maintain 91 canonical prompts across the two active sectors. Each prompt is tagged with an intent_stage:

  • Research — broad how/what questions at the top of the buyer journey.
  • Comparison — side-by-side comparisons of named products.
  • Evaluation — explicit "best tool for X" questions.
  • Agentic — long, constraint-heavy requests that look like an agent's search query.

Prompts are fixed. We do not cherry-pick prompts week-to-week to produce a desired outcome. When we add or deprecate a prompt, it is logged in the methodology changelog below.

Run cadence

  • Monday primary run — full prompt set across all active engines. Data from this run is the basis of the Monday post.
  • Thursday volatility run — full prompt set re-run to measure week-over-week volatility. Thursday data feeds into the following Monday's post; it does not trigger its own publication.
  • Total weekly volume: 91 prompts × 3 engines × 2 runs = 546 queries per week.

Citation extraction

Citations are extracted from structured API response fields only — never from the AI engine's free-text response. This matters because:

  • ChatGPT gpt-4o-search-preview exposes citations in the annotations array on each assistant message.
  • Perplexity sonar returns a citations array on the response object.
  • Claude web_search exposes web_search_tool_result blocks with explicit URLs.

We do not scrape citations out of the prose response. This keeps the dataset reproducible: given the same response JSON, the same citations are extracted every time.

After extraction, cited URLs are canonicalised (protocol, host casing, tracking parameters stripped) and joined to cited-page metadata (word count, schema.org types, content depth, media tier) via an overnight scraper. The scraper respects robots.txt and uses a descriptive user-agent that identifies AEOps.

Aggregation and analysis

Raw query results are aggregated into weekly category statistics (citation counts by domain, engine, intent stage, and sector) before any insight is generated. Insights are then derived from these aggregates — we do not cherry-pick outliers from raw data.

When a finding is flagged as statistically notable, the underlying aggregate query is made available on request to subscribers.

Limitations and caveats

We try to be honest about what the data can and cannot tell you.

  • Sample size. Two runs per week per prompt is small. Week-over-week changes can be noise. When we report a change, we also report the prior two weeks of data so you can judge for yourself.
  • Engine non-determinism. AI engines are stochastic. Running the same prompt twice can produce different citations. We do not control for this at query time; we surface variance in our reporting instead.
  • Model version changes. Vendors update their underlying models, indexes, and retrieval behaviour without notice. A change in citation pattern may reflect a model update rather than a change in the underlying web.
  • Regional bias. Queries run from a single region with a single API key per vendor. Results may differ for buyers in other regions.
  • AI-injected results. Engines sometimes recommend made-up products or hallucinated URLs. These are logged as invalid and excluded from citation counts.
  • Not a ranking service. AEOps is a research publication, not a ranking or scoring service. We do not sell visibility, boost tools on request, or accept placement fees.

Data-handling principles

  • Public data only. All underlying data is sourced from public AI engine responses and public web pages. No private dataset, customer list, or confidential information from any past or present employer, client, or third party is used.
  • No user tracking in the pipeline. The pipeline does not observe or track any readers. Reader analytics on the website use consent-gated tooling via Iubenda.
  • No paid placements. No tool, vendor, or party pays to be in the dataset or to be featured in a post. This is guaranteed in the Sponsor Disclosure policy.

Transparency

Material changes to the methodology — adding or removing a prompt, switching engine models, changing run cadence — are logged here with the effective date. The last few entries:

  • 21 April 2026 — Pivot to 91 canonical prompts across B2B SaaS and digital marketing agencies. Legal services and financial services deferred.
  • Prior to 21 April 2026 — Initial build of the pipeline across four sectors and 165 prompts.

If a methodology change might alter how you interpret earlier data, we will note it in the relevant weekly post and link back to this page.

Contact

Questions, corrections, or requests for raw aggregates: hello@aeops.io.