Methodology

Last updated: 1 May 2026

This page describes how AEOps collects, processes, and publishes data on AI engine citation behaviour. It is intended to be readable by practitioners without an engineering background. If you spot an error, email hello@aeops.io.

What we measure

AEOps tracks what the leading AI search engines — ChatGPT, Perplexity, and Claude — recommend when answering realistic buyer-intent queries in UK fintech. Within that sector we cover personal banking, savings and ISAs, investing, money transfer, credit and lending, B2B payments, insurtech, and pensions.

The publication's focus has narrowed over time. Earlier rounds of work covered B2B SaaS and digital marketing agencies; the prompts and historical data for those sectors are retained but not currently published.

The engines

Each query runs against all three engines in parallel. We log the model identifier and the date of the run with every record so any later analysis can account for vendor model updates.

We use each vendor's structured search API. Vendors update their underlying models, indexes, and retrieval behaviour frequently. All findings are conditional on each engine's behaviour on the day of the run.

Gemini is currently out of scope. If it is added we will note the date of inclusion.

The prompts

We maintain a controlled set of buyer-intent prompts. Each is tagged with an intent_stage:

  • Research — broad how/what questions at the top of the buyer journey.
  • Comparison — side-by-side comparisons of named products.
  • Evaluation — explicit "best tool for X" questions.
  • Agentic — long, constraint-heavy requests that look like an agent's search query.

Prompts are fixed for the period they are active. We do not cherry-pick prompts week-to-week to produce a desired outcome. When we add or deprecate a prompt the change is logged.

Run cadence

The pipeline runs on a weekly cadence. Findings publish each Monday on aeops.io and via the free email brief.

Citation extraction

Citations are extracted from structured API response fields only — never from the AI engine's free-text response. This matters because:

  • Each vendor exposes citations in a structured field on the response (annotations, citations, or web-search-tool result blocks).
  • Parsing from prose is unreliable and tends to over-count.

We do not scrape citations out of the prose response. This keeps the dataset reproducible: given the same response JSON, the same citations are extracted every time.

After extraction, cited URLs are canonicalised (protocol, host casing, tracking parameters stripped) and joined to cited-page metadata (word count, schema.org types, content depth, media tier) via an overnight scraper. The scraper respects robots.txt and uses a descriptive user-agent that identifies AEOps.

Aggregation and analysis

Raw query results are aggregated into weekly category statistics (citation counts by domain, engine, intent stage, and sub-category) before any insight is generated. Insights are then derived from these aggregates — we do not cherry-pick outliers from raw data.

When a finding is flagged as statistically notable, the underlying aggregate query is made available on request to subscribers.

Limitations and caveats

We try to be honest about what the data can and cannot tell you.

  • Sample size. The pipeline is a research instrument, not a market census. Week-over-week changes can be noise. When we report a change, we also report the prior weeks of data so you can judge for yourself.
  • Engine non-determinism. AI engines are stochastic. Running the same prompt twice can produce different citations. We do not control for this at query time; we surface variance in our reporting instead.
  • Model version changes. Vendors update their underlying models, indexes, and retrieval behaviour without notice. A change in citation pattern may reflect a model update rather than a change in the underlying web.
  • Regional bias. Queries run from a single region with a single API key per vendor. Results may differ for buyers in other regions.
  • AI-injected results. Engines sometimes recommend made-up products or hallucinated URLs. These are logged as invalid and excluded from citation counts.
  • Not a ranking service. AEOps is a research publication, not a ranking or scoring service. We do not sell visibility, boost tools on request, or accept placement fees.

Data-handling principles

  • Public data only. All underlying data is sourced from public AI engine responses and public web pages. No private dataset, customer list, or confidential information from any past or present employer, client, or third party is used.
  • No user tracking in the pipeline. The pipeline does not observe or track any readers. Reader analytics on the website use consent-gated tooling via Iubenda.
  • No paid placements. No tool, vendor, or party pays to be in the dataset or to be featured in a post. This is guaranteed in the Sponsor Disclosure policy.

Transparency

Material changes to the methodology — adding or removing a prompt, switching engine vendors, changing run cadence — are logged here with the effective date. The most recent entry:

  • 1 May 2026 — Publication scope narrowed to UK fintech. Earlier-sector prompts retained as inactive for historical continuity.

If a methodology change might alter how you interpret earlier data, we will note it in the relevant weekly post and link back to this page.

Contact

Questions, corrections, or requests for raw aggregates: hello@aeops.io.