Brand-level sentiment matters because real AI answers often name several brands in one response. A traditional once-per-response sentiment label collapses "Brand-A is great, Brand-B is risky" into a single number — losing the sub-signal that drives competitive positioning.

In engine 1.0.0 we run a separate sentiment classification for each distinct brand mentioned in each response. Implementation note: this is what we call F-2 ("N per-brand calls") in the engine. A single-prompt multi-brand variant (F-1) was evaluated but did not meet our acceptance criteria for label agreement and stability without live model access at the time of lock-in. We may revisit F-1 in a future engine version if its agreement with F-2 reaches our threshold under empirical testing.

Cost implication. A typical run that previously emitted 32 LLM cost events under once-per-response sentiment now emits roughly 47 events (16 query + 31 sentiment) for the same fixture, because the classifier runs once per (response × distinct brand). The engine emits a structured LLMCostEvent for every call so the bookkeeping is transparent — see Cost transparency for the per-run dollar implications.

We treat citations as their own scoring axis distinct from name-mentions because the gap between cite-and-name behavior is large and provider-specific. Some engines disproportionately cite without naming brands; others disproportionately name brands without citing them. Pass 5 measures citation reach for the target brand only — competitor URL citations show up in perProviderCitations as context (with brandAppears: false) but do not contribute to your citationScore. The score follows the formula:

citationScoreForBrand = (Σ over responses: clamp(brandCited, 0, 1)) / totalResponses × 100

The sum clamp ensures one response with three branded URLs still contributes 1 to the numerator, not 3.

When Google grounds an answer, it decomposes the user's question into multiple sub-queries (webSearchQueries) before retrieving sources. The engine captures those sub-queries verbatim from providerMetadata.google.groundingMetadata.webSearchQueries and attaches them to the result. Fan-out coverage is not scored in v1 because research suggests fan-out coverage maximization is anti-correlated with citations — naively scoring "we appeared in N fan-out sub-queries" would optimize for the wrong thing. Capturing the data lets you audit what Google thought your topic meant, without misleading you about what to do next.

ChatGPT, Claude, and Perplexity also internally decompose queries, but only Google currently exposes the decomposition. Non-Google providers therefore have no fanOutQueries key on the result at all — this is a structural absence ("we don't fake fan-out"), not an empty array. An empty fanOutQueries.google would mean "Google answered without fan-out" (rare, captured verbatim); the absence of the key entirely means "this provider does not expose fan-out."

Every result publishes a perProviderScores map breaking the headline number down by provider — { openai: { score, sampleN }, anthropic: { ... }, ... }. The engine writes these with uniform weights only: every provider counts equally in the headline aggregate. We deliberately do not bake non-uniform per-platform weighting into the core engine because weighting is a downstream business decision (e.g. "weight Perplexity higher for community-driven categories") that should be configurable read-time, not stamped into the comparability contract. If non-uniform weighting becomes a paid product feature, it will live in a separate package that reads perProviderScores and projects a re-weighted score — never in the core engine.

For location-dependent brands (brick-and-mortar, regional services), we append region context directly to the query — e.g. "best coffee shops in Osaka, Japan" rather than "best coffee shops". This is a deliberate methodology choice, not a workaround. The only regional variance we cannot control is provider-side routing (which geographic server handles the request) — that remains uncontrolled noise.

Repeat runs for the same brand are pinned to the original run's time-of-day as closely as scheduling allows, so score-over-time comparisons are not confounded by diurnal drift in LLM outputs. Small timing offsets from queue scheduling are tolerated; larger shifts are only introduced on breaking changes (engine version bump or keyphrase set change) where comparability has already reset.

For paid consumer-tier runs (tier='consumer' on ScoringRunInput), the engine selects flagship consumer-grade models per provider. These are the models real end-users encounter when they ask ChatGPT / Claude / Gemini / Perplexity directly, so this tier preserves the trust contract for "what does AI see when it sees my brand?"

For free-tier runs (tier='free' on ScoringRunInput), the engine selects cost-bounded API-tier models. Where possible these are the models a FREE consumer session actually gets, so a free AVS run mirrors free ChatGPT or the free Gemini app rather than approximating a paid one. They may diverge from what the paid consumer surfaces show — that is the point of the tier, and the free-tier results page carries a permanent disclosure label about the trade-off plus an upgrade CTA per SCOR-04 (D-14 reframe).

A run stamped tier='consumer' and a run stamped tier='free' are not directly comparable. The score is well-defined within each tier (same prompts, same engine version, same scoring rubric, same hashing discipline), but model behavior differs across tiers — a brand may score 72 on consumer-tier and 65 on free-tier without any change in its actual AI visibility.

The trust-comparison contract therefore narrows for tier-mixed runs: only compare scores stamped with the same tier AND the same engineVersion AND the same promptHash AND the same keyphraseSetHash. The tier field on every ScoringRunResult makes this boundary structurally enforceable — consumer call sites (results page, score-history projection, cohort analytics) that mix tiers without disclosure are the failure mode this stamp guards against.

Why publish both registries. Trust is the moat. We do not score real users on models we won't name, and we do not let the free-tier disclaimer hide which exact API SKU produced the cheaper score. Both registries here are the SAME pins the engine actually uses — there is no third "actually we used X" SKU lurking behind either tier.

Why consumer tier for paid. We score paid runs against the model versions real users are actually talking to when they visit chatgpt.com, claude.ai, gemini.google.com, or perplexity.ai. API-only lightweight models (e.g. gpt-5.4-mini) are cheaper and faster but can diverge meaningfully from the consumer experience. They are never used for paid runs — paid users on a budget-trip queue (D-08) wait, they don't drop in quality.

Why pinned, not auto-latest. If we silently upgraded models, your score would move for reasons that have nothing to do with your brand's actual visibility changing. Every pinned version here requires a new engine version and a changelog entry (see the Changelog) before it changes. That is the contract that keeps scores comparable over time, within a tier.

What we pin to: the model people actually get. A pin's job is to answer "what does AI say about my brand?" for the models real people are talking to. So each slot tracks what the consumer app serves by default — the consumer tier mirrors a paid ChatGPT/Claude/Gemini/Perplexity session, the free tier mirrors a free one. When a provider makes a new model the default, our pin follows it at the next engine version.

This is a deliberate trade against a tempting alternative: freezing a pin forever would hold your score perfectly steady, and would make it steadily less true. A score measured against a model nobody is served anymore is precise and worthless. Comparability is preserved instead by the stamp — every run records its engineVersion and its exact modelVersions, and scores are only comparable within a matching stamp set. That is why a pin change resets comparability rather than being forbidden by it.

Within that policy we still pin exactly. We never point at a floating family alias that a provider re-points silently (gpt-5.6, chat-latest, gemini-pro-latest) — those would change the score with no stamp change to show it, which is the one failure this contract exists to prevent. We prefer a dated snapshot when the provider publishes one; several current defaults ship bare IDs only, and the bare ID is then the complete, correct pin.

Sent to the Haiku classifier once per (response × distinct mentioned brand), with ${brandName}, ${matchedText}, and ${responseText} interpolated at call time. Bytes verbatim from packages/scoring-engine/src/prompts.ts (SENTIMENT_PROMPT_TEMPLATE):

Analyze how "${brandName}" is mentioned in this AI response. The mention was found as: "${matchedText}".

Classify the intent:
- recommendation: The brand is explicitly endorsed, suggested, or recommended to the user. Includes phrases like "I recommend", "I'd suggest", "a great option is", "one of the best", "top choice", or being presented as the answer to the user's question.
- neutral: The brand is mentioned without explicit endorsement or criticism. Includes being named as existing in a space, referenced for context, or listed among options without the response endorsing any of them.
- negative_qualified: The brand is warned about in a SPECIFIC context but not condemned overall. Example: "not ideal for large-scale events" or "can be pricey". The negative is scoped to a particular use case, feature, or attribute.
- negative_blanket: The brand is warned against OVERALL. Example: "I'd recommend avoiding them" or "there have been reliability concerns". The negative applies to the brand as a whole.

Full response:
${responseText}

Three diagnostic prompts run after main scoring on all 4 providers × 3 reps each. Probe outputs do not contribute to overallScore — they answer recommendation-fuel questions ("what does the AI think you are known for?", "who does it consider weaker than you — and why?", "who does it consider stronger — and why?") that surface in the results UI alongside the score, not inside it. See Positioning probes below for the full diagnostic-purpose / matrix / failure-handling description.

Each probe response is then fed through a Haiku generateObject extraction pass that converts the raw text into structured insights.

KNOWN_FOR_PROBE_TEMPLATE — sent to each main provider (ChatGPT, Claude, Gemini, Perplexity), 3 reps each. Bytes verbatim from packages/scoring-engine/src/prompts.ts:

What is ${brandName} known for? Please describe what ${brandName} is recognized for in the ${topic} space.

WEAKER_COMPETITORS_PROBE_TEMPLATE — sent to each main provider, 3 reps each. Bytes verbatim:

Name up to three competitors that are tangibly weaker than ${brandName} on ${topic}. Do not make a competitor seem weaker just to fill the list — if the list is short or empty because ${brandName} is genuinely weak, that is fine. For each competitor you do name, give the name and one short sentence explaining why it is weaker.

STRONGER_COMPETITORS_PROBE_TEMPLATE — sent to each main provider, 3 reps each. Bytes verbatim:

Name up to three competitors that are tangibly stronger than ${brandName} on ${topic}. Do not make a competitor seem stronger just to fill the list — if the list is short or empty because ${brandName} is genuinely strong, that is fine. For each competitor you do name, give the name and one short sentence explaining why it is stronger.

KNOWN_FOR_EXTRACTION_TEMPLATE — sent to the Haiku classifier once per probe rep. Returns { summary, phrases }. Bytes verbatim:

Extract the key claims from this AI response about what "${brandName}" is known for. Return a one-sentence summary and the top phrases.

Response:
${rawResponse}

WEAKER_COMPETITORS_EXTRACTION_TEMPLATE — sent to the Haiku classifier once per probe rep. Returns deduped { name, reason } competitor pairs (names post-processed through canonicalBrandKey to collapse case + alias variants; the reason is the response's own one-sentence explanation, empty when it gave none). Bytes verbatim:

Extract the competitors this AI response presents as weaker than "${brandName}". Only include brands the response actually says are weaker — skip brands it calls comparable, stronger, or mentions only in passing. If it names none, return an empty list. For each competitor, return the brand name and the response's one-sentence reason why it is weaker (empty string when the response gives no reason).

Response:
${rawResponse}

STRONGER_COMPETITORS_EXTRACTION_TEMPLATE — sent to the Haiku classifier once per probe rep. Same deduped { name, reason } pair contract as the weaker extraction. Bytes verbatim:

Extract the competitors this AI response presents as stronger than "${brandName}". Only include brands the response actually says are stronger — skip brands it calls comparable, weaker, or mentions only in passing. If it names none, return an empty list. For each competitor, return the brand name and the response's one-sentence reason why it is stronger (empty string when the response gives no reason).

Response:
${rawResponse}

All six template byte-strings are SHA-256 fingerprinted at module load by src/prompt-hashes.ts and published in TECHNICAL_REFERENCE.md. A byte change to any one of them surfaces as a hash diff on its own row of the PROMPT_HASHES table without invalidating the unrelated extraction or scoring hashes.

Pass 4 is deterministic arithmetic, not an LLM call. The rubric bytes are hashed alongside the sentiment prompt so any future change to the point table shows up as a hash diff on every subsequent run and forces a changelog entry.

Bytes verbatim from packages/scoring-engine/src/prompts.ts (SCORING_PROMPT_TEMPLATE):

Scoring rubric (points per mention, integer, bounded [-10, 10]):

Sole recommendation ............... 10
#1 ordered list ................... 10
#2 ordered list ...................  8
#1 unordered list .................  8
#3 ordered list ...................  6
#2-3 unordered list ...............  6
#4 ordered list ...................  4
#4-5 unordered list ...............  4
#5 ordered list ...................  3
#6-7 ordered/unordered ............  2
#8+ ordered/unordered .............  1
Passing mention ...................  4
Negative (qualified) .............. -4
Negative (blanket) ............... -10

Normalized brand score: (sum(points) / (totalResponses * 10)) * 100, clamped to [0, 100].

The rubric extension to slots 5–8 with diminishing weights (3/2/2/1 ordered, 4/2/2/1 unordered) reflects research showing AI Overview pages routinely cite 5+ sources per slot — under the previous rubric a brand at rank 6 earned the same 4 points as rank 4, flattening the deep-list gradient. Engine 1.0.0 captures the diminishing-but-nonzero contribution of mid-list visibility.

Publishing the full prompts is a deliberate trust choice. The score's trust is our moat. Anyone can read exactly what we ask and confirm we are not reframing the question behind the scenes. Fingerprints of both prompts and instructions for independent hash verification live in TECHNICAL_REFERENCE.md.

Probes are intentionally additive: they can never drag your score down, and they cannot inflate it either. The split exists because conflating position (where AIs rank you) with positioning (what AIs say about you) muddies both signals. The score answers position. The probes answer positioning.

A 5-keyphrase run already executes 60 main-score query calls on the free tier (4 providers × 5 keyphrases × 3 reps) or 100 on the consumer tier (× 5 reps) — see How we measure it. Probes layer the following on top, the same on both tiers:

• 2 probe templates (knownForProbe, weakerCompetitorsProbe)
• × 4 providers (ChatGPT, Claude, Gemini, Perplexity — same registry as main scoring)
• × 3 reps per (probe × provider)

= 24 probe LLM calls per run, plus 24 Haiku extraction calls (one per probe rep, one extraction pass each), for a deterministic upper bound of 48 additional LLMCostEvents per run when probes succeed on every (provider × rep) cell.

Each LLMCostEvent carries a purpose enum ("main" for scoring queries, "sentiment" for the per-brand classifier, "probe" for raw probe calls, "extraction" for the Haiku structured-output pass) so downstream cost dashboards can split spend by stage cleanly.

Probes interpolate ${brandName} and ${topic} placeholders. topic is derived from the run's input — preferred source is the explicit topic field on ScoringRunInput, falling back to the first keyphrase's category when not provided. The derived topic is stamped onto the result as runTopic so probe outputs are reproducible end-to-end (every probe-rep raw response can be re-paired with the exact topic string used).

Probe results are stamped on ScoringRunResult.positioningProbes:

positioningProbes: {
  knownFor: { openai: ProbeRep[], anthropic: ProbeRep[], google: ProbeRep[], perplexity: ProbeRep[] },
  weakerCompetitors: { openai: ProbeRep[], anthropic: ProbeRep[], google: ProbeRep[], perplexity: ProbeRep[] }
}

Each ProbeRep carries the raw response text, the structured extraction ({ summary, phrases } for knownFor, competitors[] for weakerCompetitors), the provider's response ID where available, and the per-rep cost. The full Zod shape lives in packages/shared-types/src/scoring.ts (PositioningProbesSchema).

Probes are wrapped in Promise.allSettled cross-provider tolerance — a probe failure on one provider never blocks the rest. A run with all 4 providers succeeding on the main score but Anthropic failing on the weakerCompetitorsProbe ships a complete overallScore plus a positioningProbes.weakerCompetitors.anthropic array containing the partial reps that did succeed (or omitted entirely if every rep failed). The failedProviders stamp distinguishes main-score failures from probe failures so retry decisions can be made per-stage.

If the same provider fails on both main score and probes, the probe entry is skipped — there's nothing useful to retry independently. If main score succeeds but probes fail, the run is still classified partial (see User-triggered resume below) so the user can choose to retry just the probe stage without re-running the more expensive main-score loop.

This is the default tier for every run the score publishes. These are the same models a real user is talking to when they open ChatGPT, Claude, Gemini, or Perplexity — the only versions whose behavior the AI Visibility Score is designed to measure.

‡ Promotional rates, both verified live on the providers' pricing pages 2026-09-07. OpenAI guarantees Sol no higher than 2026-11-21. Anthropic's Sonnet 5 promotional rate has no announced end date — it was originally slated to end 2026-08-31 and was quietly extended, so we carry it until it moves. Either may revert to list ($5.00/$30.00 and $3.00/$15.00 respectively); the weekly drift cron surfaces the change.

§ Google tiers this model at $4.00 / $18.00 above 200k input tokens. We encode the flat ≤200k rate because a scoring query plus its grounded context runs in the low thousands of tokens, three orders of magnitude below the threshold. It is a documented approximation, not an oversight.

¶ Perplexity bills tool invocations separately from tokens. Every call we make forces one web_search at $0.0025, which pricing.ts carries as toolFeePerCallUsd and adds on top of the token cost — so a Perplexity call costs more than its token rates alone imply. Both tiers pay the same rates; the consumer tier costs more per call because its deeper search pulls roughly 3.7× the input tokens (5,226 vs 1,415 on the 2026-09-08 verification calls). The flat one-invocation fee is a documented floor: it matched both tiers on those calls, but a run that reformulates into extra search rounds bills more. Each Agent API response reports its exact figure in usage.cost.tool_calls_cost; reading that instead of approximating is a tracked follow-up.

Per-run cost (ballpark). A typical 5-keyphrase consumer-tier run costs roughly $4–7, nominal ~$5.35 (re-derived 2026-09-08; down from $7.55). This is a rough figure, not a measured band like the free-tier projection below — no test asserts it. It is anchored on a real run rather than a guess: consumer validation run 87dc48b9 billed $7.55 on 2026-08-28, and re-pricing that run's implied token volume at current rates gives ~$5.35. Reconciling $7.55 against the 2026-08-28 rates implies consumer responses run about 2× the token volume of the free-tier profiles in the projection table below — consumer models answer at greater length — and that 2× factor is what carries forward; only the rates move. The dominant line is no longer Sonnet: at $2.00/$10.00 the Anthropic leg is ~$0.83, while OpenAI gpt-5.6-sol at $4.00/$20.00 is ~$2.68, half the run. Gemini 3.1 Pro adds ~$1.20, Perplexity ~$0.27 (cheap tokens, deeper search, plus the per-invocation fee), and the tier-independent judge/extraction calls ~$0.39. Free-tier runs use cheaper budget SKUs (Haiku 4.5, gpt-5.6-luna, Gemini 3.6 Flash, perplexity/sonar at reduced search depth) at 3 reps instead of 5, and project at ~$0.72 per run (asserted band $0.62–$0.83, recentered 2026-09-22 for the engine 1.10.0 rep cut — see the Changelog) — see the Phase 04.5 free-tier cost projection (2026-05-14) subsection below for the per-call breakdown derived from real prod smoke data. The sentiment-side cost is now larger than under previous engine versions: F-2 sentiment runs once per (response × distinct mentioned brand), so the classifier emits ~47 cost events for the reference TestCo fixture (16 query + 31 sentiment) versus ~32 under the legacy once-per-response classifier. The deterministic upper bound on classifier calls is keyphrases × providers × reps × distinctBrandsPerResponse — every cost event is structured and emitted via LLMCostEvent so the bookkeeping is auditable. These figures are baseline token costs only — they exclude other overhead costs.

Per-run cost projection for the default free-tier matrix (5 keyphrases × 3 reps × 4 providers = 60 main responses, plus 4 providers × 3 probes × 3 reps = 36 probe responses, plus 60 entity-extraction calls and 1 competitor-classifier call). Derived from prod smoke run bfb25892-d8b7-49a7-b443-377a94d2b40a 2026-05-14 (empirical total $0.8254, run at the then-current 5 reps).

Token profiles below are the empirical per-call averages from that smoke and have not changed since; the rate column is what moves between engine versions, and the call counts moved once, at engine 1.10.0 (5 → 3 free-tier reps).

Each provider's 24 calls are 15 main (5 keyphrases × 3 reps) plus 9 probe (3 probes × 3 reps). Sentiment is one call per response here; the real count scales with distinct brands per response.

¶ $0.0024 of tokens at $0.25/$2.50 per 1M (exactly $0.002375, so $0.004875 a call), plus the flat $0.0025 forced-web_search fee — see the ¶ footnote under the consumer-tier rate table.

Band note (2026-09-22, engine 1.10.0). The asserted band in __tests__/cost-projection.test.ts is $0.62–$0.83, the ±15% window around the $0.72 nominal above ($0.6155 → $0.62, $0.8327 → $0.83). The 5 → 3 free-tier rep cut moved the nominal from ~$1.02 to ~$0.72: 24 provider calls per provider instead of 34, and 60 sentiment and 60 entity-extraction calls instead of 100 each. Probe legs, the classifier, token profiles and rates are unchanged. This is not a loosened window — the old $0.87–$1.18 band excludes the new nominal entirely.

Band note (2026-09-08, superseded). The asserted band in __tests__/cost-projection.test.ts is $0.87–$1.18, the ±15% window around the $1.02 nominal above. It was recentered on 2026-09-08 from $0.80–$1.08, which had been drawn around the superseded $0.94 nominal and left the post-Perplexity-migration figure sitting near its top edge rather than its center. Band width is unchanged at ±15% — the window is $0.31 wide against a nominal 9% higher, the same proportion, not a wider net.

Rate history. The 2026-05-14 prod smoke priced out at $0.85 on then-current rates and a gpt-4o-mini extraction model. Engine 1.4.0 (2026-06-02) moved it to ~$1.07 (band $0.90–$1.25) when the LiteLLM sync raised Google Flash output and OpenAI budget output, and extraction migrated to gpt-5.4-nano-2026-03-17. Engine 1.5.1 (2026-08-28) moved it to ~$1.34 (band $1.10–$1.55) when the Google budget leg went to the gemini-3.7-flash list rate. Engine 1.6.0 (2026-09-07) brings it down to ~$0.94 (band $0.80–$1.08) — the first decrease in the series. Nothing about our usage got cheaper; the pins moved to models that cost less. The OpenAI leg fell $0.21 → $0.076 (gpt-5.4-mini → gpt-5.6-luna) and the Google leg fell $0.383 → $0.191 (gemini-3.6-flash at the rate we are actually billed). The 2026-09-08 Perplexity Agent API migration then took that leg the other way, $0.078 → $0.166, for a ~$1.02 nominal — the new endpoint's cheaper tokens are more than offset by its per-invocation search fee — and the band was recentered to $0.87–$1.18 the same day. Engine 1.10.0 (2026-09-22) cut free-tier reps from 5 to 3, taking the nominal to ~$0.72 (band $0.62–$0.83) — a volume change, not a rate change. See the Changelog 1.4.0, 1.5.1, 1.6.0 and 1.10.0 entries.

This band will rise again on 2027-01-01, when the Gemini Flash rates double and the Google leg goes to ~$0.27, putting the total at ~$0.86 — above the $0.83 ceiling. The changeover therefore breaks the asserted band rather than merely shifting the nominal inside it: it is a required simultaneous edit to pricing.ts, this table, and the cost-projection test, and CI will go red on the rate change if it is missed.

The wide tolerance band (~±15%) reflects per-response token-count variance — responses with web_search tool usage emit substantially more tokens than responses without, and the empirical 2026-05-14 smoke per-call averages reconstruct to in/out token profiles around 2x what the original deploy-note projection assumed for Haiku 4.5 in particular. Production runs that consistently fall outside this band signal a model-pricing or usage-pattern change worth investigating. The cost-projection test at packages/scoring-engine/__tests__/cost-projection.test.ts asserts this band in code so drift surfaces in CI.

Resolved 2026-09-07 (engine 1.6.0). The open question here used to be whether the free tier should drop its OpenAI leg (~$0.21/run) onto the cheaper model already used for entity extraction, trading response fidelity for cost. The 5.6 family settled it without a trade: OpenAI retired the separate nano class, so the cheapest available model is the free-ChatGPT default, and the scoring and extraction slots now both pin gpt-5.6-luna. The leg fell to $0.076/run and the fidelity argument never had to be made — we are scoring against the model free users actually get.

Not used for paid runs. Published here for transparency about the candidate free-tier product the funnel may offer: a cheaper, lower-fidelity score computed against API-only SKUs. Any free-tier result would ship with an explicit disclaimer that the budget tier may diverge from consumer-tier behavior.

† These rates double to $1.50 / $7.50 on 2027-01-01. Under the encode-what-we-are-billed policy above we carry the current rate and treat the changeover as a scheduled edit to pricing.ts, the projection table below, and the cost-projection test band.

Per-run cost (ballpark): roughly $0.62–$0.83 for a 5-keyphrase, 3-rep free run (the projected free-tier band — see the projection subsection above; nominal ~$0.72 since engine 1.10.0). That nominal is the c_free figure docs/product/cost-model-scenario-D.md prices the funnel on. These are baseline token costs only — they exclude other overhead costs, and the Perplexity line additionally carries the per-invocation search fee noted above.

Why rate-limited. We run on a fixed daily spend cap to keep the free tier alive on a solo-founder budget. When the cap is near, new runs queue instead of failing — users see a branded "you are in line" experience, not an error.

Content that gives unambiguous, declarative answers — without hedging — is more readily quoted by AI assistants. When an assistant is asked "is X a good fit for Y," it gravitates toward the most authoritative-sounding sentence in the source material; hedged copy ("X may be useful in some contexts") tends not to surface as the lifted answer. Write the answer as a direct claim, then qualify; do not bury the claim under qualifiers. The first-position dominance pattern in our underlying research consistently points to declarative answer-paragraphs landing in the #1 mention slot more often than equivalent hedged prose.

A question heading paired with a single-paragraph answer is the structure assistants pull verbatim. The pattern matches the way an assistant itself constructs answers — when training data already encodes Q–A pairs, retrieval lifts the pair as a unit. Schema.org FAQPage markup amplifies this further by signalling the structure explicitly to crawlers and grounded retrievers. Concretely: lead each major topic with the user's question as a heading, follow with one tight paragraph of answer, and mark the block up with FAQPage where appropriate.

Named entities — people, places, products, structured Schema.org markup, Wikidata properties — anchor a brand inside the knowledge graphs assistants are trained on and grounded against. Brands with a thin entity surface are harder for assistants to confidently name; brands with rich entity coverage become "obvious" answers. The highest-leverage moves for a brand starting from low entity coverage are usually a Wikidata entry, Schema.org Organization and Person markup on owned pages, and outbound canonical references that cross-link the brand into authoritative external entities.

Honest discussion of trade-offs — "we're great for X, less suited for Y" — reads as more trustworthy to LLM judges than one-sided puffery. The training data assistants draw from contains explicit patterns of marketing-versus-reality contrast, and content that overclaims tends to be down-weighted relative to content that names its own limitations. A page that explicitly calls out where the brand is not the right fit gets surfaced more often when users ask comparison questions, because the assistant treats it as a more credible source of judgment.

Plain-language prose at roughly an 8th-grade reading level is preferred over jargon-heavy industry-speak. Assistants draft answers at a register comfortable for a general audience; sources written at that register are integrated more readily, while dense or jargon-loaded copy is more often paraphrased away or ignored entirely. Run candidate copy through a Flesch–Kincaid checker before publication and rewrite passages that score significantly above grade 8. Shorter sentences, fewer Latinate connectives, and concrete nouns all push in the right direction.

Comparability statement. Free-tier scores do not carry across this boundary. A free (screening) run now asks each keyphrase 3 times per provider instead of 5, so a 5-keyphrase free run's rates and counts sit on 60 responses instead of 100. The dashboard's score history marks the engine change as a method change. The consumer tier is unchanged at 5 reps, but it shares the engine version, so the next consumer run after 1.10.0 is also marked as a method change in history even though its method did not move. Prompts, model pins, probe reps, tool versions and search depth are unchanged. Compare only runs sharing an engineVersion and a tier.

Changes:

• Free runs: 5 → 3 reps per keyphrase per provider; consumer stays at 5 (SCOTT-725, founder ruling 2026-09-22). Screening is framed as an educated guess: budget models, fewer asks. The paid index is the one that runs consumer models at 5 reps. repsForTier() resolves the count from the run's tier. Probe reps stay at 3 on both tiers (PROBE_REPS). (SCOTT-725)
• repsPerKeyphrase stamped on the result. The stored result records how many reps the run executed. The field is optional: rows written before 1.10.0 lack it, and those runs ran 5. (SCOTT-725)
• Free-tier cost projection recentered: ~$1.02 → ~$0.72 per run, band $0.62–$0.83 (was $0.87–$1.18). Per provider, main calls fall from 25 to 15 (probe calls stay at 9); sentiment and entity-extraction calls fall from 100 to 60 each; probe extraction (36) and the competitor classifier (1) are unchanged. Rates and token profiles did not move. See the projection table. (SCOTT-725)
• Expected free-run duration (EXPECTED_RUN_MINUTES.free, quoted at intake) 55 → 40 minutes, scaled by answer calls per provider (34 → 24). Interim: to be re-derived from prod p90 once 1.10.0 free runs have completed. (SCOTT-725)

Comparability statement. Scores do not carry across this boundary for Share of Voice or the stronger/weaker competitor probes. One counting rule changed (which extracted entities land on a tracked brand's Share of Voice row — and therefore how many rows sit in the SoV pool), and four prompts changed bytes (both competitor probes and both competitor extractions). overallScore, citationScore, pins, rep counts, tool versions and search depth are unchanged. Compare only runs sharing an engineVersion and a tier. Stored runs are not rewritten: a pre-1.9.0 run keeps the trackedEntities[] and probe answers it was scored with.

Changes:

• A brand's declared URL is an exact identifier on the Share of Voice path (SCOTT-700). The classifier's tracked-brand map was built from name + aliases only, while intake stores the brand's domain in urls — so a brand whose display name differs from its domain label got a second, "discovered" row for its own domain (run e05a15d6: "Kindred Collective" 54 mentions beside kindredcollectivewed.com 16). urls now feed the same exact canonical-key match as name and aliases — inside the 1.8.0 exact-match ruling (URL, alias, full name; no heuristics). Every aggregated row that maps to one brand folds into one row: mentions summed, provider/keyphrase breadth as the union (this also closes a latent duplicate-row case for a brand mentioned by both name and alias). validateBrandsOrThrow walks urls too, so two brands colliding through a URL fail loud at engine entry. Known limit, accepted 2026-09-21: a subdomain URL keys on its subdomain label — see Alias handling. (SCOTT-700)
• Stronger/weaker competitor probes ask for up to three, and say a short or empty list is fine (SCOTT-701). The old wording ("Can you name three competitors …") handed a genuinely strongest or weakest brand a padded list. The extraction prompts now keep only brands the response actually presents as stronger/weaker — skipping ones it calls comparable or mentions in passing — and return an empty list when it names none. New bytes are in Prompts (full text); PROMPT_HASHES.strongerCompetitorsProbe, weakerCompetitorsProbe, strongerCompetitorsExtraction and weakerCompetitorsExtraction change, the other entries are byte-stable. Expect shorter stronger/weaker lists, and empty ones, on 1.9.0 runs. (SCOTT-701)
• Dashboard display rule, not an engine change: a cited site counts once per answer (SCOTT-716). The "sources AI trusts" table used to add one per source entry, and providers repeat a site inside one answer (one entry per search result or annotation — up to 8× in one Anthropic answer), so the count measured provider verbosity: run e05a15d6 showed two hosts at 159 "citations" across 100 answers. Founder ruling 2026-09-21: a site counts at most once per AI answer, read as "cited in N of M answers" (that run: 58 and 59 of 100). Derived at render time from queryResponses[].sources, so stored runs re-render under the new rule; citationScore already counted once per response and is unchanged. (SCOTT-716)

Comparability statement. Scores do not carry across this boundary. Two counting rules changed: which discovered entities count toward the target's Share of Voice row (a multi-word brand whose shortened form used to be folded into it loses those mentions), and which entities sit in the Share of Voice pool at all (the SoV denominator for every run). No pin, prompt, rep count, tool version, or search depth changed. Compare only runs sharing an engineVersion and a tier. The 1.7.0/1.7.1 reference profiles are re-run on 1.8.0 before the scoreboard fine-tune reads them.

Changes:

• Target mentions are exact-match only, on both counters (SCOTT-634). The literal matcher behind queryResponses[].mentions checks URL, alias, and full name — nothing else — and the Share of Voice path credits the target's trackedEntities[] row only for extracted entities whose canonical key equals the brand's name or an alias. The token-subset absorption added on 2026-08-06 (a discovered "Wayan" row folded into a tracked "Wayan Miami" brand via the stored distinctiveBrandTokens) is removed; the stored tokens now serve intake keyphrase hygiene only. Replaying run 87dc48b9 (brand "Hey AI, it's Me", stored tokens ["hey","ai","me"]) showed why: no response named the brand in any form, and the two target mentions on the SoV path were the extractor returning the bare word "AI", absorbed into the target. Founder ruling 2026-09-16: there is not enough data to tune a heuristic, and missing a mention is better than over-counting one. A shortened brand name is an alias, entered on the brand, not a rule the engine infers. Accepted cost: a response that says only "Wayan" counts 0 for "Wayan Miami" on both paths until the alias exists, and a discovered "Wayan" row reaches the classifier like any other candidate.
• Answering assistants excluded from discovery (SCOTT-635). The assistants that answer the scored queries — ChatGPT, Claude, Gemini, Perplexity (seeded from the model registry's display names) plus Copilot / Microsoft Copilot, AI Overviews / Google AI Overviews, and Perplexity AI — are dropped from the entity candidate list before the competitor classifier runs, so they no longer appear in trackedEntities[] or the SoV denominator. Matching is exact on the canonical key of the product name: "Claude Code", "ChatGPT Enterprise", and the companies (OpenAI, Anthropic, Microsoft) remain discoverable. No prompt text changed (prompt hashes unchanged).

Comparability statement. Scores carry across this boundary. No pin, prompt, rep count, tool version, or search depth changed; this version records a change to failure handling only, so a 1.7.0 and a 1.7.1 run of the same tier are directly comparable.

Changes:

• In-process retry loop removed. The engine's own withRetry (3 attempts, exponential or 30/60/120/180 s overload backoff, per-loop wall-clock budgets) is gone from all four provider runners and from the Haiku analysis calls (sentiment, entity and probe extraction, competitor classifier, keyphrase suggester, distinctive-token derivation). Retries now happen in one place — the Inngest job layer, one retry per provider-call step from a fresh process — with the AI SDK's client retry still absorbing momentary HTTP blips inside a call. The parent coordinator's classify-competitors step is the one exception: it runs once and degrades to tracked-only on failure (see Retry and timeout policy). See Retry and timeout policy.
• Why. The loop's 429/5xx/network branch was dead: the AI SDK wraps those failures in a RetryError that carries no status code, so only the overload keyword match ever fired. And a timed-out call was labelled transient but never retried. One layer with one rule removes both contradictions, and the worst case for a single job falls from ~281 s to ~200 s (one 180 s attempt plus the ≤20 s sentiment/entity tail).
• Inert step cap removed. stopWhen: stepCountIs(3) on the OpenAI, Anthropic, and Google runners — grounding runs server-side, so every call was already a single step (the SDK default). Perplexity's max_steps stays: the Agent API defaults to 1 when the field is omitted, so the consumer tier's 3 is a real setting and the tier's documented depth axis.
• Dead fallback pin removed from the Google consumer slot (decision log 2026-09-09: a withdrawn pin fails loudly and is re-ratified — no silent model swap). Nothing ever read the field.
• not_found error class added on 2026-09-15 (no engine bump — failure classification is not part of the score). classifyError maps HTTP 404 and "not found" / "does not exist" messages to not_found instead of unknown, so a withdrawn model pin fails the run on the first pass rather than backing off for ~15h (SCOTT-655; see User-triggered resume).

Comparability statement. Scores do not carry across this boundary on the Anthropic consumer leg. Pins, prompts, reps, and provider set are unchanged; what changed is the web-search tool version the Anthropic consumer runner requests, and with it the number of citations a Sonnet response returns. citationScore, perProviderCitations.anthropic, and the citation row of dimensionBreakdown may all rise at 1.7.0 with no change in a brand's actual visibility. The free tier is unaffected (it was already on this tool version). Compare only runs sharing an engineVersion and a tier.

What changed and why. Since the Sonnet 5 pin (2026-08-28) the consumer runner requested web_search_20260209. On Sonnet 5 that tool version defaults to programmatic tool calling: the model opens a code-execution sandbox and runs the search from inside it. Measured on the scoring prompt, every sampled call stalled ~97s inside that sandbox before the first result, made 5–6 sandbox round-trips, consumed 32–42K input tokens, and returned 8–9 sources, for 123–220s per call. That stall is what produced every Anthropic transient timeout on the 2026-09-08/09 production runs; no client-side timeout value could have fixed it. The runner now requests web_search_20250305 on both tiers: one direct search, 15–20s, ~14K input tokens, 23–27 sources on the same prompt. This is also the closer mirror of claude.ai, which does not route search through a sandbox.

Changes:

• Anthropic consumer web-search tool web_search_20260209 → web_search_20250305 (budget tier already there). Source shape is identical through one SDK code path; direct search additionally emits inline result-location citations the sandbox path never produced, hence the citation-count shift.
• Anthropic consumer cost per call falls from ~$0.087 to ~$0.043 in modeled tokens (the $0.010 per-search fee, not modeled in pricing.ts for Anthropic, is unchanged). No rate changed.
• Provider call timeout raised 45s → 90s (PROVIDER_CALL_TIMEOUT_MS, all four runners) on 2026-09-09 ahead of this fix. It is a hang guard, ~2x the post-fix observed maximum, not a tuning knob.
• Provider call timeout raised again 90s → 180s on 2026-09-11 (no engine bump — timeouts are not part of the score). The two 1.7.0 reference runs completed clean (272 consumer calls) but their legitimate tail reached 85.4s (OpenAI multi-search) / 72.3s (Gemini reasoning) / 72.0s (Perplexity), ~1.05x the guard; 180s is 2x the observed maximum.

Comparability statement. Scores do not carry across this boundary on the OpenAI, Google, or Perplexity legs, on either tier. Six of the eight registry slots changed model or API surface, so a brand's overallScore, perProviderScores.openai, perProviderScores.google, and perProviderScores.perplexity may all move at 1.6.0 with no change whatsoever in its actual AI visibility. Only the Anthropic pins are unchanged, and only that leg remains comparable. Compare only runs sharing an engineVersion and a tier; the per-run modelVersions stamp is the record of exactly which models produced any given score.

The Perplexity leg deserves its own warning. Pre-1.6.0, free and consumer Perplexity runs used two different models (sonar and sonar-pro); they now use the same model at two search depths. That removes the model confound but does not make the two tiers comparable, because the depth split lands directly on what the Perplexity leg measures: consumer retrieves up to 15 results per call and free up to 5, and citationScore, perProviderCitations, and the citation row of dimensionBreakdown all count branded URLs among the sources a response returns. A deeper-searching tier gets more chances to surface a brand's URL from the same underlying visibility. Free- and consumer-tier Perplexity scores are still not comparable to each other — the axis moved from model to retrieval depth. And neither is comparable to anything stamped before this release.

Why the pins moved. This release states the pin policy explicitly for the first time: pins track the model each consumer app actually serves by default, not the model that is most stable to pin. The consumer tier mirrors a paid ChatGPT/Claude/Gemini/Perplexity session; the free tier mirrors a free one. A frozen pin would hold scores steady while making them progressively less true, so comparability is carried by the stamp instead of by pin immobility. Full reasoning in Pinned models.

Changes:

• OpenAI consumer gpt-5.5-2026-04-23 → gpt-5.6-sol, the default on paid ChatGPT Plus/Pro. GPT-6 Astra shipped 2026-09-03 but is opt-in and usage-capped rather than any plan's default, so it is deliberately not pinned.
• OpenAI budget gpt-5.4-mini-2026-03-17 → gpt-5.6-luna, the free-chatgpt.com default since 2026-08-06.
• Entity extraction gpt-5.4-nano-2026-03-17 → gpt-5.6-luna, now sourced from the OpenAI budget slot rather than repeated as a literal. The 5.6 family has no separate nano class, so the cheapest available model and the budget pin are the same model; the collapse is deliberate and structural, so a future budget re-pin carries extraction with it.
• Google consumer gemini-2.5-pro → gemini-3.1-pro-preview, the only Pro-class model in the 3.x line. Preview grade is Google's constraint — there is no GA or dated Pro variant to pin. This slot also gains a named fallback (gemini-3.8-flash), recorded in the registry but not yet wired into the runners; deliberately not the budget pin, since falling back onto it would silently collapse the two tiers onto one model.
• Google budget gemini-3.7-flash → gemini-3.6-flash, the free Gemini app default (GA).
• Anthropic pins unchanged — claude-sonnet-5 / claude-haiku-4-5-20251001 were re-verified as still-current defaults.
• Perplexity migrated to the Agent API (POST /v1/agent), replacing the Sonar chat-completions endpoint that Perplexity retires 2026-09-27. Both tiers now pin perplexity/sonar — sonar-pro does not exist on the new API, and the presets that would otherwise stand in for a pro tier (fast/low/medium/high/xhigh) resolve to third-party OpenAI models, so pinning one would publish an OpenAI score under Perplexity's name. See the next bullet for what replaces the tier split, and Pinned models for the full doctrine.
• Perplexity tiers now differ on search depth, not model. Consumer buys 15 results at high page context over 3 research steps; free buys 5 results at low context over 1 step. This is a deliberate choice rather than a consolation prize: retrieval depth is the dimension Perplexity actually sells (its consumer product is an unpinnable router whose paid tier buys more search, not a different LLM), so depth is what a representative tier split should track. Free-tier search is never disabled — an unsearched Perplexity answer has no citations, and citations are most of what the Perplexity leg measures.
• Perplexity search is now forced, not merely offered. Verification on 2026-09-08 found the Agent API's default tool_choice: "auto" letting the model skip search entirely and return a fluent, wholly ungrounded answer with no citations — which would have scored as a real Perplexity response with zero sources. Both tiers now send tool_choice: {type: "web_search"}.
• Perplexity citations moved channel. They arrive inside the response's output[] array as a search_results item, not through the AI SDK's normalized sources channel (the runner no longer uses an SDK at all). The assistant message's own annotations array is empty on live responses, so reading citations from it — the obvious-looking choice — yields none. This is the third Perplexity citation-channel change; see 1.1.0 for the first.
• Perplexity now stamps a response ID. The Agent API returns a canonical resp_… id, so providerResponseIds is no longer sparse on this provider.
• Perplexity RATE corrected as part of the migration: consumer $3.00/$15.00 and free $1.00/$1.00 → $0.25/$2.50 per 1M on both tiers, plus a flat $0.0025 per forced web_search invocation. Verified against live billing on 2026-09-08 (a 1,415-token input billed $0.00035; a 550-token output billed $0.00138). Neither old line described an Agent API call. pricing.ts gained a toolFeePerCallUsd field to carry the invocation fee, which no other provider uses — token-only pricing understated every Perplexity cost event by roughly the fee.
• Anthropic consumer RATE corrected (pin untouched): $3.00/$15.00 → $2.00/$10.00. The 2026-08-28 refresh encoded Sonnet 5's list rate and recorded the promotional rate as expiring 2026-08-31. Verification against Anthropic's live pricing and platform pages on 2026-09-07 found the promotional rate still in force with no announced end date and no list figure published — the expiry note was simply wrong, and we had been over-stating every Sonnet cost event since 2026-08-28 by 50%. Scores are unaffected; cost events and per-run totals on the Anthropic consumer leg are not comparable across this boundary. Caught by the weekly LiteLLM drift cron, which had been flagging this slot correctly.

Pricing policy reversed. 1.5.1 encoded list rates and deliberately ignored promotional ones. 1.6.0 encodes the rate we are actually billed, promotional or not, because cost events exist to make spend true and a list rate we are not paying overstates every one of them. The Anthropic correction above is the clearest argument for the change: under the old policy we knowingly billed ourselves 50% over actual on that leg for ten days, and the drift cron's weekly objection was being closed unmerged by design. Three promotional rates are now live: OpenAI Sol at $4.00/$20.00 per 1M (guaranteed no higher than 2026-11-21), Anthropic Sonnet 5 at $2.00/$10.00 (no announced end date), and Gemini Flash at $0.75/$3.75 (doubling 2027-01-01). Both are scheduled edits, not set-and-forget. Google's gemini-3.1-pro-preview is encoded at its flat ≤200k rate ($2.00/$12.00); its >200k tier ($4.00/$18.00) is a documented approximation we never reach at per-query prompt sizes. Consequently the standing instruction to close the Google drift-cron PR without merging is retired — our Flash rate and LiteLLM's should now agree, so a drift PR on that slot is real signal.

• Free-tier projection recalibrated: ~$1.34 → ~$0.94 per run, the first decrease in the series and entirely a rate effect — token profiles are unchanged. The Perplexity migration then lifted the nominal to ~$1.02, and the asserted band was recentered on it: $0.80–$1.08 → $0.87–$1.18, still ±15%. The cost-projection test asserts the new band (see the band note under the projection table).
• Consumer-tier ballpark re-derived: $7.55 measured → ~$5.35 (rough range $4–7), by re-pricing the 2026-08-28 consumer validation run 87dc48b9 at current rates. gpt-5.6-sol is now the dominant line at ~half the run; Sonnet, long the driver, is ~$0.83. No test asserts this figure.
• Downstream cost docs re-derived off these numbers: docs/product/cost-model-scenario-D.md (c_free $0.91 → $0.72 at 3 reps, c_normal $5.00 → $5.35, funnel-loaded COGS $24.20 → ~$20.75) and the per-run figures in POSITIONING.md, HOW-PRESENTATION.md, LAUNCH-PUNCHLIST.md, FRONTEND-REBUILD.md.

Resume pin-drift guard fixed (two bugs). assertNoPinDrift compared every stamped run against .consumer pins regardless of the tier it ran on. For free-tier runs this was wrong in both directions: it raised a false PinDriftError on every clean free partial (budget stamps never equal consumer pins), locking resumable free runs out of resuming at all, and it waved through genuine budget-pin drift whenever a stamp happened to match the consumer pin. Separately, the resume path rebuilt its input without carrying tier, so a resumed free-tier run silently re-ran the retried providers on consumer models and stamped consumer pins over a result whose surviving responses came from budget models — one runId carrying two tiers, with nothing raising an error. Both are fixed and covered by tests.

Comparability statement. No scoring-behavior change — prompts, rubric, model pins, reps, and provider set are all unchanged. What changes is cost accounting: the Google budget leg (gemini-3.7-flash) was billed at the carried-over gemini-2.5-flash rates ($0.30/$2.50 per 1M in/out) between the 2026-08-28 pin refresh and this release, under-billing that leg 5x on input and 3x on output. Cost events and per-run totals are not comparable across this boundary on the Google leg; scores are.

Changes:

• Google budget pricing re-pinned to the verified gemini-3.7-flash list rate: $1.50/$7.50 per 1M in/out (ai.google.dev). Google's intro rate ($0.75/$3.75, through 2026-12-31) is deliberately not encoded — we pin list rates (sonnet-5 precedent). Note: LiteLLM carries the intro rate, so the weekly drift cron will disagree with this pin until LiteLLM flips to list.
• Free-tier projection recalibrated: ~$1.07 → ~$1.34 per run (band $1.10–$1.55); the cost-projection test asserts the new band.

Comparability statement. overallScore, citationScore, and perProviderScores are unchanged at this boundary — positioning probes are a diagnostic surface, never a score input, and the scoring/sentiment prompt bytes did not move. What changes is the probe diagnostic content: pre-1.5.0 runs carry competitor names only; 1.5.0+ runs additionally carry a one-sentence per-competitor reason. Probe outputs across the boundary describe the same brands but are not byte-comparable (different probe prompt → different response distribution).

Changes:

• Competitor probe prompts reworded (WEAKER_COMPETITORS_PROBE_TEMPLATE, STRONGER_COMPETITORS_PROBE_TEMPLATE): still ask for three competitor names, now also ask for one short sentence per name explaining why it is weaker/stronger (previously "List names only."). Hash diffs on both rows.
• Competitor extraction templates reworded (WEAKER_COMPETITORS_EXTRACTION_TEMPLATE, STRONGER_COMPETITORS_EXTRACTION_TEMPLATE): extract { name, reason } pairs instead of bare names. The extracted competitor shape gains an optional reason field inside each probe rep's extracted bag — additive; pre-1.5.0 reps parse and render unchanged (votes-only).
• Extraction timeout raised 12s → 20s per Haiku extraction call — reason sentences lengthen completions.
• KNOWN_FOR_* templates, scoring rubric, sentiment prompt, model pins, reps, and provider set unchanged.

Comparability statement. Post-refresh runs are not directly comparable with pre-refresh runs on the Anthropic leg (consumer tier) — the Anthropic flagship pin moved claude-sonnet-4-6 → claude-sonnet-5 — or on the Google leg (free/budget tier) — the Google budget pin moved gemini-2.5-flash → gemini-3.7-flash. Per-provider scores on those legs can move across this boundary for reasons that are model-behavior change, not brand-visibility change. Prompts, scoring rubric, reps, and the other pins are unchanged. No ENGINE_VERSION bump: pins are soft pre-launch, and the per-run modelVersions stamp records exactly which pins produced each result — that stamp, not the engine version, is the boundary marker for this refresh.

Changes (live-verified against provider docs 2026-08-28):

• Anthropic flagship pin: claude-sonnet-4-6 → claude-sonnet-5. claude-sonnet-5 is the complete ID — Anthropic publishes no dated snapshot for Sonnet 5. List pricing unchanged at $3.00/$15.00 per 1M in/out (an intro rate of $2/$10 runs through 2026-08-31; we pin list rates).
• Google budget pin: gemini-2.5-flash → gemini-3.7-flash (latest stable GA Flash). Pricing was not verified at refresh time — pricing.ts carried the 2.5-flash rates ($0.30/$2.50) forward until engine 1.5.1 (same day) re-pinned the verified list rate $1.50/$7.50; see the 1.5.1 entry above.

Deliberately kept (verified, not stale):

• OpenAI flagship stays gpt-5.5-2026-04-23 — the GPT-5.6 family shipped, but OpenAI's docs still recommend 5.5 for production and publish no dated 5.6 snapshots; ambiguous, so the current pin holds.
• OpenAI budget/entity stay gpt-5.4-mini-2026-03-17 / gpt-5.4-nano-2026-03-17 — no 5.5-class mini/nano exist.
• Google flagship stays gemini-2.5-pro — Gemini 3.1 Pro is preview-only; 2.5 Pro remains the latest stable Pro.
• Anthropic budget / sentiment classifier stays claude-haiku-4-5-20251001 — Haiku 4.5 is still the latest Haiku.
• Perplexity stays sonar-pro / sonar — note: Perplexity retires the Sonar endpoints 2026-09-27 in favor of its Agent API; that is an API-surface migration tracked separately, not a pin choice.

Comparability statement. Pre-boundary result blobs are not directly comparable with post-boundary blobs for trackedEntities[] discovered-row membership and any Share of Voice math derived from it. Before this change, the classifier prompt offered no "this isn't a business" outcome, so dishes, procedures, and abstract concepts (observed in production: Botox, CoolSculpting, satay, Rendang) were FORCED into a competitor role and admitted as discovered-direct or discovered-adjacent rows. After this change, those rows are rejected before the tracked-entities pool. A brand whose prior run picked up non-business "competitors" will see its discovered-entity count (and any SoV denominator built from it) shrink at this boundary — the new number is the more defensible measure, not a brand-visibility change. Scoring rubric, model pins, and every other prompt template are unchanged.

• 2026-08-06 (instrument fixes, Plan A): CLASSIFIER_PROMPT_TEMPLATE gains a reject outcome — isBusinessEntity distinguishes actual businesses from dishes/procedures/concepts, which previously were FORCED into a competitor role (the prompt offered no "not an entity" option; observed: Botox, CoolSculpting, satay, Rendang admitted as discovered-direct competitors). Rejected rows are dropped before the tracked-entities pool.

Comparability statement. Pre-boundary and post-boundary keyphrase sets are not directly comparable for self-serve and admin-suggested runs. KEYPHRASE_SUGGESTER_TEMPLATE now asks for 8 candidate keyphrases instead of 5, before the post-filter .slice(0, 5) cap still returns 5 — this is a different LLM generation, not just "more room to filter from the same list," so a post-boundary run against the identical brand/businessType can receive a different keyphrase set than a pre-boundary run would have, independent of any brand-contamination filtering. Separately, the new distinctive-token filter (backed by distinctiveBrandTokens) can now reject a keyphrase — or the whole /api/runs/start request, 400 keyphrase_contains_brand — that a pre-boundary run would have accepted verbatim: a submitted keyphrase list containing a token-split leak of the brand name (e.g. "where to eat traditional Wayan dishes in Miami") now fails intake instead of entering the scoring pipeline unfiltered. Scoring rubric and every other prompt template are unchanged.

• 2026-08-06 (instrument fixes, Plan A): KEYPHRASE_SUGGESTER_TEMPLATE revised — requests 8 candidate keyphrases (post-filter still returns ≤5; the Rules bullet's count reference was synced from "across the 5" to "across the 8" alongside item 1) and additionally returns distinctiveBrandTokens (brand identity tokens, excluding location/category/generic words) used by the keyphrase-contamination filter. New DISTINCTIVE_TOKENS_TEMPLATE registered (PROMPT_HASHES.distinctiveTokens) for intake paths that skip the suggester. Rationale: the prior whole-string brand filter let every multi-word brand leak token-split contamination ("where to eat traditional Wayan dishes in Miami").

Comparability statement. Pre-1.4.0 result blobs are not directly comparable with 1.4.0 blobs for the OpenAI leg (consumer tier). The OpenAI flagship pin moved gpt-5.4-2026-03-05 → gpt-5.5-2026-04-23, so per-provider OpenAI scores (and any aggregate they feed) can move across this boundary for reasons that are model-behavior change, not brand-visibility change. Prompts, scoring rubric, reps, and the other three provider pins are unchanged — Anthropic/Google/Perplexity per-provider scores remain comparable across the boundary.

• OpenAI flagship pin: gpt-5.4-2026-03-05 → gpt-5.5-2026-04-23 (dated snapshot, same reproducibility policy). Consumer-tier OpenAI pricing moves with the model: $2.50/$10.00 → $5.00/$30.00 per 1M in/out tokens.
• Entity-extraction model: gpt-4o-mini-2024-07-18 → gpt-5.4-nano-2026-03-17 (gpt-4o-mini announced shutdown 2026-10-23). Engine-internal surface — extraction output shape and prompts unchanged; per-call rate now $0.20/$1.25 per 1M in/out.
• Budget-tier pricing sync (2026-06-01 LiteLLM): Google Flash output $0.30 → $2.50 per 1M (8×), OpenAI gpt-5.4-mini output $3.00 → $4.50 per 1M (+50%). Pins unchanged — rates only.
• Free-tier per-run projection recalibrated ~$0.85 ± $0.13 → ~$1.07 (band $0.90–$1.25), driven mostly by the pricing sync rather than the pin changes. The band is asserted in CI (__tests__/cost-projection.test.ts).

Comparability statement. Pre-1.3.0 result blobs are not directly comparable with 1.3.0 result blobs for Share of Voice. Engine 1.2.0 (and earlier) computed Share of Voice from a points-weighted aggregate over summary.perKeyphrase.byBrand, which only included the input Brand[] list (target-only in production). Engine 1.3.0 computes Share of Voice as entity.mentionCount / sum(all trackedEntities[].mentionCount) over a unified pool that includes the target, every specified competitor, and every LLM-discovered direct or adjacent entity. The math, the inputs, and the units differ. Other dimensions (Frequency, Position, Sentiment, Citations) are unchanged at this boundary — pre-1.3.0 vs 1.3.0 scores for those dimensions remain comparable.

Schema changes (clean rename — no backward-compat aliases):

• Renamed classifiedCompetitors[] → trackedEntities[] on ScoringRunResultSchema. The old field's isDirectCompetitor: boolean is replaced by a role: z.enum(['target', 'specified-competitor', 'discovered-direct', 'discovered-adjacent']) enum on the renamed schema.
• Deleted the shareOfVoiceAggregate result slot, its Zod schema, the orchestrator IIFE that populated it, and every downstream consumer. The new derivation helper (apps/web/lib/results/tracked-entities-share.ts) reads trackedEntities[] directly; pre-1.3.0 runs that lack trackedEntities[] render the empty-state ("0% — not mentioned this run") rather than back-compat-translating the legacy aggregate.
• Engine version bumped 1.2.0 → 1.3.0. Every fresh run stamps result.engineVersion === "1.3.0". Verified live in free-tier production smoke prior to ship.

Engine mechanics:

• Target brand is no longer excluded at entity extraction. The per-response extractEntities call receives excludedBrandNames: [] so the target appears in the aggregated set alongside discovered entities.
• New classifyEntities (in packages/scoring-engine/src/analysis/competitor-classifier.ts) partitions aggregated rows into tracked (target + specified competitors, stamped directly from Brand[]) vs. discovered (everything else, sent to the Haiku 4.5 classifier for direct-vs-adjacent labeling). Only discovered rows hit the LLM.
• trackedEntities[] is capped at 30 with tracked rows never evicted — see Top-30 cap with eviction protection.
• Brand[] canonical-key collisions are rejected at construction by validateBrandsOrThrow (packages/scoring-engine/src/analysis/validate-brands.ts).

Why this is a comparability boundary. Two changes compound at the SoV math: (a) the target row is now in the denominator alongside every other entity; (b) the denominator itself grew from Brand[].length (typically 1 for target-only) to up to 30 trackedEntities. A historical 1.2.0 run that reported "100% target SoV" against a target-only Brand[] will, on 1.3.0 schema, show the target as one row in a pool of up to 30 — its SoV percentage drops to a fraction matching its actual share of cross-entity mentions. The new number is the more defensible measure; the old number was a degenerate artifact of a target-only denominator. Documented here so any reader auditing a pre-1.3.0 vs 1.3.0 SoV delta knows the boundary is methodology, not brand-visibility change.

What did NOT change. Score formula, prompts, scoring rubric, prompt hashes, the per-brand sentiment classifier, the partial-state contract, the resume mechanism, the model pins. The four-axis headline trust contract (Mention reach, List position, Per-brand sentiment, Citation reach) is unchanged.

Comparability statement. Pre-1.2.0 result blobs ARE directly comparable with 1.2.0 result blobs in terms of score math — hash values are unchanged. Only stamp field names changed. A historical 1.1.0 run and a fresh 1.2.0 run that scored the same brand against the same keyphrase set will produce the same overallScore, the same keyphraseSetHash value, and the same promptHash.

What changed (field names only — values preserved):

• ScoringRunInput.keywords → ScoringRunInput.keyphrases
• ScoringRunResult.keywords → ScoringRunResult.keyphrases (self-sufficiency stamp)
• ScoringRunResult.keywordSetHash → ScoringRunResult.keyphraseSetHash
• QueryResponse.keyword → QueryResponse.keyphrase
• analytics."run.requested".properties.keywordCount → keyphraseCount
• Engine API: hashKeywordSet → hashKeyphraseSet, classifyKeyword → classifyKeyphrase, KeywordSummary → KeyphraseSummary, KeywordBrandSummary → KeyphraseBrandSummary, KeywordIntent → KeyphraseIntent, REPS_PER_KEYWORD → REPS_PER_KEYPHRASE, file src/keywords.ts → src/keyphrases.ts
• DB column scoring_run_responses.keyword → keyphrase; scoring_runs.keyword_set_hash → keyphrase_set_hash. UNIQUE INDEX uniq_scoring_run_responses_tuple preserved via PG RENAME COLUMN (index is auto-updated, no drop-recreate)
• Public API: POST /api/runs/start body field keywords → keyphrases

Historical row backfill. Drizzle migration 0009_keyword_to_keyphrase_rename.sql rewrites jsonb keys on all existing scoring_runs.result, scoring_runs.input, and scoring_runs.partial_result blobs so historical rows carry the new field names directly. No backward-compat Zod alias is in place — pre-1.2.0 result shapes must be backfilled to parse against 1.2.0+ schemas.

Why this is in the changelog despite preserving math. The reproducibility contract requires every engineVersion bump to be documented. A reader auditing a 1.1.0 run against a 1.2.0 run needs to know that the stamp-field names are the only difference. Score values, hash values, and prompt bytes are unchanged at this boundary.

Origin: quick-260513-kfj (commits 01f07af … e3f3c0c). Driver: project-wide terminology pin (project_terminology_keyphrase memory). UI surface had already shifted to "keyphrase" exclusively via the keyphrase-not-keyword-in-ui.test.ts guard test; this entry brings code identifiers, schema field names, and the public API contract into the same shape.

Three observable changes to scoring-run output. None alter the headline overallScore math; all change what we record on each response's sources[] slice.

• Google source URLs are now bare publisher domains (e.g. "zapier.com"), not Vertex grounding-redirector wrappers (vertexaisearch.cloud.google.com/grounding-api-redirect/...). The redirector wrapper has unverified TTL and adds no recoverable signal once it expires, so we discard it at the runner boundary. Pre-1.1.0 Google source rows that stored the wrapper remain in storage unchanged.
• Perplexity citations are now populated. From 1.0.0 through pre-1.1.0, Perplexity's source URLs were silently dropped — the runner read result.providerMetadata.perplexity.citations which is undefined under @ai-sdk/perplexity@3.0.23 + ai@6.0.116. The fix switches to result.sources[] (the AI SDK's normalized channel). Empirically verified against a live capture on 2026-05-12.
• OpenAI and Anthropic source titles are now retained. The runners previously discarded the article/page title from each source; 1.1.0 preserves it under sources[i].title for downstream display + provenance.

See .planning/decisions/2026-05-12-citation-source-schema.md for the empirical evidence, schema decision, and per-provider shape table. Captured fixtures in .planning/research/c2-citation-data/ are pinned by packages/scoring-engine/__tests__/runners.citations.fixtures.test.ts.

Comparability statement. Engine version unchanged at 1.0.0 — this entry is additive within 1.0.0. The model pins did not move. What changed is how the engine selects between two pre-existing pinned tiers: free vs. consumer.

Changes:

• MODEL_REGISTRY_API published as a first-class tier alongside MODEL_REGISTRY_CONSUMER. Both registries were already in the codebase (the budget-tier pins were "reserved — not used in any scoring run today" pre-Phase 3); this release activates MODEL_REGISTRY_API for the free-tier product and publishes both pin tables side-by-side under Pinned models.
• ScoringRunInput.tier field added — z.enum(["free","consumer"]).default("consumer"). Additive default preserves the engine 1.0.0 baseline; pre-tier fixtures still parse without modification. The Phase 3 Inngest worker reads users.tier from the DB and stamps it onto the input before calling engine.run(). Engine remains stateless re: persistence.
• Engine selects MODEL_REGISTRY[provider][tier] at every access site — orchestrator's emitProviderCost (model + modelVersion stamp on every LLMCostEvent), the modelVersions field on ScoringRunResult, and main-loop's ScoredResponse.modelName. Sentiment classifier untouched: always Haiku regardless of run tier.
• ScoringRunResult.tier stamp added — additive optional on the result schema. Records the tier the engine ran so cross-tier comparison can be hard-blocked at consumer call sites without re-deriving from modelVersions.
• Tier Comparability Caveat published under Pinned models. Cross-tier comparison silently invalidates the trust contract; the tier stamp is the structural guard.

What did NOT change: the score formula, the prompts, the scoring rubric, the hashes, the per-brand sentiment classifier (still Haiku, still F-2), the partial-state contract, the resume mechanism. A consumer-tier 1.0.0 run from before this release and a consumer-tier 1.0.0 run from after this release ARE directly comparable. Free-tier results were not produced before this release, so no comparability boundary applies there.

Comparability statement. Pre-1.0.0 POC scores are not comparable with 1.0.0 scores. The POC engine (0.1.0) was never shipped to real users; 1.0.0 is the first baseline against which scores should be compared going forward. Any historical 0.1.0 fixture or score is invalidated at this boundary — comparability resets.

Why a single big-bang version. Four legs of the comparability contract change in this release simultaneously: engine version, model registry shape, prompt template bytes, and pricing rates. A chain of compatible-ish minor bumps would have been more confusing than one hard cut. The 0.1.0 → 1.0.0 boundary makes the reset explicit.

Changes:

• Model pins refreshed (consumer tier — what we actually score):
  • OpenAI: gpt-4o-2024-11-20 → gpt-5.4-2026-03-05
  • Anthropic: claude-sonnet-4-5-20251001 → claude-sonnet-4-6 (alias only — SDK exposes no dated form for Sonnet 4.6 yet)
  • Google: gemini-2.5-pro (unchanged version string; new comparability epoch all the same)
  • Perplexity: sonar-pro (unchanged version string; new comparability epoch all the same)
• Sentiment classifier: stays at claude-haiku-4-5-20251001. A swap to gpt-5.4-nano / gpt-4o-mini was evaluated; without LIVE_LLM access at lock-in time the bake-off acceptance threshold (≥accuracy − 5pp) could not be empirically verified, so the locked rule "if no candidate passes: keep Haiku" applied.
• Budget-tier pins added to MODEL_REGISTRY for all four providers (gpt-5.4-mini-2026-03-17, claude-haiku-4-5-20251001, gemini-2.5-flash, sonar). Reserved for future free-tier product — not used in any scoring run today.
• MODEL_REGISTRY shape changed from flat MODEL_REGISTRY[provider] to nested MODEL_REGISTRY[provider][tier] (.consumer / .budget). Every callsite reads .consumer for v1 scoring; budget tier is a separately-keyed reserved future surface.
• Pass 5 — Citation reach (NEW). Independent scoring axis for the target brand. Computed from per-response source URLs (webSearch / groundingMetadata / providerMetadata.perplexity.citations). Adds two stamp fields: citationScore (target-brand reach × 100 / totalResponses, capped at 100) and perProviderCitations (full per-provider citation list with brandAppears flags including competitor URL context).
• Google fan-out capture (NEW, capture-only). webSearchQueries from providerMetadata.google.groundingMetadata are stamped under fanOutQueries.google. Non-Google providers have no key in the record at all — structural absence, not empty array. Capture-only by design; coverage is not scored in v1.
• Per-brand sentiment (F-2). Sentiment classifier now runs once per (response × distinct mentioned brand), replacing the legacy once-per-response/target-only approach. Repeated mentions of the same brand within one response collapse to one label. The F-1 alternative (single multi-brand prompt) was evaluated but did not meet the locked acceptance criteria (≥90% per-brand label agreement with F-2, no cross-polarity flips, F-1 ≥85% stability over 3 reruns) without LIVE_LLM access at lock-in. Cost-event count rises ~47% per run as a result; deterministic upper bound preserved.
• perProviderScores transparency surface (NEW). Every result publishes per-provider sub-scores ({ score, sampleN }) with uniform weights only. Architectural boundary: non-uniform per-platform weighting is never permitted in the core engine — it is a downstream read-time projection (future pro feature).
• Pass-4 rubric extended to slots 5–8 with diminishing weights: ordered 3/2/2/1, unordered 4/2/2/1. Previous rubric capped meaningful contribution at rank 4. Source: AI Overview slot research (Surfer median 5 / AWR 7.2 / Authoritas 8.2 cited URLs per AIO).
• Schema additivity (Pitfall 4 contract). All four new ScoringRunResultSchema fields (perProviderScores, citationScore, perProviderCitations, fanOutQueries) are .optional(). Existing pre-1.0.0 fixtures still parse cleanly through the extended schema; downstream consumers reading 1.0.0 results without the new keys see the legacy shape.
• New prompt hashes (engine 1.0.0):
  • SCORING_PROMPT_TEMPLATE = 4b967dc95fca4dede22522d92a51d13b023c19f226eca8ef241d55a07f45e8da
  • SENTIMENT_PROMPT_TEMPLATE = 41804e2b6171e339e74790ad45a29096d0f255efdd62b52fb6c3c6eb9682c988 (unchanged from 0.1.0 — the rewrite proposal was rejected, F-2 ships with the original prompt bytes)

Documentation:

• The pre-1.0 draft banner was removed. The doc is now audience-ready; engine 1.0.0 scores are the first numbers we expect to publish to real users.
• The private internal synthesis informing these decisions ("Methodology Bible" — Obsidian vault, 96 article summaries plus tool deep-dives) is not republished here. This document is the audit-ready public surface; the Bible is the working notebook behind it.

• Ported the AEO-Project POC pipeline into packages/scoring-engine. No ongoing dependency on the source repo.
• Pinned all four providers to exact dated API version strings (no aliases like gpt-4o that route to provider-chosen latest). Pins: gpt-4o-2024-11-20, claude-sonnet-4-5-20251001, gemini-2.5-pro, sonar-pro. Sentiment classifier pinned to claude-haiku-4-5-20251001.
• Removed the "no brands mentioned" pseudo-competitor classifier that was in the POC. It was mis-classifying responses that named secondary competitors as though no brand appeared at all, dragging down legitimate competitor scores. Fuzzy-matched secondary-competitor counting (handles "Engage Summits" ↔ engagesummits.example equivalence) replaces the signal; empty-mention responses now earn zero points for all brands.
• Added per-LLM-call LLMCostEvent emission covering both the query call and the sentiment classifier call. Previously the POC accumulated cost internally; externalizing it enables hard spend caps and per-variant cost dashboards downstream.
• Added SHA-256 hashing of both prompt templates. Every run is stamped with the prompt hash, and the hashes are published for independent verification.
• Added optional provider response ID stamping (Perplexity does not expose one).
• Removed the POC's hardcoded Kindred Collective brand data, the brand-discovery.ts GPT-4o-mini extraction pipeline, the writeSnapshot file I/O, and the top-of-module dotenv/config import — all either reassigned to a later phase or replaced by the monorepo's lazy-env pattern.

The 0.1.0 engine was never shipped to real users — it was the audit-first POC migration baseline. All 0.1.0 fixtures and scores are invalidated by the 1.0.0 boundary.