# GEO Lab 03: response to the external review

This revision evaluates the whole supplied review, regardless of its priority labels. It adds no model queries and preserves the 24 complete answers, all 27 prompt submissions, technical failures and original strict coding. New calculations are explicitly post-hoc. The earlier report and analysis are archived locally; the public repository retains the preceding release in version history.

## Itemized decisions

| Review recommendation | Assessment and action |
| --- | --- |
| Keep the existing experiment instead of recollecting 24 answers | Accepted. No new target-model requests or API charges. The collection design and original strict labels remain intact. |
| Distinguish consumer ChatGPT from the OpenAI API throughout | Accepted and strengthened. Titles near the results, methods, figures and environment labels identify **OpenAI API + web search** and **Grok Expert · consumer interface**. API results cannot represent the ChatGPT web app. Interface, model and later API collection time remain confounded. |
| Compare within- and cross-wording variation with the same metric | Accepted. Added six planned within-wording pairs and nine planned cross-wording pairs per intent/environment. Show pair IDs, numeric denominators, both-empty N/A and descriptive percentage-point differences. Pooled-union summaries remain a separate statistic, not the repeat baseline. |
| Verify the reviewer's numerical estimates | Recomputed from the frozen strict sets. Grok M: 35.4663% within versus 26.0303% cross, difference −9.4359 points. Grok V: 45.5303% versus 27.0282%, difference −18.5021 points. The M difference rounds to −9.4 points from unrounded values; −9.5 results from subtracting the separately rounded displayed means. No significance test or causal conclusion was added. |
| Add product/family/category normalization sensitivity | Accepted. A public, versioned 52-name mapping adds family and broad category lenses without replacing strict coding. Google Analytics and GA4 group only in the family view; other Google services stay distinct. Vendor visibility offerings such as Ahrefs are grouped only under explicitly declared family rules. |
| Separate specialized GEO products, general tools and channels | Accepted. Added four explicit categories and strict-name comparisons filtered by category. This separates a changing GEO-product list from changing analytics, recordkeeping or channel suggestions. Category membership is not itself a recommendation. Coarse grouping can increase overlap mechanically. |
| Keep the repeat-variation finding with a concrete scoring example | Accepted. The reader retains the M1 0–3 versus −1/0–4 example, clearly marked generated advice rather than a validated scoring instrument. |
| Reduce emphasis on intent predicting workflow versus audit answers | Accepted. The 6/6 M-workflow and 6/6 V-checklist pattern per environment remains descriptive context and supplementary evidence. It is expected from the tasks and is not evidence of a newly discovered retrieval mechanism. |
| Reduce cross-platform source-preference claims | Accepted. Source composition is supplementary and coverage-qualified. Three early Grok source maps do not meet the strict condition. Unknown categories and exact-href limitations remain visible; no inherent platform preference is inferred. |
| Clarify the C/M/R denominator and move it to supplementary analysis | Accepted. Renamed **C/M/R among coded named entities**. The 30 and 126 rows are entity-by-answer observations selected from named-entity inventories, all already M=true, not general brand-mention rates. Own-content C is separate from overall source-citation frequency. Full tables remain in the appendix. |
| State AI coding limitations and obtain independent human spot-checking | Partly completed, with the remaining work explicit. Assistant subjectivity and lack of independent human double-coding are prominent. An eight-case targeted packet and blank decision fields are supplied. **Zero cases have been independently reviewed by a human in this revision.** A person must provide those judgments; no assistant recheck is relabeled human validation or representative accuracy. |
| Add URL-level comparison alongside domain overlap | Accepted. The API M case has 3/3 normalized domains but only 4/9 comparison URLs in common. The URL lens removes declared tracking/session/signed parameters, preserves paths and fragments, and does not infer article identity or resolve redirects. Strict source coverage and qualified observed-link views remain separate. |
| Remove private runtime information and audit evidence-file access | Accepted as a verification requirement, not evidence of a prior public leak. The review examined a local report with private diagnostic links; the previous public edition was separately redacted and deployed. This revision retains only safe public records and repeatable analysis, strips opaque IDs, private paths, account context and sensitive URL parameters, and rechecks JSON, HTML, archives and relative links. Private originals remain local. |
| Shorten the public report and move technical records to an appendix | Accepted. A concise reader leads with three bounded findings and two core figures. The complete answers, coding, source composition, C/M/R and failures remain in a separate Technical Appendix. No evidence is deleted to make the page look cleaner. |
| Default-collapse all raw records | Adapted to the existing user agreement. The user previously requested all headings expanded. The short reader and separate appendix reduce initial information load while the appendix remains fully expanded. This addresses readability without silently changing that preference. |
| Verify rendering and relative links through the actual website | Accepted. Offline path/hash/archive checks are supplemented by actual-browser checks of the deployed reader and appendix, including narrow and desktop layouts and legacy fragment navigation. The old local-file preview restriction is not treated as a public-page failure. Deployment evidence is recorded separately. |
| Spot-check whether source links open and support the attached claims | Accepted with a bounded scope. Six purposively selected source claims were checked by the assistant: five original source URLs were readable, one unavailable through the retrieval tool. Access and claim support are separate fields. The arXiv citation only partially supports its attached sentence; the Markets source has a commercial-origin qualification. This is not a fact-check of all sources or a validation of commercial scores, prices or free-plan operation. |
| Link the experiment to a practical GEO Prompt Bank | Accepted. Added a short workflow for separating intent/variant IDs, recording budget cues, freezing exact wording and environment conditions, repeating tests, declaring normalization and preserving failures/review status. These are measurement-design implications, not experimentally proven GEO improvements. |
| Use a restrained English core summary and an explicit cannot-conclude section | Accepted in substance, with wording tied to recomputed results. The reader foregrounds sampled variation, dependent comparisons, model/interface/time confounds, conditional entity denominators and pending human review. It does not imply the earlier lessons proved GEO efficacy. |

## What remains unresolved

The original experiment is small, non-causal and does not validate a commercial visibility score. No change to layout or additional calculation removes those limits. The eight prepared human decisions still require a genuine reviewer. One selected source URL remains inaccessible through the retrieval tool; related primary-origin material is identified without pretending it verifies that exact copy. The six-source sample is purposive and supplies no corpus-wide fact-check rate.

## Reproducible supporting files

- `entity-normalization.json`: explicit strict-name, family and category decisions.
- `human-review.json` and `human-review.md`: eight pending independent-human decisions.
- `source-audit.json` and `source-audit.md`: selected URL-access and claim-support checks.
- `prompt-bank.md`: practical implications and a minimal record schema.
- `analysis_extensions.py`: deterministic matched comparisons, normalization and URL sensitivity.

> Original answers and original strict coding are evidence, not endorsed recommendations. The reader/appendix split changes presentation; it does not turn partial captures into negative results, generated examples into observations, or assistant coding into a human gold standard.
