Technical appendix

Stage 2: complete prompts, answers, source roles, judgments and collection history.

Return to the concise reader report

Original authored findings · supporting observations

  • Direct access and discovery are different outcomes. Grok retrieved body-specific facts when given the URL. Neither selected exact-excerpt nor natural-query answer returned the target X post. The complete ChatGPT C is a later manual supplement with qualified control evidence. A direct-URL pass does not show proactive discovery.
  • The direct-URL result differs across these two platform runs. Grok met the content-access criterion; ChatGPT explicitly said it could not access the post. Grok’s summary added “on the same platforms,” an unsupported qualifier that is recorded separately from access.
  • Exact quotation did not guarantee discovery. Both complete B answers reported no matching original post. The experimenter independently verified that the public post exists and matches the quote; a no-result answer does not establish that it is absent or unindexed. The supplied 86-character excerpt is an exact continuous fragment from the latter part of a sentence, not a full sentence. A Unicode-level comparison found no transcription difference; Grok also tried shorter and punctuation-free variants. ChatGPT reports variants, but its actual B search queries were not independently captured.
  • The answer dataset is complete, with one qualified manual replacement. Ten submissions cover six planned cells, now all with complete answer bodies. The user supplied C-R5; the complete native original corroborates it and supplies 11 URLs. Four earlier C technical/partial captures remain intact. Model/time are user-reported and session-isolation preflight was not independently recorded.
  • A citation can accompany an explicit exclusion. ChatGPT C cites and mentions Artur Ferreira while placing him among sources to exclude; Grok C presents him as a research candidate to read. ChatGPT’s conclusion instead prioritizes Rank Prompt, mapou, Zeeshan Yaseen and Charles Cousyn. These are observed selection differences, not evidence of GEO causality.
  • Six planned cells, 10 actual submissions, 6 complete selected answers. Four earlier ChatGPT C submissions had technical or body-capture problems before the complete manual replacement was obtained. Selection completeness does not mean the collection had no failures. These are collection counts, not retrieval success rates.
  • A · Direct URL access. ChatGPT Search: Not demonstrated; Grok: Demonstrated.
  • B · Exact-excerpt discovery. ChatGPT Search: Not demonstrated; Grok: Not demonstrated.
  • C · Natural discovery. ChatGPT Search: Not demonstrated; Grok: Not demonstrated.

Scope: The comparison contains six planned cells. Actual submissions and supplemental attempts are tracked separately below; a recovered capture is not a new query. There is no generalizable success rate. A supplied URL, a repeated excerpt, or facts recovered from GitHub do not demonstrate access to the X post.

Results below display recorded reviewer judgments. The generator does not classify answers from keywords, links, or a model's claims of verification. Missing assessments remain explicit.

Redacted public edition: Questions, observed outcomes, returned source URLs, and failure history are preserved. Private conversation and share links, screenshots, account-interface material, local paths, and original diagnostic captures are excluded. Complete originals remain local. Captured non-English UI tokens are preserved as Unicode escape sequences.

Download Stage 2 public data · Publication scope · Learning notes and data guide

Collection summary: 6 planned cells · 10 actual submissions · 6 complete selected answers. Four earlier ChatGPT C attempts retain technical/capture problems. Each cell has one selected answer; no stable success or failure rate is estimated.

Six-cell comparison

A · Direct URL

The URL is supplied. The test concerns demonstrated access to the post's content.

B · Exact excerpt

An exact continuous excerpt is supplied. Discovery requires the original post and matching retrieved content.

C · Natural query

No target identity, URL, or excerpt is supplied. Other experiments and GitHub-only results are separate outcomes.

Condition · platformCollection statusX post content accessExact discoveryUnprompted discoveryGitHub onlyActual search
A · ChatGPT SearchRecovered capture · completeNot demonstratedN/AN/ANoObserved
A · GrokCompletedDemonstratedN/AN/ANoObserved
B · ChatGPT SearchRecovered capture · completeNot assessedNot demonstratedN/ANoNot recorded
B · GrokCompletedNot assessedNot demonstratedN/ANoObserved
C · ChatGPT Search
Selected: S2-CHATGPT-C-R5
CompletedNot assessedN/ANot demonstratedNoObserved
C · GrokCompletedNot assessedN/ANot demonstratedNoObserved

One selected answer per platform × condition. This is a six-cell observation table, not an estimate of stable success/failure rates. ChatGPT B search queries were not independently captured. ChatGPT C is the later user-operated C-R5 replacement; its exact question and actual search were corroborated, but generation model/effort and time are user-confirmed and pre-submit session isolation was not independently recorded.

Reading the matrix: Demonstrated / Not demonstrated are explicit recorded judgments, not automatic scores. N/A marks a discovery task that the condition did not test. Unknown, Not assessed, and technical failure are not negative discovery evidence. “GitHub only” applies only when explicitly coded; it is never inferred from a GitHub URL.

Search enabled in the interface and search actually observed are different fields. External experiments described as “verified” inside an answer are still platform claims unless a separate assessment records verification.

Test conditions and account controls

Protocol registered: 2026-10-31 11:35:43 Asia/Shanghai (UTC+08:00).

chatgpt

conversation
Temporary Chat
personalization
Not personalized; UI states no memory, plugins or custom instructions
model
GPT-5.6 Sol (UI selection; composer label 5.6 Medium)
reasoning effort
Medium
search
Web Search tool chip selected before submission
model choice
GPT-5.6 Sol / Medium was explicitly selected in the UI; backend routing was not independently verified.

grok

conversation
Private Chat
mode
Expert
backend model version
Not disclosed in visible selector
customization
Custom style and all listed styles unchecked
personalize using x test
No
personalize using history test
No
restoration
Original personalization settings restored after tests; original preference values omitted.

manual chatgpt c replacement

collector
User
exact prompt
Verified against original native user message after separating Search tool marker
search
Actual native activity: 29 websites searched
model and reasoning
GPT-5.6 Sol / Medium, user-confirmed; generation UI not independently captured
time
User says now (around16:49Asia/Shanghai); native source turn16:47:24–16:47:36 and UI thought36s retained separately
personalization isolation
Not independently recorded before submission
interpretation
Complete answer capture does not establish identical pre-submit control evidence.

Recorded post-test restoration

restored at
2026-10-31T03:50:56.650Z
completed
Yes
scope
Original Grok personalization settings restored. Original preferences and account UI are not published.
other settings changed
No

Temporary/private conversations and visible personalization controls reduce known account-context paths. They do not establish audited backend isolation or complete independence. See each run for its actual model, mode, search evidence, and any contamination evidence.

Method and interpretation limits

Design and execution order

planned submissions
6
platforms
[
  "ChatGPT Search",
  "Grok"
]
conditions
[
  "A",
  "B",
  "C"
]
repetitions per cell
1
execution order
[
  "S2-CHATGPT-C",
  "S2-GROK-C",
  "S2-CHATGPT-B",
  "S2-GROK-B",
  "S2-CHATGPT-A",
  "S2-GROK-A"
]
order rationale
Natural discovery before exact excerpt before direct URL, limiting identity leakage within this batch. Each uses a fresh temporary/private conversation; only one user prompt, no steering follow-up.
publication
Redacted public edition authorized on 2026-11-01; private originals remain local.
actual execution order
[
  "S2-CHATGPT-C",
  "S2-GROK-C",
  "S2-GROK-B",
  "S2-CHATGPT-B",
  "S2-GROK-A",
  "S2-CHATGPT-A",
  "S2-CHATGPT-C-R2",
  "S2-CHATGPT-C-R3",
  "S2-CHATGPT-C-R4",
  "S2-CHATGPT-C-R5"
]
actual submission attempts
10
deviations
[
  "Grok B and A were submitted before ChatGPT B and A because of ChatGPT access interruption. Within each original platform sequence the order remained C→B→A.",
  "ChatGPT C was supplemented twice after the six planned attempts with explicit user authorization, retaining the initial Cloudflare failure.",
  "The C supplement was submitted in fresh non-personalized temporary mode before A/B were saved to ordinary history. No follow-up or seeded prompt was added.",
  "Existing A/B chats were saved after generation under explicit user authorization to recover full body text; these are capture recovery actions, not additional submissions.",
  "After C-R2 recovery exposed only a truncated prefix, C-R3 repeated the unchanged prompt/settings in a new non-personalized temporary chat. A/B/C-R2 were then already saved in account history; visible memory/plugins/custom-instruction access remained disabled for C-R3. Backend isolation is not audited.",
  "A single delayed C-R4 retry ran at the user-requested later time on 2026-10-31, using the same visible Sol Medium model and native Web Search in a fresh non-personalized temporary chat. Forty-five searched websites were observed. No complete answer body was recovered.",
  "C-R5 was user-operated. The source contains the exact C question plus the native Search marker and no steering; original activity verifies 29 searched websites. User confirms GPT-5.6 Sol / Medium and approximate time. No pre-submit temporary/personalization screenshot exists; the manual control evidence is qualified."
]
selected attempt policy
Six planned cells now have complete answer captures. ChatGPT C selects the first complete manual replacement (C-R5), with its collection provenance qualified. No outcome-based cherry-picking: all four earlier C failures/partial captures and all ten submissions remain in attempts.json.
publication at collection
Local collection only; no X post, public share, repository push or website update is authorized for this stage.

Exact-excerpt verification: The B prompt calls the quotation a sentence. It is an 86-character continuous passage from the latter part of a longer sentence, verified character for character against the captured public post, including case, spaces, numbers, and punctuation. The question is preserved unchanged. No transcription mismatch was observed.

A · Direct URL access

Pass requires two distinct specific findings supported by the actual post, the original URL as source, and visible evidence of post-content retrieval. Repeating the supplied URL or claiming to have read it is insufficient. Correct facts obtained only from the linked report are tracked separately, not as demonstrated access to the X post.

B · Exact-excerpt discovery

Pass requires the real target X status URL and correct author, plus evidence that retrieved page content matches the excerpt. Repeating the supplied excerpt or guessing a plausible URL is insufficient.

C · Natural discovery

Target-discovery pass requires proactively finding and citing the target X status without supplied identity, URL or excerpt. Finding the GitHub report, author, or another genuine experiment is tracked separately; it does not establish discovery of this X post.

Missing or incomplete capture is not coded as no discovery. Reading a preview or source panel does not prove full stored content.

Target and reference facts

https://x.com/bird_helen67403/status/2116080254523445676

status id
2116080254523445676
author display
HelenBird
author handle
bird_helen67403
published ui
4:10 PM · Oct 30, 2026
ground truth
User supplied full post; verified against the rendered X page before tests.

Ground-truth evidence: target-post.json

  • design: 3 GEO questions × 2 platforms × 2 runs = 12 answers.
  • creator_overlap: For practical data-backed tutorials for indie builders, 2 of 20 unique recommended entities were shared: about 10% cross-platform overlap.
  • cmr_distinction: Citation, mention and recommendation can occur separately; citation does not necessarily mean brand recommendation.
  • source_variation: The same query can return different retrieval results/source picks across platforms.
  • answer_variation: Recommendations vary; one answer is insufficient to measure GEO.

Boundaries

  • Only one run per platform and condition; no estimate of stable retrieval rates or GEO causal effectiveness.
  • Temporary/private UI settings reduce visible personalization; backend isolation is not independently audited.
  • The post was published the previous day; indexing delay may affect results.
  • X-content access, exact matching, natural discovery, and GitHub-report retrieval are separate outcomes.
  • Grok source-panel page-anchor inventories include query/answer/activity links and are not fully scoped retrieval-result sets. Consent UI links are excluded from derived analysis while original captures remain unchanged. UI query labels can concatenate result counts; source badge counts are not distinct citations.
  • Selected ChatGPT C is a user-operated replacement. Its full answer, exact question and actual search were corroborated, but the generation model is user-confirmed, time is approximate and pre-submit memory/temporary controls were not captured. Comparisons must retain this collection difference.

These tests cannot establish causal GEO effectiveness, stable indexing or retrieval rates, a platform's inherent superiority, or that an undiscovered post does not exist. The conditions provide different amounts of identifying information and should not be treated as equivalent tasks.

Full prompts, answers, and evidence

All sections are expanded initially. Public prompt and answer copies are escaped and preserve their wording apart from marked privacy edits. Original private capture files are excluded. A planned protocol prompt is identified when no submitted prompt was recorded.

S2-CHATGPT-A · ChatGPT Search · Direct URL accessRecovered capture · complete

Input: runs.json + selected public record. Submitted: 2026-10-31 11:48:50 Asia/Shanghai (UTC+08:00). Captured: 2026-10-31 12:04:40 Asia/Shanghai (UTC+08:00).

Full answer recovered without resubmission. Main UI rendering failure and failed Copy remain in capture history; the response itself reports no successful access/discovery. Saved after generation under explicit user authorization.

Recorded assessment

DimensionJudgmentEvidence / qualification
X post content accessNot demonstratedNo supporting assessment recorded.
Exact discoveryN/ANo supporting assessment recorded.
Unprompted discoveryN/ANo supporting assessment recorded.
GitHub onlyNoNo supporting assessment recorded.
outcome
fail
rationale
The complete recovered answer explicitly says it cannot access the X post and supplies no experimental findings. The original URL was already supplied in the prompt, so repeating it does not establish retrieval.
access error claim
The answer reports HTTP 403 from X; this is a platform-reported cause, not an independently captured HTTP response.
Full prompt

Recorded prompt; retained verbatim.

Read this public X post: https://x.com/bird_helen67403/status/2116080254523445676

Summarize two specific experimental findings from the post. Provide the original URL as a source. If you cannot access the actual post content, say so explicitly. Do not infer or invent findings.

Observed composer text

\u7f51\u9875\u641c\u7d22 Read this public X post: 
https://x.com/bird_helen67403/status/2116080254523445676

Summarize two specific experimental findings from the post. Provide the original URL as a source. If you cannot access the actual post content, say so explicitly. Do not infer or invent findings.
Captured answer
I’m unable to access the actual content of the X post.

I attempted to retrieve the original post, but X returned **403 Forbidden**\. Searches for the post ID and account did not reveal an indexed copy of its contents.

**Original source\:**
urlhttps\://x.com/bird\_helen67403/status/2116080254523445676https://x.com/bird_helen67403/status/2116080254523445676

**Experimental findings\:**

1. **First finding\:** Cannot verify because the post content is inaccessible.
2. **Second finding\:** Cannot verify because the post content is inaccessible.

I won't invent or infer experimental results without seeing the original evidence.

**If you paste the post text or upload a screenshot,** I can identify and summarize exactly two specific experimental findings, preserving the reported results and quantitative details.
Model, mode, search, and capture evidence

Actual search: Observed

model label
GPT-5.6 Sol
reasoning effort
Medium
search enabled ui
Yes
temporary
Yes
personalization
No
body complete
Yes
main page error
Unable to load this response
copy result
\u65e0\u6cd5\u52a0\u8f7d\u6b64\u56de\u590d
tool activity
[
  "Visible Python analysis decodes the X status ID into a timestamp; this is not evidence of reading the post body."
]
selected attempt id
S2-CHATGPT-A
attempt number
1
capture method
Saved existing generated temporary conversation, then read its full agentMessage through native read_thread; page.hasMore=false, no truncation marker. No prompt resubmission.
capture provenance
Original generation used fresh non-personalized temporary mode. Saving after generation created an ordinary-history copy with a new conversation ID. Original UI/copy failures are retained. Returned native agentMessage text is preserved verbatim, including escaped Markdown/citation serialization; it is not a retrieval summary.
search evidence
Expanded native activity shows search query and “searched 12 websites”; separate code analysis decodes status timestamp only.
search queries
[
  "\"https://x.com/bird_helen67403/status/2116080254523445676\" OR \"2116080254523445676\" 2026"
]

Original timestamps: { "submitted_at": "2026-10-31T03:48:50.116Z", "captured_at": "2026-10-31T04:04:40.909814+00:00" }

Capture and recovery history
Capture timeMethodCapture statusFileEvidence and notes
2026-10-31 11:54:37 Asia/Shanghai (UTC+08:00)Original temporary UI and Copycapture_incompleteNot recorded
{
  "body_complete": false,
  "notes": "Unable to load response; Copy exposed only error text."
}
2026-10-31 12:03:18 Asia/Shanghai (UTC+08:00)User-approved save existing chatsaved_copy_createdNot recorded
{
  "notes": "Ordinary-history copy; no new question or generation."
}
2026-10-31 12:04:40 Asia/Shanghai (UTC+08:00)Native full turn message read of saved copyrecoveredNot recorded
{
  "body_complete": true,
  "notes": "Native full agentMessage, not preview alone; original generated_at unknown.",
  "recorded_at": "2026-10-31T04:04:40.909814+00:00"
}
Returned answer links and recorded sources

Answer links and retrieved sources have different roles. A URL appearing in either list does not by itself establish access to the X post or satisfy an assessment.

Links in the answer

Label / evidenceURLDomainRecorded role
Untitled sourcehttps://x.com/bird_helen67403/status/2116080254523445676x.comPrompt-supplied original URL repeated in answer; not proof of post access

Explicitly recorded answer citations

No links recorded in this field.

Retrieval-panel sources · not automatically answer citations

No links recorded in this field.

S2-GROK-A · Grok · Direct URL accessCompleted

Input: runs.json + selected public record. Submitted: 2026-10-31 11:46:31 Asia/Shanghai (UTC+08:00). Captured: 2026-10-31 11:49:00 Asia/Shanghai (UTC+08:00).

No notes recorded.

Recorded assessment

DimensionJudgmentEvidence / qualification
X post content accessDemonstratedNo supporting assessment recorded.
Exact discoveryN/ANo supporting assessment recorded.
Unprompted discoveryN/ANo supporting assessment recorded.
GitHub onlyNoNo supporting assessment recorded.
outcome
pass
rationale
Direct content access demonstrated: native post-reading activity, correct author/title, and accurate body-specific observations (2/20≈10% entity overlap; citation, mention and recommendation can occur separately). The 12-answer design is corroborating context. A supplied URL does not establish discovery. The added “on the same platforms” qualifier remains unsupported by the supplied text.
faithfulness
{
  "verdict": "partly_supported",
  "caveat": "The answer adds “on the same platforms” to the variability finding. That qualifier is not explicit in the target post; this excerpt alone does not establish within-platform repeat variability."
}
Full prompt

Recorded prompt; retained verbatim.

Read this public X post: https://x.com/bird_helen67403/status/2116080254523445676

Summarize two specific experimental findings from the post. Provide the original URL as a source. If you cannot access the actual post content, say so explicitly. Do not infer or invent findings.

Observed composer text

Read this public X post: https://x.com/bird_helen67403/status/2116080254523445676

Summarize two specific experimental findings from the post. Provide the original URL as a source. If you cannot access the actual post content, say so explicitly. Do not infer or invent findings.
Captured answer
Post content successfully retrieved via X tools (no access issues).

Source: https://x.com/bird_helen67403/status/2116080254523445676

Author: HelenBird (@bird_helen67403)

Posted: Fri, 30 Oct 2026 08:10:22 GMT

Title/theme: GEO Lab #01: Why do AI search engines recommend different sources?

Two specific experimental findings stated in the post
Low entity overlap between platforms. In an experiment comparing ChatGPT Search vs. Grok (3 GEO questions × 2 platforms × 2 runs = 12 answers), when the query asked for practical, data-backed GEO tutorials for indie builders, the two platforms shared only 2 of 20 unique recommended entities — roughly 10% overlap.
High run-to-run variability. Even near-simultaneous searches on the same platforms produced totally different recommendation lists.

The post frames these as exploratory observations (not causal claims) from a small first run and notes related takeaways such as “citation ≠ mention ≠ recommendation” and that a single answer is insufficient for measuring GEO. It points to a full write-up and repo for more detail but does not expand the experimental numbers further in the post itself.
Model, mode, search, and capture evidence

Actual search: Observed

model label
Exact backend version not disclosed
mode label
Expert
search mode
Expert native web/X retrieval; direct public URL provided
private
Yes
personalize using x
No
personalize using history
No
body complete
Yes
selected attempt id
S2-GROK-A
attempt number
1
search evidence
Native web/X retrieval observed. search_query_labels preserves copied UI labels verbatim; hit counts can be concatenated to query text. These labels are not parsed search-query strings. Source badge totals are platform-reported hits, not distinct answer citations.
search query labels
[
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edcsite:x.com bird_helen67403 21160802545234456765"
]
source panel declared hits
5

Original timestamps: { "submitted_at": "2026-10-31T03:46:31.827Z", "captured_at": "2026-10-31T03:49:00.560Z" }

Capture and recovery history

No capture or recovery events recorded in this field.

Returned answer links and recorded sources

Answer links and retrieved sources have different roles. A URL appearing in either list does not by itself establish access to the X post or satisfy an assessment.

Links in the answer

Label / evidenceURLDomainRecorded role
https://x.com/bird_helen67403/status/2116080254523445676https://x.com/bird_helen67403/status/2116080254523445676x.comlink in answer

Explicitly recorded answer citations

No links recorded in this field.

Retrieval-panel sources · not automatically answer citations

No links recorded in this field.

Source-inventory scope

Whole-page anchor inventory is not included in the public edition. It was captured while source panels were expanded and includes query, answer and activity links; it is not a scoped retrieval-result set or a list of answer citations. Original captures remain local. Source badge totals are reported hits, not distinct cited URLs. The answer links and search-activity labels are retained below.

S2-CHATGPT-B · ChatGPT Search · Exact-excerpt discoveryRecovered capture · complete

Input: runs.json + selected public record. Submitted: 2026-10-31 11:46:11 Asia/Shanghai (UTC+08:00). Captured: 2026-10-31 12:04:40 Asia/Shanghai (UTC+08:00).

Full answer recovered without resubmission. Main UI rendering failure and failed Copy remain in capture history; the response itself reports no successful access/discovery. Saved after generation under explicit user authorization.

Recorded assessment

DimensionJudgmentEvidence / qualification
X post content accessNot assessedNo supporting assessment recorded.
Exact discoveryNot demonstratedNo supporting assessment recorded.
Unprompted discoveryN/ANo supporting assessment recorded.
GitHub onlyNoNo supporting assessment recorded.
outcome
fail
rationale
The complete recovered answer explicitly says the original X post URL was not found, the author was not verified, and quote-to-link verification was impossible. No target URL returned.
access error claim
The answer reports HTTP 403 from X; this is a platform-reported cause, not an independently captured HTTP response.
Full prompt

Recorded prompt; retained verbatim.

Search the web and X for the original public post containing this exact sentence:

"the two platforms shared only 2 of 20 unique recommended entities — about 10% overlap."

Return the original X post URL and author. Verify the link matches the quoted text. If you cannot find it, say so explicitly.

Observed composer text

\u7f51\u9875\u641c\u7d22 Search the web and X for the original public post containing this exact sentence:

"the two platforms shared only 2 of 20 unique recommended entities — about 10% overlap."

Return the original X post URL and author. Verify the link matches the quoted text. If you cannot find it, say so explicitly.
Captured answer
**I could not find or verify the original X post containing that exact sentence.**

I searched for\:

- The complete sentence in quotation marks.
- The distinctive phrase `"2 of 20 unique recommended entities"`\.
- Variations of `"about 10% overlap"` and `"shared only 2 of 20"`\.
- Indexed X/Twitter posts using both `x.com` and `twitter.com` domain searches.

**Verification result**

| Requested information | Result |
|--------|--------|
| Exact sentence found | No |
| Original X post URL | Not found |
| Author | Not verified |
| Link-to-quote verification | Not possible |

I also attempted to access X's search page directly, but the request returned HTTP 403 (access denied).

**Conclusion\:** I cannot reliably identify the original post or its author. I won't invent an X URL or attribute the quotation to someone without evidence.

The original public post remains unverified.
Model, mode, search, and capture evidence

Actual search: Not recorded

model label
GPT-5.6 Sol
reasoning effort
Medium
search enabled ui
Yes
temporary
Yes
personalization
No
body complete
Yes
main page error
Unable to load this response
copy result
\u65e0\u6cd5\u52a0\u8f7d\u6b64\u56de\u590d
selected attempt id
S2-CHATGPT-B
attempt number
1
capture method
Saved existing generated temporary conversation, then read its full agentMessage through native read_thread; page.hasMore=false, no truncation marker. No prompt resubmission.
capture provenance
Original generation used fresh non-personalized temporary mode. Saving after generation created an ordinary-history copy with a new conversation ID. Original UI/copy failures are retained. Returned native agentMessage text is preserved verbatim, including escaped Markdown/citation serialization; it is not a retrieval summary.
search evidence
Native Web Search chip selected; final answer reports phrase/domain searches. No independent native search activity panel was captured for B.
search queries
[]

Original timestamps: { "submitted_at": "2026-10-31T03:46:11.104Z", "captured_at": "2026-10-31T04:04:40.944638+00:00" }

Capture and recovery history
Capture timeMethodCapture statusFileEvidence and notes
2026-10-31 11:50:25 Asia/Shanghai (UTC+08:00)Original temporary UI and Copycapture_incompleteNot recorded
{
  "body_complete": false,
  "notes": "Unable to load response; Copy exposed only error text."
}
2026-10-31 12:01:18 Asia/Shanghai (UTC+08:00)User-approved save existing chatsaved_copy_createdNot recorded
{
  "notes": "Ordinary-history copy; no new question or generation."
}
2026-10-31 12:04:40 Asia/Shanghai (UTC+08:00)Native full turn message read of saved copyrecoveredNot recorded
{
  "body_complete": true,
  "notes": "Native full agentMessage, not preview alone; original generated_at unknown.",
  "recorded_at": "2026-10-31T04:04:40.944638+00:00"
}
Returned answer links and recorded sources

Answer links and retrieved sources have different roles. A URL appearing in either list does not by itself establish access to the X post or satisfy an assessment.

Links in the answer

No links recorded in this field.

Explicitly recorded answer citations

No links recorded in this field.

Retrieval-panel sources · not automatically answer citations

No links recorded in this field.

S2-GROK-B · Grok · Exact-excerpt discoveryCompleted

Input: runs.json + selected public record. Submitted: 2026-10-31 11:42:12 Asia/Shanghai (UTC+08:00). Captured: 2026-10-31 11:45:51 Asia/Shanghai (UTC+08:00).

No notes recorded.

Recorded assessment

DimensionJudgmentEvidence / qualification
X post content accessNot assessedNo supporting assessment recorded.
Exact discoveryNot demonstratedNo supporting assessment recorded.
Unprompted discoveryN/ANo supporting assessment recorded.
GitHub onlyNoNo supporting assessment recorded.
outcome
fail
rationale
The complete captured answer did not return the target original X URL or author. The captured page-anchor inventory likewise contains no target X URL or own GitHub/Pages URL, but is not claimed to be the full underlying retrieval set. This is one observed failure to discover, not evidence the post is absent or unindexed.
Full prompt

Recorded prompt; retained verbatim.

Search the web and X for the original public post containing this exact sentence:

"the two platforms shared only 2 of 20 unique recommended entities — about 10% overlap."

Return the original X post URL and author. Verify the link matches the quoted text. If you cannot find it, say so explicitly.

Observed composer text

Search the web and X for the original public post containing this exact sentence:

"the two platforms shared only 2 of 20 unique recommended entities — about 10% overlap."

Return the original X post URL and author. Verify the link matches the quoted text. If you cannot find it, say so explicitly.
Captured answer
I could not find any public post (on X or the web) containing that exact sentence.

I searched:

Exact phrase matches for “the two platforms shared only 2 of 20 unique recommended entities — about 10% overlap.”
Close variants (“2 of 20 unique recommended entities,” “unique recommended entities” + overlap, “platforms shared only 2 of 20,” “10% overlap” with recommended/entities, etc.).
Keyword and semantic searches on X.
Broader web searches for related concepts (platform overlap of recommended entities/items).

None returned a matching post, article, or source. No X post with that wording (or a near-identical version) appeared in results.

Possible reasons it is not surfacing:

The post is very recent and not yet indexed.
It was deleted, set to private, or from a protected account.
Minor wording/punctuation differences (e.g., hyphen vs. em dash, “approx.” instead of “about,” slight rephrasing).
It originated elsewhere (newsletter, private chat, non-public document) and was only partially quoted.

If you have additional context (author name, approximate date, platform names involved, surrounding text, or a screenshot/link fragment), I can search more specifically.
Model, mode, search, and capture evidence

Actual search: Observed

model label
Exact backend version not disclosed
mode label
Expert
search mode
Expert native web/X retrieval; prompt explicitly requests web and X search
private
Yes
personalize using x
No
personalize using history
No
body complete
Yes
selected attempt id
S2-GROK-B
attempt number
1
search evidence
Native web/X retrieval observed. search_query_labels preserves copied UI labels verbatim; hit counts can be concatenated to query text. These labels are not parsed search-query strings. Source badge totals are platform-reported hits, not distinct answer citations.
search query labels
[
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"the two platforms shared only 2 of 20 unique recommended entities — about 10% overlap.\"15",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"two platforms shared only 2 of 20 unique recommended entities\"10",
  "\\u5df2\\u641c\\u7d22 𝕏\"the two platforms shared only 2 of 20 unique recommended entities\"",
  "\\u5df2\\u641c\\u7d22 𝕏\"2 of 20 unique recommended entities\"",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"2 of 20 unique recommended entities\"10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"unique recommended entities\" overlap platforms10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"about 10% overlap\" recommended entities OR recommendations platforms10",
  "\\u5df2\\u641c\\u7d22 𝕏\"recommended entities\" overlap",
  "\\u5df2\\u641c\\u7d22 𝕏the two platforms shared only 2 of 20 unique recommended entities about 10% overlap9",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"2 of 20\" overlap recommended OR recommendations platforms OR systems10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"10% overlap\" \"recommended\" entities OR items platforms10",
  "\\u5df2\\u641c\\u7d22 𝕏\"2 of 20\" (overlap OR recommended OR recommendations)5",
  "\\u5df2\\u641c\\u7d22 𝕏\"unique recommended\"5",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"platforms shared only 2 of 20\"10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"2 of 20 unique\" recommended10",
  "\\u5df2\\u641c\\u7d22 𝕏\"shared only 2 of 20\"",
  "\\u5df2\\u641c\\u7d22 𝕏\"10% overlap\" (recommended OR recommendations OR entities)",
  "\\u5df2\\u641c\\u7d22 𝕏two platforms shared only 2 of 20 unique recommended entities about 10% overlap4",
  "\\u5df2\\u641c\\u7d22 𝕏\"recommended entities\"4",
  "\\u5df2\\u641c\\u7d22 𝕏\"unique recommended entities\"",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"recommended entities\" \"10%\" OR \"2 of 20\" OR \"overlap\"10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"two platforms\" \"recommended\" overlap OR \"10%\" entities OR items OR accounts10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"the two platforms shared only\"10",
  "\\u5df2\\u641c\\u7d22 𝕏\"platforms shared only\""
]
source panel declared hits
152

Original timestamps: { "submitted_at": "2026-10-31T03:42:12.672Z", "captured_at": "2026-10-31T03:45:51.332Z" }

Capture and recovery history

No capture or recovery events recorded in this field.

Returned answer links and recorded sources

Answer links and retrieved sources have different roles. A URL appearing in either list does not by itself establish access to the X post or satisfy an assessment.

Links in the answer

No links recorded in this field.

Explicitly recorded answer citations

No links recorded in this field.

Retrieval-panel sources · not automatically answer citations

No links recorded in this field.

Source-inventory scope

Whole-page anchor inventory is not included in the public edition. It was captured while source panels were expanded and includes query, answer and activity links; it is not a scoped retrieval-result set or a list of answer citations. Original captures remain local. Source badge totals are reported hits, not distinct cited URLs. The answer links and search-activity labels are retained below.

S2-CHATGPT-C · ChatGPT Search · Natural discoveryCompleted

Input: runs.json + selected public record. Submitted: Around 2026-10-31 16:49 Asia/Shanghai (user said “now”; approximate, not a measured submission time). Captured: 2026-10-31 16:59:29 Asia/Shanghai (UTC+08:00).

  • This manual submission fills the missing answer body and returned-URL inventory; all four earlier C attempts remain retained.
  • Model/reasoning and approximate question time are user-confirmed. Native turn timestamps and UI thought duration are recorded separately and must not be treated as interchangeable timing measurements.
  • Eleven cited/linked sources comprise eight included studies and three exclusions. Artur Ferreira is cited and mentioned but explicitly excluded, so Recommendation=false.
  • This is a qualified manual replacement, not proof that all pre-submit controls matched the earlier automated collection.

Recorded assessment

DimensionJudgmentEvidence / qualification
X post content accessNot assessedNo supporting assessment recorded.
Exact discoveryN/ANo supporting assessment recorded.
Unprompted discoveryNot demonstrated
{
  "evidence": "The complete supplied/native answer and its 11 explicit URLs contain no target X post, status ID or author. Other experiments are returned; the target-specific criterion was not met in this selected answer."
}
GitHub onlyNoNo supporting assessment recorded.
outcome
fail
rationale
Complete manual C answer collected and original links recovered. It identifies other GEO research but does not proactively discover the target X post. Citation-source inclusion and exclusion are coded separately; cited excluded candidates are not recommendations.
candidate verification
The answer’s study statistics and “verified” labels are platform claims, not independent verification by this experiment.
protocol qualification
Manual supplement: exact prompt and live-search activity verified; generation model is user-confirmed and pre-submit personalization isolation was not recorded.
Full prompt

Recorded prompt; retained verbatim.

Can you find independent builders who have published original experiments comparing GEO recommendations across ChatGPT Search and Grok?

Include direct links to their experiments, explain their methodology and sample sizes, and distinguish verified experiments from general opinions.
Captured answer
Independent GEO experiments: ChatGPT Search vs Grok

Research cutoff: October 31, 2026

I found several published first-party studies that measure AI brand visibility across ChatGPT and Grok, including experiments by smaller GEO research teams and independent operators.

The key distinction is between recommendation experiments (which brands an AI suggests), citation experiments (which websites an AI references), and GEO intervention experiments (whether changing content causes AI visibility to improve).

These test different hypotheses. In particular, a study comparing ChatGPT and Grok does not automatically establish that it tested the actual consumer-facing ChatGPT Search interface.

1. Strongest published cross-platform experiments

1. mapou — State of AI Search 2026⁠

Original observational benchmark · May 2026

Compares which brands ChatGPT, Grok and three other engines recommend or cite across buyer-intent prompts.

Sample: 500 brands, 25 competitive segments, 5 engines, 20 canonical prompts per segment and 12 buyer personas. The authors report approximately 30,000 baseline brand-level verdicts per monthly run.

Method: Fixed prompts spanning discovery, filtered discovery, comparison and evaluation. Measures brand visibility, engine agreement and the effects of buyer personas, with Wilson 95% confidence intervals.

Finding: Mean cross-engine brand-ranking agreement was reported at 52%. AI visibility changes materially by engine and buyer persona.

Evidence status: Published comparative measurement with a methodology. ChatGPT Search interface equivalence is not established by the headline report. mapou

2. AI Visibility Report Group — Six AI Platforms, Same Eight Questions⁠

Original repeated citation test · August 29, 2026

Direct comparison of ChatGPT, Grok, Gemini, Claude, Perplexity and Meta AI.

Sample: Eight unbranded buying questions, six platforms, two rounds. Of 96 planned responses, 87 were successful (39 in round one, 48 in round two).

Method: Ask identical buyer questions separately, extract citation URLs, normalize sources to domains and compare overlap.

Finding: In the complete second round, 468 of 551 cited domains (85%) appeared on just one platform. Grok cited 256 domains and ChatGPT 50.

Evidence status: Verified published experiment on citation-source diversity. Small sample, restricted to Dutch-market AI visibility questions. The figures do not measure the number of brands recommended. research

3. GOSH AI — The Search vs Learned Divergence⁠

Original multi-model observational study · March–May 2026

Tests whether brands surfaced through live retrieval differ from brands generated using learned model knowledge.

Sample: Six AI-model families, including ChatGPT and Grok; more than 23,800 model executions, 80,300 citations and 8,600 brands identified across the reported baseline.

Method: Apply equivalent prompts with retrieval enabled and disabled, then classify observed brand-category pairs by presence in each path.

Finding: Only 8.28% of observed brand-category pairs appeared in both the live-search and learned-knowledge paths.

Evidence status: Published original research with substantial reported sample size. The headline result is aggregated across models, and the comparable dual-path category analysis is narrower. It should not be presented as a ChatGPT-vs-Grok-specific effect. GOSH AI

2. Individual builders running original tests

These are particularly relevant if your goal is to identify people publishing their own experiments rather than relying on generic GEO advice.

4. Zeeshan Yaseen — Two GEO experiments challenging conventional AI visibility advice

First-person field experiments · Published September 14, 2026

Read the original experiment write-up⁠

Method and sample: Two sequential tests, each using 15 commercial-intent keywords. The first compared four engines and the second added Grok and Google AI Mode, for six engines total. The second was a 30-day cold-start campaign for a SaaS link-building agency, with manual checks and tracked content placements.

Result: In the second experiment, the tracked brand appeared 32 times on ChatGPT versus 4 times on Grok, out of 298 appearances across all six platforms. Both experiments together logged 775 citation events.

Assessment: A genuine first-person GEO intervention study involving both engines. It has commercial interventions and before/after measurements, but no randomized control or established ChatGPT Search interface matching. Search Engine Land

5. Charles Cousyn / Weftd — ChatGPT, Instinct, Grok Bot: Our Test

Independent real-world recommendation test · September 23, 2026

Read the restaurant recommendation experiment⁠

Method and sample: The builder gave the same practical mission to ChatGPT, Grok Bot and Instinct: identify three restaurants near Lyon Part-Dieu station for a business lunch with a specified reservation requirement. Five agent configurations were tested on September 22–23.

Result: The configurations suggested 12 different restaurants, with only one appearing in more than two configurations. The one reservation verified in depth exposed contradictory availability information.

Assessment: A first-person, real-world comparison of AI recommendations, especially useful for local GEO. The sample is just five configurations for one mission. The research tests agent behavior, not a broad SEO optimization intervention. Weftd

6. Queraid — AI Visibility Study 2026

Original brand recommendation benchmark · May 25, 2026

Read the 70+ brand experiment⁠

Method and sample: Over 70 brands across more than 50 industries, evaluated using six prompts per brand on each of five engines, including ChatGPT and Grok. The authors report more than 2,000 responses.

The prompts cover brand identity, recommendations, comparisons, reputation, use cases and best-in-class selections. Scores combine mention rate (40%), sentiment (30%), competitor parity (20%) and cross-model consistency (10%).

Assessment: Published first-party quantitative benchmarking. The composite score is proprietary to the researchers, and the reported headline rankings do not separately isolate ChatGPT Search versus Grok. The authors say the underlying dataset is available through their API. Queraid

3. Evidence ranking

Study	Original observations?	ChatGPT + Grok?	Direct user-facing Search comparison established?
mapou	Yes	Yes	Unclear
AI Visibility Report Group	Yes	Yes	Unclear
GOSH AI	Yes	Yes	No, tests retrieval modes
Zeeshan Yaseen	Yes, plus interventions	Yes, second study	Unclear
Charles Cousyn / Weftd	Yes	Yes	Agent comparison; not isolated
Queraid	Yes	Yes	Unclear

“Verified” here means a first-party experiment write-up with identifiable testing procedures and reported observations. It does not mean the findings have been independently audited or replicated. The studies above are also self-published by people or businesses active in the AI visibility market, so their commercial interests should be considered.

4. Two additional studies worth including

7. Nazareno Castro Bay / Rank Prompt — Same brand, different AI surfaces

Original test and methodology⁠

This is one of the closest matches to your request because the authors explicitly separate ChatGPT Search, ChatGPT without search, and Grok.

On July 30, 2026, they tested one B2B brand using 70 prompts across seven engine surfaces, producing 490 prompt-by-engine observations. They held the country, language, date and brand constant.

They reported:

* 5.71% brand presence on Grok versus 0% on ChatGPT overall.
* 25.5 citations per answer on Grok versus 4.2 on ChatGPT Search.
* 87.45% of cited URLs appeared on exactly one engine for the same question.

Evidence status: Original, documented experiment with explicit search-mode separation. However, the test concerns one brand and one market category, and the published collection uses the company’s API rather than a verified consumer-interface session. Rank Prompt

8. Columbia University Tow Center — ChatGPT Search vs Grok source accuracy

Read the Tow Center’s original study⁠

This is an independent academic-journalism benchmark, published March 6, 2025.

Sample: 20 news publishers × 10 articles × 8 AI search tools = 1,600 queries, with 200 queries per tool.

Method: Researchers supplied article excerpts and asked each AI search engine to identify the article, publisher and URL. They manually assessed accuracy.

Results: ChatGPT Search incorrectly identified 134 of 200 articles (67%). Grok-3 answered 94% incorrectly.

Evidence status: Strong direct evidence about retrieval and citation accuracy, with explicitly named ChatGPT Search and Grok versions. It does not test commercial GEO brand recommendations. Columbia Journalism Review

5. General opinions and weaker claims to exclude

Source	Why it doesn’t qualify as strong experimental evidence
Livesov — Who Gets Recommended	Explicitly says its figures are illustrative, with no fixed completed sample established in the article.
AstroFabric — ChatGPT vs Grok	Claims zero citation overlap, but the social post alone does not establish an adequate sample size or repeated-run methodology.
Artur Ferreira — GEO Experiment Files	Describes actual experiments, but the referenced cross-engine comparison uses ChatGPT, Gemini and Perplexity; Grok is absent.

Livesov Blog +2

Conclusion

For your specific research question, I would prioritize Rank Prompt, mapou, Zeeshan Yaseen, and Charles Cousyn.

Rank Prompt provides the clearest reported distinction between ChatGPT Search and Grok. mapou has the broader brand recommendation panel. Yaseen documents an actual GEO campaign intervention. Cousyn provides a concrete, first-person comparison of real-world local business recommendations.

The evidence supports measurable differences across AI systems in brand visibility and citation selection. It does not yet establish a universal GEO tactic that reliably increases recommendations on both ChatGPT Search and Grok. A stronger demonstration would require identical pre-registered buyer prompts, repeated runs, documented interface versions, publicly available raw responses and a controlled content intervention.
Model, mode, search, and capture evidence

Actual search: Observed

model label
GPT-5.6 Sol
model evidence
User explicitly confirms GPT-5.6 Sol / Medium. The generation-time model selection was not independently captured; recorded as user-confirmed.
reasoning effort
Medium
mode label
Manual collection; original single-prompt conversation, pre-submit temporary/personalization controls not captured
fresh session
Yes
search enabled ui
Yes
temporary
Not recorded
personalization
Not recorded
body complete
Yes
selected attempt id
S2-CHATGPT-C-R5
attempt number
5
replacement for
S2-CHATGPT-C-R4
capture method
User-pasted complete answer corroborated by the complete original native agentMessage; all 11 URLs recovered from native URL tags. Read-only original UI verifies search activity.
capture provenance
The pasted and native versions correspond in content, with different formatting and citation representations; they are not byte-identical. Both run through the final conclusion. The public share and original web views still show a loading error, but the native original message is complete. The agent sent no new prompt, created no share link and made no public repository/site change.
memory evidence
Native conversation contains one user prompt and one complete answer, with no added target identity or steering. Pre-submission memory/personalization state was not independently recorded.
collector
User-operated manual supplement; imported by agent
submitted time reported
Around 2026-10-31 16:49 Asia/Shanghai (user said “now”; approximate, not a measured submission time)
source turn started at
2026-10-31T08:47:24.314577+00:00
source turn completed at
2026-10-31T08:47:36.202617+00:00
condition verification
Exact question and actual web search verified. Model/reasoning and approximate question time are user-reported. Session memory isolation and original temporary mode remain unverified; this manual supplement must be distinguished from agent-operated preflight captures.
source inventory complete
Yes
source inventory scope
All 11 explicit URLs and corresponding citation markers in the complete native answer: 8 included studies and 3 excluded candidates. This is not an inventory of all searched results; external study claims were not independently verified.
search evidence
Original user message contains the native Search marker. Expanded original UI activity says searched 29 websites and thought 36s. No search-result URL inventory was exposed by the activity panel.
searched websites reported
29

Original timestamps: { "submitted_at": "Around 2026-10-31 16:49 Asia/Shanghai (user said “now”; approximate, not a measured submission time)", "captured_at": "2026-10-31T08:59:29.987331+00:00" }

Capture and recovery history
Capture timeMethodCapture statusFileEvidence and notes
2026-10-31 16:59:29 Asia/Shanghai (UTC+08:00)User supplied complete paste and existing share URLcomplete_user_pasteNot recorded
{
  "body_complete": true
}
2026-10-31 16:59:29 Asia/Shanghai (UTC+08:00)Read original native conversationcomplete_native_corresponding_answerNot recorded
{
  "body_complete": true,
  "notes": "11,699 native characters; complete through conclusion; 11 explicit URL tags."
}
2026-10-31 16:59:29 Asia/Shanghai (UTC+08:00)Original web view and shared view inspectionweb_render_failureNot recorded
{
  "notes": "Both body views fail to render. Native original text and user paste supply the complete body. Original activity verifies 29 searched websites."
}
Returned answer links and recorded sources

Answer links and retrieved sources have different roles. A URL appearing in either list does not by itself establish access to the X post or satisfy an assessment.

Links in the answer

Label / evidenceURLDomainRecorded role
mapou — State of AI Search 2026https://mapou.ai/research/state-of-ai-searchmapou.aiIncluded study / suggested reading
AI Visibility Report Group — Six AI Platforms, Same Eight Questionshttps://aivisibilityreport.group/en/six-ai-platforms-same-questions-different-sourcesaivisibilityreport.groupIncluded study / suggested reading
GOSH AI — The Search vs Learned Divergencehttps://www.mygosh.ai/search-learned-divergencewww.mygosh.aiIncluded study / suggested reading
Read the original experiment write-uphttps://searchengineland.com/geo-experiments-challenge-conventional-ai-visibility-advice-488342searchengineland.comIncluded study / suggested reading
Read the restaurant recommendation experimenthttps://www.weftd.com/resources/use-cases/same-mission-three-ais-chatgpt-instinct-grok-botwww.weftd.comIncluded study / suggested reading
Read the 70+ brand experimenthttps://queraid.com/blog/ai-visibility-study-2026/queraid.comIncluded study / suggested reading
Original test and methodologyhttps://rankprompt.com/how-to-measure-ai-search-visibility/rankprompt.comIncluded study / suggested reading
Read the Tow Center's original studyhttps://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.phpwww.cjr.orgIncluded study / suggested reading
Livesov — Who Gets Recommendedhttps://livesov.com/blog/ai-visibility-study-who-gets-recommendedlivesov.comExplicitly excluded candidate; cited and mentioned, not recommended
AstroFabric — ChatGPT vs Grokhttps://www.linkedin.com/posts/astrofabric_why-chatgpt-and-grok-cite-different-sources-activity-7496921932441743360-YebHwww.linkedin.comExplicitly excluded candidate; cited and mentioned, not recommended
Artur Ferreira — GEO Experiment Fileshttps://www.linkedin.com/posts/arturgeo_three-weeks-ago-i-started-publishing-the-activity-7494675142204125185-djT0www.linkedin.comExplicitly excluded candidate; cited and mentioned, not recommended

Explicitly recorded answer citations

The same 11 URLs are listed above and explicitly recorded as citations. Their inclusion or exclusion roles are preserved per link.

Retrieval-panel sources · not automatically answer citations

No links recorded in this field.

S2-GROK-C · Grok · Natural discoveryCompleted

Input: runs.json + selected public record. Submitted: 2026-10-31 11:37:26 Asia/Shanghai (UTC+08:00). Captured: 2026-10-31 11:40:47 Asia/Shanghai (UTC+08:00).

No notes recorded.

Recorded assessment

DimensionJudgmentEvidence / qualification
X post content accessNot assessedNo supporting assessment recorded.
Exact discoveryN/ANo supporting assessment recorded.
Unprompted discoveryNot demonstratedNo supporting assessment recorded.
GitHub onlyNoNo supporting assessment recorded.
outcome
fail
rationale
The complete captured answer did not return the target original X URL or author. The captured page-anchor inventory likewise contains no target X URL or own GitHub/Pages URL, but is not claimed to be the full underlying retrieval set. This is one observed failure to discover, not evidence the post is absent or unindexed.
candidate verification
Other experiments and sample sizes described by Grok are platform claims, not independently verified by this experiment.
Full prompt

Recorded prompt; retained verbatim.

Can you find independent builders who have published original experiments comparing GEO recommendations across ChatGPT Search and Grok?

Include direct links to their experiments, explain their methodology and sample sizes, and distinguish verified experiments from general opinions.

Observed composer text

Can you find independent builders who have published original experiments comparing GEO recommendations across ChatGPT Search and Grok?

Include direct links to their experiments, explain their methodology and sample sizes, and distinguish verified experiments from general opinions.
Captured answer
Independent builders have published a handful of original, data-backed experiments comparing GEO (Generative Engine Optimization) tactics and outcomes across ChatGPT Search and Grok. Most public discussion remains opinion or anecdotal. The verified experiments below are distinguished by disclosed methods, sample sizes, and direct links to primary write-ups or data. Larger academic or vendor studies (e.g., Princeton GEO-bench, Fudan fabricated-concept paper, Peec AI fanout analysis) are noted only where they contextualize, as they are not from independent builders.

Kasra Dash (@Kasra_Dash) — Self-referencing listicles, comparisons, and alternatives

Direct link: X post with setup, results, and data images (April 2026; also covered in interviews such as Ryan Doser’s video).

Methodology: Published 50 self-referencing articles on a mid-age domain in three batches (roughly 10, then 20, then 20), with ~3 weeks between drops. Formats: comparison pages (e.g., brand A vs brand B), listicles (“best X for Y”), and alternative pages (“Z alternatives”). Tracked brand mentions/citations across ChatGPT, Grok, Perplexity, Claude, and Google AI Overviews. Later 410’d (removed) all pages and monitored retention of citations.

Sample size: 50 articles; multi-week observation with batch-level tracking of mentions (e.g., Grok mentions scaled from ~7 → 25 → 41 across batches).

Key verified findings (vs. ChatGPT Search): Grok and Perplexity picked up the brand fastest and most readily (high single-digit to low double-digit “out of 10” style pickup rates in summaries; Grok highly responsive). ChatGPT and Claude were slower/more moderate (~6/10 range in secondary reports) but retained references to deleted pages for weeks, while Grok and Perplexity dropped faster after 410s. Google AI was least responsive. This is presented as a live, tracked test rather than theory; data visuals are in the primary post.

This is a clear original experiment by an independent SEO/AI-search practitioner.

Lennart Vallo / Cited (wearecited.com) — New-domain niche content to #1 citations

Direct link: Full results write-up (April 2026 case study on BaliPropertyRules.com).

Methodology: Registered a blank domain (early February 2026), published 18 original articles on Bali property regulations/costs/processes with technical/strategic optimization and no link building, paid placements, or ads. Tracked citations via 10 audits over ~6–8 weeks (Feb 12–Mar 29, 2026) using a fixed set of prompts. Measured coverage (% of prompts citing the site), total citations, share of voice, win rate (first-position citations), domain rank, unique URLs, and referral traffic (GA4 + GSC). Also noted volatility and response to a Google Core Update.

Sample size: 18 articles; 73 prompts across the engines; 10 audits; 202 total citations recorded.

Key verified findings (ChatGPT Search vs. Grok):

Grok: Higher coverage (~72.6%), more citations (68), broader unique URLs (~20), strong/high win rate (cited first in a large majority of cases when it cited the site), #1 domain rank. More volatile but loyal and broad in trust from early on.
ChatGPT/OpenAI: Lower coverage (~28.8%), fewer citations (25), fewer unique URLs (~11), still #1 domain rank on the prompts it covered, but narrower (favored information-scarcity/time-sensitive angles) and more correlated with Google signals. Drove more tracked referral sessions in this test despite lower coverage.

Site reached #1 domain rank on all four engines tested (including ChatGPT and Grok). Data is summarized in tables in the public write-up (no raw CSV download noted, but methodology and metrics are disclosed). This is an original controlled-style experiment by an independent/small-team builder.

Additional relevant independent or small-lab work
Second Wind (Marty Coleman, secondwind.cloud): Original 360-probe study asking ChatGPT, Claude, Gemini, Perplexity, and Grok to rank/recommend/describe each other. Direct link. Sample: 360 probes. GEO takeaways include strong “niche-boxing” for Grok (high specialty association, low general-mention rate ~13.5%, niche-boxing ~78.7%) versus ChatGPT’s default/generalist strength. More observational ranking experiment than content-tactic test, but original and quantified.
The GEO Lab (Artur Ferreira, thegeolab.net): Multiple pre-registered/public experiments and logs (e.g., Experiment 001 on narrative vs. declarative structure producing a ~23-percentage-point citation-rate gap on Perplexity; 330-query tests across engines including ChatGPT). GEO Log and related pages. Stronger on controlled single-variable tests and methodology transparency than pure ChatGPT-vs-Grok head-to-heads, but independent and data-driven.
Verified experiments vs. general opinions

Verified experiments (above) publish concrete methods, sample sizes, tracking periods, and (at least summarized) results or data visuals. They allow inspection of what was actually tested (listicle formats, new-domain niche content, structural changes, etc.) and how ChatGPT Search and Grok differed in speed of pickup, breadth of citation, retention after content removal, volatility, or ranking style.

General opinions dominate elsewhere: blog posts, X threads, and agency advice asserting that “listicles work,” “add statistics/quotes,” “Grok is easier,” or “ChatGPT prefers X” without disclosed sample sizes, controls, tracking windows, or primary data. Many recycle the original Princeton GEO paper’s simulated findings (top-5 Google results fed to an older model) without live multi-engine tests. Company tools (Qwairy, VectorGap, AthenaHQ, Promptwatch, etc.) produce large observational datasets that include both engines but are product/vendor reports rather than independent builder experiments.

Caveats that apply across the independent work: Sample sizes are modest by academic standards (tens of articles or prompts, not tens of thousands); engines update frequently, so results are time-bound; most are not peer-reviewed; and “success” is usually measured in citations/mentions rather than downstream traffic or conversions (though some track the latter). Retrieval behavior and model knowledge often matter more than on-page tweaks alone, per larger studies.

These are the clearest publicly documented independent-builder experiments that directly compare GEO outcomes on ChatGPT Search versus Grok. Primary sources remain the Kasra Dash X thread and the Cited results page.
Model, mode, search, and capture evidence

Actual search: Observed

model label
Exact backend version not disclosed
mode label
Expert
search mode
Expert supports native web/X retrieval; no separate force-search toggle exposed; actual search to be checked
private
Yes
personalize using x
No
personalize using history
No
body complete
Yes
selected attempt id
S2-GROK-C
attempt number
1
search evidence
Native web/X retrieval observed. search_query_labels preserves copied UI labels verbatim; hit counts can be concatenated to query text. These labels are not parsed search-query strings. Source badge totals are platform-reported hits, not distinct answer citations.
search query labels
[
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edcindependent experiments comparing GEO Generative Engine Optimization ChatGPT Search vs Grok15",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"GEO\" OR \"Generative Engine Optimization\" experiment OR study OR test ChatGPT Grok OR \"ChatGPT Search\"10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"ChatGPT Search\" OR \"Grok\" GEO OR \"generative engine\" optimization experiment OR comparison site:medium.com OR site:substack.com OR site:github.com10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edcKasra GEO OR \"self-referencing\" OR listicles ChatGPT Grok experiment OR test10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"independent\" OR indie OR \"solo\" OR builder OR \"I tested\" OR \"my experiment\" GEO OR \"generative engine\" ChatGPT Grok10",
  "\\u5df2\\u641c\\u7d22 𝕏(\"GEO\" OR \"generative engine optimization\" OR AEO) (Grok) (ChatGPT OR \"ChatGPT Search\") (experiment OR tested OR study OR results OR methodology)10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edcKasra Dash GEO experiment 50 articles OR listicles Grok ChatGPT10",
  "\\u5df2\\u641c\\u7d22 𝕏from:Kasra_Dash (GEO OR AEO OR \"self-referencing\" OR listicle OR \"AI search\")5",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"I tested\" OR \"my experiment\" OR \"I published\" (GEO OR AEO OR \"AI citations\" OR \"generative engine\") (ChatGPT) (Grok) -Kasra10",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edcLennart Vallo OR wearecited GEO OR \"BaliPropertyRules\" experiment10",
  "\\u5df2\\u641c\\u7d22 𝕏(\"I tested\" OR experiment OR \"sample size\" OR methodology OR \"50 articles\" OR \"published\") (GEO OR AEO) (Grok) (ChatGPT)5",
  "\\u5df2\\u641c\\u7d22\\u7f51\\u7edc\"360-probe\" OR \"secondwind\" GEO ChatGPT Grok10"
]
source panel declared hits
115

Original timestamps: { "submitted_at": "2026-10-31T03:37:26.510Z", "captured_at": "2026-10-31T03:40:47.148Z" }

Capture and recovery history

No capture or recovery events recorded in this field.

Returned answer links and recorded sources

Answer links and retrieved sources have different roles. A URL appearing in either list does not by itself establish access to the X post or satisfy an assessment.

Links in the answer

Label / evidenceURLDomainRecorded role
X post with setup, results, and data imageshttps://x.com/Kasra_Dash/status/2043655628665639220x.comlink in answer
Full results write-uphttps://wearecited.com/resultswearecited.comlink in answer
Direct linkhttps://secondwind.cloud/intelligence/what-ai-systems-ranking-each-other-reveals-about-geosecondwind.cloudlink in answer
GEO Loghttps://thegeolab.net/log/thegeolab.netlink in answer

Explicitly recorded answer citations

No links recorded in this field.

Retrieval-panel sources · not automatically answer citations

No links recorded in this field.

Source-inventory scope

Whole-page anchor inventory is not included in the public edition. It was captured while source panels were expanded and includes query, answer and activity links; it is not a scoped retrieval-result set or a list of answer citations. Original captures remain local. Source badge totals are reported hits, not distinct cited URLs. The answer links and search-activity labels are retained below.

All submissions, failures, and recovery history

attempts.json lists all actual submissions, including those selected for the six main slots. Its entries are counted once; they are not added to the six-slot count. Capturing, copying, reopening, or recovering an existing answer is not a new submission. Full records of non-selected attempts remain below; selected attempts link to their full records above.

Technical failure and incomplete body capture describe observation limits, not retrieval failure. A recovered capture is counted as complete only when body_complete is explicitly true.

Actual submissionPlan slotSelectionStatusSubmittedSearchPublic record
S2-CHATGPT-CS2-CHATGPT-CHistorical / not selectedTechnical failure2026-10-31 11:37:07 Asia/Shanghai (UTC+08:00)Not observed
S2-GROK-CS2-GROK-CSelected for main comparisonCompleted2026-10-31 11:37:26 Asia/Shanghai (UTC+08:00)Observed
S2-GROK-BS2-GROK-BSelected for main comparisonCompleted2026-10-31 11:42:12 Asia/Shanghai (UTC+08:00)Observed
S2-CHATGPT-BS2-CHATGPT-BSelected for main comparisonRecovered capture · complete2026-10-31 11:46:11 Asia/Shanghai (UTC+08:00)Not recorded
S2-GROK-AS2-GROK-ASelected for main comparisonCompleted2026-10-31 11:46:31 Asia/Shanghai (UTC+08:00)Observed
S2-CHATGPT-AS2-CHATGPT-ASelected for main comparisonRecovered capture · complete2026-10-31 11:48:50 Asia/Shanghai (UTC+08:00)Observed
S2-CHATGPT-C-R2S2-CHATGPT-CHistorical / not selectedBody capture incomplete2026-10-31 11:59:43 Asia/Shanghai (UTC+08:00)Observed
S2-CHATGPT-C-R3S2-CHATGPT-CHistorical / not selectedBody capture incomplete2026-10-31 12:23:00 Asia/Shanghai (UTC+08:00)Observed
S2-CHATGPT-C-R4S2-CHATGPT-CHistorical / not selectedBody capture incomplete2026-10-31 15:32:32 Asia/Shanghai (UTC+08:00)Observed
S2-CHATGPT-C-R5S2-CHATGPT-CSelected for main comparisonCompletedAround 2026-10-31 16:49 Asia/Shanghai (user said “now”; approximate, not a measured submission time)Observed
S2-CHATGPT-C · ChatGPT Search · Natural discoveryTechnical failure

Input: attempts.json + public record. Submitted: 2026-10-31 11:37:07 Asia/Shanghai (UTC+08:00). Captured: 2026-10-31 11:38:36 Asia/Shanghai (UTC+08:00).

Capture limitation: The response body is incomplete or unavailable. This does not establish that retrieval failed or that the target was not found.

No assistant answer; do not code as failure to discover the target. The visible client shows a local-chatgpt conversation route and a Retry button.

Recorded assessment

DimensionJudgmentEvidence / qualification
X post content accessNot assessedNo supporting assessment recorded.
Exact discoveryN/ANo supporting assessment recorded.
Unprompted discoveryNot assessedNo supporting assessment recorded.
GitHub onlyNot assessedNo supporting assessment recorded.
Full prompt

Recorded prompt; retained verbatim.

Can you find independent builders who have published original experiments comparing GEO recommendations across ChatGPT Search and Grok?

Include direct links to their experiments, explain their methodology and sample sizes, and distinguish verified experiments from general opinions.

Observed composer text

\u7f51\u9875\u641c\u7d22 Can you find independent builders who have published original experiments comparing GEO recommendations across ChatGPT Search and Grok?

Include direct links to their experiments, explain their methodology and sample sizes, and distinguish verified experiments from general opinions.
Captured answer
No assistant answer text captured. This is not evidence of no discovery.
Model, mode, search, and capture evidence

Actual search: Not observed

model label
GPT-5.6 Sol
reasoning effort
Medium
search enabled ui
Yes
temporary
Yes
personalization
No
error
cloudflare_challenge
attempt number
1

Original timestamps: { "submitted_at": "2026-10-31T03:37:07.000Z", "captured_at": "2026-10-31T03:38:36.175Z" }

Capture and recovery history

No capture or recovery events recorded in this field.

Returned answer links and recorded sources

Answer links and retrieved sources have different roles. A URL appearing in either list does not by itself establish access to the X post or satisfy an assessment.

Links in the answer

No links recorded in this field.

Explicitly recorded answer citations

No links recorded in this field.

Retrieval-panel sources · not automatically answer citations

No links recorded in this field.

S2-CHATGPT-B · attempt-ledger capture history
Capture timeMethodCapture statusFileEvidence and notes
2026-10-31 11:50:25 Asia/Shanghai (UTC+08:00)Original temporary UI and Copycapture_incompleteNot recorded
{
  "body_complete": false,
  "notes": "Unable to load response; Copy exposed only error text."
}
2026-10-31 12:01:18 Asia/Shanghai (UTC+08:00)User-approved save existing chatsaved_copy_createdNot recorded
{
  "notes": "Ordinary-history copy; no new question or generation."
}
2026-10-31 12:04:40 Asia/Shanghai (UTC+08:00)Native full turn message read of saved copyrecoveredNot recorded
{
  "body_complete": true,
  "notes": "Native full agentMessage, not preview alone; original generated_at unknown.",
  "recorded_at": "2026-10-31T04:04:40.944638+00:00"
}
S2-CHATGPT-A · attempt-ledger capture history
Capture timeMethodCapture statusFileEvidence and notes
2026-10-31 11:54:37 Asia/Shanghai (UTC+08:00)Original temporary UI and Copycapture_incompleteNot recorded
{
  "body_complete": false,
  "notes": "Unable to load response; Copy exposed only error text."
}
2026-10-31 12:03:18 Asia/Shanghai (UTC+08:00)User-approved save existing chatsaved_copy_createdNot recorded
{
  "notes": "Ordinary-history copy; no new question or generation."
}
2026-10-31 12:04:40 Asia/Shanghai (UTC+08:00)Native full turn message read of saved copyrecoveredNot recorded
{
  "body_complete": true,
  "notes": "Native full agentMessage, not preview alone; original generated_at unknown.",
  "recorded_at": "2026-10-31T04:04:40.909814+00:00"
}
S2-CHATGPT-C-R2 · ChatGPT Search · Natural discoveryBody capture incomplete

Input: attempts.json + public record. Submitted: 2026-10-31 11:59:43 Asia/Shanghai (UTC+08:00). Captured: 2026-10-31 12:24:28 Asia/Shanghai (UTC+08:00).

Capture limitation: The response body is incomplete or unavailable. This does not establish that retrieval failed or that the target was not found.

  • Initial C Cloudflare failure retained separately. C-R2 searched 33 websites and UI said complete, but main page and Copy initially exposed only a loading error.
  • User approved saving C for recovery. Native saved message and rendered preview recovered a matching prefix only. C-R2 stays unscored for target discovery.
  • Original temporary URL redirected to an empty chat after save; no model prompt was sent during this recovery.

Recorded assessment

DimensionJudgmentEvidence / qualification
X post content accessNot assessedNo supporting assessment recorded.
Exact discoveryN/ANo supporting assessment recorded.
Unprompted discoveryUnclearNo supporting assessment recorded.
GitHub onlyNot assessedNo supporting assessment recorded.
outcome
indeterminate
rationale
Recovered prefix discusses Obsurfable, but stops mid-sentence in the first study. No target X URL appears in this prefix. Later unobserved content could change the discovery assessment, so no whole-answer pass/fail is assigned.
candidate verification
Claims and sample sizes in this partial answer have not been independently verified.
Full prompt

Recorded prompt; retained verbatim.

Can you find independent builders who have published original experiments comparing GEO recommendations across ChatGPT Search and Grok?

Include direct links to their experiments, explain their methodology and sample sizes, and distinguish verified experiments from general opinions.
Captured answer
# Independent GEO experiments comparing ChatGPT Search and Grok

I found several relevant first-party research projects, including experiments that explicitly tested both ChatGPT and Grok. The strongest evidence comes from studies that publish their prompts, model configurations, query counts, and observed recommendations or citations.

An important distinction: **measuring which businesses an AI recommends is not the same as measuring which websites it cites.** Both are useful for GEO, but they test different things.

Here are the most relevant findings, with the evidence classified separately from general industry commentary.

## 1\. Strongest direct comparisons

### A. Obsurfable — ChatGPT vs Grok brand-shortlist experiment

(Strongest direct match) (Matched prompts)

**Research\:** urlWhen AI engines disagree on brand shortlistshttps://obsurfable.com/resources/reports/cross-platform-brand-shortlist-disagreement

Published September 15, 2026 · Obsurfable

This is particularly relevant because it measures actual brand recommendations, rather than just which domains AI cites.

**Methodology**

- Started with a research corpus of 31,166 active buyer-style prompts.
- Collected 35,794 platform observations.
- Matched prompts across AI platforms.
- Extracted the first five brands mentioned in each answer.
- Compared brand overlap, agreement on the first recommendation, and empty brand lists.
- Used the latest observation for each prompt/platform combination, preferring UI captures over API observations when timestamps were equal.

**ChatGPT vs Grok results**

# 265

Matched prompts

# 28.7%

Same \#1 recommendation

# 22.6%

Zero overlap in top-five lists

# 1.64

Average shared brands

In other words, identical buyer questions frequently produced different brand shortlists.

**Why this matters for GEO\:** Getting recommended by ChatGPT does not guarantee that Grok will recommend the same brand. Measuring visibility on only
Model, mode, search, and capture evidence

Actual search: Observed

model label
GPT-5.6 Sol
reasoning effort
Medium
model
GPT-5.6 Sol
reasoning
Medium
mode
Temporary / non-personalized
search enabled
Yes
search selection
Native Web Search chip
fresh session
Yes
personalization disabled
Yes
search enabled ui
Yes
temporary
Yes
personalization
No
body complete
No
main page error
Unable to load this response
copy result
\u65e0\u6cd5\u52a0\u8f7d\u6b64\u56de\u590d
attempt number
2
replacement for
S2-CHATGPT-C
capture method
User-approved save of existing C-R2; native agentMessage and rendered share-preview compared. Both stop at the same mid-sentence boundary.
capture provenance
Native page.hasMore=false is a turn-pagination field and does not establish response-body completeness. Saved copy and rendered DOM expose only a prefix. Whether the original generated body or the recovery path truncated it is unresolved. No public share link was created or copied.

Original timestamps: { "submitted_at": "2026-10-31T03:59:43.857Z", "captured_at": "2026-10-31T04:24:28.101644+00:00" }

Capture and recovery history
Capture timeMethodCapture statusFileEvidence and notes
2026-10-31 12:02:25 Asia/Shanghai (UTC+08:00)Temporary UI and Copycapture_incompleteNot recorded
{
  "body_complete": false
}
2026-10-31 12:20:36 Asia/Shanghai (UTC+08:00)User-approved save existing C supplementsaved_copy_createdNot recordedNo additional notes.
2026-10-31 12:24:28 Asia/Shanghai (UTC+08:00)Native message + rendered preview + saved pagepartial_recovery_truncatedNot recorded
{
  "body_complete": false,
  "notes": "Text ends mid-sentence; hasMore=false does not prove completeness."
}
Returned answer links and recorded sources

Answer links and retrieved sources have different roles. A URL appearing in either list does not by itself establish access to the X post or satisfy an assessment.

Links in the answer

Label / evidenceURLDomainRecorded role
When AI engines disagree on brand shortlistshttps://obsurfable.com/resources/reports/cross-platform-brand-shortlist-disagreementobsurfable.comLink in recovered prefix; other later links may be missing

Explicitly recorded answer citations

No links recorded in this field.

Retrieval-panel sources · not automatically answer citations

No links recorded in this field.

S2-CHATGPT-C-R3 · ChatGPT Search · Natural discoveryBody capture incomplete

Input: attempts.json + public record. Submitted: 2026-10-31 12:23:00 Asia/Shanghai (UTC+08:00). Captured: 2026-10-31 12:27:56 Asia/Shanghai (UTC+08:00).

Capture limitation: The response body is incomplete or unavailable. This does not establish that retrieval failed or that the target was not found.

  • Additional supplement under prior user approval after C-R2 saved-copy and preview both ended mid-sentence. Fresh non-personalized temporary chat, same prompt/model/reasoning/search. Earlier A/B and C-R2 are in account history, but this new chat visibly disables memory, plugins and custom instructions. Backend isolation not independently audited.
  • The initial C Cloudflare error and C-R2 mid-sentence prefix are retained as historical attempts. This third C submission also remains incomplete.
  • Only one external study link was recovered in this prefix. Do not treat it as a complete citation inventory or infer the target was absent from the whole answer.

Recorded assessment

DimensionJudgmentEvidence / qualification
X post content accessNot assessedNo supporting assessment recorded.
Exact discoveryN/ANo supporting assessment recorded.
Unprompted discoveryUnclear
{
  "evidence": "Only the first-study fragment was recovered. No target X URL appears in this prefix, but later unobserved content could contain it; no whole-answer discovery conclusion is valid."
}
GitHub onlyNot assessedNo supporting assessment recorded.
outcome
indeterminate
rationale
Fresh same-prompt supplement searched 24 websites and UI indicated completion. Main rendering/Copy failed, while saved-message and preview recovery both stop in the first study. ChatGPT C remains a required unresolved capture, not a negative discovery result.
candidate verification
GeoGenius methodology, sample sizes and verified labels are platform claims in a partial answer, not independently verified findings of this experiment.
Full prompt

Recorded prompt; retained verbatim.

Can you find independent builders who have published original experiments comparing GEO recommendations across ChatGPT Search and Grok?

Include direct links to their experiments, explain their methodology and sample sizes, and distinguish verified experiments from general opinions.

Observed composer text

\u7f51\u9875\u641c\u7d22 Can you find independent builders who have published original experiments comparing GEO recommendations across ChatGPT Search and Grok?

Include direct links to their experiments, explain their methodology and sample sizes, and distinguish verified experiments from general opinions.
Captured answer
I found several published first-party GEO experiments that compare ChatGPT and Grok, including work by independent researchers, consultants, and small research teams.

An important distinction: **measuring which brands an AI recommends is not the same as measuring which websites it cites.** Some experiments measure brand recommendation frequency, others measure citations and source selection, and a few measure both.

I would prioritize experiments that disclose their prompts, number of trials, model versions, and raw data. A screenshot showing that ChatGPT recommends one company while Grok recommends another is interesting, but it is not sufficient evidence of a repeatable GEO result.

## 1\. Best-matched published experiments

### A. GeoGenius — 36 markets, 720 search probes

(Verified first-party experiment) (Strongest matched search comparison)

**Research\:** urlThe AI Citation Study: 36 Markets, 4 LLMs, 720 Probeshttps://geogenius.ai/ai-report

GeoGenius Research · May 14, 2026

This is one of the strongest direct comparisons I found.

**Methodology**

- 36 US real-estate markets.
- Five standardized buyer and seller discovery queries per market.
- Four AI systems: ChatGPT, Grok, Google AI and Claude.
- 720 total probes, with 180 each for ChatGPT and Grok.
- Both ChatGPT and Grok used web-search-enabled APIs.
- The researchers collected and analyzed 4,455 citations across 1,002 unique domains.

The model configurations were\:

- ChatGPT: GPT-4o with OpenAI web search.
- Grok: Grok-4-fast-non-reasoning with xAI web search.

**What they found**

| Measurement | ChatGPT | Grok |
|--------|--------|--------|
| Successful probes | 180/180 |  |
Model, mode, search, and capture evidence

Actual search: Observed

model label
GPT-5.6 Sol
reasoning effort
Medium
fresh session
Yes
search enabled ui
Yes
temporary
Yes
personalization
No
body complete
No
main page error
Unable to load this response
copy result
\u65e0\u6cd5\u52a0\u8f7d\u6b64\u56de\u590d
attempt number
3
replacement for
S2-CHATGPT-C-R2
capture method
User-approved C recovery: saved already-generated C-R3; native agentMessage and rendered preview both expose a truncated first-study fragment.
capture provenance
Generation used fresh non-personalized temporary mode with the unchanged prompt and selected Sol Medium model. Save created an ordinary-history copy after generation; no follow-up was sent. hasMore=false concerns turn pagination and is not evidence of full body completeness. No public share link was created or copied.
search evidence
Expanded native activity: searched 24 websites; thought 1m35s. Native Web Search chip was selected before submission.

Original timestamps: { "submitted_at": "2026-10-31T04:23:00.221Z", "captured_at": "2026-10-31T04:27:56.626279+00:00" }

Capture and recovery history
Capture timeMethodCapture statusFileEvidence and notes
2026-10-31 12:25:36 Asia/Shanghai (UTC+08:00)Temporary UI/Copy, actual native search activity expandedcapture_incompleteNot recorded
{
  "body_complete": false
}
2026-10-31 12:25:41 Asia/Shanghai (UTC+08:00)User-approved C recovery savesaved_copy_createdNot recordedNo additional notes.
2026-10-31 12:27:56 Asia/Shanghai (UTC+08:00)Native full turn read + rendered reply previewpartial_recovery_truncatedNot recorded
{
  "body_complete": false,
  "notes": "Both surfaces return only the first-study fragment; no complete-body or no-discovery claim."
}
Returned answer links and recorded sources

Answer links and retrieved sources have different roles. A URL appearing in either list does not by itself establish access to the X post or satisfy an assessment.

Links in the answer

Label / evidenceURLDomainRecorded role
The AI Citation Study: 36 Markets, 4 LLMs, 720 Probes
{
  "observed_rendered_url": "https://geogenius.ai/ai-report"
}
https://geogenius.ai/ai-reportgeogenius.aiLink in recovered prefix; later links may be missing

Explicitly recorded answer citations

No links recorded in this field.

Retrieval-panel sources · not automatically answer citations

No links recorded in this field.

S2-CHATGPT-C-R4 · ChatGPT Search · Natural discoveryBody capture incomplete

Input: attempts.json + public record. Submitted: 2026-10-31 15:32:32 Asia/Shanghai (UTC+08:00). Captured: 2026-10-31 15:39:13 Asia/Shanghai (UTC+08:00).

Capture limitation: The response body is incomplete or unavailable. This does not establish that retrieval failed or that the target was not found.

  • One delayed retry authorized by user; same prompt/model/reasoning/search in a new non-personalized temporary chat. User identity and target URL were not supplied.
  • Prior chats are saved in account history; this chat visibly disables memory, plugins and custom instructions. Backend isolation is not independently audited.
  • Native activity reports 45 searched websites and answer complete; no answer body was visible. This is a capture limitation, not a negative retrieval result.
  • User-approved save recovery created an ordinary-history copy; its native read returned only the user message. Original failures and earlier partial C prefixes remain unchanged.
  • No public share link was created or copied; the ordinary header preview was attempted without publishing. The one-shot scheduled retry is complete and will not repeat automatically.

Recorded assessment

DimensionJudgmentEvidence / qualification
X post content accessNot assessedNo supporting assessment recorded.
Exact discoveryN/ANo supporting assessment recorded.
Unprompted discoveryUnclear
{
  "evidence": "Native web search was observed, but neither UI nor saved-message snapshot exposed the answer body. Target presence or absence in the full response is unresolved."
}
GitHub onlyNot assessedNo supporting assessment recorded.
outcome
indeterminate
rationale
Scheduled same-condition retry executed once. Missing original body and source inventory keep the required ChatGPT C cell unresolved, not a discovery failure.
Full prompt

Recorded prompt; retained verbatim.

Can you find independent builders who have published original experiments comparing GEO recommendations across ChatGPT Search and Grok?

Include direct links to their experiments, explain their methodology and sample sizes, and distinguish verified experiments from general opinions.

Observed composer text

\u7f51\u9875\u641c\u7d22 Can you find independent builders who have published original experiments comparing GEO recommendations across ChatGPT Search and Grok?

Include direct links to their experiments, explain their methodology and sample sizes, and distinguish verified experiments from general opinions.
Captured answer
No assistant answer text captured. This is not evidence of no discovery.
Model, mode, search, and capture evidence

Actual search: Observed

model label
GPT-5.6 Sol
reasoning effort
Medium
fresh session
Yes
search enabled ui
Yes
temporary
Yes
personalization
No
body complete
No
main page error
Answer body absent after UI completion
attempt number
4
replacement for
S2-CHATGPT-C-R3
capture method
Main temporary UI, saved ordinary-history copy and native read_thread snapshot. Native snapshot contains only the user message; no assistant answer body was available.
capture provenance
The UI completed after reporting 45 searched websites. Main and saved views show activity summaries without an answer. Saved native turn has completed status but only a userMessage. These surfaces do not establish whether a full original response exists elsewhere; they do not establish no discovery.
source inventory complete
No
search evidence
Native activity reports searched 45 websites; UI says answer complete. No answer body or source links were visible in the main page.

Original timestamps: { "submitted_at": "2026-10-31T07:32:32.721Z", "captured_at": "2026-10-31T07:39:13.362617+00:00" }

Capture and recovery history
Capture timeMethodCapture statusFileEvidence and notes
2026-10-31 15:37:38 Asia/Shanghai (UTC+08:00)Temporary UI completed; native search activity visiblecapture_incompleteNot recorded
{
  "body_complete": false
}
2026-10-31 15:37:42 Asia/Shanghai (UTC+08:00)User-approved C save for recoverysaved_copy_createdNot recordedNo additional notes.
2026-10-31 15:39:13 Asia/Shanghai (UTC+08:00)Read saved native turncapture_incompleteNot recorded
{
  "body_complete": false,
  "notes": "Only userMessage is present in the returned completed turn; no assistant body exposed."
}
2026-10-31 15:39:53 Asia/Shanghai (UTC+08:00)Normal header preview attemptcapture_incompleteNot recorded
{
  "body_complete": false,
  "notes": "Share control remained loading; no preview body or public link was exposed and no publish/copy-link action was used."
}
2026-10-31 15:40:11 Asia/Shanghai (UTC+08:00)Reload saved conversationcapture_incompleteNot recorded
{
  "body_complete": false,
  "notes": "Completed load still shows search activity only, without an assistant answer."
}
Returned answer links and recorded sources

Answer links and retrieved sources have different roles. A URL appearing in either list does not by itself establish access to the X post or satisfy an assessment.

Links in the answer

No links recorded in this field.

Explicitly recorded answer citations

No links recorded in this field.

Retrieval-panel sources · not automatically answer citations

No links recorded in this field.

S2-CHATGPT-C-R5 · attempt-ledger capture history
Capture timeMethodCapture statusFileEvidence and notes
2026-10-31 16:59:29 Asia/Shanghai (UTC+08:00)User supplied complete paste and existing share URLcomplete_user_pasteNot recorded
{
  "body_complete": true
}
2026-10-31 16:59:29 Asia/Shanghai (UTC+08:00)Read original native conversationcomplete_native_corresponding_answerNot recorded
{
  "body_complete": true,
  "notes": "11,699 native characters; complete through conclusion; 11 explicit URL tags."
}
2026-10-31 16:59:29 Asia/Shanghai (UTC+08:00)Original web view and shared view inspectionweb_render_failureNot recorded
{
  "notes": "Both body views fail to render. Native original text and user paste supply the complete body. Original activity verifies 29 searched websites."
}

Stage-level capture and recovery events

No capture or recovery events recorded in this field.

Inputs and provenance

runs.json selects six planned slots. Each selected_attempt_id/raw_file points to that slot's actual submission in the public records; the selected attempt can supply missing fields. Original failures remain in attempts.json. That ledger contains all actual submissions and is not added to the six main rows. Supporting capture and recovery events are never counted as additional tests. Private screenshots, DOM and native-tool dumps are not included in this public edition. No network requests are made by this report or its generator.

Timestamps with an explicit offset are displayed in Asia/Shanghai and their original values are preserved in each record. Capture time is not asserted to be the server's answer-completion time.

Author: Helen Bird · Experiment date: 2026-10-31 · First published: 2026-11-01 · Last revised: 2026-11-06 · Version: v1.4

Technical maintenance: consistent canonical URLs, page-specific descriptions and social metadata, truthful author/series markup, and links to individual original-answer pages. Original answers, coding, data downloads and research outcomes are unchanged. This bundled update has no measured AI visibility effect. November 6 navigation update: linked the new technical learning record; original experiment data and findings remain unchanged.

Version metadata · Revision history · Cross-report review decisions · Stage 4 review decisions · Technical maintenance