MPWhat The Shift Is
AI search is a new category, and it is moving faster than anything search has done in twenty years. The platforms change how they choose sources without announcing it. Google ships updates that reach AI Overviews and blue links at the same time. New measurement tools appear monthly, and the research about them contradicts the research from the month before. A business owner cannot follow all of that, and should not have to.
So every Monday morning we read what changed across Google AI Overviews, ChatGPT, Perplexity, Copilot and the studies around them, and we write down the handful of things that matter to a business trying to get found, understood and recommended. Plain English first. The detailed brief underneath for the marketing people. If you have ever wished somebody would just tell you what changed and what to do about it, without a webinar, this is that.
Two labels do the honest work in every issue. Confirmed means a platform documented it or the data is public. Our read means it is our interpretation, which is worth exactly what you paid for it, and we say so.
In Plain English: What Changed and What To Do
One line in robots.txt will soon say no to AI training, for everyone at once
On October 30, OpenAI, Google and Microsoft each announced they will honor a shared signal in robots.txt, the small file that tells crawlers what they may do on your site, for whether your content can be used to train AI models. The proposal was filed with the Internet Engineering Task Force the same day by an engineer at Anthropic. Instead of writing a separate rule for every company's crawler, a site writes one rule under a shared name, __usage_ai-training, and every participating company reads it. Google says it plans to support the signal in 2027, and that an opt-out which does not mention Google-Extended will be treated as if the site had blocked Google-Extended. OpenAI says it will implement the signal in the coming months; Microsoft says it starts reading it on February 1, 2027. Each company's existing crawler controls keep working. The proposal is a draft, not a standard.
Two different questions keep getting mixed up. Training is whether your words help build the next version of a model. Being cited is whether an AI answer quotes or recommends you today. This signal is about the first one. Google's own documentation says Google-Extended does not affect inclusion in Google Search and is not a ranking signal. For a business whose whole goal is to be the name AI recommends, opting out of training is a choice with a cost, because what a model already knows about you shapes whether it mentions you at all. It should be a decision, not a default someone else picked.
Nothing to add to your site yet; no company reads the new line today. This week, open yourwebsite.com/robots.txt and write down what it blocks. If you see GPTBot, Google-Extended, CCBot or a blanket rule you did not choose, find out who added it (a host, a security plugin, a past developer) and decide on purpose. If you want AI to recommend you, allowing is usually the right call.
Microsoft splits 'train on my site' from 'quote my site', and says blocking the second costs you
Also on October 30, Microsoft published how it will read site preferences from February 1, 2027. Training is set in robots.txt or per page and defaults to allowed. Grounding, which is Microsoft's word for using your page to answer a question live in Bing, Copilot and Microsoft 365, is set per page only and also defaults to allowed. A page can limit grounding to Microsoft's own products, or block it entirely. Microsoft states that either limit significantly reduces brand visibility in these AI experiences, that allowing it makes a page eligible to be cited with a link, and that none of this affects whether the page can appear in regular Bing results. One catch: old NoArchive and NoCache tags stop counting as AI preferences after March 1, 2027, and a temporary grounding setting for sites that use them opens in Bing Webmaster Tools on December 15.
This is the first time a major platform has written down, in its own documentation, that opting out of AI answers has a visibility price. It also gives a name to the distinction that matters most for a business: being trained on and being quoted are separate switches. The NoArchive detail is the sleeper. Those tags were often added years ago for reasons that had nothing to do with AI, and some sites carry them without anyone remembering why.
Ask your developer, or view the source of your homepage and search for noarchive and nocache. If either is there and you want to be cited in Copilot, plan to remove it well before March 1, 2027. Put December 15 on the calendar to look at the new setting in Bing Webmaster Tools, and verify your site there if you have not already.
Search your own name. Google has probably written a summary of you
SE Ranking's SE Visible team published a study on October 22 that checked 1,000 US brand searches three times each on desktop in October. A Google AI Overview appeared on 82.5 percent of them on average. The summaries leaned on the brand's own website far more than the regular results do: brand sites were 33.8 percent of AI Overview citations but 17 percent of top-ten results. Only 36.8 percent of those brand-site citations were homepages; product, service and category pages made up 22.9 percent and About and company pages 11.7 percent, and 62.2 percent of the cited brand pages were not in the top ten results for that search. Google Business Profiles were 2.7 percent of US citations. Then, from about October 31, several SEOs reported brand-search AI Overviews moving into the right-hand column on desktop, the spot the knowledge panel used to hold. Google has not commented on that change.
When someone hears your name and Googles it, the first thing they read is increasingly a paragraph Google wrote about you, assembled from pages you may not have opened since launch. It reaches past the homepage to service pages and the About page, so a thin or out-of-date page is exactly what gets summarized. If the right-column placement sticks, that summary becomes the new business card beside your results. Nobody hands out a business card they have not read.
Google your business name on a desktop and on a phone. Read the AI Overview as a stranger would, check every claim, and click every cited link. Then fix the pages it pulls from: service pages that say what you do, where and for whom; an About page with real facts such as founding year, team and credentials; and the same facts, word for word where it makes sense, on your Google Business Profile.
The case against chasing AI traffic, and what it does and does not prove
On October 21, Marie Haynes published an analysis of 12 months of Google Analytics data from 100 randomly sampled websites, more than 145 million sessions in all. ChatGPT sessions came to 0.62 percent of Google organic sessions, and all AI assistants together to 0.73 percent. ChatGPT visitors converted at 13.8 percent against 3.02 percent for Google organic, but Google organic still drove 964 conversions to 36 from every AI assistant combined, about 27 times as many. She also reports that ChatGPT referral visits have declined since GPT-5 launched in August 2025, and concludes that chasing AI citations is a distraction from Google Search.
An AI search agency is about to tell you that Google still sends most of the customers. It is true, and we would rather say so than sell you a fantasy. Two things are worth holding next to it. First, the study counts clicks, and a lot of AI influence never becomes a click: someone reads a recommendation, then Googles the business or calls it, and that visit is credited to Google or Direct. Second, most of what earns an AI citation, clear pages, accurate facts, a good reputation elsewhere, is the same work that ranks in Google. The useful conclusion is not to pick a side; it is to stop paying twice for the same work.
Do not move budget out of Google to chase AI. In Google Analytics, compare the conversion rate of ChatGPT visitors with organic visitors on your own site; small numbers that convert well are still worth protecting. And in Search Console, track impressions for searches that include your business name, which is the closest free proxy for people who heard about you somewhere first.
Your To-Do List This Week
- Google yourself like a stranger.Search your business name on a desktop and a phone, read the AI summary, click every source it cites, and fix the service and About pages it is built from.
- Read your robots.txt before anyone else edits it.Write down what it blocks today, and check your page source for noarchive and nocache tags, so any AI opt-out you make is a decision rather than an inheritance.
- Measure AI by more than referrals.Compare the conversion rate of ChatGPT visitors with organic visitors, and track Search Console impressions for searches that include your name.
The Full Brief
The detailed version, for marketing directors and anyone who has to defend a number in a meeting. Every item carries its source. Where we are interpreting, it says so.
Google, OpenAI and Microsoft back a shared robots.txt signal for AI training preferences
On October 30, 2026, OpenAI announced it "will honor a common signal" in robots.txt for preferences about the use of content "for foundational generative AI training," aims to implement it "in the coming months," and said the signal "will not replace the controls we already provide through robots.txt." OpenAI described the work as drawing on the IETF AI Preferences working group and encouraged other AI providers to honor the signal. OpenAI: Making it easier for websites to express AI training preferences
Google published "Supporting a shared AI training control for websites" on its Search Central blog the same day. Google plans to support the shared control in 2027; if a site uses it to opt out of training without explicitly specifying Google-Extended, Google plans to interpret that as if the publisher had disallowed Google-Extended, and Google-Extended and the existing Search Console control for Search generative AI keep working alongside it. Google's crawler documentation states that Google-Extended does not impact a site's inclusion in Google Search and is not used as a ranking signal. Google Search Central Blog: Supporting a shared AI training control for websites
The proposal is Internet-Draft draft-kelley-aipref-training-preference-signal-00, dated October 30, 2026, by Kevin Kelley of Anthropic, intended for the Standards Track but with no formal IETF standing yet. It adds no new robots.txt syntax: a group with the User-Agent policy token __usage_ai-training uses ordinary Disallow and Allow rules to express a preference against or in favor of AI training for matching paths, wildcard groups do not count as expressing the preference, and the draft states the preference does not control crawling. IETF Datatracker: Machine-Readable Training Preference Signal
This is a simplification for publishers who already want out of training, and it does not change citation or ranking for anyone today. The risk for businesses that want AI visibility is the opposite one: a one-line opt-out is easy to add by accident, through a host default, a security plugin or a developer copying a template, and model knowledge of a brand is one of the inputs that decides whether that brand is mentioned at all. Treat training preference as a business decision with an owner.
Audit robots.txt for every client now and record the current state for GPTBot, Google-Extended, CCBot, ClaudeBot and wildcard rules, with the reason for each block. Add the __usage_ai-training token to the change-control checklist so it cannot appear without sign-off. Revisit when each provider confirms implementation, Microsoft on February 1, 2027 and Google during 2027.
Microsoft separates training and grounding preferences and states the visibility cost of opting out
On October 30, 2026, Microsoft Bing published new preference controls taking effect from February 1, 2027 as Microsoft crawls. Training can be set in robots.txt for a site or path, using the shared __usage_ai-training token that Microsoft says brings forward the work of the IETF AI Preferences working group, or per page, and defaults to Allow; per-page settings override robots.txt. Grounding is set per page only and defaults to Allow; DisallowExtended limits use to Microsoft-operated experiences and excludes third-party grounding services such as Web IQ and Foundry API, and Disallow blocks grounding entirely. Grounding covers web-grounded responses in Bing, Copilot and Microsoft 365, and Microsoft states that choosing DisallowExtended or Disallow significantly reduces brand visibility in these AI experiences. Allowing grounding makes content eligible to inform responses with links and citations, guarantees no inclusion, traffic or payment, and Bing search result eligibility is not affected. Temporary grounding preferences for sites using NoArchive or NoCache open in Bing Webmaster Tools on December 15, 2026; after March 1, 2027, NoArchive and NoCache are no longer read as training or grounding preferences; temporary preferences expire September 30, 2027. Bing Search Blog: Your content, your choice
Microsoft has put in writing what practitioners have argued for two years: being trained on and being quoted are separate permissions, and refusing the second has a cost. Expect other platforms to be pushed toward the same distinction. The NoArchive and NoCache change matters most for older sites and for some legal, medical and financial sites where those tags were added for caching or compliance reasons long before AI answers existed.
Crawl client sites for noarchive and nocache in meta robots tags and X-Robots-Tag headers. Where present, document why, and agree with the client whether to remove them or set a temporary grounding preference in Bing Webmaster Tools from December 15. Keep grounding at Allow for any client whose goal includes AI recommendations, and verify every client in Bing Webmaster Tools before December 15.
AI Overviews appear on 82.5 percent of US brand searches and cite deep brand pages; brand AI Overviews move to the right column
SE Visible (SE Ranking) published a study on October 22, 2026 of 1,000 US and 1,000 German brand keywords across 17 industries, each checked three times on desktop in October 2026, with source analysis of 827 US and 405 German AI Overviews. In the US, AI Overviews appeared on an average of 82.5 percent of brand searches, against 40.4 percent in Germany. Brand websites held 17 percent of US organic top-ten results but 33.8 percent of AI Overview citations. Homepages were 71.1 percent of brand-site URLs in the organic top ten but 36.8 percent of brand-site AI Overview citations; product, service and category pages rose to 22.9 percent and About and company pages to 11.7 percent. 62.2 percent of US brand-site citations pointed to URLs outside the organic top ten for the same search. Google Business Profiles were 2.7 percent of US citations and 6.4 percent in Germany. The study notes it is a snapshot and does not measure accuracy, traffic or conversions. SE Visible: AI Overviews on branded searches
Search Engine Roundtable reported on November 2, 2026 that Google moved AI Overviews for many brand searches from the main column to the right-hand column on desktop, effectively in place of the brand knowledge panel, citing screenshots shared by Chris Long and an October 31 post by David McSweeney. Google has not confirmed the change. Search Engine Roundtable: Google AI Overviews for brands moved to right column
Branded search used to be the safe query: your homepage first, your knowledge panel beside it. It is now a summary that Google writes, drawing on interior pages that rarely get the attention homepages do. The right-column move, if it holds, makes that summary the de facto knowledge panel. For local and service businesses, the low Business Profile share in the US suggests the website, not the profile, is doing most of the talking.
Add branded queries (name, name plus city, name plus reviews, name plus pricing) to monthly AI visibility checks on desktop and mobile, capturing the AI Overview text and cited URLs. Prioritize service, product and About pages that appear in citations for factual accuracy, clear entity information and consistency with the Business Profile.
Marie Haynes: ChatGPT sends 0.62 percent of Google organic sessions across 100 sites
Marie Haynes published "GEO is a distraction" on October 21, 2026, based on 12 months of GA4 data from 100 randomly sampled websites with more than 145 million sessions; 55 of the sites tracked conversions. ChatGPT sessions were 0.62 percent of Google organic sessions, and all LLMs combined 0.73 percent. ChatGPT converted at 13.8 percent against 3.02 percent for Google organic, while Google organic drove 964 conversions to 36 from all LLMs combined, about 27 times more. She reports ChatGPT referral visits grew until GPT-5 launched in August 2025 and have declined since, calls the dataset small, and recommends focusing on Google Search, tracking key events and preparing for agentic search. Marie Haynes: GEO is a distraction
The numbers are a fair correction to anyone selling AI referral traffic as a replacement for Google. They measure last-click sessions, which undercount AI influence that turns into branded searches, calls and direct visits, and they say nothing about how often a brand is recommended inside answers that produce no click. The higher ChatGPT conversion rate is consistent with visitors arriving further along in their decision. Our position has not changed: the overlap between ranking in Google and being cited by AI is large, so the work should be planned once and measured in both places.
Report AI referrals alongside conversion rate, not sessions alone. Add branded search impressions from Search Console and, where available, call tracking and self-reported attribution ("How did you hear about us?") to AI visibility reporting. Do not reallocate Google budget to AI work; reallocate within the program toward pages that serve both.
Questions People Ask
Can I stop AI companies from training on my website?
Partly, and it is getting easier. Today you block each company's training crawler in robots.txt, such as GPTBot for OpenAI or Google-Extended for Google. On October 30, 2026, OpenAI, Google and Microsoft said they will also honor one shared robots.txt rule, __usage_ai-training, so a single line reaches all of them. Microsoft starts reading it on February 1, 2027, Google during 2027 and OpenAI in the coming months. It is a draft proposal, not yet a standard.
Will blocking AI training remove my business from ChatGPT or Google AI Overviews?
Not directly. Training and citing are separate. Google says Google-Extended does not affect inclusion in Google Search or rankings, and Microsoft says its training preference is separate from grounding, the setting for being quoted in Bing and Copilot answers. Blocking grounding does reduce visibility, Microsoft says, and opting out of training can mean a model knows less about you, which can make it less likely to mention you.
How often does Google show an AI Overview when someone searches a business name?
Often. An SE Ranking study published October 22, 2026 found an AI Overview on 82.5 percent of US brand searches on average, checked three times each on desktop. The summaries cited the brand's own site more than the regular results did, including service, product and About pages, and in late October 2026 many of them began appearing in the right-hand column on desktop.
Is traffic from ChatGPT worth it for a small business?
It is small but it converts. In Marie Haynes's 100-site study published October 21, 2026, ChatGPT sent 0.62 percent as many sessions as Google organic but converted at 13.8 percent against 3.02 percent. Google still drove about 27 times as many conversions. The practical answer is to do work that helps both, and not to judge AI visibility by referral clicks alone.
Was there a Google algorithm update in late October 2026?
Google has not confirmed one. Its Search Status Dashboard lists nothing after the September 2026 spam update, which finished in early October. Rank trackers and forums reported volatility around October 20 and 21 and again on October 24. A traffic dip during unconfirmed volatility is weather, not a verdict; check Search Console before acting.
What is The Shift?
The Shift is Agency AEO's weekly brief on AI search news. Every Monday morning we review what changed across Google AI Overviews, ChatGPT, Perplexity, Copilot and the research around them, explain in plain English what it means for a business, and separate what is confirmed from what is our read. It exists because AI search is a new category that changes weekly.
Sources & Further Reading
- OpenAI: Making it easier for websites to express AI training preferences (October 30, 2026)
- Google Search Central Blog: Supporting a shared AI training control for websites (October 30, 2026)
- IETF Datatracker: Machine-Readable Training Preference Signal, draft 00 (October 30, 2026)
- Bing Search Blog: Your content, your choice: clearer preferences for training and grounding (October 30, 2026)
- Google Search Central: Control what you share with Google Search
- SE Visible: AI Overviews on branded searches (October 22, 2026)
- Search Engine Roundtable: Google AI Overviews for brands moved to right column (November 2, 2026)
- Marie Haynes: GEO is a distraction (October 21, 2026)
- Search Engine Roundtable: November 2026 Google webmaster report (November 2, 2026)
- Google Search Status Dashboard