How to scrape Bing Copilot Search (Bing AI Mode) in 2026

How to scrape Bing Copilot Search (Bing AI Mode) in 2026

Ricardo Batista

Founder, cloro

November 6, 2026

•

7 min read

•

Web Scraping

AI Search

Tutorial

TL;DR

To scrape Bing Copilot Search with Python, open bing.com/copilotsearch?q= in Playwright, find the answer iframe, wait until the answer stops growing, then read the frame’s DOM. The outer page holds the search chrome and none of the answer.

Sources sit in the “Reading sources” list as data-url , data-title and data-snippet attributes. The pages behind “+N more” are already in the DOM, so you do not need to click.

The number on a citation chip is a count, not an index: a chip labelled “2” covers two sources. In our 12 live samples, 234 of 242 chips could be resolved to URLs from the page alone.

Throttle your browsers. On 2026-11-07, 11 of 12 requests sent at once to our own fleet failed after 171 seconds. The same 11 succeeded when we sent them 3 at a time.

If you would rather not scrape Bing Copilot yourself, cloro’s Bing AI Mode API returns the text, sources, citation pills and markdown in one POST, at 4 credits per async request.

The page goes by several names. Microsoft calls it Copilot Search in Bing and serves it at bing.com/copilotsearch . Microsoft is testing the label “AI Mode” for the Copilot tab in Bing, cloro sells its endpoint as Bing AI Mode, and other vendors sell it as Bing Copilot.

All three names point at the same page, so when you scrape Bing Copilot, bing.com/copilotsearch is what you load. The Copilot app at copilot.microsoft.com is a separate chat product with its own transport and session handling. This guide does not cover it; our Copilot app scraping guide does.

Everything below comes from 12 Copilot Search pages we scraped on 2026-11-07 through cloro’s Bing AI Mode API, plus a timing run of 10 more prompts on 2026-11-08. The 12 prompts covered shopping, local, recipe, news, health, finance, sports, travel and how-to queries. We stored the outer page and the answer iframe for each one, and every selector in this post comes from those files.

Copilot Search is a search page, not the Copilot app

The difference decides how you scrape Bing Copilot. The app streams its answer over a socket. Copilot Search writes its answer into a normal web page, inside an iframe, and you read it from the DOM.

Copilot Search in Bing Copilot app
URL bing.com/copilotsearch?q=... copilot.microsoft.com
Where the prompt goes The q parameter of the URL A chat message
Where the answer arrives An iframe on a Bing results page WebSocket frames
What you parse The frame’s DOM The event stream
Guide This post How to scrape Copilot

The prompt rides in the URL, so very long prompts run into URL length limits. cloro’s API caps a Bing AI Mode prompt at 3,500 characters after URL encoding for that reason, and a DIY scraper should check the encoded length before it opens the page.

Copilot Search answers a query with a short essay and a handful of cited pages. If you track AI visibility, you need to know which prompts cite your pages and which cite a competitor. Microsoft offers no API for this page (see the FAQ below), so the rendered page is the only source. Once you can scrape Bing Copilot reliably, Copilot rank tracking is the next step.

What a Copilot Search page contains

Take the prompt “best italian restaurants in chicago”. The page opens with a reasoning panel: Searching for "best italian restaurants in chicago" , then “Reading sources and understanding” above six site names and a “+3 more” button. Under it, the answer starts with a one-sentence summary, breaks into sections with headings, and carries inline chips such as “The Infatuation +2” or “Bing”. A table lists restaurants, and a row of source cards closes the answer.

All 12 pages had this same layout, including the shopping, local and video prompts. Bing showed no product, place, map, video or ad module in any of them. So if you scrape Bing Copilot for one query type, the same parser works for every type we tried.

Page element Where it sits in the frame Seen in our 12 samples cloro API field
Search query h2.b_ts_ls_hdrtext 12/12, equal to the prompt every time none
“Reading sources” list, with “+N more” ul.b_ts_sources > li.b_ts_source_pill 12/12, 6 to 11 pages each sources (same order, folded pages included)
Inline citation chips ( 1 , The Infatuation +2 ) div.gs_sup_cit > a 11/12 (health had none), 242 in total citationPills
Source cards below the answer div.gs_cit[data-citnumb] 12/12, 2 or 3 cards each already in sources
Per-section “N Sources” groups .gs_cit_stry_preview_txt 12/12 none (same pages as the chips)
Summary sentence, headings, lists .gs_heroTextHeader , h3 , h4 12/12 text , markdown
Tables table inside the answer 6/12 markdown only
Follow-up question the last block of the answer 12/12 end with an offer, 6 under a heading markdown only

Local answers change how you count citations. In the local-restaurants sample, Bing cited its own local listings with chips labelled “Bing”: 18 of the 42 citation pills in that answer point at bing.com, and 5 of its 14 sources are Bing search pages, 4 of them local listings ( filters=local_ypid ). If you rank domains by citations, decide up front whether bing.com counts.

Bing also wrote the restaurants’ addresses and phone numbers as plain text, labelled “Places API result”, with no structured place object behind them.

Almost everything you want is in data-* attributes and short, readable class names. Tables and the follow-up question exist only as rendered HTML, and you convert them yourself if you need them.

How to scrape Microsoft Copilot answers programmatically from Bing with Python

To scrape Bing Copilot, the code below runs five steps for each prompt:

Open bing.com/copilotsearch?q= with the prompt in the URL.

Find the answer iframe and wait until the answer stops changing.

Read the frame’s DOM for the query, the answer text and the tables.

Collect the sources from the “Reading sources” list, folded pages included.

Map each citation chip to its sources.

Step 1: open the page and wait for the answer to finish

In the outer page, the answer frame sits at #b_copilot_search li.ca_turn div.frame_cont iframe , and its aria-label is the query followed by “result container”. In the pages we stored, the iframe carries the answer document in its srcdoc attribute. In a live browser, read the frame through Playwright instead of parsing the attribute. Scripts inside the frame write the answer one token at a time: in the stored pages, every word is its own span.gs_tkn .

We found no “done” flag in the DOM. Finished pages still say “Reading sources and understanding” in the reasoning panel. So the code below polls the size of the answer container and treats three quiet seconds as the end.

import

asyncio

from

urllib.parse

import

quote_plus

from

playwright.async_api

import

async_playwright

COPILOT_SEARCH

=

"https://www.bing.com/copilotsearch?q="

FRAME

=

"#b_copilot_search .frame_cont iframe"

SIZE_JS

=

"""() => {

const el = document.querySelector('.b_genserp_container');

return el ? el.innerHTML.length : 0;

}"""

async

def

wait_until_stable

(frame, quiet_s

=

3.0

, timeout_s

=

90

):

last, quiet

=

1

,

0.0

for

_

in

range

(

int

(timeout_s

/

0.5

)):

size

=

await

frame.evaluate(

SIZE_JS

)

if

size

>

0

and

size

==

last:

quiet

+=

0.5

if

quiet

>=

quiet_s:

return

else

:

quiet

=

0.0

last

=

size

await

asyncio.sleep(

0.5

)

raise

TimeoutError

(

"Copilot Search answer did not settle"

)

Step 2: read sources, chips, cards and text

Run the extraction inside the frame, so one round trip returns everything. The “Reading sources” items behind “+N more” carry the class b_hide . They are in the DOM with the same attributes as the visible ones, so the code keeps them and marks them as folded.

EXTRACT_JS

=

"""() => {

const all = (sel, root = document) => Array.from(root.querySelectorAll(sel));

const answer = document.querySelector('.b_genserp_container');

const query = document.querySelector('h2.b_ts_ls_hdrtext')?.innerText.trim();

const sources = all('li.b_ts_source_pill').map((li, i) => ({

position: i + 1,

title: li.dataset.title,

url: li.dataset.url,

site: li.dataset.domain,

snippet: li.dataset.snippet,

folded: li.classList.contains('b_hide'),

}));

const cards = all('.gs_cit[data-citnumb]', answer).map(c => ({

num: c.dataset.citnumb,

url: c.dataset.url,

site: c.dataset.displayname,

}));

const chips = all('.gs_sup_cit > a', answer).map(a => {

const dom = a.querySelector('.gs_sm_cit_dom');

return {

href: a.href,

label: dom ? Array.from(dom.children).map(n => n.innerText).join(' ') : '',

sups: (dom?.dataset.sups || '').split(',').filter(Boolean),

};

});

const hero = answer.querySelector('.gs_heroTextHeader')?.innerText.trim();

// Remove chips and source widgets so the text and tables read cleanly.

all('.gs_sup_cit, .gs_cit_cont, .gs_cit_stry_preview', answer).forEach(n => n.remove());

const tables = all('table', answer).map(t =>

Array.from(t.rows).map(r => Array.from(r.cells).map(c => c.innerText.trim())));

return { query, sources, cards, chips, hero, tables, text: answer.innerText.trim() };

}"""

Step 3: resolve each chip to its URLs

A chip’s link holds one URL, but the chip can stand for several sources, and a scraper that reads only the link loses the rest. The data-sups attribute on .gs_sm_cit_dom lists the citation numbers the chip covers, and the badge shows how many there are. Across our samples, a chip labelled “1” always covered one source, “2” covered two, and “The Infatuation +2” covered three.

To turn numbers into URLs, build a map from the source cards ( data-citnumb to data-url ) and from every chip that covers a single number. That map resolved 234 of the 242 chips in our samples. The other 8, six in local-restaurants and two in recipe, include a number that no single link on the page spells out.

“The Infatuation +2” is one of them: it covers numbers 6, 7 and 8, and number 7 never appears alone. cloro’s API returned all three pages for that chip (The Infatuation, Secret Chicago and Tripadvisor).

def

resolve_chips

(data):

by_num

=

{card[

"num"

]: card[

"url"

]

for

card

in

data[

"cards"

]}

for

chip

in

data[

"chips"

]:

if

len

(chip[

"sups"

])

==

1

:

by_num.setdefault(chip[

"sups"

][

0

], chip[

"href"

])

for

chip

in

data[

"chips"

]:

chip[

"urls"

]

=

[by_num.get(n)

for

n

in

chip[

"sups"

]]

# None: not on the page

return

data

async

def

scrape

(browser, prompt):

page

=

await

browser.new_page()

try

:

await

page.goto(

COPILOT_SEARCH

+

quote_plus(prompt),

timeout

=

30_000

)

iframe

=

await

page.wait_for_selector(

FRAME

,

timeout

=

30_000

)

frame

=

await

iframe.content_frame()

await

wait_until_stable(frame)

return

resolve_chips(

await

frame.evaluate(

EXTRACT_JS

))

finally

:

await

page.close()

async

def

main

(prompts, max_parallel

=

3

):

gate

=

asyncio.Semaphore(max_parallel)

async

with

async_playwright()

as

p:

browser

=

await

p.chromium.launch()

async

def

one

(prompt):

async

with

gate:

return

await

scrape(browser, prompt)

results

=

await

asyncio.gather(

*

(one(q)

for

q

in

prompts),

return_exceptions

=

True

)

await

browser.close()

return

results

if

__name__

==

"__main__"

:

print

(asyncio.run(main([

"best italian restaurants in chicago"

])))

Run three at a time, not twelve

Each answer takes about 20 seconds to write, so when you scrape Bing Copilot in parallel, open browsers pile up fast. On 2026-11-07 we sent all 12 sample prompts to our own fleet at once, and 11 of them failed after 171 seconds. When we sent the same 11 three at a time, every one succeeded, in 19 to 43 seconds each.

That is what we saw on our infrastructure on one day, and we cannot state a Bing limit from it. Start your own scraper at three concurrent pages, as max_parallel does above, and raise it only while you watch the failure rate.

Country targeting comes from the exit IP

When you scrape Bing Copilot yourself, the location comes from where the browser connects. Our US request for “emergency plumber near me” came back about East Haven, Connecticut, and 9 of its 10 sources were pages about that town. For each country you want, you need a proxy exit in that country. For state-level local answers, you need exits in each state.

What breaks when you scrape Bing Copilot in production

The class names are short and readable ( b_ts_* , gs_* ). That suggests people write them by hand and no build step generates them, so they should last longer than hashed names. Avoid the parts that are clearly generated: the indexed id #b_gsc_0 , the insertion-id attributes (a new UUID on every render) and the data-bm counters.

Matching on text is fragile. The “+3 more” button’s aria-label reads “3 more page sources are avaiable”, with Bing’s spelling. The follow-up heading appeared as “Follow-up question”, “Quick Follow-Up Question” and “One helpful follow-up” in our samples. Select by structure.

Without a done flag, a stability check can stop early if the stream pauses. Log answers that end mid-sentence and widen the quiet window if you see them.

8 of 242 chips in our samples had a citation number with no URL on the page. Store None for those numbers and do not guess.

Local answers can cite bing.com search pages and local listings. Filter or keep them on purpose.

If you scrape Bing Copilot through a pool of mixed exit IPs, “near me” prompts come back about a mix of towns, which makes results hard to compare day to day.

The API path: one POST to cloro

If you do not want to scrape Bing Copilot yourself, the API does the same work: you send a prompt and a country, cloro opens Copilot Search, waits for the answer and parses it, and you get JSON back.

import

requests

resp

=

requests.post(

"https://api.cloro.dev/v1/monitor/bing/aimode"

,

headers

=

{

"Authorization"

:

"Bearer YOUR_API_KEY"

},

json

=

{

"prompt"

:

"best running shoes for flat feet under $150"

,

"country"

:

"US"

,

"include"

: {

"markdown"

:

True

,

"html"

:

True

},

},

timeout

=

300

,

)

result

=

resp.json()[

"result"

]

print

(result[

"text"

][:

200

])

for

source

in

result[

"sources"

]:

print

(source[

"position"

], source[

"url"

])

for

pill

in

result[

"citationPills"

]:

print

(pill[

"citationPillId"

], pill[

"domain"

], pill[

"position"

])

This is the response for that prompt from our samples, trimmed to one entry per list. The full answer had 9 sources and 23 citation pills.

{

"success"

:

true

,

"result"

: {

"text"

:

"The ASICS Gel Pulse 15, Brooks Glycerin 21, and Saucony Guide 19 are the strongest under-$150 choices for flat feet, offering wide bases, stability features, and cushioning suited to flat arches. ..."

,

"sources"

: [

{

"position"

:

1

,

"label"

:

"Best Running Shoes Under $150 in 2026: 8 Picks Worth the Money"

,

"url"

:

"https://runninggearlab.com/best-running-shoes-under-150/"

,

"description"

:

"The best running shoes under $150 in 2026, including Saucony Ride 19, Puma Velocity Nitro 4, Brooks Ghost 18, New Balance 880v15 and more."

}

],

"citationPills"

: [

{

"citationPillId"

:

1

,

"position"

:

3

,

"label"

:

"Best Running Shoes for Flat Feet (2026): Tested Picks"

,

"url"

:

"https://www.alastairrunning.com/best-running-shoes-for-flat-feet/"

,

"domain"

:

"alastairrunning.com"

}

],

"markdown"

:

"The **ASICS Gel Pulse 15**, **Brooks Glycerin 21**, and **Saucony Guide 19** are the strongest under-$150 choices ..."

,

"html"

:

"https://cdn.cloro.dev/results/4fbaeeae-0453-4283-8bb7-528f70d81c6d/page-1.html"

}

}

sources lists the pages Bing read, in the order of the “Reading sources” list, then any other page a chip links to. citationPills has one entry per chip and source pair, and its position points back into sources . Headings, tables and the follow-up question are only in markdown , and html is a link to the rendered page.

On 2026-11-08 we timed 10 prompts through the sync endpoint, US, two at a time. All 10 returned an answer, in 15.0 to 26.5 seconds (median 19.4). Each answer had 9 to 11 sources and 14 to 39 citation pills.

A Bing AI Mode request costs 4 credits on the async API, the same as Google AI Mode. A sync request adds 2 credits, and a US state adds 2 more. On the Hobby plan ($100 a month for 250,000 credits), 4 credits is $1.60 per 1,000 async requests.

The free plan includes 500 credits a month. For batches, send the work to POST /v1/async/task or /v1/async/task/batch with taskType: "BING_AIMODE" , and collect results by webhook or polling. See pricing for the other plans.

cloro is not the only API for this page. SerpApi and SearchApi both read bing.com/copilotsearch with engine=bing_copilot .

Which to choose

Scrape Bing Copilot yourself if you need a few hundred answers from one country, you already run browsers and proxies, and you can accept a few chips without URLs. The code above is enough to start.

Use an API if you need many countries or US states, a steady daily volume, or complete chip-to-source pairs for citation reporting. Our Bing AI Mode endpoint does the waiting, the parsing and the location for 4 credits a request. Our comparison of Bing Copilot APIs sets it beside SerpApi, SearchApi and the other vendors.

About the author

Ricardo Batista

Founder, cloro

Ricardo is one of the founders and engineers behind its SERP and AI-search scraping infrastructure. Before cloro he scaled a financial comparison site to $7M ARR and ran the full-country operations of a unicorn to $65M ARR, then went back to building. He writes about search engine scraping, generative-engine optimization, and turning live search and AI-answer data into something teams can act on.

More articles by Ricardo

Connect on LinkedIn

Frequently asked questions

Is there an official Bing Copilot Search API?

No. Microsoft retired the Bing Search APIs on August 11, 2025 and pointed customers to Grounding with Bing Search in Azure AI Agents. Grounding feeds Bing web results into an answer that your own agent writes. It does not return the Copilot Search page, its sources or its citation chips, so to see what Bing shows a user you have to render the page or use a scraping API such as cloro's Bing AI Mode endpoint .

Is scraping Copilot Search the same as scraping Copilot?

No. Copilot Search in Bing is a search results page at bing.com/copilotsearch, and its answer is written into an iframe that you read from the DOM. The Copilot app at copilot.microsoft.com is a chat product that streams its answer over WebSockets. The two need different scrapers. Our Copilot app guide covers the app.

How long does a Copilot Search answer take?

In a timing run on 2026-11-08, 10 prompts through cloro's Bing AI Mode API (sync, US, two at a time) took 15.0 to 26.5 seconds each, with a median of 19.4 seconds. All 10 returned an answer. A DIY scraper should allow at least that long before it parses, and cap the wait at about 90 seconds.

Can I target a country or a US state?

With cloro, yes: set `country` (every country except BY, CN, IR, RU, SY and VE) and, for the US, an optional `state`. A DIY scraper gets its location from the browser's exit IP, so you need a proxy exit in each country, and in each state for state-level results. Location changes local answers: our US request for "emergency plumber near me" came back about East Haven, Connecticut.

How many sources does a Copilot Search answer cite?

Across our 12 samples on 2026-11-07, the "Reading sources" list held 6 to 11 pages, with Bing showing six and folding the rest behind a "+N more" button. In the 10-prompt timing run, cloro returned 9 to 11 sources and 14 to 39 citation pills per answer.

Why does my scraper find no answer on the Copilot Search page?

Because the answer is not in the outer page. Bing renders it inside an iframe whose label ends in "result container", and scripts inside that frame write the answer one word at a time. Read the frame's document, not the outer HTML, and wait until the answer stops growing before you parse it.

Related reading

AI Search

How to Scrape Copilot: WebSocket Events and Citations

Learn how to scrape Microsoft Copilot: capture WebSocket events, parse Bing-grounded citations, and budget for running it yourself.

Google

How to scrape Google AI Mode: citation pills + place cards

Scrape Google AI Mode in 2026: extract citation pills, parse place cards and shopping carousels, and intercept the conversational SERP network calls.

AI Search

Microsoft Copilot API: Official APIs vs Scraping

There's no single 'ask Copilot' API. Here's what the official Microsoft 365 Copilot APIs actually return, who can use them, and when you have to scrape instead.

AI SEO Scraper API

Turn AI results into data

Get results in seconds

Enterprise ready

All endpoints included

Get free API key

Includes 500 credits

We use cookies for analytics, advertising, and to credit partners who refer you. You can turn any of it off.

More articles by Ricardo

Web Scraping

AI Search

Tutorial

Microsoft Copilot scraping and WebSocket parsing diagram AI Search How to Scrape Copilot: WebSocket Events and Citations Learn how to scrape Microsoft Copilot: capture WebSocket events, parse Bing-grounded citations, and budget for running it yourself.

Connect on LinkedIn

Google AI Mode scraping and citation pill extraction diagram Google How to scrape Google AI Mode: citation pills + place cards Scrape Google AI Mode in 2026: extract citation pills, parse place cards and shopping carousels, and intercept the conversational SERP network calls.

Microsoft Copilot API surfaces and enterprise grounding diagram AI Search Microsoft Copilot API: Official APIs vs Scraping There's no single 'ask Copilot' API. Here's what the official Microsoft 365 Copilot APIs actually return, who can use them, and when you have to scrape instead.

Get free API key

cloro's Bing AI Mode endpoint

testing the label “AI Mode”

Copilot rank tracking

pricing

SerpApi

SearchApi

Our comparison of Bing Copilot APIs

retired the Bing Search APIs on August 11, 2025