How to scrape Avito

Avito (avito.ru) is Russia's largest classifieds site: phones, cars, real estate, clothing, services — listed by private sellers and shops. For price monitoring or market research, the useful data is right on the public pages: titles, prices and discounts, location, item condition and characteristics, and who is selling.

Getting that data is harder than loading HTML and running CSS selectors over it:

• Avito sits behind its own firewall. It answers too many requests from one address with a «Доступ ограничен: проблема с IP» page. A plain HTTP client never gets past it: curl with full browser headers got HTTP 429 on 6 of 6 attempts from fresh addresses.
• The firewall can ask for a captcha. Every request measured from 2026-10-06 to 2026-10-09 got a GeeTest v4 slide puzzle.
• One URL, two renderings. The same listing page sometimes arrives as plain markup and sometimes with an embedded JSON state that carries more fields.
• The obvious price field is the wrong one. On a discounted item, Avito's bare price value is the pre-discount price, not the one the buyer sees.

In this guide we'll walk through the open-source project 2scraper/avito-scraper: how it reaches the pages, where it reads the data from, and how it handles each firewall state. We'll focus on the Playwright engine, which the repository recommends.

Unless a measurement says otherwise, all figures below come from the repository's live runs on 2026-10-06 through a Russian residential IP.

What We'll Build

A scraper with three modes:

1. listing — a category or search page, paginated with ?p=N. One row per card.
2. item — individual item pages, either from URLs or straight from a listing run's output.
3. seller — a seller profile plus all of that seller's listings.

A listing row includes: sku (Avito's item id), title, price, currency, original_price and discount_pct (when the site states them), region_slug, city, location, summary_params, published_at, seller_name, seller_type, seller_id, seller_rating, seller_reviews, promoted, badges, delivery, page and position.

Item mode adds description, params (all characteristics), images, category_path, address, lat/lng, views_total, views_today, seller_since, seller_reply_time and is_active.

What the scraper does not collect: phone numbers (Avito shows them only after login), anything behind login, and the masked IMEI some categories list among item parameters. Private sellers are anonymised by Avito itself, so for them seller_name stays empty and seller_type is private. Company and shop names are collected, so if you store or process the output, the personal-data law where you operate applies to it — in Russia, Federal Law 152-FZ.

Every run also writes a metadata file next to the data. It tells you whether the result is complete, partial or blocked.

Requirements

• Python 3.9+
• Playwright (recommended). The repository also ships pyppeteer and Selenium engines. Install one engine per virtualenv: their dependencies conflict.
• beautifulsoup4 and requests

And, because of the firewall:

• A Russian residential proxy with a sticky session (2Captcha residential proxies work). The browser and the captcha solver must leave from the same IP (more on why below).
• A 2Captcha API key (TWOCAPTCHA_KEY) to solve the GeeTest captcha on the firewall page.
• A 2Captcha Browser API endpoint (AVITO_CDP_ENDPOINT), a remote browser over the Chrome DevTools Protocol. The repository recommends it, and the latest measurements make it practically required: since 2026-10-14, captcha solutions for a local browser were refused every time (details below).

Unlike many sites, Avito leaves no free path here. Plain HTTP is refused, and a fresh local browser runs into the captcha on every new session (5 of 5 in tests). The parsing techniques below apply no matter how you connect; getting the pages at all needs a browser, a Russian IP address and, more often than not, a solved captcha.

Project Setup

Clone the repository and install the core dependencies plus the Playwright engine:

git clone https://github.com/2scraper/avito-scraper.git
cd avito-scraper
python3 -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

pip install -r requirements.txt
pip install -r requirements-playwright.txt
playwright install chromium

cp .env.example .env

The scraper reads exactly four variables from .env:

Keep credentials in .env, not on the command line: anything on the machine can read argv with ps, and it ends up in shell history and CI logs. A password containing @, : or / works in either proxy form. An IP-whitelisted gateway URL without a login is refused: the session id that pins the exit lives inside the login.

If you use the Browser API, the endpoint and the proxy must be built together, so the remote browser and the captcha solver share one IP. The repository includes a helper that does it in one step and writes both values into .env without printing them:

python3 tools/browser_profile_client.py accounts
python3 tools/browser_profile_client.py connection --account-id <id> --mint-session --write-env

A proxy session lives for a limited number of minutes (-sessTime- in the login, 120 by default). When it expires, the endpoint built on it fails on connect with 500 proxy_error, and you rerun the same command.

python3 env_config.py shows which variables were picked up, without printing their values.

Quick Start

# A category listing, three pages, JSON and CSV
python3 playwright_scraper.py --mode listing \
  --url "https://www.avito.ru/moskva/telefony/mobilnye_telefony/apple-ASgBAgICAkS0wA3OqzmwwQ2I_Dc" \
  --pages 3 --out out/iphones

# A search
python3 playwright_scraper.py --mode listing --url "https://www.avito.ru/moskva?q=iphone+15" --pages 2 --out out/search

# Item pages for what the listing run found
python3 playwright_scraper.py --mode item --from out/iphones.json --max-products 20 --out out/items

# A seller profile and its listings
python3 playwright_scraper.py --mode seller --url "https://www.avito.ru/brands/<seller>" --out out/seller

Add --verbose to see, page by page, what the scraper met and what cleared it.

Each run writes <out>.json and/or <out>.csv, plus two metadata files:

• <out>.meta.json describes the data next to it.
• <out>.latest_attempt.meta.json is written on every ending — success, block, connection error or crash — with exit_code and error_class.

The second file matters for automation. A run that finds nothing does not overwrite the previous good output with [] (--allow-empty is the opt-out), so the attempt file is how a scheduler learns that the latest run failed.

Avito's robots.txt disallows sort and filter parameters (?s=, ?f=, ?radius=, ?view=) and /items/phone. The scraper refuses such URLs before fetching anything, naming the rule. Narrow a listing by path instead, the way the site's own URLs do: city → category → subcategory. ?q= search and ?p= pagination are allowed; a plain ?p=2 returned 50 new item ids in tests.

How the Scraper Works

The repository separates when to fetch from what a page says:

The flow for one listing page:

1. Build the page URL (?p=N for page 2 onward).
2. Navigate and wait, bounded, until the page is ready. For a listing, "ready" means more than one card is present ([data-marker="item"][data-item-id]), because one match can resolve on an unrelated card-shaped node before the grid renders.
3. Classify the document: content, challenge, captcha, empty, blocked or unknown.
4. Apply the policy for that state (next section).
5. Parse the cards, deduplicate by sku (Avito repeats some promoted cards across pages), move to the next page.
6. Stop at the site's last page, when a page adds no new items, or at --max-products.

The policy lives in page_flow.py as data, so the three engines can't disagree on it:

STATE_POLICY = {
    "content":   {"retry": False, "solve": False, "wait": False, "rotate": False, "blocked": False},
    "challenge": {"retry": False, "solve": False, "wait": True,  "rotate": True,  "blocked": True},
    "captcha":   {"retry": False, "solve": True,  "wait": True,  "rotate": True,  "blocked": True},
    "empty":     {"retry": False, "solve": False, "wait": False, "rotate": False, "blocked": False},
    "blocked":   {"retry": False, "solve": False, "wait": False, "rotate": True,  "blocked": True},
    "unknown":   {"retry": True,  "solve": False, "wait": False, "rotate": False, "blocked": False},
}

An empty listing is treated as a correct answer, not a fault. A blocked page means "try a different address", not "try again".

Why a browser and not requests? The firewall refuses plain HTTP clients outright, and one of its states is a proof-of-work page that only a browser can run. The data itself is in the served HTML, so once a browser gets past the firewall, parsing is ordinary HTML/JSON work.

Data Extraction

Listings: two renderings of the same URL

The same listing URL comes back in one of two forms:

• Plain markup. The grid is in the DOM only: 50 cards per page, each with schema.org microdata (itemprop="price", priceCurrency, name, url).
• Markup plus embedded JSON. The same cards, plus a <script type="mime/invalid" data-mfe-state> block whose catalog.items carry fields the DOM doesn't have: timestamps, the old price of a discounted item, the seller type.

The parser reads both. The JSON wins where it exists, and every row records its source in price_source (state or dom):

def listing_state(html):
    m = re.search(r"<script[^>]*data-mfe-state[^>]*>(.*?)</script>", html, re.S)
    if not m:
        return None
    data = json.loads(m.group(1))
    return data["loaderData"]["data"]["catalog"]     # catalog["items"] is the list

The DOM path reads the card attributes and microdata:

for card in soup.select('[data-marker="item"][data-item-id]'):
    sku = card["data-item-id"]
    title = card.select_one('[data-marker="item-title"]') or card.select_one('[itemprop="name"]')
    price = card.select_one('[itemprop="price"]')            # its `content` attribute
    location = card.select_one('[data-marker="item-location"]')
    params = card.select_one('[data-marker="item-specific-params"]')
    rating = card.select_one('[data-marker="seller-rating/score"]')

A card that the JSON doesn't know about is still taken from the DOM, so a partial JSON block never loses rows.

The price trap

On a discounted item, Avito's top-level priceDetailed.value in the listing JSON (and item.price on the item page) holds the old price — for example 69 989 while the page shows 62 990. The price the buyer sees sits in the PriceStep "price" component, and the old price is stated separately in the "discount" component:

for step in item["iva"]["PriceStep"]:
    component = step["componentData"]["component"]
    if component == "price":
        price = step["payload"]["priceDetailed"]["value"]       # displayed price
    elif component == "discount":
        original_price = step["payload"]["old_string"]          # e.g. "69 989 ₽"
        discount_pct = step["payload"]["percent"]

On the DOM side, the itemprop="price" microdata already holds the displayed price. All three sources — the JSON, the DOM microdata and the item page — agreed on the displayed price on 50 of 50 cards. The scraper never computes original_price: it is filled only from the site's own figure, so plain-markup rows simply leave it empty.

Results that are not results

Two kinds of cards on a listing page have the same markup as real results but aren't:

• A search with no matches still returns about fifty cards, under a heading «Похожие объявления рядом и в других городах» ("similar listings nearby and in other cities"). The parser cuts the page at that heading. A no-match search ends as an empty result (exit code 4) instead of a complete-looking answer to a different question.
• A «Подобрали для вас» ("picked for you") recommendation carousel can sit inside the results grid with cards of the same data-marker="item" shape. The parser drops any card inside [data-marker="itemsCarousel"]. In the JSON state these cards live in a separate list, so only the DOM path needs the filter.

cards = [c for c in cards if c.find_parent(attrs={"data-marker": "itemsCarousel"}) is None]

Item pages

An item page carries its data in a hydration payload:

window.__staticRouterHydrationData = JSON.parse("…")

It's a JavaScript string literal that contains JSON, so the parser decodes it twice — the literal first, then the JSON inside it — and takes buyerItem from loaderData:

k = html.find("__staticRouterHydrationData")
q = html.find('JSON.parse("', k)
literal, _ = json.JSONDecoder().raw_decode(html, q + len("JSON.parse("))
data = json.loads(literal)
buyer_item = next(v["buyerItem"] for v in data["loaderData"].values()
                  if isinstance(v, dict) and "buyerItem" in v)

The displayed price on the item page is item.formattedPrice.value, again not item.price.

Seller profiles

A seller page (/brands/<slug>) is parsed from the DOM: name, rating, review count, «На Авито с …» tenure, subscribers and the active-listings counter. The profile itself renders only the first 15 listings. To get the rest, the scraper opens the seller's feed (/brands/<slug>/all?sellerId=…), clicks its «Показать все» button and scrolls; every scroll loads 15 more. On 2026-10-09 this collected 353 of 353 listings in 22 scrolls, every row priced. A shop whose profile carries no sellerId gets it from /brands/<slug>/items (checked on 2026-10-10 on a shop with 7,331 listings).

A run that collects fewer listings than the profile's own counter is reported as partial (exit code 6), not complete: compare listings_on_page with active_ads.

One robots.txt caveat. The scraper never requests a path Avito's robots.txt disallows, but scrolling the feed makes the page's own script fetch /web/1/profile/items, and /web/ is disallowed for robots. This is a deliberate choice: it is the only way to a seller's listings beyond the first 15. Use --max-products 15 to stay on the profile page alone.

Working with Avito's Anti-Bot Protection

Avito's protection is its own firewall, «Доступ ограничен: проблема с IP». Its text says why it appears: too many requests from this address. In other words, it is a request-rate decision tied to the IP.

The repository recognises three states of that page:

The third state is not a captcha, and sending it to a solver is pointless. The scraper classifies it as blocked and switches to a new IP address when it can. Left alone, the wall didn't clear by itself within 140 s in 3 of 3 tests.

Detection is ordered by how much each signal proves:

WALL_TITLE = "Доступ ограничен"
WAIT_TEXT = "подождите немного и обновите страницу"

def page_state(html):
    if WALL_TITLE not in title_of(html):
        return "content" if has_listing_cards(html) else ...
    if re.search(r"<form\b[^>]*\bjs-firewall-form\b", html):
        return "captcha"          # the firewall's captcha form is on the page
    if WAIT_TEXT in html:
        return "blocked"          # rejected solution: switch to a new IP
    return "challenge"            # proof-of-work: let the browser clear it

Two details are worth copying:

• Look for the form element, not a substring. The firewall page's own script mentions .js-firewall-form in a querySelector whether or not the form is on the page.
• Don't test for the word "captcha" in the HTML. The Browser API injects its own …/captcha/… scripts into every page it loads, so a substring test would flag every page as blocked.

Because the firewall reacts to the request rate from one address, the repository refuses --concurrency above 1 and waits 2 s between pages by default (--delay).

Using Proxies

Proxy support is built in. AVITO_PROXY is a single proxy URL:

http://user:pass@host:port

For a 2Captcha residential gateway, the login selects the region and pins the session:

http://{login}-zone-custom-region-RU-session-{id}-sessTime-120:{password}@<gateway-host>:<port>

How the proxy is used depends on how the browser runs:

• Local browser: all browser traffic goes through it. Playwright gets the credentials through its own proxy fields rather than --proxy-server=, so they never land on the browser's command line:

  python Copy

  launch["proxy"] = {"server": "http://host:port", "username": user, "password": password}
  browser = pw.chromium.launch(**launch)

  pyppeteer passes them through page.authenticate. Chrome under Selenium can't take a proxy password at all, so the scraper points it at proxy_forwarder.py, a loopback proxy inside the scraper process that adds the credentials upstream. A gateway login without -session- would rotate the IP on every request, so the scraper pins one session for the run automatically.

• Browser API: the remote browser comes with its own IP address. AVITO_PROXY is then used only by the captcha solver, and it has to be the exact session the endpoint was built with.

When a page stays blocked in a local browser, the scraper starts a new browser with a new IP address (--proxy-block-retries, default 1). It never moves a live session to a new IP: cookies the firewall issued to one address, replayed from another, look more suspicious than either IP alone. New IP addresses come from a --proxy-file pool or, for a 2Captcha gateway, from newly minted sessions. --proxy-rotate per-page uses a new IP address for every page, at the cost of a browser relaunch per page.

Working with CAPTCHA (GeeTest v4)

Every captcha the firewall served from 2026-10-06 to 2026-10-09 was GeeTest v4 (slide), including the page's own "new picture" request. Here is how the repository solves it. Task parameters are described in the GeeTest docs.

1. Detect it

The firewall page contains blocks for three captcha types, and the page's script un-hides the one the server picked. wall_variant() returns the first block that isn't display: none. For GeeTest that is #geetest_captcha, and the GeeTest id must also be present.

2. Extract the parameters

GeeTest v4 needs one parameter, the captcha_id. It is hard-coded in the firewall page's inline script:

captcha_id = re.search(r"captchaId\s*=\s*['\"]([0-9a-f]{32})['\"]", html).group(1)

3. Send it to the solver — through the browser's IP

This is the detail that decides success. A token solved from a different IP than the browser's is rejected: in a test without the browser's proxy, Avito answered with a second captcha. So the task is a GeeTestTask with proxy fields, carrying the same sticky-session proxy the browser uses. It is created with createTask and polled with getTaskResult:

task = {
    "type": "GeeTestTask",
    "websiteURL": page_url,
    "version": 4,
    "initParameters": {"captcha_id": captcha_id},
    "proxyType": "http",
    "proxyAddress": proxy_host,
    "proxyPort": proxy_port,
    "proxyLogin": proxy_login,
    "proxyPassword": proxy_password,
}
task_id = requests.post("https://api.2captcha.com/createTask",
                        json={"clientKey": API_KEY, "task": task}).json()["taskId"]

while True:
    time.sleep(5)
    result = requests.post("https://api.2captcha.com/getTaskResult",
                           json={"clientKey": API_KEY, "taskId": task_id}).json()
    if result.get("status") == "ready":
        solution = result["solution"]
        # {captcha_id, lot_number, pass_token, gen_time, captcha_output}
        break

The repository refuses to build a proxyless task by default (solve_geetest_v4 raises an error when there is no proxy), because paying for a token the site will reject is worse than reporting the page as blocked. It also waits up to 240 s for the verdict, longer than the solver may take to give up on a hard puzzle. A shorter timeout abandons the task before you learn why it failed. A page buys at most one paid solve per run: a fresh IP re-fetches a page that stayed blocked, but doesn't buy a second token for it.

4. Submit it through the page's own form

Rather than calling Avito's verify endpoint directly, the scraper puts the solution where the page's own code would and triggers the form's own submit handler. The handler adds the headers the site expects, posts to /web/3/firewallCaptcha/verify and reloads the page, the same path a person's solve takes:

value = json.dumps({"captcha_id": captcha_id, "lot_number": solution["lot_number"],
                    "pass_token": solution["pass_token"], "gen_time": solution["gen_time"],
                    "captcha_output": solution["captcha_output"]})

page.evaluate("""value => {
  const inp = document.querySelector('input[name=captcha-response]');
  const form = document.querySelector('.js-firewall-form');
  inp.value = value;
  form.dispatchEvent(new Event('submit'));
}""", value)

Then it watches what comes back:

• The page's content (cards, an item, a profile, or a genuinely empty listing): the solution was accepted.
• The «подождите…» page: the solution was rejected, so the scraper switches to a new IP address.
• Another captcha: not accepted this time. The page stays blocked and the run reports it.

"The firewall page is gone" alone doesn't count as success: the page passes through intermediate documents while it reloads. While it is navigating, Playwright's page.content() can even throw "page is navigating", so read the HTML inside a try while you wait.

What the measurements say

A correct solution doesn't guarantee that Avito accepts the session. The repository's live results:

Why local solves stopped working is not proven. In a control on 2026-10-14, GeeTestTask returned a solution even for a proxy with a wrong password and for a proxy host that doesn't exist, so the solver evidently did not use the browser's proxy, and a token from another IP is the likeliest reason it was refused. The practical conclusion is the same either way: use the Browser API for Avito. Once a profile has passed the firewall, its cookies carry it through later pages.

Fallbacks: picture captcha and hCaptcha

The firewall page's code also contains two other captcha types: Avito's own picture captcha and hCaptcha. Neither appeared in testing, but the scraper handles both in case the server starts assigning them:

• Picture captcha (#inner-captcha): the image is a data: URL, solved with ImageToTextTask. The answer goes into #form-input, then the same form submit.
• hCaptcha (#h-captcha): the sitekey comes from the page's data-sitekey, solved with HCaptchaTask through the browser's IP address. The token goes into #h-captcha-response.

transport_facts.solves[].variant in the run metadata records which type was met (geetest, image or hcaptcha). If you see one of the latter two, that's new behaviour worth checking.

Using 2Captcha Browser API (CDP)

Instead of launching a local Chromium, the Playwright and pyppeteer engines can connect to a remote browser over CDP:

browser = pw.chromium.connect_over_cdp(cdp_endpoint)
context = browser.contexts[0] if browser.contexts else browser.new_context()
page = context.pages[0] if context.pages else context.new_page()

With AVITO_CDP_ENDPOINT set in .env, the CLI does this automatically. The commands from Quick Start don't change. Setup and session options are in the Browser API docs.

This is the option the repository recommends for Avito:

• The profile is persistent. Once it has passed the firewall, its cookies let it through on later requests: on 2026-10-06 a dozen consecutive listing, item and seller pages came back without meeting the firewall again, and on 2026-10-19 eleven runs of every mode met no firewall at all.
• It clears the proof-of-work variant by itself.
• It brings its own fingerprint, so the scraper sets no user agent.

On connect, the scraper also turns on the remote browser's built-in captcha auto-solve (Captcha.setAutoSolve) and gives it 25 s. On Avito's GeeTest it didn't succeed (0 of 3 on 2026-10-06), so the scraper falls back to the GeeTestTask flow above. That fallback is why AVITO_PROXY must be the session the endpoint was built on.

A few operational facts the code accounts for:

• One live connection per profile. A second one gets 500 profile_locked until the first is released; the scraper retries 3 times, 10 s apart. A cleanly closed run releases it quickly; a killed one can hold it much longer. The driver always closes the browser handle at the end.
• The endpoint expires with its proxy session. The answer is 500 proxy_error on connect; rebuild it with browser_profile_client.py connection --mint-session --write-env.
• Profiles can be rate-limited too. Keep the default delay between pages.
• Selenium can't use it. Chromedriver has nowhere to put the endpoint's credentials, so the Selenium engine runs a local Chrome only.

Scaling the Scraper

The repository gives you the building blocks. Production orchestration is up to you:

• Parallelism means more IP addresses, not more threads. --concurrency above 1 is refused, because parallel fetches from one address make the firewall appear more often. A Browser API profile also allows only one connection at a time. To go faster, run separate processes, each with its own IP address or profile.

• Keep a delay. --delay defaults to 2 s between pages. The repository makes no claim about a "safe" request rate for Avito, so measure how often the firewall appears on your own traffic.

• Reuse sessions. For periodic monitoring, a persistent Browser API profile that has already passed the firewall beats a fresh session per run.

• Separate a timeout from a refusal. A navigation timeout is retried with the same IP (--retries, --retry-delay). A dead proxy (ERR_CONNECTION_RESET, ERR_PROXY_…) or a blocked page gets a new IP address (--proxy-block-retries).

• Use the exit codes:

  Code	Meaning
  0	Complete
  1	Crash — a bug in the code, not the site or the connection
  2	Bad usage — wrong host or mode, robots-disallowed URL, unreadable --from file
  3	Blocked — the firewall stayed up after every step
  4	Zero results — the listing was read to its end and is empty, including a search with no matches
  5	Content never obtained — timeout, dead proxy, profile_locked after 3 retries, proxy_error, an item page with no item on it, or a seller profile that never parsed
  6	Partial — some pages fetched, then stopped; in seller mode, fewer listings than the profile's own counter

  3 and 5 call for different fixes: 5 means rebuild the endpoint, retry or check the URL, not "the firewall won".

• Check completeness. status in <out>.meta.json is complete, partial or failed. transport_facts lists every firewall page met, which step cleared it, every solve with its time, cost and 2Captcha task_id, and every rotation.

• Compare runs safely. diff_runs.py --old A.json --new B.json reports added, removed and changed items by sku. It refuses (exit 2) to compare incomplete runs, runs of different modes, or runs over a different listing, page count or --max-products cap, which would otherwise show up as items "disappearing"; --force compares anyway. It also doesn't report the difference between the two renderings as a price change.

• "Removed" means "not on these pages any more", not "delisted". Avito re-orders listings constantly as ads are raised and added. Two runs of the same 3-page category on 2026-10-19 differed by 47 added and 42 removed out of 145 — listings that moved past page 3. To tell the two apart, run --mode item on the removed ones: an ad that is gone ends with exit code 5.

Troubleshooting

Example Output

A listing row (--mode listing) from the 3-page run on 2026-10-06. This one comes from the plain-markup rendering, so the JSON-only fields are null; seller name and slug are placeholders:

{
  "source": "avito.ru",
  "scraped_at": "2026-10-06T11:43:07.634024+00:00",
  "url": "https://www.avito.ru/moskva/telefony/iphone_17_256_gb_sim_esim_8212942358",
  "sku": "8212942358",
  "title": "iPhone 17, 256 ГБ, SIM + eSIM",
  "image_url": "https://…img.avito.st/image/1/…",
  "price": 64935.0,
  "currency": "RUB",
  "category": null,
  "price_source": "dom",
  "original_price": null,
  "discount_pct": null,
  "region_slug": "moskva",
  "city": "Москва",
  "location": "Багратионовская, 6–10 мин.",
  "summary_params": "Новый",
  "published_at": null,
  "published_text": null,
  "seller_name": "Продавец 1",
  "seller_type": null,
  "seller_id": "seller1",
  "seller_url": "https://www.avito.ru/brands/seller1",
  "seller_rating": 5.0,
  "seller_reviews": 624,
  "promoted": true,
  "badges": ["Реквизиты проверены", "яПомогаю"],
  "delivery": null,
  "page": 1,
  "position": 1
}

An item row (--mode item, shortened) adds, among others, the full characteristics:

{
  "sku": "7924452716",
  "price": 39990.0,
  "original_price": null,
  "params": {
    "Состояние": "Новое",
    "Производитель": "Apple",
    "Модель": "iPhone 15",
    "Встроенная память": "128 ГБ",
    "SIM-карты": "SIM + eSIM",
    "Цвет": "Розовый"
  },
  "views_total": 4503,
  "seller_type": "company",
  "is_active": true
}

And the run metadata (<out>.meta.json, shortened):

{
  "source": "avito.ru",
  "mode": "listing",
  "status": "complete",
  "stop_reason": "completed",
  "pages_requested": 3,
  "pages_completed": 3,
  "pages_failed": [],
  "products": 147,
  "transport_facts": {
    "over_cdp": true,
    "walls": [],
    "solves": [],
    "rotations": 0
  }
}

Complete Compact Code Example

Below is a single-file version of the repository's core: open a listing in Playwright (over a Browser API CDP endpoint, or locally through a proxy), classify the firewall state, solve GeeTest v4 through the browser's IP address if the firewall shows it, and parse the cards with the price trap, the carousel and the "similar listings" cut-off handled. It leaves out pagination, IP rotation, item and seller pages, and the metadata files. For those, use the repository. Given the measurements above, run it with AVITO_CDP_ENDPOINT set: on a local browser the captcha step is unlikely to be accepted.

"""Minimal avito.ru listing scraper — the core of 2scraper/avito-scraper in one file.

    pip install playwright beautifulsoup4 requests && playwright install chromium
    export AVITO_PROXY='http://user:pass@host:port'   # Russian residential, sticky session
    export TWOCAPTCHA_KEY='...'                        # only needed if Avito shows the GeeTest captcha
    python3 avito_minimal.py "https://www.avito.ru/moskva?q=iphone+15"

Set AVITO_CDP_ENDPOINT instead of launching a local Chromium to use a remote browser.
"""
import json
import os
import re
import sys
import time
from urllib.parse import urlparse

import requests
from bs4 import BeautifulSoup
from playwright.sync_api import sync_playwright

WALL_TITLE = "Доступ ограничен"
WAIT_WALL_TEXT = "подождите немного и обновите страницу"
CARD = '[data-marker="item"][data-item-id]'


# ---------------------------------------------------------------- page state
def page_state(html: str) -> str:
    title = re.search(r"<title[^>]*>(.*?)</title>", html, re.S)
    if not title or WALL_TITLE not in title.group(1):
        return "content" if 'data-marker="item"' in html else "unknown"
    if re.search(r"<form\b[^>]*\bjs-firewall-form\b", html):
        return "captcha"            # GeeTest v4 behind the «Продолжить» button
    if WAIT_WALL_TEXT in html:
        return "blocked"            # follows a rejected solution: change IP
    return "challenge"              # proof-of-work, the browser clears it itself


# ------------------------------------------------------------ GeeTest solving
def solve_geetest(api_key: str, captcha_id: str, page_url: str, proxy: str) -> dict:
    p = urlparse(proxy)
    task = {"type": "GeeTestTask", "websiteURL": page_url, "version": 4,
            "initParameters": {"captcha_id": captcha_id},
            # Solve from the browser's own IP, or Avito rejects the token.
            "proxyType": p.scheme, "proxyAddress": p.hostname, "proxyPort": p.port,
            "proxyLogin": p.username, "proxyPassword": p.password}
    r = requests.post("https://api.2captcha.com/createTask",
                      json={"clientKey": api_key, "task": task}, timeout=30).json()
    if r.get("errorId"):
        raise RuntimeError(r.get("errorCode"))
    for _ in range(48):                                    # up to 240 s
        time.sleep(5)
        res = requests.post("https://api.2captcha.com/getTaskResult",
                            json={"clientKey": api_key, "taskId": r["taskId"]}, timeout=30).json()
        if res.get("errorId"):
            raise RuntimeError(res.get("errorCode"))
        if res.get("status") == "ready":
            return res["solution"]
    raise TimeoutError("no solution within 240 s")


SUBMIT_JS = """value => {
  const inp = document.querySelector('input[name=captcha-response]');
  const form = document.querySelector('.js-firewall-form');
  if (!inp || !form) return false;
  inp.value = value;
  form.dispatchEvent(new Event('submit'));   // the page's own handler posts /verify
  return true;
}"""


def content(page) -> str:
    try:
        return page.content()
    except Exception:                                      # the page is mid-reload
        return ""


def pass_wall(page, url: str, proxy: str) -> str:
    html = content(page)
    for _ in range(25):                                    # proof-of-work: just wait
        if page_state(html) != "challenge":
            break
        page.wait_for_timeout(1000)
        html = content(page)
    if page_state(html) != "captcha":
        return html
    key = os.environ.get("TWOCAPTCHA_KEY")
    captcha_id = re.search(r"captchaId\s*=\s*['\"]([0-9a-f]{32})['\"]", html)
    if not (key and proxy and captcha_id):
        return html
    sol = solve_geetest(key, captcha_id.group(1), url, proxy)
    value = json.dumps({k: sol.get(k) for k in
                        ("lot_number", "pass_token", "gen_time", "captcha_output")}
                       | {"captcha_id": captcha_id.group(1)})
    page.evaluate(SUBMIT_JS, value)
    for _ in range(20):                                    # wait for the reload
        page.wait_for_timeout(1500)
        html = content(page) or html
        if page_state(html) in ("content", "blocked"):
            break
    return html


# ------------------------------------------------------------------- parsing
def listing_state(html: str) -> dict:
    m = re.search(r"<script[^>]*data-mfe-state[^>]*>(.*?)</script>", html, re.S)
    if not m:
        return {}
    try:
        catalog = json.loads(m.group(1))["loaderData"]["data"]["catalog"]
    except (ValueError, KeyError, TypeError):
        return {}
    return {str(i["id"]): i for i in catalog.get("items", []) if i.get("type", "item") == "item"}


def prices(item: dict):
    """(displayed price, old price). The bare `priceDetailed` is the PRE-discount
    price; the one the buyer sees is in the `PriceStep` "price" component."""
    shown, old = (item.get("priceDetailed") or {}).get("value"), None
    for step in (item.get("iva") or {}).get("PriceStep") or []:
        component = (step.get("componentData") or {}).get("component")
        if component == "price":
            shown = step["payload"]["priceDetailed"]["value"]
        elif component == "discount":
            digits = re.sub(r"\D", "", step["payload"].get("old_string") or "")  # "69 989 ₽"
            old = float(digits) if digits else None
    return shown, old


def parse_listing(html: str) -> list:
    soup = BeautifulSoup(html, "html.parser")
    state = listing_state(html)
    # A search with no matches still returns cards, under «Похожие объявления…».
    cut = [i for i in (html.find("extraTitle-"), html.find("Похожие объявления")) if i >= 0]
    results = set(re.findall(r'data-item-id="(\d+)"', html[:min(cut)])) if cut else None
    rows, seen = [], set()
    for card in soup.select(CARD):
        if results is not None and card["data-item-id"] not in results:
            continue                                       # substitutes, not results
        if card.find_parent(attrs={"data-marker": "itemsCarousel"}):
            continue                                       # «Подобрали для вас», not results
        sku = card["data-item-id"]
        if sku in seen:
            continue
        seen.add(sku)
        if sku in state:
            item = state[sku]
            price, old = prices(item)
            rows.append({"sku": sku, "title": item.get("title"),
                         "price": price, "original_price": old, "price_source": "state",
                         "url": "https://www.avito.ru" + item.get("urlPath", "").split("?")[0]})
        else:
            title = card.select_one('[itemprop="name"]')
            price = card.select_one('[itemprop="price"]')
            link = card.select_one('a[itemprop="url"], a[data-marker="item-title"]')
            rows.append({"sku": sku,
                         "title": title.get_text(" ", strip=True) if title else None,
                         "price": float(price["content"]) if price and price.get("content") else None,
                         "original_price": None, "price_source": "dom",
                         "url": "https://www.avito.ru" + link["href"].split("?")[0] if link else None})
    return rows


# ---------------------------------------------------------------------- main
def main(url: str) -> int:
    proxy = os.environ.get("AVITO_PROXY")
    cdp = os.environ.get("AVITO_CDP_ENDPOINT")
    with sync_playwright() as pw:
        if cdp:
            browser = pw.chromium.connect_over_cdp(cdp)
            context = browser.contexts[0] if browser.contexts else browser.new_context()
        else:
            launch = {"headless": True}
            if proxy:
                p = urlparse(proxy)
                launch["proxy"] = {"server": f"{p.scheme}://{p.hostname}:{p.port}",
                                   "username": p.username, "password": p.password}
            browser = pw.chromium.launch(**launch)
            context = browser.new_context(locale="ru-RU")
        page = context.pages[0] if context.pages else context.new_page()
        page.goto(url, wait_until="domcontentloaded", timeout=60_000)
        try:
            page.wait_for_function(
                "() => document.title.includes('Доступ ограничен') || "
                "document.querySelectorAll('[data-marker=\"item\"][data-item-id]').length > 1",
                timeout=20_000)
            page.wait_for_load_state("load", timeout=20_000)   # the listing keeps loading after the first cards
        except Exception:
            pass
        html = pass_wall(page, url, proxy)
        state = page_state(html)
        browser.close()

    if state != "content":
        print(f"blocked: page state is {state!r}", file=sys.stderr)
        return 3
    rows = parse_listing(html)
    print(json.dumps(rows[:5], ensure_ascii=False, indent=2))
    print(f"{len(rows)} rows", file=sys.stderr)
    return 0 if rows else 4


if __name__ == "__main__":
    sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else
                  "https://www.avito.ru/moskva?q=iphone+15"))

The example returns exit code 3 when the page is still blocked. In production that is the point to start a fresh browser with a different IP address, which is what the repository's fetch_with_rotation does.

Testing and Limitations

The repository ships an offline suite that runs with no network and no browser:

python3 smoke_test.py
python3 -m pytest -q

It checks the deterministic parts against fixtures cut from real pages: page-state classification for all three firewall states, both listing renderings, the price trap, item and seller parsing, solver task construction, proxy handling and output/metadata writing. CI also runs it once per engine in its own virtualenv and builds the Docker image. A daily canary scrapes three pages through the Browser API on a Russian IP and is green only when that run is complete and its rows pass the data checks.

Live runs, Moscow category «Мобильные телефоны → Apple», 2026-10-06:

And the search from Quick Start (?q=iphone+15 --pages 2), 2026-10-10:

Limitations to keep in mind:

• A local browser meets the GeeTest captcha on every new session, and from 2026-10-14 to 2026-10-20 none of its 15 solutions were accepted. Use the Browser API.
• The picture captcha and hCaptcha branches are implemented from the firewall page's code but were never served in testing.
• The seller feed relies on the page's own request to /web/1/profile/items, a path robots.txt disallows (see Seller profiles).
• The firewall's thresholds are not documented anywhere, and the repository doesn't claim any.

FAQ

Is it legal to scrape Avito?
The scraper reads public pages only: it doesn't log in, doesn't open contact details and doesn't message sellers. It respects robots.txt for every URL it requests, with the one seller-feed exception described above. The output contains sellers' names, so the personal-data law where you operate applies — in Russia, 152-FZ. Check Avito's terms as well. This is not legal advice.

Can I scrape Avito without a browser?
No. Plain HTTP clients got HTTP 429 on 6 of 6 attempts, and the proof-of-work page can only be cleared by a browser.

What captcha does Avito use?
Its firewall served GeeTest v4 on every measured request. The page also contains Avito's own picture captcha and hCaptcha, which the scraper can solve but which never appeared in testing.

How much does solving cost?
A GeeTest v4 solve cost $0.00299 in the repository's runs, and the scraper buys at most one solve per page per run. With a Browser API profile that has already passed the firewall, most runs need no solve at all: 11 runs on 2026-10-19 bought none.

Why do I need a Russian IP?
The firewall decides by IP and request rate, and the solver must use the same IP as the browser. A Russian residential exit with a sticky session keeps both on one address.

Conclusion

Scraping Avito comes down to two things: reading the data where the frontend actually keeps it, and treating the firewall as a normal part of the workflow.

On the data side: listing cards come from the DOM's microdata or, when present, the embedded data-mfe-state JSON. Item pages come from __staticRouterHydrationData. Always take the displayed price, never Avito's bare price field. Recommendation carousels and "similar listings" blocks look like results and aren't.

On the access side, Avito offers no plain-HTTP route. You need a browser with a Russian IP address. The proof-of-work page clears itself, GeeTest v4 needs a solution submitted through the page's own form, and the solution only counts when it comes from the browser's own IP. The «подождите…» page is not a captcha at all, just a signal to change the IP address.

Given the latest measurements, use a persistent Browser API profile even for one-off pulls: local-browser solutions have stopped being accepted. For regular monitoring, combine it with run metadata and diff_runs.py, so a temporary block or a re-ordered listing doesn't show up in your data as listings disappearing.

Get started: create a 2Captcha account for an API key and try the 2Captcha Browser API.

Source code

• Repository: github.com/2scraper/avito-scraper (MIT)
• All scrapers: github.com/2scraper

Documentation

• Quick start: API key, first request, first result.
• GeeTest: GeeTestTask parameters, including version 4.
• createTask and getTaskResult: the two calls behind every solve.
• Error codes and request limits.
• Browser API docs: CDP endpoint and profile options.
• Proxy API: generating residential proxy lists.
• 2captcha-python: the official Python SDK.

Related guides

• How to scrape SHEIN: solving grid and coordinates puzzles on SHEIN's risk gateway.
• How to scrape Transfermarkt: AWS WAF and pagination traps.
• How to scrape Google Play: per-country ratings and full review histories.
• How to scrape Lidl: product grids and unit prices.
• How to scrape YouTube: comments and metadata via the Innertube API.