scrape
Pay-per-use web scraping API for AI agents: any public https URL as Markdown, HTML, or a screenshot, from $0.003 a page. No account, no API key.
scrape is dvmkit's pay-per-use web scraping API for AI agents. Give it a public https URL and it returns the rendered page as Markdown, HTML, a screenshot, and structured metadata. It costs from $0.003 a page, with no account and no API key. It is a Digital Vending Machine (DVM): an HTTPS service an agent finds, pays, and calls on its own.
- What it does: fetches one public https URL and returns the rendered page in the formats you ask for.
- Price: From $0.003 per page on the fast tier; $0.005 per page on the default auto-routed tier. A screenshot adds $0.002. Pages over 1 MB, LLM extraction, and polish add metered surcharges, listed under Pricing.
- Pays with: USDC on Base, USDC.e on Tempo, or Bitcoin ecash (Cashu), per job or from a prepaid credit. The credit can also be funded over Bitcoin Lightning, or by card through Stripe-hosted checkout or Stripe Link with no wallet; see card top-ups.
- Endpoint:
https://scrape.dvmkit.ai, with one capability:fetch. A smaller one-request x402 fetch is also offered for stock x402 clients.
It runs two tiers: a fast in-process headless browser handles most pages, and a hardened anti-bot tier takes over only when a page fights back. Either way, the caller pays one price and never manages a browser.
Published reference. This page is the source of truth for scrape's documented inputs, outputs, prices, and errors. The running service publishes its current capability schema and advertised price in its live descriptor. Full request and quote envelope shapes are in the dvm CLI reference.
fetch
Fetch one URL. The only required field is url; everything else tunes format, rendering, and optional LLM post-processing.
Input
| Field | Type | Default |
|---|---|---|
url | string (https only, public host) | required |
format | "markdown" | "html" | "both" | "markdown" |
screenshot | boolean | false |
viewport | "mobile" | "desktop" | "desktop" |
country | string (lowercase ISO-3166-1 alpha-2) | unset |
wait_for_selector | string (CSS selector, XOR with ms) | unset |
wait_for_ms | number (≤ 30000, XOR with selector) | unset |
cache_max_age | number (seconds, ≤ 6 days) | 3600 |
tier | "fast" | "hardened" | unset (auto-route) |
extract | { schema | rows, instruction? } | unset |
polish | boolean (LLM-clean the markdown) | false |
integrity | "best_effort" | "strict" | "best_effort" |
Notes: country forces geo-routed fetching through the hardened tier, so it is incompatible with tier: "fast" (rejected as invalid_menu). wait_for_selector and wait_for_ms are mutually exclusive. tier: "fast" means never escalate to the hardened tier: the request surfaces fetch_blocked instead of silently paying the hardened price.
When the site itself answers with a 4xx, such as a 404 or 410, the job completes on either tier at the normal price. The result carries that status_code, plus the page body when one comes back.
integrity decides what happens when the delivered content might not actually satisfy the request, whether that's a redirect to a homepage, a lost query constraint, or a page in the wrong locale. best_effort (the default) still returns the bytes it got, along with diagnostics. strict fails the job instead, with no charge, whenever the derived verdict says the content can't safely be treated as satisfying what was asked for. This changes what happens on a bad match, not how the page is fetched or cached.
Response
Only the fields relevant to the request are present. Illustrative shape:
{
"final_url": "https://example.com/article",
"status_code": 200,
"metadata": { "title": "...", "canonical": "...", "og": {}, "jsonld": [] },
"markdown": "...",
"html": "...",
"screenshot_url": "https://.../scrape/<job_id>.png",
"truncated": false,
"semantic": { "type": "article", "title": "...", "byline": "...", "published": "..." },
"links": [{ "url": "...", "text": "..." }],
"outline": [{ "level": 1, "text": "..." }],
"images": [{ "url": "...", "alt": "..." }],
"extracted": { "title": "...", "author": "..." },
"rows": { "rows": [], "rows_returned": 24, "duplicates_removed": 2, "completeness": "partial" },
"polished_markdown": "..."
}Conditional fields: markdown / html per format; screenshot_url when screenshot: true; truncated when rendered HTML exceeded the 10 MB cap; extracted when extract was requested and the LLM call succeeded; rows when extract.rows was declared; polished_markdown when polish: true; semantic / links / outline / images present only when the page has matching content.
Always-on free surfaces
Every successful fetch also returns, at no extra cost:
links: everya[href]deduped by absolute URL, in document order, capped at 1000.outline:h1..h6headings in document order, for navigating long pages.images: every<img src>deduped by absolute URL, capped at 1000.semantic: flat structured data when the page exposes a recognisable shape (product,video,place,forum,search,article,feed, and more), extracted deterministically from JSON-LD / Open Graph / per-site selectors, no LLM involved. The most specific shape wins; absence ofsemanticmeans no shape matched.
Bien’ici property searches
Bien’ici town searches such as https://www.bienici.com/recherche/location/saint-germain-en-laye-78100/maisonvilla?prix-max=3500&prix-min=2500 use its public listings feed. The response includes listing links, prices, surfaces and search results. Signed callers can use extract.rows to read the listing cards. A search with no matches succeeds with empty results.
The feed route supports location and achat, single property types including maisonvilla and appartement, and the filters prix-min, prix-max, surface-min. It returns the first 24 listings and states the total when more exist. Unsupported filters use the browser route. Leave tier: "hardened", country, screenshot, HTML formats and explicit waits unset to allow the feed route.
CLI example
dvm request -d dvmkit--scrape/fetch --data '{"url":"https://example.com/article"}'
dvm request -d dvmkit--scrape/fetch --data '{"url":"https://example.com","format":"both","screenshot":true}'
dvm quote -d dvmkit--scrape/fetch --data '{"url":"https://example.com","polish":true}'Auth
Signed requests support every payment rail and prepaid credit. Every signed quote and job carries a secp256k1 + BIP-340 Schnorr signature over the canonical-JSON body, at the descriptor level so all capabilities share one auth slot. Drift window ±5 minutes; replay window 10 minutes. A stale or replayed signed request rejects with a structured 401 auth_error before payment is taken.
The CLI handles this for you: dvm init creates a default signing identity and dvm quote / dvm request sign with it automatically. Pass --as <identity> only when you want to sign as a different one. If you see auth_error, run dvm identity list to check you have one.
One-request x402 fetch
A stock x402 client, such as @x402/fetch, can fetch a page as markdown in one paid request, with no dvmkit identity. Send POST https://scrape.dvmkit.ai/v1/x402/fetch with a flat JSON body such as {"url":"https://example.com"}. The only other field is an optional viewport ("desktop" or "mobile"). The unpaid request returns HTTP 402 with a PAYMENT-REQUIRED header asking for $0.003 in USDC on Base. The client signs that payment and sends the same request again.
A URL that scrape won't fetch is refused before any payment, with HTTP 400 and no 402 challenge, so nothing is charged. The body is an error object with code, display, and hint. An http:// URL answers invalid_url. A private, loopback, or unresolvable host answers unsafe_url. When the DNS lookup times out or a nameserver fails, the 400 is unsafe_url with sub_reason: "dns_temporary": retry in a few seconds. The fetch checks the address again after payment, because DNS can change in between. A host that has turned private by then fails as an HTTP 500 that is not charged.
This is a separate, smaller definition from the metered fetch on /v1/job. It uses only the fast browser tier and never escalates to the hardened tier. It has no size or AI surcharges, and it runs for at most 25 seconds. It costs $0.003, collected only if the fetch succeeds. A busy browser pool, an unreachable page, a non-2xx page, a page with no readable markdown, and a page that runs past the time limit all fail without a charge. The browser starts in the background after a machine boots, so a call that wakes a stopped machine may spend part of its 25 seconds waiting for it. A heavy page on that call is the likeliest to run out of time. Paywall, login-wall, and app-shell pages fail with the same codes as on /v1/job, as uncharged failures.
A successful response carries status: "completed", a signed receipt if the DVM issues them, and a PAYMENT-RESPONSE header naming the settlement transaction. The page is in the inline JSON artifact in messages, with two fields: markdown and truncated. The page is cut to 512 KiB before conversion, and the markdown is capped at 160,000 UTF-8 bytes. truncated is true when either cut removed content. An oversized page still succeeds, truncated, and is charged.
Let the HTTP client wait at least 60 seconds: the request can include waking a stopped machine and settling the payment. Payments explains charge_outcome, including what to do when it reads unknown.
Unsigned paid calls (NUT-24)
A generic Cashu wallet can pay for fetch with one unsigned NUT-24 payment, without a dvmkit identity. Leave extract unset and polish unset or false; all other fetch options are available at an upfront price. Send the fetch body to POST /v1/job without payment first. The HTTP 402 response carries X-Cashu: creqA…, a payment request naming the amount, accepted mints, and lock. Pay that request and resend the same body with X-Cashu: cashuB…, without X-Cashu-Request-Id.
The paid retry returns HTTP 200 with the page inline in messages when the fetch finishes within the inline wait (10 seconds by default) and the messages total at most 1,048,576 serialised UTF-8 bytes. scrape's JSON artifact contains the markdown. Above that 1 MB cap, the response omits messages and keeps payload_in: "messages"; read the full result from GET /v1/job/:id/messages with X-Job-Token. If the fetch is still running, it returns HTTP 202 with job_id and job_token; poll GET /v1/job/:id with X-Job-Token as usual. Repeating the same token and body returns the original job without another charge. See the wire contract.
Send the unpaid request first, then have a NUT-24 wallet pay the X-Cashu payment request from challenge.headers. Set cashu_token to the provider-locked token it returns. Keep the body unchanged for the paid retry:
DVM_URL=https://scrape.dvmkit.ai
body='{"capability":"fetch","data":{"url":"https://example.com/","tier":"fast"}}'
curl -sS -D challenge.headers -o challenge.json \
-H 'Content-Type: application/json' --data "$body" "$DVM_URL/v1/job"
# The wallet prepares cashu_token from the challenge's X-Cashu payment request.
curl -sS -o paid.json -H 'Content-Type: application/json' \
-H "X-Cashu: $cashu_token" --data "$body" "$DVM_URL/v1/job"
jq -r '.messages[] | select(.type == "artifact" and .content.mime_type == "application/json") | .content.data | fromjson | .markdown' paid.jsonThe jq command reads the markdown directly from an inline completed response. If messages is absent, use job_id and job_token from paid.json to read /v1/job/:id/messages; a running job may need polling first.
Unsigned Cashu calls cost $0.002 more than signed calls and cover rendered HTML up to 3 MB without a size surcharge. They cannot use extraction, polish, or prepaid credit. Keep the job token for polling or later result reads when the response is not inline. An unsigned Cashu payer gets no refund if the job fails.
Prepaid credit
scrape offers a prepaid balance, so an agent making repeated calls funds once instead of paying per job. The funding menu rides on every quote and every 402.
| Field | Value |
|---|---|
| Minimum funding | $0.10 |
| Maximum residual balance | $20.00 |
| Credit lifetime | 90 days from the most recent funding |
dvm credit fund dvmkit--scrape --amount 2.00
dvm credit balance dvmkit--scrape
dvm credit drain dvmkit--scrape # reclaim it, works after expiry tooCard top-ups. scrape takes card top-ups in US dollars. Callers anywhere can pay a Stripe-hosted checkout link when scrape advertises it. Stripe Link is another option for agents in the US and Canada. Use its advertised card minimum; the SDK defaults to US$5. Each top-up needs your approval. With CLI 0.19.0 or later, dvm credit fund dvmkit--scrape --rail stripe --amount '$5' --no-wait returns a checkout link and next_step when the menu has credit.stripe.checkout: true. Pay the link, run next_step, and check credit balance. Without checkout, it returns Link approval steps. dvm credit drain dvmkit--scrape sends unspent card credit back to the card, less a proportional share of Stripe’s processing fee. Paying by card covers the flow.
A job priced above the maximum still clears: the ceiling binds how much balance you may hold, not what you may spend. A failed job is charged only for the pieces it finished. The rest of its charge is released back onto your credit rather than settled. See Prepaid credit for the full model.
Pricing
Minimum upfront + metered surcharges. The caller commits a small fixed fee at job start, which buys the fetch. Every other charge is a mid-job payment request, and each one buys the next piece of work. The size surcharge is asked once the page size is known. extract and polish are each asked at their cap before the LLM call, and you are charged what the call actually cost, with the rest released. The /v1/quote response includes the upfront amount and a hint enumerating every potential surcharge with its rate and cap, so a caller can size their token before committing.
Upfront commitment (locked in at job start):
| Component | USD |
|---|---|
Base (auto-route or tier: "hardened") | $0.005 |
tier: "fast" (cheap / API tier; never escalates) | $0.003 |
| Screenshot (any tier) | +$0.002 |
Metered surcharges (asked mid-job in this order, each only when it applies):
| Surcharge | Rate | Cap / worst case |
|---|---|---|
| Size (beyond 1 MB rendered HTML) | +$0.001 per additional MB (ceiling-rounded) | max +$0.009 (10 MB truncation cap) |
extract (either schema or rows) | $0.003 base + $0.005 / 1K input tokens + $0.025 / 1K output tokens | 50K in / 8K out → $0.453 |
polish | same rate card as extract | $0.453 |
The one-request x402 fetch is priced separately: a flat $0.003, collected only on success, with no surcharges.
Worst-case total if every knob fires at its ceiling: $0.922. Typical (no extract/polish, small page, no screenshot): $0.005. Amounts convert to sats at quote-time FX. Declining a surcharge stops the job at that stage. You keep what was delivered before the request and are charged for those finished pieces only: the upfront fee, any size surcharge you paid, and the measured cost of any extract or polish that finished. The declined surcharge is never taken. The page is delivered before the size request, so declining the size surcharge charges the upfront fee and you keep the whole page. Cancelling during an extract or polish call charges that piece's full cap, because it is lowered to the measured cost only when the call finishes.
LLM extract & polish
extract: { schema, instruction? }runs the rendered markdown through an LLM with your JSON Schema and returns the result inextracted. Tool-use forces schema-shaped output, which doubles as the primary defence against prompt injection in scraped content.schemais capped at 10 KB;instructionis an optional ≤ 500-char hint; source markdown is capped at ~50K tokens (extract_input_too_largebeyond that).extract: { rows, instruction? }is the search-page mode of the same call: same price, same cache, one LLM call. Declare the columns you want ({ name, type, description?, unit?, min?, max? }, 1–12 of them) and get one normalized row per result card inrows, alongside the markdown. Each row carries its values grounded against its own card, aconfidenceband, and theissuesbehind it; the set carries duplicate counts and whether these are the whole result set or one page of it. Numeric columns declared with aunitare checked against that dimension's plausible range and against the other rows on the page, so a value that is grounded but implausible, an impossible 3600 m² for a two-bedroom flat, comes back flagged rather than as fact. A page with no repeated-card structure runs no LLM call and is charged nothing for extract. Mutually exclusive withschema.polish: truereturns an LLM-cleaned markdown inpolished_markdownalongside the rawmarkdown. It drops residual nav chrome, fixes broken list/heading hierarchy, and tightens whitespace, without inventing content.
Both cache separately from the fetch (keyed by content hash + schema/model + instruction); cache hits bill the originally-recorded token cost without re-running the LLM.
Error codes
When Scrapfly supplies an error code and message, the failure reason preserves them as scrapfly_<CODE>: <message>. Its echoed request configuration is omitted. Scrapfly reports both malformed and unmatched selectors as ERR::SCRAPE::DOM_SELECTOR_NOT_FOUND. A local syntax check then distinguishes invalid_selector from selector_not_found. Passing this check does not establish that the selector identifies the intended content.
Failures surface via the CLI error envelope with a stable code. The HTTP-status column is documentary; codes are decoded client-side.
| Code | HTTP | Trigger |
|---|---|---|
auth_error | 401 | Request signature, clock, or nonce invalid; see Auth |
invalid_input | 400 | Request failed validation, or asked for a feature this instance isn't configured for (a screenshot with no blob storage wired, extract or polish with no LLM key). Rejected before payment |
invalid_url | 400 | URL malformed or non-https scheme |
unsafe_url | 400 | Host is private/internal/metadata, or DNS could not resolve it. On the x402 route, a DNS timeout or failing nameserver adds sub_reason: "dns_temporary": retry in a few seconds |
invalid_menu | 400 | XOR violation or other menu-shape violation |
wait_too_long | 400 | wait_for_ms > 30000 |
extract_schema_invalid | 400 | extract.schema empty, oversized, or rejected |
payment_required | 402 | Cashu token invalid / insufficient |
provider_allowance_exhausted | 402 | The operator's own anti-bot or LLM vendor plan is out of credit. Operator issue, not the caller's fault |
paywall_detected | 402 | Page loaded but content is gated |
invalid_selector | 400 | The hardened selector wait failed and the local CSS parser rejected its syntax. Correct wait_for_selector before retrying |
selector_not_found | 422 | The requested selector did not appear before the hardened rendering wait ended. Check the selector against the page or use another source; avoid an unchanged retry |
fetch_blocked | 403 | Hard tier failed without a more specific classification |
cloudflare_unsolved | 403 | Couldn't get past the target site's bot challenge |
captcha_unsolved | 403 | Couldn't bypass a CAPTCHA |
login_wall_detected | 403 | Page loaded but returned a logged-out login/onboarding shell instead of the content |
job_timeout | 408 | 90s wall-clock exceeded |
page_too_large | 413 | Rendered HTML > 10 MB and normalisation failed |
extract_input_too_large | 413 | Rendered markdown exceeds the 50K-token extract cap |
polish_input_too_large | 413 | Rendered markdown exceeds the 50K-token polish cap |
app_shell_detected | 422 | Page loaded 2xx but its content region never populated: chrome without content. A genuine zero-result page is not flagged |
request_integrity_failed | 422 | integrity: "strict" was set and the delivered content can't safely be treated as satisfying the request: a homepage fallback, a canonicalized or path-collapsed resource, a lost query constraint, or the wrong place. No charge; see the integrity field above |
screenshot_unavailable | 422 | screenshot: true, the target site answered with a 4xx, and no screenshot came back with it. Nothing is charged; retry with screenshot: false to receive the 4xx page |
rate_limited | 429 | Substrate quota exhausted |
rate_limited_by_target | 429 | Origin target returned 429 |
geo_blocked | 451 | Content is region-locked |
substrate_unavailable | 502 | Anti-bot backend unreachable (after one retry) |
extract_validation_failed | 502 | LLM output failed schema validation twice |
extract_llm_failed | 502 | Extract LLM call threw a non-recoverable error |
polish_llm_failed | 502 | Polish LLM call threw a non-recoverable error |
fetch_timeout | 504 | Both cheap and hard tiers timed out |
internal_error | 500 | Unexpected DVM-side fault |
See also
- Explore the catalog · dvm CLI reference · Caller quickstart
- Other first-party DVMs: scribe · narrate · cast · discover · satoshi-quotes
satoshi-quotes
Check a quote or claim against the Satoshi Nakamoto Institute (SNI) public archive. Get a verdict, source passages, and links for $0.005 per check.
scribe
Pay-per-use speech-to-text API for AI agents: audio or video to a timestamped transcript, $0.02 a job plus from $0.005 a minute. No account, no API key.