Simple server-side integration
Begin with api_key and a URL-encoded url. Add method, location, rendering, response, or asynchronous options only when the workload requires them.
API de scraping
Scraper API for reliable
web data retrieval.
Send a URL and receive the page body, a JSON response, or selected fields while WebScrapingAPI operates proxy routing, access handling, optional JavaScript rendering, and internal target retries.
- api_key + url Smallest valid request
- Managed routing Proxy rotation and access handling
- Optional render_js Browser rendering when required
- Body · JSON · selectors Documented response choices
One managed request
Scraper API keeps retrieval infrastructure
out of your application.
Provide the target and documented request options. WebScrapingAPI operates the access layer and returns a response your existing pipeline can inspect.
Managed access handling
WebScrapingAPI operates proxy routing, rotation, access-handling strategies, and internal target retries behind the API request.
Review managed featuresDocumented response choices
Return the page body, a JSON envelope, a JSON DOM, customer-defined selected fields, or an explicit documented error state.
Review response formatsYour application providesTarget URL + request context
WebScrapingAPI operatesRouting + access handling + rendering
Your application receivesBody + JSON + selected fields
Automated access features
Move repeatable retrieval work behind one Scraper API request.
Configure the context your workflow needs. WebScrapingAPI operates the supporting access infrastructure while keeping unsuccessful states visible to your client.
01
Proxy routing and rotation
The service selects and operates the proxy route behind each supported request context.
02
Access handling and retries
Automated access strategies and internal target retries reduce the infrastructure your application has to run.
03
Observable access outcomes
Validation, access, rate, and service errors remain explicit so your application can handle unsuccessful requests deliberately.
04
Optional JavaScript rendering
Set render_js=1 when the target needs client-side execution before the page response is returned.
05
Geographic request context
Request a documented country and supported city, state, or ASN context when location affects the response.
06
Request and output controls
Use supported methods, an optional body, timeout, asynchronous retrieval, JSON modes, or extraction rules.
Response design
Return the level of response your workflow can use.
Keep the page body, wrap it with request metadata, execute selectors at the API boundary, or retrieve a queued result later.
Raw page response
Use the default page body with an existing parser.
JSON envelope
Inspect body, target headers, initial status, type, cost, and metadata.
Selected fields
Execute customer-owned CSS or XPath selectors for text, HTML, attributes, tables, or nested records.
Queued retrieval
Submit work asynchronously and retrieve the completed response using its identifier.
Extraction boundary: WebScrapingAPI executes the supplied rules. Your team owns the schema, selectors, business-field validation, and source-change maintenance. Choose Data API or Managed Data when those obligations should move to WebScrapingAPI.
How it works
From target URL to application-ready response in three steps.
Your application defines the request. WebScrapingAPI handles retrieval. Your client receives the requested output or an explicit state it can act on.
- 01Send the request
Provide the API key, target URL, and only the documented context or output options the workload needs.
- 02WebScrapingAPI retrieves the page
The service validates the request, operates proxy routing, applies access handling and internal retries, and renders JavaScript when requested.
- 03Receive the response
Use the returned page body, JSON envelope, selected fields, or documented error inside your existing application.
- Request metadata
- Resolved URL and scraped response headers are available through documented response headers.
- Usage context
- Per-call and current credit usage can be inspected through the documented WSA-prefixed headers.
- Error contract
- Validation, access, rate, and service errors remain explicit inputs to client-side handling.
Developer integration
Call one endpoint from the language
already running your data pipeline.
Keep the API key server-side, URL-encode the target, begin with a basic request, and add rendering, geography, or extraction only when the workload requires it.
01 · integration brief
Begin with the smallest valid request.
Send api_key and a URL-encoded url. Add render_js=1, geographic context, a response mode, or extraction rules only when the target and downstream workflow require them.
- Secrets
- Keep
WSA_API_KEYserver-side - Target
- URL-encode every target URL
- Réponse
- Inspect status and type before parsing
- Failures
- Use bounded backoff for retryable states
curl --get --fail-with-body --max-time 120 \
"https://api.webscrapingapi.com/v2" \
--data-urlencode "api_key=$WSA_API_KEY" \
--data-urlencode "url=https://example.com/public-page" \
--data-urlencode "json_response=1"import os
import requests
response = requests.get(
"https://api.webscrapingapi.com/v2",
params={
"api_key": os.environ["WSA_API_KEY"],
"url": "https://example.com/public-page",
"json_response": 1,
},
timeout=120,
)
response.raise_for_status()
print(response.json())const params = new URLSearchParams({
api_key: process.env.WSA_API_KEY,
url: "https://example.com/public-page",
json_response: "1",
});
const response = await fetch(
"https://api.webscrapingapi.com/v2?" + params,
{ signal: AbortSignal.timeout(120_000) }
);
if (!response.ok) throw new Error("API status " + response.status);
console.log(await response.json());<?php
$query = http_build_query([
"api_key" => getenv("WSA_API_KEY"),
"url" => "https://example.com/public-page",
"json_response" => 1,
]);
$client = curl_init("https://api.webscrapingapi.com/v2?" . $query);
curl_setopt_array($client, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_TIMEOUT => 120,
]);
$body = curl_exec($client);
$status = curl_getinfo($client, CURLINFO_RESPONSE_CODE);
$error = curl_error($client);
curl_close($client);
if ($body === false) throw new RuntimeException($error);
if ($status >= 400) throw new RuntimeException("API status " . $status);
echo $body;package main
import (
"fmt"
"io"
"net/http"
"net/url"
"os"
"time"
)
func main() {
params := url.Values{}
params.Add("api_key", os.Getenv("WSA_API_KEY"))
params.Add("url", "https://example.com/public-page")
params.Add("json_response", "1")
client := &http.Client{Timeout: 120 * time.Second}
response, err := client.Get(
"https://api.webscrapingapi.com/v2?" + params.Encode(),
)
if err != nil { panic(err) }
defer response.Body.Close()
if response.StatusCode >= 400 {
panic(fmt.Sprintf("API status %d", response.StatusCode))
}
body, err := io.ReadAll(response.Body)
if err != nil { panic(err) }
fmt.Println(string(body))
}import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
String query = "api_key=" + URLEncoder.encode(
System.getenv("WSA_API_KEY"), StandardCharsets.UTF_8
) + "&url=" + URLEncoder.encode(
"https://example.com/public-page", StandardCharsets.UTF_8
) + "&json_response=1";
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.webscrapingapi.com/v2?" + query))
.timeout(Duration.ofSeconds(120))
.GET().build();
var response = HttpClient.newHttpClient()
.send(request, HttpResponse.BodyHandlers.ofString());
if (response.statusCode() >= 400) {
throw new RuntimeException("API status " + response.statusCode());
}
System.out.println(response.body());using System.Net;
var apiKey = WebUtility.UrlEncode(
Environment.GetEnvironmentVariable("WSA_API_KEY")
);
var target = WebUtility.UrlEncode(
"https://example.com/public-page"
);
var requestUrl =
"https://api.webscrapingapi.com/v2?api_key=" + apiKey +
"&url=" + target + "&json_response=1";
using var client = new HttpClient();
client.Timeout = TimeSpan.FromSeconds(120);
var response = await client.GetAsync(requestUrl);
response.EnsureSuccessStatusCode();
Console.WriteLine(await response.Content.ReadAsStringAsync());require "net/http"
require "uri"
uri = URI("https://api.webscrapingapi.com/v2")
uri.query = URI.encode_www_form(
api_key: ENV.fetch("WSA_API_KEY"),
url: "https://example.com/public-page",
json_response: 1
)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.open_timeout = 10
http.read_timeout = 120
response = http.get(uri.request_uri)
raise "API status #{response.code}" unless response.is_a?(Net::HTTPSuccess)
puts response.bodyOperating ownership
WebScrapingAPI operates retrieval. Your team owns the data product.
WebScrapingAPI operates routing, optional rendering, and internal target retries. Your team owns target eligibility, the request specification, downstream extraction, storage, and use.
WebScrapingAPI
Managed request infrastructure
- API request validation and execution
- Proxy routing and rotation
- Automated access handling and internal target retries
- Optional browser-backed rendering
- Documented response and error delivery
Votre équipe
Source and data-product decisions
- Target eligibility, intended use, and request policy
- URL, method, context, timeout, and output options
- API-level failure handling and business validation
- Downstream extraction, selector maintenance, and storage
- Scheduling, lineage, retention, and downstream use
Source and purposeCustomer defines eligible sources and intended use.
Request contractCustomer configures the documented request; WebScrapingAPI validates and executes it.
Couche d'accèsWebScrapingAPI operates proxy routing, access handling, internal retries, and optional rendering.
RéponseWebScrapingAPI returns the requested output or documented error; the customer validates content for its application.
ExtractionWebScrapingAPI executes supplied rules; the customer owns selectors, schema, and change maintenance.
Downstream operationCustomer owns scheduling, storage, monitoring, retention, and decisions.
Use the service only for eligible public webpages. Review source terms, applicable requirements, and the Accord de service, then apply appropriate request, retention, and data-use policies.
Use cases
Use the returned page inside workflows that still need custom logic.
Scraper API is the access layer. Your application decides how the page becomes an observation, record, alert, test result, or model input.
Product choice
Choose the product by the part of the workflow your team wants to keep.
Start from operating ownership, not feature overlap. Scraper API fits one managed page request; adjacent products move a different boundary.
Infrastructure de proxys
Les proxies
Maximum request-level controlWeb access APIYou operate the HTTP client, request policy, retries, parsing, and storage. WebScrapingAPI provides the network route.
API de scraping
One managed page requestWeb access APIProvide a URL and request options. WebScrapingAPI operates retrieval and returns the page body or documented output.
API de navigateur
Guided browser requestWeb access APISend a browser-backed REST request with documented waits, interactions, navigation steps, and output controls.
API SERP
Search-result recordsGuided evaluationSend search parameters and receive structured results from supported search engines.
API d’exploration
Scoped page collectionStructured APIEvaluate a bounded multi-page collection when the unit of work spans related pages rather than one request.
API de données
Maintained structured recordsLivraison géréeQuery a supported source while WebScrapingAPI operates access, target-specific extraction, and parser maintenance.
Données gérées
Recurring data deliveryDefine sources, schema, cadence, quality, and destination while WebScrapingAPI operates the contracted program.
Pricing orientation
Evaluate with representative targets before you size a plan.
Current pricing remains the source of truth. Use a realistic request sample to understand the options and operating behavior your production workload will use.
Self-serve evaluation
Move a representative request into the API.
Start with the basic HTTP path, then test only the controls your production target mix requires.
- Representative URLs and target methods
- Static versus browser-rendered request mix
- Geography, timeout, and response requirements
- Observed error states and client retry policy
FAQ
Scraper API questions for a grounded evaluation.
Use these answers to choose the product and request path, then review the current documentation for parameter and error details.
What is Scraper API?
Scraper API is a server-side request endpoint that retrieves an eligible public webpage while WebScrapingAPI operates the access infrastructure behind the request. Your application supplies the URL and documented options, then receives the requested page response.
What does Scraper API return?
The default response is the retrieved page body. Documented options can return a JSON envelope, a JSON DOM representation, fields selected with extraction rules, or an asynchronous result that your application retrieves later.
Does Scraper API render JavaScript?
Yes. Set render_js=1 when a page requires client-side JavaScript execution before the response is returned. Use Browser API when the workflow needs a documented sequence of browser interactions, page-state waits, clicks, or navigation steps.
What access work does Scraper API manage?
WebScrapingAPI operates proxy routing, rotation, access-handling strategies, and internal target retries within the service. Explicit API errors remain possible, so your application should still handle documented unsuccessful states.
Which HTTP methods are supported?
Scraper API supports GET, POST, PUT, and PATCH requests. Include a request body when the selected method and target workflow require one.
Can I request location-specific content?
Documented controls include country and supported city, state, or ASN context. Exact availability can depend on the current plan, geography, and route, so confirm the locations required by your workload.
Can Scraper API return selected fields?
Yes. Customer-supplied CSS or XPath extraction rules can return selected text, HTML, attributes, tables, lists, or nested records. Your team owns the selectors, schema, validation, and source-change maintenance for those rules.
Can requests run asynchronously?
Yes. Queue a request with the documented asynchronous option, retain the returned identifier, and retrieve the completed result later by passing that identifier as snapshot_id.
How should my application handle unsuccessful requests?
Inspect the documented HTTP status and error body, use bounded retry and backoff where appropriate, and distinguish an API failure from content that is present but incomplete for your business schema.
How is Scraper API different from Proxies, Browser API, and Data API?
Scraper API manages one page retrieval. Proxies leave the HTTP client and collection logic with your team, Browser API executes documented browser interactions within a server-side REST request, and Data API returns maintained structured records for supported sources.
Your first request
Move one representative page into the API.
Start self-serve, or bring us the target mix, required response, geography, and expected request profile for a production review.