Scrape
Extracts named values from the downloaded page using ScrapeParams selectors and optional conversions, then completes the current task. If processing is pending, poll the scrape-result URL
Endpoint
POST /api/v3/tasks/{taskId}/scrape
Path Parameters
| Name | Type | Description |
|---|---|---|
| taskId | string | Required. Download task ID returned by job start, in download task records from crawl requests, or embedded in a 202 Location URL from async task endpoints |
Request Body
Required application/json body containing an array of ScrapeParams.
Responses
200 (OK)
Returns TaskScrapeResult[]. Each item uses the public ScrapeResult shape.
202 (Accepted)
Task processing is pending. Follow the Location header to the crawl-result or scrape-result endpoint and retry after the number of seconds in Retry-After (currently 1). Continue until the response is no longer 202 (Accepted).
400 (Bad Request)
Invalid request parameters. Refer to the response text for more information.
403 (Forbidden)
Access restricted. Refer to the response text for more information.
404 (Not Found)
Task not found.
422 (Unprocessable Content)
There is an issue with processing the page content. Refer to the response text for more information.
Selector Format
The selector argument is a selector of the following format: CSS|XPATH: selector. The first part defines the selector type, the second one should be a selector in the corresponding type.
Supported types:
Data Contracts
ScrapeParams
Defines one named field extracted from pages by selector, attribute, and optional conversion
Fields:
| Name | Type | Description |
|---|---|---|
| Name | string | Name of the output field populated from this extraction rule. Names must be unique within one scrape request or Traversal level |
| Selector | string | Selector used to read matching elements. Use ‘CSS: |
| AttributeName | string or null | Optional. Attribute read from each matched element. Use ‘val’, or omit/pass null or empty, to return inner text |
| Convert | string or null | Optional. Conversion applied to scraped values. Use ‘md()’ for Markdown or ‘sr()’ for main readable content using Mozilla Readability. Omitted or null applies no conversion |
ScrapeResult
Values extracted for one ScrapeParams entry
Fields:
| Name | Type | Description |
|---|---|---|
| Name | string or null | Optional. Field name copied from the matching ScrapeParams entry |
| Values | array of string | Extracted values returned by the selector, attribute, and optional conversion |