ScrapeAll

Scrapes all data elements from a web page by the specified CSS selector

Syntax

wds.ScrapeAll( downloadTask, selector, [attributeName] )

Arguments

Remarks

The selector argument is a selector of the following format: CSS|XPATH: selector. The first part defines the selector type, the second one should be a selector in the corresponding type. Supported types:

• CSS
• XPATH

Return type

TABLE (Data NVARCHAR(MAX))

Return value

List of found data or the empty list if nothing found

Examples

Creating a job and getting all product names string from the first page of the section Armor And Accessories on the Playground

DECLARE @jobConfig wds.JobConfig = 'JobName: TestJob1; Server: wds://localhost:2807; StartUrls: http://playground.svc';
SELECT  
    section.Task.Url as URL,
    (SELECT STRING_AGG(Data, ', ') FROM wds.ScrapeAll(section.Task, 'css: table tr td:first-child', DEFAULT)) Products
FROM wds.Start(@jobConfig) root
    OUTER APPLY wds.Crawl(root.Task, 'css: ul.nav li a[href^="/armor_and_accessories"]') section

Creating a job and getting all product names list from the first page of the section Armor And Accessories on the Playground

DECLARE @jobConfig wds.JobConfig = 'JobName: TestJob1; Server: wds://localhost:2807; StartUrls: http://playground.svc';
SELECT  
    section.Task.Url as URL,
    products.Data as ProductName
FROM wds.Start(@jobConfig) root
    OUTER APPLY wds.Crawl(root.Task, 'css: ul.nav li a[href^="/armor_and_accessories"]') section
    OUTER APPLY wds.ScrapeAll(section.Task, 'css: table tr td:first-child', DEFAULT) products