Crawl

Performs a download task to collect subsequent pages of web resources

Syntax

wds.Crawl( downloadTask, selector, [attributeName] )

Arguments

Name	Type	Description
downloadTask	DownloadTask	Required. A download task from a previous command result set
selector	String	Required. Selector for getting interesting links on a web page
attributeName	String	Optional. Attribute name to get data from. Use `val` to get inner text. Default value: `href`

Remarks

The selector argument is a selector of the following format: CSS|XPATH: selector. The first part defines the selector type, the second one should be a selector in the corresponding type. Supported types:

CSS
XPATH

Return type

TABLE (Task wds.DownloadTask)

Return value

List of DownloadTask (one per a found URL) in a form of table.

Examples

DECLARE @jobConfig wds.JobConfig = 'JobName: TestJob1; Server: wds://localhost:2807; StartUrls: http://playground.svc';
SELECT 
    nav.Task.Url URL
FROM wds.Start(@jobConfig) root
    OUTER APPLY wds.Crawl(root.Task, 'css: ul.nav a', null) nav

URL
http://playground.svc/
http://playground.svc/armor_and_accessories/1/
http://playground.svc/beast_and_creature_items/1/
http://playground.svc/elemental_and_nature_items/1/
http://playground.svc/magical_artifacts/1/
http://playground.svc/potions_and_elixirs/1/
http://playground.svc/rings_and_amulets/1/
http://playground.svc/wands_and_staffs/1/
http://playground.svc/weapons/1/