WDS for MS SQL Server

Bring web crawling and scraping into T‑SQL. WDS for MS SQL Server is a CLR library with user‑defined types (UDTs) and functions that let you start jobs, discover pages, extract data, and check task status directly from SQL. See Plans for feature availability.

What You Can Do

• Start: launch a job with a JobConfig and receive initial DownloadTasks.
• Crawl: discover follow‑up pages and get new DownloadTasks.
• Scrape: extract one value (ScrapeFirst), all values (ScrapeAll), or multiple fields in one call (ScrapeMultiple).
• Inspect: query DownloadTaskStatus to monitor progress and debug issues.

Prerequisites

• SQL Server: tested with SQL Server 2022 (Windows).
• WDS API Server: running and reachable (see Deployments).

Components

• CLR Functions: ServerStatus, Start, Crawl, ScrapeFirst, ScrapeAll, ScrapeMultiple, TaskStatus, and ToStringsTable — see CLR Functions.
• UDT contracts: 23 installed SQL types configure jobs and carry results. Their source-backed descriptions are embedded in the function pages that use them; start with JobConfig, DownloadTask, and DownloadTaskStatus.

Examples

Explore end‑to‑end scripts for common scenarios — see Examples

Install

Step‑by‑step instructions to enable CLR and load the library — see Install

Support

If you hit issues, please use GitHub Issues.