WDS v3.1 Released — Inspect. Retain. Deploy.

Collecting web data is only part of the challenge. Inspecting downloaded content, managing recurring crawl history, and simplifying deployment are just as important.

Web Data Source v3.1 makes these workflows easier to operate.

What’s new?

🔹 HTML & Markdown Content Browsing
Inspect downloaded pages directly through REST endpoints, with rewritten links for navigation and dedicated evaluation entry points.

🔹 Traversal Run Retention
Use MaxRuns to automatically prune older completed Traversal runs before new runs begin, keeping execution history manageable while preserving data shared by other runs.

🔹 Stateless MCP Server
Streamable HTTP replaces session-dependent MCP communication, simplifying integration with AI agents and MCP clients.

🔹 Infrastructure & API Improvements
Updated container distributions, improved request tracing, configurable identifier hashing, and more consistent REST task polling.

For existing users: v3.1 introduces API and internal gRPC changes that require attention during upgrades. Installed MSSQL CLR function signatures remain unchanged.

The goal remains the same: turn web resources into structured, reusable data for databases, automation, and AI agents — with more control over the entire lifecycle.

WDS v3.1 is now available.

Explore the release: WDS v3.1 documentation