sliced-resume

Crawl a web resource by slices and build a resume of what it is about and what it offers based on the currently available slices.

How to call

In supported clients the command syntax can vary. If the WDS MCP server is registered as wds, use:

Arguments

Returned Prompt Text

The block below contains the complete prompt returned to the MCP client. Runtime placeholders map to arguments as follows:

• <url> — url

ROLE:
Act as a web developer with experience in designing multi-page web resources, including paged data, templating, and expertise in pure CSS as well as in modern CSS frameworks: Bootstrap, Tailwind CSS, Foundation, Bulma, UIkit, Tachyons, Materialize, Metro, Spectre.css, Milligram, Pure.css, Skeleton, NES.css, W3.CSS, Picnic CSS, Basscss, Shoelace, etc.
Your experience includes working with web resources that have complex structures, such as those with paged data, hierarchical menus, and leaf pages with data accessible from various navigation elements. You are skilled in identifying and extracting data fields from web pages using CSS selectors and XPath expressions.

OBJECTIVE:
Crawl through all pages of TARGET_URL website using WDS MCP tools, gather its content, and prepare a resume of what the website is about and what it offers.

RESTRICTIONS:
    -   The agent must use the exact values returned by MCP tools as inputs for later steps. Do not rewrite or guess job names, URLs, run numbers, cursors, or selectors.
    -   MCP structured tool results are serialized as lower-camel JSON fields. Use returned field names such as runNum, completeDateUtc, failedDownloadTasks, data, and dataCursor when reading tool results.
    -   The agent must perform all steps of the PROCEDURE in order, without skipping or parallel execution.
    -   Call MCP tools with named arguments matching the tool parameter names.
    -   WdsFetchChunked returns the download task ID as a string. It does not return raw page text.
    -   Use WdsGetChunk(taskId=taskId, startIndex=startIndex, length=length) to read task content in chunks. Do not use shell/file tools to recover hidden or truncated content from MCP output.
    -   Resolve relative links against the current page URL before passing them to WdsFetchChunked.
    -   To change a JobConfig, read the current object with WdsGetJobConfig when it exists, create or modify the complete object in memory, then persist it with WdsUpsertJobConfig(jobName=jobName, jobConfig=jobCfg). Preserve all unrelated existing properties.
    -   When setting nested JobConfig properties, create missing nested objects or arrays before setting their fields.
    -   To change a TraversalConfig, create or modify the complete object in memory, then persist it with WdsUpsertTraversalConfig(jobName=jobName, traversalConfig=traversalConfig).
    -   WdsRunTraversal and WdsTraversalAll return TraversalRunResult, not crawl data. Use runNum with WdsGetTraversalRunInfo, WdsGetTraversalRunErrors, and WdsGetTraversalRunData.
    -   On any MCP tool error, the agent must STOP WITH REASON and surface the error response.

GLOBAL CONSTANTS:
    TARGET_URL = '<url>'
    CONVERT = 'md()'

PROCEDURE (EXECUTE IN EXACT ORDER)
    1.  Identify host
    -	Extract authority/host from TARGET_URL -> lowercase, normalized -> tgtHost
    2.  Get or create job
    -   jobs = WdsGetJobs()
    -   Find job where job.jobName equals tgtHost (case-insensitive)
    -   If not found:
        -   jobName = tgtHost
        -   Do not call WdsGetJobConfig before creating the job; a missing job is expected on this path.
        -   Determine jobType:
            -   'Intranet' if host is single-label or ends with .local, .internal, .svc.cluster.local, or lacks a TLD
            -   Otherwise 'Internet'
        -   Create jobCfg as a complete JobConfig:
            -   startUrls = [TARGET_URL]
            -   type = jobType
            -   restart.jobRestartMode = 'Continue'
        -   Persist jobCfg with WdsUpsertJobConfig(jobName=jobName, jobConfig=jobCfg)
        -   jobCfg = WdsGetJobConfig(jobName=jobName)
    -   If found:
        -   jobName = job.jobName
        -   jobCfg = WdsGetJobConfig(jobName=jobName)
    3.  Configure job
        -   Update jobCfg in memory, preserving all unrelated properties:
            -   jobCfg.restart.jobRestartMode = 'Continue'
        -   Persist jobCfg with WdsUpsertJobConfig(jobName=jobName, jobConfig=jobCfg)
        -   jobCfg = WdsGetJobConfig(jobName=jobName)
    4.  Start crawling
        -   convert = CONVERT
        -   maxDepth = 0
        -   traversalRunResult = WdsTraversalAll(jobName=jobName, convert=convert, maxDepth=maxDepth)
        -   runNum = traversalRunResult.runNum
        -   Do not infer runNum from job state, run lists, previous output, or arithmetic.
    5.  Validate crawl completion
    -   Wait for the traversal run to complete:
        -   maxPolls = 60
        -   pollIntervalSec = 1
        -   pollCount = 0
        -   lastTraversalRunInfo = null
        -   REPEAT while pollCount < maxPolls:
            -   pollCount = pollCount + 1
            -   traversalRunInfo = WdsGetTraversalRunInfo(jobName=jobName, runNum=runNum)
            -   lastTraversalRunInfo = traversalRunInfo
            -   If traversalRunInfo.completeDateUtc is not null:
                -   BREAK
            -   Wait pollIntervalSec seconds before the next poll.
            -   Continue polling with WdsGetTraversalRunInfo(jobName=jobName, runNum=runNum) until traversalRunInfo.completeDateUtc is not null or pollCount reaches maxPolls.
        -   If lastTraversalRunInfo.completeDateUtc is null:
            -   STOP WITH REASON because traversal run did not report completion after maxPolls polls; include runNum and lastTraversalRunInfo in the reason.
    -   traversalRunErrors = WdsGetTraversalRunErrors(jobName=jobName, runNum=runNum)
    -   If traversalRunErrors.failedDownloadTasks is NOT empty:
        -   For each downloadTask in traversalRunErrors.failedDownloadTasks:
            -   For each failedDownloadTask in downloadTask.failedDownloadTasks:
                -   status = WdsGetDownloadTaskStatus(taskId=failedDownloadTask.id)
                -   If status.result is null:
                    -   Collect status.url, null, null
                -   If status.result is not null:
                    -   Collect status.url, status.result.httpStatusCode, status.result.reasonPhrase
        -   Output a table: | URL | HttpStatusCode | ReasonPhrase |
    -   Proceed with successfully gathered data
    6.  Get all gathered data
        -   cursor = null
        -   traversalAllData = []
        -   REPEAT:
            -   traversalData = WdsGetTraversalRunData(jobName=jobName, runNum=runNum, path=null, cursor=cursor, limit=100)
            -   Append every item in traversalData.data to traversalAllData without dropping or overwriting previously fetched items
            -   cursor = traversalData.dataCursor
        -   UNTIL cursor is null
    7.	Analyze the first part and answer
        -   Analyze all retrieved and fetched data in traversalAllData.
        -   Build the first part of a resume of the website with the following structure:
            -   Main Topic: a brief description of the main topic of the website;
            -   Contact Information: a list of contact information available on the website, including phone numbers, email addresses, and social media links;
            -   FAQ: a list of frequently asked questions found on the website, with answers to each question;
        -   SHOW the partial resume obtained so far to the user.
    8.  Continue crawling the rest of the data without confirmation.
        -   Inform users that you are continuing crawling the rest of the data automatically.
        -   Do not ask users for confirmation, just continue automatically.
    9.  Check robots.txt and inform user about CrawlDelay if any
        -   robotsUrl = scheme and authority from TARGET_URL + '/robots.txt'
        -   Read robots.txt content:

            -   taskId = WdsFetchChunked(jobName=jobName, url=robotsUrl)
            -   REPEAT:
                -   contentResult = WdsGetChunk(taskId=taskId, startIndex=0, length=1024)
                -   If contentResult is not null: BREAK
                -   Continue polling WdsGetChunk with the same arguments until contentResult is not null.
            -   robots = contentResult.contentSpan if it is not null; otherwise ''
            -   contentLength = contentResult.contentLength if it is not null; otherwise length(robots)
            -   If more unseen content is required for the current analysis:
                -   REPEAT:
                    -   startIndex = length(robots)
                    -   If startIndex >= contentLength: BREAK
                    -   contentResult = WdsGetChunk(taskId=taskId, startIndex=startIndex, length=1024)
                    -   If contentResult is null: continue polling WdsGetChunk with the same arguments until contentResult is not null
                    -   Append contentResult.contentSpan to robots
                -   UNTIL enough relevant content has been inspected or startIndex >= contentLength
            -   Analyze only the content present in robots. If required evidence is in an unread span, read the next span before making conclusions.
        -   If robots is not null and robots is not empty:
            -   Parse robots as plain text. Find the first case-insensitive 'Crawl-delay:' directive and set crawlDelay to its trimmed value.
            -   If crawlDelay is not null:
                -   SHOW user the Crawl-delay found in robots.txt: crawlDelay seconds.
        -   Update jobCfg in memory, preserving all unrelated properties:
            -   If crawlDelay is not null, ensure jobCfg.crawlersProtectionBypass.crawlDelays contains { host: tgtHost, delay: crawlDelay }
            -   If crawlDelay is null, ensure jobCfg.crawlersProtectionBypass.crawlDelays contains { host: tgtHost, delay: 'robots' }
            -   jobCfg.restart.jobRestartMode = 'Continue'
        -   Persist jobCfg with WdsUpsertJobConfig(jobName=jobName, jobConfig=jobCfg)
    10. Continue crawling
        -   maxDepth = null
        -   traversalRunResult = WdsTraversalAll(jobName=jobName, convert=convert, maxDepth=maxDepth)
        -   runNum = traversalRunResult.runNum
        -   Do not infer runNum from job state, run lists, previous output, or arithmetic.
    11. Validate crawl completion
    -   Wait for the traversal run to complete:
        -   maxPolls = 60
        -   pollIntervalSec = 1
        -   pollCount = 0
        -   lastTraversalRunInfo = null
        -   REPEAT while pollCount < maxPolls:
            -   pollCount = pollCount + 1
            -   traversalRunInfo = WdsGetTraversalRunInfo(jobName=jobName, runNum=runNum)
            -   lastTraversalRunInfo = traversalRunInfo
            -   If traversalRunInfo.completeDateUtc is not null:
                -   BREAK
            -   Wait pollIntervalSec seconds before the next poll.
            -   Continue polling with WdsGetTraversalRunInfo(jobName=jobName, runNum=runNum) until traversalRunInfo.completeDateUtc is not null or pollCount reaches maxPolls.
        -   If lastTraversalRunInfo.completeDateUtc is null:
            -   STOP WITH REASON because traversal run did not report completion after maxPolls polls; include runNum and lastTraversalRunInfo in the reason.
    -   traversalRunErrors = WdsGetTraversalRunErrors(jobName=jobName, runNum=runNum)
    -   If traversalRunErrors.failedDownloadTasks is NOT empty:
        -   For each downloadTask in traversalRunErrors.failedDownloadTasks:
            -   For each failedDownloadTask in downloadTask.failedDownloadTasks:
                -   status = WdsGetDownloadTaskStatus(taskId=failedDownloadTask.id)
                -   If status.result is null:
                    -   Collect status.url, null, null
                -   If status.result is not null:
                    -   Collect status.url, status.result.httpStatusCode, status.result.reasonPhrase
        -   Output a table: | URL | HttpStatusCode | ReasonPhrase |
    -   Proceed with successfully gathered data
    12. Get all gathered data
        -   cursor = null
        -   traversalAllData = []
        -   REPEAT:
            -   traversalData = WdsGetTraversalRunData(jobName=jobName, runNum=runNum, path=null, cursor=cursor, limit=100)
            -   Append every item in traversalData.data to traversalAllData without dropping or overwriting previously fetched items
            -   cursor = traversalData.dataCursor
        -   UNTIL cursor is null
    13.	Analyze and answer
        -   Analyze all retrieved and fetched data in traversalAllData.
        -   Build a resume of the website with the following structure:
            -   Main Topic: a brief description of the main topic of the website;
            -   Contact Information: a list of contact information available on the website, including phone numbers, email addresses, and social media links;
            -   Target Audience: a brief description of the target audience of the website, including their interests and needs.
            -   FAQ: a list of frequently asked questions found on the website, with answers to each question;
            -   Services: a list of services found on the website (if they are), with a full description of each service.
            -   Products: a list of products found on the website (if they are), with a full description of each product;
        -   SHOW the whole resume obtained to the user.