The ZAP Spider: Automated Discovery Engine

The Spider, also called the crawler, automatically discovers content within web applications by following links, submitting forms, and parsing responses for additional URLs. Unlike simple web crawlers that merely follow hyperlinks, ZAP's Spider understands modern web applications, parsing JavaScript, handling AJAX requests, and discovering content that traditional crawlers miss. This sophisticated discovery mechanism ensures comprehensive coverage of your target application before vulnerability testing begins.

Understanding how the Spider works helps optimize its configuration for different applications. The Spider starts from seed URLs—typically your application's homepage—and recursively discovers new content. It parses HTML responses for links, form actions, JavaScript references, and comments that might contain URLs. The Spider maintains a queue of discovered URLs, systematically visiting each while respecting depth limits and scope constraints. Modern web applications often generate URLs dynamically, requiring the Spider to execute JavaScript and observe DOM changes.

The traditional Spider handles conventional web applications effectively but struggles with modern single-page applications (SPAs) that heavily rely on JavaScript. For these applications, ZAP offers the Ajax Spider, which uses browser automation to discover content. The Ajax Spider launches a real browser, interacts with the application as a user would, and observes resulting DOM changes. This approach discovers content invisible to traditional crawling but requires more resources and time.