Professional

Crawl paths

• Last updated: June 18, 2024

• Read time: 4 Minutes

By default, Burp Scanner uses an embedded Chromium browser to navigate around a target during the crawl phase of a scan. This enables it to act much as a skilled manual tester would, discovering dynamic content that a regular scanning engine might miss.

The Crawl paths tab shows the results of this process in real time, displaying the actions taken by Burp Scanner to reach each location discovered in the target site, and any issues found in those locations.

Accessing the Crawl paths tab

You can access the Crawl Paths tab from two locations:

• To view combined path information for all regular, non-isolated scans in the current project, go to Target > Crawl paths. Any new non-isolated scans that you run add to the information displayed in this tab.
• To view path information for an isolated scan, go to the Dashboard tab and select the scan from theTasks list. In the main panel, go to the Target > Crawl paths tab. This tab doesn't include information from any other scans. You only see this tab for scans that have the Run isolated scan setting selected.

Viewing crawl paths

The First crawl path to location panel shows each location discovered by Burp Scanner during the crawl phase of the scan. The tree structure of this panel enables you to see the path that Burp Scanner took to discover each location. Paths include actions such as clicking links, submitting forms, and making requests to background URLs.

To view a screenshot of Burp's browser at any crawl location, select a node then click Show screenshot.

Example crawl path

For example, suppose that Burp Scanner is crawling an online shop and finds that each product page has a "more information" link. The path to this link may be displayed as something like:

Requested URL "example-shop.net" > Clicked "Products" > Clicked "Example Product" > Clicked "More Information"

Any additional actions performed by Burp Scanner from the "Example Product" page (such as clicking any additional links or submitting a stock information form) would be displayed underneath the Clicked "Example Product" node in the tree.

Viewing HTTP requests

To view further information on the HTTP requests and responses sent at a particular point, select a node from the First crawl path to location panel. The HTTP messages tab displays details of the messages that were sent to reach the selected location, including:

• Host
• Method
• URL
• Params
• Status code
• Length
• MIME type
• Title
• Time requested

Select a message to view the raw request and response in the tabs below.

All destination URLs include fragments. This makes it easier to understand what outcome a particular action leads to.

You can customize and sort the table contents. For more information, see Customizing Burp's tables.

Viewing issues

If the scan is configured to audit as well as crawl, the First crawl path to location panel also shows details of any issues discovered.

Where an issue is found, the panel displays an icon next to the relevant node in the tree. This enables you to see the path taken by Burp Scanner to discover the issue, helping you to understand how you can reproduce it if required.

Where a node has one or more children, it displays the icon of the most significant issue that can be found among its child nodes. For example, if a node's children have one high-severity issue and ten information-only issues, the parent node displays a high-severity issue icon.

To view further information on the issues discovered at a particular location, select a location from the First crawl path to location panel and click the Issues tab. This tab contains a list of the issues found at that location in this scan. Select an issue to view the specific requests and resources that identified it, as well as a tab containing advisory information.

Viewing outlinks

To view details of all the possible navigation actions that the crawler was able to take from a given location in the crawl path, select the Outlinks tab. Actions may include, for example, clicking links, submitting forms, and making requests.

For each action the crawler completed from the selected location, further details are displayed, including the:

• Destination URL - The URL that the crawler navigates to.
• Title - The title of the destination page, if there is one present.
• Issues - Any issues found on the destination page. These are only displayed if the scan is configured to audit as well as crawl.
• Requests - The number of requests made as a result of taking the action.
• Time visited - The time at which the crawler took the action.