Overview

The Crawled URLs feature helps website managers see which pages have been successfully scanned by the site crawler. This insight is crucial for maintaining good SEO health, confirming content visibility, and ensuring all key webpages are properly tagged and indexed.

For teams managing multiple sites or domains, this feature provides a centralized way to monitor website coverage and track crawl results across environments.

Step-by-Step Process:

Once your URLs have been uploaded to Schema Manager, you can view them in the 'Tag Schema' section of Schema Manager.

The page will then display the web page URLs that have been discovered and grouped together. 

URLs with a certain number against them and highlighted in blue are those that have been grouped.

You will see the schemas deployed under the ‘Schemas Deployed’ column, and the  sign under the ’Status’ column indicates that the schema has been successfully published.

However, you will see this message until the discovery is complete.

Glossary of Terms

Crawled URL: A page successfully scanned by the crawler for content and structure.

Schema Markup: Code that helps search engines understand a webpage’s content (e.g., product, review, FAQ).

Orphaned Page: A page with no inbound internal links, making it difficult for crawlers to find.

Auto-Discovery: A system-driven process that scans site links without manual input.

Audit Report: A post-crawl summary showing coverage, errors, and schema application across URLs.