Append/Recrawl URLs

The ‘Append URLs’ option gives users the autonomy of adding new URLs to the existing set of URLs that have been crawled.  Once selected, click on ‘Upload File’ to append URLs.  In addition, the ‘Recrawl URL’ option can be used to recrawl all website URLs and ensure the website content is updated.

In addition, users can recrawl specific pages on the ‘Tag Schema’ page to update the content as per the live version of those pages. To do so, click on the URLs that you want to update and click on ‘Recrawl’.

Note: Only once that pages have been crawled can any action of tagging, generating, and publishing of schemas be carried out. Your existing live schema is not affected by the recrawled pages. If there is a content update in the pages after recrawl, it is recommended to review and republish the schema so that the web page always has the schema referring to the latest content.