Quartz 4

robots.txt

1 min read

Syntax

  • User-agent- Following instructions are for a specific bot (or * for all)
  • Allow- Encourages bot to call following URL
  • Disallow- Robot should not crawl pages

Graph View

Backlinks