It reads a few pages. We read the domain.
Point one at a competitor and it follows a handful of links, then stops. We follow every link off the home page, then every link off those, until the domain runs out.
- Depth 0 The page you handed it 1
- Depth 1 Everything linked from the home page 8
- Depth 2 Product pages, plans, comparisons 41
- Depth 3 Customer stories, changelog, the fine print 100
- Followed to the end, every run 150