Sites we can report on, and sites we cannot
Most sites are fine. A few cannot be read at all, and one of them might be yours, so this is the list, before you pay rather than after. We also check the site before taking any money, and refuse the sale rather than take it.
Sites that are fine
- An ordinary business site of five, fifty or five hundred pages
- A one-page site, a holding page, or a site that is mostly a contact form
- A shop with thousands of product pages
- A site built on WordPress, Squarespace, Wix, Shopify, Webflow, or hand-written HTML
- A site that renders its content with JavaScript, which we read in a real browser
- A site in a language other than English
What happens with a site that is not in English is its own page, because the answer has two halves.
Sites we cannot read
These are the four things that stop a report, in the words a buyer is shown if the check catches them:
- We could not find that site
- Nothing answered at that address. Check the spelling, and if the site is new check that its domain is pointing at your host yet.
- That site did not answer
- The address is real but the server did not send us a page. If it is down, it is worth trying again once it is back.
- That site is turning us away
- Something in front of the site is refusing us, so we cannot read a single page of it. It is usually a firewall or a bot filter, and whoever runs the site can let us through. It is worth knowing either way: whatever turned us away is deciding the same thing about search engines.
- That address has no home page
- The domain answers, but its front page is missing. If your site lives somewhere else on the domain, that is the address to give us.
- That address is not a website
- Whatever is at that address is not a web page, so there is nothing here for us to read.
If any of those happens before payment, the sale is refused and nothing is charged. If one happens after, and the check is a single request that cannot see everything, we either finish the report by hand or refund you in full.
A firewall is the common one, and it is worth knowing about
Cloudflare, a bot filter or an aggressive WAF can turn us away. We arrive as an ordinary browser, from one address, at a few pages a minute, and we identify ourselves in a header on every request, so whoever runs the site can let us through, and the crawler page is written for exactly that conversation.
That is the part worth knowing either way. A filter strict enough to refuse a browser reading a few pages a minute is a filter that is making decisions about crawlers you do want. It is a finding rather than an inconvenience, and it is one most owners have never been told.
If your site is very small
It works, and it is arguably where the report earns the most. The smallest site we have produced a report for had four pages. It came back with 18 pages to write, and you can read the whole thing: that is the sample.
A true holding page, one screen with a logo and an email address, is the honest edge. There is enough to report on, because what the report is mostly doing is working out what a business like yours has to say, but a great deal of it will be *this does not exist yet* and you will already suspect as much. If that is your site, answering the four questions is what turns the report from a description into a plan.
If your site is very large
We open at most 100 pages. On a bigger site that is not the first 100 we stumble into: every address is discovered and scored first, and the crawl spends its budget on the pages a buyer actually lands on: what the site says it sells, what the sitemap treats as important, what has been kept up to date. Legal boilerplate and deep archive pages go last.
Pages under a repeating template, a thousand products in one catalogue shape, are counted rather than opened one at a time, because reading the nine-hundredth tells us nothing the first ten did not.
The report says which of those things happened. It prints how many pages it read against how many it found worth reading, rather than implying the whole site was seen, and a figure it could not complete is printed as a floor rather than a total. What is outside that count is the part of a multilingual site we did not root the report on, and the page on languages says why and how much.
What the check before payment actually does
One request for your home page, made while you are answering the questions, so the answer is usually back before you reach the card. It asks for https:// first and falls back to http:// only if https could not be reached at all.
It is deliberately one request and no more: we are fetching a site a stranger typed, before any money has moved and with nobody's authorisation, so a crawl at that point would be the wrong thing to do. That is also why it cannot see everything, and why the refund above exists.
If it refuses you, your answers are kept, the address goes back in an editable box, because it is usually a typo, and nothing has been charged.