TakesNoBot

What it is, and who runs it

TakesNoBot is the scanner behind TakesNo. The bot is operated by TakesNo.

It does not roam the web looking for sites. Each scan starts from one page address that a person submitted to TakesNo.

What it does

TakesNoBot opens the submitted page in a real browser. It loads the page several times: once leaving the cookie banner untouched, once choosing accept, once choosing reject. Each load starts as a new visitor, with nothing stored from earlier visits. There is one exception: after the accept choice it may open the page once more with the cookies and browser storage that choice left behind, to record what a returning visitor is sent. After each load it records what the page did: the requests it made, the cookies it set, what it wrote to browser storage, the scripts and frames it loaded, and the consent signals it published. It also keeps a full-page screenshot and a copy of the page's HTML as the browser rendered it. The record sets what the banner said beside what the network did. It is technical evidence, not a verdict on your site.

What it does not do

• It does not log in. It holds no credentials for your site and never enters any.
• It does not fill in forms. It types nothing and fills in no fields. It clicks the buttons and links of the cookie banner, and of the preferences panel that banner opens. Where clicking does not register a choice, it may call the accept or reject command that the banner's own script provides. It also scrolls down the page, so that content further down loads as it would for a visitor.
• It does not crawl. It loads the one page address it was given and does not follow the links in your pages. If a control in the cookie banner or its preferences panel is itself a link, it is operated like any other control there. If your server redirects that address, it follows the redirect, as any browser would.
• It does not search for weaknesses. It behaves as a visitor's browser does and records what that browser was sent.

Besides the page and what the page itself loads, it asks your server for two small files: /robots.txt, to check whether it is welcome, and /.well-known/gpc.json, to see whether your site declares support for Global Privacy Control.

How to recognise it

TakesNoBot gives its name when it asks for your robots.txt file. That request carries this user agent:

TakesNoBot/1.0 (+https://takesno.com/bot)

Its robots.txt token is TakesNoBot.

The page itself is loaded in a Chrome browser, which sends a Chrome user agent. Page requests from a scan therefore appear in your logs as Chrome, after the robots.txt request from TakesNoBot.

When an image is sent without a Content-Length header, it fetches that image a second time, outside the browser, to measure its size. One scan can make several of these requests. They also carry the TakesNoBot user agent.

How to turn it away

Add these two lines to the robots.txt file at the root of your site:

User-agent: TakesNoBot
Disallow: /

TakesNoBot reads robots.txt before it first opens your page in each scan. Where the file turns it away, it stops there: the page is not opened and nothing is recorded from it.

• If you paste the full user agent from your logs on the User-agent: line instead of the short name, that works too.
• A rule for all robots (User-agent: *) applies to TakesNoBot too, unless the file also has a section naming TakesNoBot, in which case that section is the one it follows.
• To keep it out of part of a site, give the path: Disallow: /account/ covers every address beginning with /account/. Paths are matched from their start. * matches any sequence of characters and $ marks the end of an address, so Disallow: /* turns TakesNoBot away from the whole site too.
• TakesNoBot first asks for /robots.txt on the same host name as the submitted page. It follows up to five redirects, including redirects to another host name, and checks each address before requesting it. If it cannot reach a file within those redirects, receives an error response or gets no answer within five seconds, the scan goes ahead.
• Only the first 500 KiB of the file are read. Keep the TakesNoBot section within that limit in a very long file.

The person who asked for the scan is told that the site's robots.txt blocked it. No report is produced.

Contact

If our bot visited your site and you have a question or want it to stop, email bot@takesno.com.