OusanBot
OusanBot visits the home pages of websites to record what they run: server, technologies, page title, certificate and favicon. The results feed the site profiles and technology statistics on ousan.io.
User agents
- Website visits
- Mozilla/5.0 (compatible; OusanBot/1.0; +https://ousan.io/bot)
- Certificate Transparency logs
- Ousan-CT/1.0 (+https://ousan.io/bot)
What it fetches
The home page of a hostname, following at most 5 redirects, reading at most 2 MB, giving up after 10 seconds.
Sometimes the favicon, and whether /robots.txt exists.
It does not run scripts, submit forms or follow links into a site.
How often
A site that answers is visited about once a week, one that does not about once a month. A site may be checked sooner when its DNS changes. Requests to one domain are spaced out, so a site never sees a burst from us.
Stop the visits
Write to [email protected] with the domains you want left out. We stop visiting them and remove what we recorded.