Vision
AI-generated content is spreading across the internet at a scale never seen before. KrausBot reads the web to study this shift — analyzing text, detecting patterns, and building a clear picture of what the modern web actually contains.
What KrausBot does
Systematically discovers and visits publicly accessible web pages, following links across the internet to reach as much content as possible.
Extracts and processes the textual content of each page — studying writing style, structure, vocabulary, and linguistic patterns.
Identifies signals that distinguish human-written text from AI-generated content — building a dataset that reflects the real composition of the modern web.
Transparency
Respects robots.txt
KrausBot strictly follows the Robots Exclusion Protocol. If you've disallowed crawling in your robots.txt — we won't crawl.
Polite crawl delays
We apply crawl delays between requests to minimise server load. KrausBot is designed to be a good citizen of the web.
Open User Agent
We identify ourselves clearly in every HTTP request so server operators always know who is visiting their resources.
No personal data
KrausBot only reads publicly accessible text content. No user data, cookies, or private information is ever collected or stored.