Your server already writes down every request. Drop the access log into this page and see, for ten AI crawler user agents, how many requests each one made, which pages it asked for, the status codes it got back, and how long it waits before coming back to the same page.
Filtering a raw log for a handful of user agents, splitting it by page and working out repeat visits usually means a long command line or a spreadsheet that chokes on the file. This does it in one pass, on your own machine.
GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot, Perplexity-User, Meta-ExternalAgent and Bingbot, each labelled self-declared.
The median time between repeat requests for the same page, per crawler, plus first seen and last seen in UTC whatever offset your server logs in.
Paste your sitemap and get, for each crawler, the sitemap pages it did not request in the log you loaded. Any status code counts as a request.
Click a crawler for every page it asked for, with counts and status codes. Download a summary, a per-page file and the never-requested list.
Built by Mulkern AI Systems. Your log is read inside your browser. Nothing is uploaded.
Bot names come from the user-agent string each request declared. This tool does not check IP addresses, so any request can claim to be any bot.
Pay once, keep it. Works offline once installed, on desktop or phone. No account, no sign-in, no upload.
Buy Now, $19