Splash
Headless browser · Scrapinghub
Headless browser
Type
Scrapinghub
Made by
No
Verifiable
None
robots.txt token
What Splash is
Splash JavaScript rendering service.
It has no robots.txt token. Documentation: project site .
Why it cannot be verified
Crawlers run by a company can be checked: the company publishes the addresses or DNS names its crawler uses, and a request from anywhere else is not really it. Splash is software anyone can run, so there is no such list. The User-Agent tells you which software sent a request, or which software the sender chose to name. Changing it is one line of code, so the default User-Agent mostly identifies people who did not bother.
Handling it
Before blocking it, check whether you rely on it: your own scripts, uptime checks, webhooks and partner integrations often use tools like this. Then judge requests by what they do rather than the name they give:
- How fast they arrive, and from how many addresses
- Which paths they ask for, such as admin pages, config files or every product page in order
- Whether they touch links or fields no person could see
robots.txt does not help here: it only works on a client that chooses to read and obey it.
That is the approach WebDecoy takes: hidden decoy links and traps that only automated clients follow, so a request is judged by its behaviour, whatever User-Agent it sends.
Frequently Asked Questions
What is Splash? +
Splash is a headless browser made by Scrapinghub. Splash JavaScript rendering service.
Can I verify a request from Splash? +
No. Splash is software anyone can run, so there is no operator whose IP ranges or DNS names a request could be checked against. Its User-Agent says which software sent the request, or which software the sender chose to name, not who sent it.
Should I block Splash? +
It depends on your site. Your own scripts, monitoring and integrations may use it. Blocking on the User-Agent alone only stops clients that leave the default in place, and anyone can change it. Judge requests by behaviour instead: request rate, the paths they ask for, and whether they touch pages no person would.
Related tools
- Browserbase Browserbase
- Browserless Browserless
- HeadlessChrome Google
- PhantomJS Open Source
- Playwright Microsoft
- Puppeteer Google
- Selenium Open Source
Catch automated clients, whatever they call themselves
WebDecoy sets hidden decoys only automated clients follow, and tracks them as persistent actors rather than disposable IPs.