WebDecoy Is on the Vercel Marketplace: See Every Crawler

WebDecoy is now on the Vercel Marketplace. Press Connect Account, choose your projects, and WebDecoy starts reporting the automated traffic reaching your site on the very next request. No package to install, no middleware to add, no redeploy.

The listing makes one promise: WebDecoy tells you which automated traffic reaches your site, and whether it is really who it claims to be. This post covers how the install keeps that promise, and then walks through everything else WebDecoy can do on a Vercel project, from the zero-code drain to the Next.js middleware to rules in your Vercel Firewall.

Install in three steps

1. Open the WebDecoy listing and press Connect Account, or go straight to vercel.com/integrations/webdecoy/new. Choose the team, then all projects or specific ones.
2. Vercel sends you back to WebDecoy. No account yet? Create one on the spot. The install waits for you and picks up where you left off.
3. Tell WebDecoy which site each project is, and press Connect. If a project serves its own domain, WebDecoy offers to create a site named after that production domain, so two projects on two domains never merge into one report.

That is the whole install. WebDecoy provisions a log drain, and the next request your site answers is reported.

To see it work, request your robots file from a terminal:

curl https://your-site.com/robots.txt

A detection for cURL appears on the Detections page a few seconds later.

Every capability at a glance

The rest of this post is the detail behind each row.

The drain: detection with zero code

A script tag only sees visitors that run JavaScript. The crawlers most site owners now care about do not. GPTBot, ClaudeBot, CCBot, and PerplexityBot fetch your HTML and leave. Googlebot’s crawl pass is a plain HTTP fetch, with rendering deferred to a separate pass that may never happen for a given URL. None of them ever run your tag.

A Vercel log drain has no such blind spot. Vercel’s edge records every request it answers, including responses served straight from its cache and static files that never invoke a function, and pushes those records to WebDecoy. The drain sees traffic your application code never does.

WebDecoy creates one drain per installation, scoped to the projects you connected. Connect another project and the drain widens. Disconnect one and it narrows. Uninstall and it is deleted. Before the drain goes live, WebDecoy validates its configuration with Vercel’s drain test, so a problem shows up during connect instead of as silence afterwards.

It asks for request logs only. Build output is filtered out on Vercel’s side, so you are not paying to deliver logs WebDecoy would throw away, and it never asks for traces, Web Analytics, Speed Insights, or audit logs.

What gets reported

A drained request becomes a detection when any of these is true:

• Its user agent matches the agent registry. That is more than 170 known crawlers, AI agents, and HTTP tools, and the same registry behind every WebDecoy sensor, so GPTBot is the same GPTBot whichever edge saw it.
• It has no user agent at all. Browsers always send one.
• It asks for a path only automation asks for: /robots.txt, /sitemap.xml, /llms.txt, /.well-known/ai.txt, or /ads.txt.

Everything else is ordinary browser traffic, and it is dropped before anything is stored. A drain carries every human visitor too. WebDecoy does not keep them. Query strings are removed from what is kept, because the path is what identifies what was asked for.

Claimed, verified, or forged

A user agent is a claim, and a free one. Anyone can put Googlebot in a header.

The drain hands WebDecoy the client IP exactly as Vercel’s edge saw it, which is not something the client can set. WebDecoy checks every crawler claim against what the operator publishes: its IP ranges and forward-confirmed reverse DNS. Each crawler request comes out as one of three answers:

Forged crawlers are the interesting ones. Something that dresses up as Googlebot to get past your rules is telling you what it intends to do.

JA4 fingerprints and cache status

Each drained request carries a JA4 digest: the TLS fingerprint Vercel’s edge computed during the handshake. WebDecoy records it with the detection. A script can write any user agent it likes, but its TLS handshake is produced by the library it actually runs on, and the fingerprint sits right next to the claim it confirms or contradicts.

The cache result is recorded too, so you can tell whether a crawler was served from Vercel’s cache or reached your functions.

Signed, and built to stay on

Every delivery is signed by Vercel with a secret WebDecoy generated for your drain alone. WebDecoy checks the signature in constant time and rejects anything unsigned or mis-signed.

A correctly signed delivery is always acknowledged, even one full of traffic that is not worth keeping. Vercel disables a drain that fails too often, and a drain that quietly switched itself off would look exactly like a site with no bots.

Coverage you can trust

Two settings on your side can thin what a drain carries. WebDecoy tells you about both instead of showing you a quiet site.

• IP Address Visibility. Under Settings → Security & Privacy, a Vercel team can remove client IPs from drains. WebDecoy cannot attribute those requests to an actor, so it counts them and marks your coverage as limited.
• Sampling. If the drain samples at 10%, WebDecoy sees 10%.

The dashboard only says a site is reporting when a drain is provisioned and a project is linked. Authorizing the integration is not the same as receiving traffic, and WebDecoy never treats it as if it were.

Hobby teams

Vercel offers log drains on Pro and Enterprise teams, and its Pro trial does not include them. On Hobby, the connect still completes, WebDecoy tells you drains are not available on your plan, and the Next.js middleware below is the way in.

What WebDecoy can reach in your account

Three scopes, and nothing else:

It cannot read your source, your deployments, your domains, or any logs beyond the drain it created.

Connecting writes nothing to your project. Earlier versions of the install set three environment variables (WEBDECOY_SITE_KEY, WEBDECOY_SCANNER_ID, and WEBDECOY_SENSOR_KEY) that nothing ever read, and the drain made them unnecessary. If your project was connected before that changed, disconnecting it removes them, and that cleanup is the only reason the environment variable scope is still requested.

Removing it

Disconnect a project and WebDecoy narrows the drain to the projects that remain. On a project connected by an earlier version, it also deletes the variables that version wrote, by the ids Vercel gave them. A variable with the same name that you set yourself is left alone.

Disconnect the integration in WebDecoy and it deletes the drain and uninstalls itself from your Vercel account. We do not offer an install we cannot cleanly reverse.

The Next.js middleware: in-request decisions, on any plan

The drain reports after a request is answered. When you want a decision before your route runs, or you are on a Hobby team, add @webdecoy/nextjs. It runs in Next.js middleware on Vercel, including the Edge runtime.

npm install @webdecoy/nextjs @webdecoy/node

// middleware.ts
import { withWebDecoy } from '@webdecoy/nextjs';
import { NextResponse } from 'next/server';

export default withWebDecoy({
  apiKey: process.env.WEBDECOY_API_KEY!,
  threshold: 70,
  mode: 'monitor', // switch to 'enforce' once you have read what it would block

  onBlocked: (request, detection) =>
    NextResponse.json({ error: 'Blocked', threat: detection.threat_level }, { status: 403 }),
});

export const config = {
  matcher: ['/api/:path*', '/login', '/checkout/:path*'],
};

Create the API key under Settings → API Keys in WebDecoy and add it to your project as WEBDECOY_API_KEY.

Three defaults matter on Vercel:

• Monitor mode is the default. The middleware records what it would have blocked and changes nothing until you set mode: 'enforce'. Watch first, then act.
• The client IP is read correctly for Vercel. The middleware reads X-Forwarded-For from the right, trusting exactly one proxy by default, which is Vercel itself. A client cannot pick its own address by sending the header.
• It fails open. If WebDecoy cannot be reached, the request goes through and the error is logged. Your site does not depend on ours.

To protect one route instead of a whole matcher, wrap the handler with withBotProtection.

Rules that run in process

The middleware takes the same rules as the rest of the SDK, evaluated inside the request with no round trip:

Shared rate limits. Vercel runs your middleware on many instances at once, so an in-memory counter only sees a slice of the traffic. upstashRateLimitStore() keeps the counters in Upstash Redis over HTTP, which works on the Edge runtime, and Upstash is one click away on the Vercel Marketplace:

import { rateLimit, upstashRateLimitStore } from '@webdecoy/node';

rateLimit({
  max: 100,
  window: 60,
  store: upstashRateLimitStore({
    url: process.env.UPSTASH_REDIS_REST_URL!,
    token: process.env.UPSTASH_REDIS_REST_TOKEN!,
  }),
});

One bot policy. botPolicy() builds your robots.txt from the same object the middleware enforces. A robots file that disallows GPTBot while your code lets it through is a policy you believe is in force and is not. One source means the two cannot drift.

A honeytoken in one line

honeytokenLink() derives a hidden link from your API key, so every instance and every render agrees on the same path without coordinating. Put it in your root layout and arm it as a tripwire in middleware. Humans never see or follow it. A crawler that does has told you exactly what it is. It is a helper rather than something injected automatically, because Next.js streams App Router responses and there is no finished document for middleware to rewrite.

Self-hosted captcha

createCaptchaHandler() mounts WebDecoy’s proof-of-work captcha as a catch-all App Router route, paired with the @webdecoy/client widget. Scoring runs in your own deployment, with no third-party script on your page and no call out to a captcha vendor.

Tracing and testing

The SDK emits OpenTelemetry spans through the tracer you pass it, without taking a dependency on OpenTelemetry itself, so detection time shows up in the traces you already collect.

To verify an install, send the reserved test user agent to a route your matcher covers:

curl -A "WebDecoy-Test/1.0" http://localhost:3000/api/hello

It fires a real detection through the whole pipeline, works on next dev before you deploy, and appears labeled Test in the dashboard, excluded from stats and billing.

Vercel Firewall: acting on what you found

Detection tells you who is there. When you want Vercel itself to act, connect your Vercel Firewall as an enforcement target. Vercel publishes no firewall scope for marketplace integrations, so this takes a Vercel API token, and it lives under Enforcement targets on the Integrations page, separate from the marketplace install under Detection sources.

Press Test and WebDecoy reads your project’s firewall config from Vercel, writing nothing. If Vercel refuses, you see Vercel’s own reason. A 403 can mean a bad token, a token without access to the project, or a plan that does not expose the firewall, and those are three different fixes, so WebDecoy does not guess which one it is.

WebDecoy writes IP rules into your project’s firewall with one of three actions:

Apply a rule yourself. From the Vercel Firewall page in WebDecoy, block or challenge an address for a set length of time. You choose; WebDecoy carries it out. The control appears when your firewall integration is set to block or challenge; one set to log only is configured to record, not act, so it is never offered as protection. A few guardrails come with it:

• A bounded lifetime. Every rule has an expiry, and WebDecoy removes it from your firewall when that time is up.
• No locking yourself out. If the address is the one you are connecting from, WebDecoy stops and tells you before writing anything.
• Scoped to your site. A rule for one site is written for that site’s hostname, and WebDecoy first checks that your Vercel project actually serves it. A rule naming a host the project does not answer for would look active and match nothing.
• A recorded author. Every manual rule records who applied it and why.

Every change is proven, not assumed. Manual or automatic, each rule goes through the same path:

• Recorded before it is sent, so a retry never creates a second rule for the same IP.
• Read back from Vercel, so “applied” means Vercel lists the rule, not that the API call returned 200.
• Rechecked while active. If someone removes the rule in the Vercel dashboard, WebDecoy notices and says so instead of claiming protection that is no longer there.
• Reversible by hand. Any rule can be removed early from the list of changes on the Vercel Firewall page.

One caution. Disable a firewall integration rather than deleting it if you want its rules to keep expiring. Deleting it takes the API token with it, so WebDecoy can no longer remove the rules it applied. Reverse the ones you want gone first, or remove them in the Vercel dashboard afterwards.

The evidence reads the two protective actions the right way round. Traffic from an IP after a block means the rule is not holding. Traffic after a challenge means a client passed it.

Automatic rules are gated deliberately. WebDecoy withholds firewall changes driven by a score until that score has a measured false-positive rate behind it. It fails closed, because a bot product that blocks your customers is worse than none. A rule you apply yourself rests on your decision, so it is not held back.

What it costs

WebDecoy’s Free plan includes 5,000 stored events a month with seven days of detail, needs no card, and does not expire. The drain stores only automated traffic, so your human visitors do not count toward it. Your paid-capability trial starts when WebDecoy verifies your first real installation, not when you create an account.

On the Vercel side, drain delivery is billed as part of your plan’s usage. That is why WebDecoy asks for request logs only and filters out build output.

Get started

From Vercel: open the WebDecoy listing and press Connect Account.

From the docs: the Vercel installation guide covers the drain, and the Next.js middleware guide covers the SDK.

The drain and the middleware complement each other. The drain sees everything Vercel’s edge answers, including the crawlers that never touch your code. The middleware decides inside the request, on the routes you choose. Run the drain for visibility, then add the middleware where a decision has to happen before the response.

Start free or read how WebDecoy verifies agent identity.