What Changed on Our Website? Bot Defense Edition

When traffic quality changes after a release, the hard part is proving what changed. A vendor migration, CDN rule edit, JavaScript update, tag manager publish, cache policy, or product release can all shift your website bot defense changes without anyone filing a security ticket.

Treat the question like a production investigation: compare before and after behavior, isolate the change window, and avoid assuming the newest vendor is the cause. Bot defense is a posture, not a single switch. OWASP recommends layered controls because login, checkout, search, account creation, and content pages face different automated threats (OWASP Bot Management and Anti-Automation Cheat Sheet).

Start With the Change Window

Define the smallest useful time box first. Use deployment logs, CDN audit history, DNS changes, tag manager publishes, WAF updates, consent banner changes, and bot vendor events to identify the exact before-and-after period. If the investigation starts with “last week,” every chart will look plausible.

Then build a baseline from signals you already trust:

• Total requests by route, method, country, ASN, user agent family, and status code.
• Bot scores, challenge rates, block rates, allow rates, and verified crawler rates.
• Business outcomes such as login failures, account creation conversion, checkout errors, search volume, inventory holds, and form spam.
• SEO and crawler indicators, including robots.txt fetches, sitemap access, crawl errors, and load from known search crawlers.

Keep good bots and abusive automation separate. Search engines, monitoring, partners, and internal QA should not be mixed with account takeover traffic or scraping. Google’s robots.txt documentation is a useful reminder that robots.txt manages cooperating crawlers, but it is not access control for sensitive content (Google Search Central).

Compare Defense Surfaces, Not Just Vendors

A bot-defense posture can change even when the named vendor does not. The common failure mode is measuring “Vendor A versus Vendor B” while missing surrounding surfaces.

Check the edge first. CDN rules, cache keys, header normalization, protocol settings, redirects, geo rules, and rate limits can change what reaches the origin and what telemetry the bot system sees. A rule that bypasses inspection for cached HTML or selected paths may make defenses look quieter without lowering risk.

Next, check browser-side signals. JavaScript changes can affect challenge execution, first-party cookies, device signals, consent gating, CSP, script ordering, and single-page app routes. Keep tests defensive by using approved synthetic clients, internal QA sessions, and known-good monitoring traffic.

Finally, check application behavior. A release that adds a login endpoint, moves an API behind GraphQL, or changes error handling can alter both attacker interest and detector confidence. OWASP’s automated threats project classifies many abuse patterns, which helps reframe the question from “did bot traffic go up?” to “which automated behavior changed?” (OWASP Automated Threats to Web Applications).

Sample Investigation Checklist

Use this checklist when a stakeholder asks what changed on the website.

• Confirm the investigation window: start time, end time, release IDs, CDN changes, vendor cutover time, and rollback points.
• Snapshot the control plane: active WAF rules, bot rules, rate limits, allowlists, blocklists, challenge modes, and logging settings.
• Compare route telemetry: request volume, status codes, latency, origin hit rate, challenge outcomes, bot scores, and cache status by path.
• Review crawler handling: robots.txt, sitemap availability, canonical pages, verified search crawler traffic, and SEO monitoring alerts.
• Review login and account endpoints: failed login rate, password reset volume, MFA prompts, account creation attempts, session creation, and suspicious success rates.
• Review revenue or content workflows: checkout errors, inventory spikes, coupon abuse, search scraping, pricing traffic, and form spam.
• Check browser changes: JavaScript bundle hash, tag manager version, consent behavior, CSP headers, cookie attributes, and third-party scripts.
• Check edge and origin differences: forwarded headers, IP preservation, TLS settings, rate-limit keys, cache bypasses, and routing.
• Validate known-good traffic: monitoring, uptime checks, QA automation, partner integrations, search crawlers, and accessibility scanners.
• Document the decision: what changed, what did not change, evidence, remaining uncertainty, owner, rollback option, and follow-up monitoring.

Where possible, retain raw logs long enough to replay the analysis. Many bot platforms expose fields such as scores, verified bot flags, or detection identifiers; Cloudflare’s bot-management field documentation is one example of telemetry that can explain request classification (Cloudflare Ruleset Engine docs).

Make Future Changes Easier to Explain

Current pressure makes this discipline worth the effort. Akamai’s 2025 fraud and abuse research reported a sharp rise in AI-driven bot activity, especially around scraping and fraud workflows (Akamai). That does not mean every spike is malicious, but teams need evidence when traffic behavior moves.

BotScope can help by giving product, security, and marketing teams a shared before-and-after view of website bot defense changes, so investigations do not depend on screenshots from four consoles and a Slack thread nobody can find.