1 post with tag #web-scraping

Blog post cover image

Blocking AI Crawlers on Istio: AuthorizationPolicy's Ceiling and Envoy's Full-String Regex Trap

Blocking ~160 known AI-crawler user agents at an Istio ingress gateway without Lua: why AuthorizationPolicy cannot do substring header matching, and why a single safe_regex alternation would silently never match anyway.

Blocking AI Crawlers on Istio: AuthorizationPolicy's Ceiling and Envoy's Full-String Regex Trap