1 post with tag #web-scraping

Blocking AI Crawlers on Istio: AuthorizationPolicy's Ceiling and Envoy's Full-String Regex Trap
Blocking ~160 known AI-crawler user agents at an Istio ingress gateway without Lua: why AuthorizationPolicy cannot do substring header matching, and why a single safe_regex alternation would silently never match anyway.
Blocking AI Crawlers on Istio: AuthorizationPolicy's Ceiling and Envoy's Full-String Regex Trap