Install with a top-level collection
6 collections include itsudo cscli collections install crowdsecurity/openbsdInstalls 6 parsers and 6 scenarios, via Web Crawlers - Whitelist.
View OpenBSD →Installation
sudo cscli postoverflows install crowdsecurity/seo-bots-whitelistThen reload CrowdSec: sudo systemctl reload crowdsec
Sign in to see which of your engines run it →Configuration
Hover a line or a note to see what each field does.
name: crowdsecurity/seo-bots-whitelistdescription: "Whitelist good search engine crawlers"whitelist:reason: "good bots (search engine crawlers)"expression:- "any(File('rdns_seo_bots.txt'), { len(#) > 0 && evt.Enriched.reverse_dns endsWith #})"- "RegexpInFile(evt.Enriched.reverse_dns, 'rdns_seo_bots.regex')"- "any(File('ip_seo_bots.txt'), { len(#) > 0 && IpInRange(evt.Overflow.Alert.Source.IP ,#)})"data:- source_url: https://hub-data.crowdsec.net/whitelists/benign_bots/search_engine_crawlers/rdns_seo_bots.txtdest_file: rdns_seo_bots.txttype: string- source_url: https://hub-data.crowdsec.net/whitelists/benign_bots/search_engine_crawlers/rnds_seo_bots.regexdest_file: rdns_seo_bots.regextype: regexp- source_url: https://hub-data.crowdsec.net/whitelists/benign_bots/search_engine_crawlers/ip_seo_bots.txtdest_file: ip_seo_bots.txttype: string
Installed on
Know where it runs
Sign in to see the Security Engines that run this postoverflow, the version they run and whether it was modified locally.
Documentation
SEO Bots Whitelist
Configuration based on crowdsecurity/rdns to whitelist following:
Search Engine:
- duckduckBot
- googlebot
- yandex
- bing
- baidu
- yahoo
- qwant
- sogou
- yandex
SEO bots:
- babar
- Amazon
Social: