cscli parsers install Zaulao/aws-albA parser for AWS Application Load Balancer (ALB) access logs. Extracts HTTP request fields from the raw message, which are used for further enrichment and processing. ALB log format is well defined here.
Application Load Balancers access logs are stored in S3 buckets, as indicated in the official documentation. Therefore, the collection of these logs must start from this data source initially.
The user can collect the logs using the AWS S3 data source, available from version 1.5 of Crowdsec, or use alternatives, such as, for example, other log collection tools that allow saving in a file or any other destinations (in this case, the data source must be specified according to the chosen alternative).
The use of this parser follows the default configuration, that is, it is performed from the acquis.yaml file. Therefore, after configuring the correct data source of the logs, its type must be specified as indicated below:
1labels:2 type: aws-alb
The information extracted from the raw log is:
time: The time when the load balancer generated a response to the client;remote_addr: The IP address of the requesting client;elb_status_code: The status code of the response from the load balancer;host: The host name from the request;port: The port of the request;request: The request line URI (i.e. the path) from the client;verb: The request HTTP method from the client;http_user_agent: A User-Agent string that identifies the client that originated the request.This information is exported for use in common threat detection scenarios from HTTP requests.
1onsuccess: next_stage2filter: "evt.Line.Labels.type == 'aws-alb'"3name: crowdsecurity/aws-alb4description: "Parse AWS ALB access logs"5grok:6 pattern: '%{DATA:type}\s+%{TIMESTAMP_ISO8601:time}\s+%{DATA:elb}\s+%{IP:remote_addr}:%{BASE10NUM:remote_port}\s+(-|(%{IP:target_addr}:%{BASE10NUM:target_port}))\s+%{BASE10NUM:request_processing_time}\s+%{DATA:target_processing_time}\s+%{BASE10NUM:response_processing_time}\s+%{BASE10NUM:elb_status_code}\s+%{DATA:target_status_code}\s+%{BASE10NUM:received_bytes}\s+%{BASE10NUM:sent_bytes}\s+\"%{WORD:http_method}\s+%{DATA:protocol}://%{DATA:host}:%{NUMBER:port}%{DATA:request}\s+HTTP/%{NUMBER:http_version}\"\s+\"%{DATA:http_user_agent}\"\s+%{DATA:ssl_cipher}\s+%{DATA:ssl_protocol}\s+%{DATA:target_group_arn}\s+\"%{DATA:trace_id}\"\s+\"%{DATA:domain_name}\"\s+\"%{DATA:chosen_cert_arn}\"\s+%{DATA:matched_rule_priority}\s+%{TIMESTAMP_ISO8601:request_creation_time}\s+\"%{DATA:actions_executed}\"\s+\"%{DATA:redirect_url}\"\s+\"%{DATA:error_reason}\"\s+\"%{DATA:target_list}\"\s+\"%{DATA:target_status_code_list}\"\s+\"%{DATA:classification}\"\s+\"%{DATA:classification_reason}\"(\s+%{DATA:conn_trace_id}\s+\"%{DATA:transformed_host}\"\s+\"%{DATA:transformed_uri}\"\s+\"%{DATA:request_transform_status}\")?'7 apply_on: Line.Raw8 statics:9 - meta: log_type10 value: alb_access_log11statics:12 - meta: service13 value: http14 - meta: log_type15 value: http_access-log16 - target: evt.StrTime17 expression: evt.Parsed.request_creation_time18 - meta: source_ip19 expression: "evt.Parsed.remote_addr"20 - meta: http_status21 expression: "evt.Parsed.elb_status_code"22 - meta: http_host23 expression: "evt.Parsed.host"24 - meta: http_port25 expression: "evt.Parsed.port"26 - meta: http_path27 expression: "evt.Parsed.request"28 - meta: http_verb29 expression: "evt.Parsed.http_method"30 - meta: http_user_agent31 expression: "evt.Parsed.http_user_agent"