apache2-logs Configuration by crowdsecurity

Name: apache2-logs
Author: crowdsecurity

Installation

cscli parsers install crowdsecurity/apache2-logs

Description

This apache2 parser support access and error logs in the HTTPD COMBINED LOG standard format, with the following possible modifications :

An optional IP or FQDN can be present as the first element of line, and will be stored in target_fqdn. This is meant for multi-tenant / aggregated logs.
referrer and user_agent have been made optional for more epurated logging formats

note : If you are aggregating logs from several domains, prefix your logline with the target FQDN. HTTP based scenarios should take this into account so that buckets are per source IP per target FQDN, limiting false positives due to logs multiplexing.

YAML Configuration

1#Apache access/errors logs
2#debug: true
3filter: "evt.Parsed.program startsWith 'apache2'"
4onsuccess: next_stage
5name: crowdsecurity/apache2-logs
6description: "Parse Apache2 access and error logs"
7#log line can be prefixed by a target_fqdn
8nodes:
9  - grok:
10      pattern: '(%{IPORHOST:target_fqdn}(:%{INT:port})? )?%{COMMONAPACHELOG}( "%{NOTDQUOTE:referrer}" "%{NOTDQUOTE:http_user_agent}")?'
11      apply_on: message
12      # these ones apply for both grok patterns
13      statics:
14        - meta: log_type
15          value: http_access-log
16        - target: evt.StrTime
17          expression: evt.Parsed.timestamp
18        - meta: service
19          value: http
20        - meta: source_ip
21          expression: evt.Parsed.clientip
22        - meta: http_status
23          expression: evt.Parsed.response
24        - meta: http_path
25          expression: "evt.Parsed.request != '' ? evt.Parsed.request : evt.Parsed.rawrequest"
26        - meta: http_verb
27          expression: "evt.Parsed.verb"
28        - meta: http_user_agent
29          expression: "evt.Parsed.http_user_agent"
30        - meta: target_fqdn
31          expression: "evt.Parsed.target_fqdn"
32    onsuccess: next_stage
33  - grok:
34      pattern: '%{HTTPD_ERRORLOG}'
35      apply_on: message
36    onsuccess: next_stage
37    pattern_syntax:
38      NOT_DOUBLE_POINT: '[^:]+'
39      NOT_DOUBLE_QUOTE: '[^"]+'    
40    nodes:
41      - filter: "evt.Parsed.module == 'auth_basic'"
42        onsuccess: next_stage
43        pattern_syntax:
44          EXTRACT_USER_AND_PATH: 'user %{NOT_DOUBLE_POINT:username}: authentication failure for "%{NOT_DOUBLE_QUOTE:target_uri}": Password Mismatch'
45          EXTRACT_USER_AND_PATH2: 'user %{NOT_DOUBLE_POINT:username} not found: "?%{NOT_DOUBLE_QUOTE:target_uri}"?'
46        grok:
47          pattern: '%{EXTRACT_USER_AND_PATH}|%{EXTRACT_USER_AND_PATH2}'
48          apply_on: message
49          # these ones apply for both grok patterns
50        statics:
51          - meta: username
52            expression: evt.Parsed.username
53          - meta: http_path
54            expression: evt.Parsed.target_uri
55          - meta: sub_type
56            value: "auth_fail"
57      - filter: "evt.Parsed.module == 'core' && evt.Parsed.message contains 'Invalid URI'"
58        onsuccess: next_stage
59        pattern_syntax:
60          EXTRACT_URIVERB: 'Invalid URI in request %{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})'
61        grok:
62          pattern: '%{EXTRACT_URIVERB}'
63          apply_on: message
64        statics:
65          - meta: http_path
66            expression: evt.Parsed.request
67          - meta: sub_type
68            value: "invalid_uri"
69      - filter: "evt.Parsed.module == 'authz_core' && evt.Parsed.message contains 'client denied'"
70        onsuccess: next_stage
71        pattern_syntax:
72          EXTRACT_PATH: 'client denied by server configuration: %{GREEDYDATA:target_uri}'
73        grok:
74          pattern: '%{EXTRACT_PATH}'
75          apply_on: message
76        statics:
77          - meta: http_path
78            expression: evt.Parsed.target_uri
79          - meta: sub_type
80            value: "permission_denied"
81      - filter: "evt.Parsed.module == 'ssl' && evt.Parsed.message contains 'SSL handshake stopped'"
82        statics:
83          - meta: sub_type
84            value: "ssl_handshake_stopped"
85            
86    statics:
87      - meta: log_type
88        value: http_error-log
89      - target: evt.StrTime
90        expression: evt.Parsed.timestamp
91      - meta: service
92        value: http
93      - meta: source_ip
94        expression: evt.Parsed.client
95      - meta: http_status
96        expression: evt.Parsed.response
97
98      
99

Hub configuration

« apache2-logs »
v1.5 by crowdsecurity
Attack scenario

Hub configuration

« apache2-logs »v1.5 by crowdsecurityAttack scenario

« apache2-logs »
v1.5 by crowdsecurity
Attack scenario