Using Logstash to Parse IPtables Firewall Logs

Published: 2021-02-13
Last Updated: 2021-02-13 21:33:44 UTC
by Guy Bruneau (Version: 1)
0 comment(s)

One of our reader submitted some DSL Modem Firewall logs (iptables format) and I wrote a simple logstash parser to analyze and illustrate the activity, in this case it is all scanning activity against this modem. An iptables parser exist for Filebeat[2], but for this example, I wanted to show how to create a simple logstash parser using Grok[3] to parse these logs and send them to Elastic.

The Logstash Configuration Parser

# Guy Bruneau, guybruneau@outlook.com
# Date: 13 Feb 2021
# Version: 0.1
#
# Parse to Elastic Common Schema (ECS) format
# https://www.elastic.co/guide/en/ecs/1.7/ecs-field-reference.html
#
# This custom parser is parsing iptables type firewall logs dump data

# Wed Feb 10 23:59:29 2021 kern.debug kernel: [288253.168004] Firewall WAN DROP (SRC): IN= OUT= MAC=20:b0:01:6d:51:c4:a0:f3:e4:a3:1d:ba:08:00 SRC=xxx.xxx.xxx.xxx DST=xx.xxx.xx.xx LEN=40 TOS=0x00 PREC=0x00 TTL=245 ID=6607 PROTO=TCP SPT=55035 DPT=6981 WINDOW=1024 RES=0x00 SYN URGP=0 MARK=0x8000000

# Grab the logs file from filebeat which match the pattern /home/guy/logs

filter {
  if [log][file][path] =~ "logs" {
    grok {
      match => { "message" => "^%{HTTPDERROR_DATE:timestamp}.*?%{LOGLEVEL:loglevel}\s+%{WORD:event.provider}:.*Firewall\s+%{WORD:interface.name}\s+%{WORD:event.action}\s+\(%{WORD:network.direction}\).*SRC=%{IP:source.ip}\s+DST=%{IP:destination.ip}\s+.*ID=%{WORD:event.id}\s+PROTO=%{WORD:network.transport}\s+SPT=%{INT:source.port}\s+DPT=%{INT:destionation.port}.*" }
    }
  }
}

# Format: Thu Feb 11 08:46:32 2021
# https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html

filter {
  date {
    match => ["timestamp", "EEE MMM dd HH:mm:ss yyyy"]
    target => "@timestamp"
  }
}

# Add GeoIP information to destination hostname

filter {
  if [log][file][path] =~ "logs" {
    geoip { source => "source.ip" }
  }
}

Now that the data is in Elasticsearch, I can create various reports based on the activity logged by iptables. The dashboards I prepared shows the number of records, the Top 10 targeted ports (services) and a table with the Top 10 sources with its location (GeoIP), action taken by the firewall and a total.

It is always a good idea to monitor and review the activity logged by the modem. If you aren't sure what to look for, we have tips published here as well as how you can participate and submit your logs to DShield, the service is open and free.

[1] https://www.elastic.co/downloads/logstash
[2] https://www.elastic.co/guide/en/beats/filebeat/7.11/filebeat-module-iptables.html
[3] https://grokdebug.herokuapp.com/
[4] https://isc.sans.edu/howto.html
[5] https://isc.sans.edu/forums/diary/Secure+Communication+using+TLS+in+Elasticsearch/26902/

-----------
Guy Bruneau IPSS Inc.
My Handler Page
Twitter: GuyBruneau
gbruneau at isc dot sans dot edu

0 comment(s)
vSphere Replication updates address a command injection vulnerability (CVE-2021-21976) - https://www.vmware.com/security/advisories/VMSA-2021-0001.html

Comments

What's this all about ..?
password reveal .
<a hreaf="https://technolytical.com/">the social network</a> is described as follows because they respect your privacy and keep your data secure:

<a hreaf="https://technolytical.com/">the social network</a> is described as follows because they respect your privacy and keep your data secure. The social networks are not interested in collecting data about you. They don't care about what you're doing, or what you like. They don't want to know who you talk to, or where you go.

<a hreaf="https://technolytical.com/">the social network</a> is not interested in collecting data about you. They don't care about what you're doing, or what you like. They don't want to know who you talk to, or where you go. The social networks only collect the minimum amount of information required for the service that they provide. Your personal information is kept private, and is never shared with other companies without your permission
https://thehomestore.com.pk/
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> public bathroom near me</a>
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> nearest public toilet to me</a>
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> public bathroom near me</a>
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> public bathroom near me</a>
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> nearest public toilet to me</a>
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> public bathroom near me</a>
https://defineprogramming.com/
https://defineprogramming.com/
Enter comment here... a fake TeamViewer page, and that page led to a different type of malware. This week's infection involved a downloaded JavaScript (.js) file that led to Microsoft Installer packages (.msi files) containing other script that used free or open source programs.
distribute malware. Even if the URL listed on the ad shows a legitimate website, subsequent ad traffic can easily lead to a fake page. Different types of malware are distributed in this manner. I've seen IcedID (Bokbot), Gozi/ISFB, and various information stealers distributed through fake software websites that were provided through Google ad traffic. I submitted malicious files from this example to VirusTotal and found a low rate of detection, with some files not showing as malware at all. Additionally, domains associated with this infection frequently change. That might make it hard to detect.
https://clickercounter.org/
Enter corthrthmment here...

Diary Archives