How do you know when someone probing your network?

How do you know if someone is probing your network?  Logs are a great place to start, especially your FW-1 logs.  Unfortunately, those logs can be difficult and time consuming to sift through (especially if you are logging 400+ B every day). Wouldn’t it be great if there was an automated method of alerting you?

There are a variety of different probes and attacks black-hats will attempt. he types we will be focusing on is port scans. Port scans are where an individual attempt to connect to a variety of ports to identify what services a system is running.  The scans can be used on a specific target or used to scan entire IP ranges, often chosen at random.  This is one of the most popular information gathering methods used by black-hats today as it identifies what ports and services are open.

The goal is to receive an email alert whenever someone is attempting to scan or probe your network.  We want to automate the entire process, saving you time and energy.  We also want to archive this information in a database for future use.  Our definition of an email alert would look as follows:

Date: Sat, 02 Dec 2019 15:40:01 -0600 (CST)
From: ids@example.net
To: fwadmin@example.net
Subject: #### Firewall ALERT ####

You have received this message because someone is potentially scanning your systems.  The information below is the packet that was denied and logged by the Firewall. This is email alert number 3, with a limit of 5 from evil.example.org.

—– CRITICAL INFORMATION —–

Date:        02Dec2019
Time:        15:39:59
Source:      evil.example.org
Destination: firewall
Service:     domain-tcp

—– ACTUAL LOG ENTRY —–

02Dec2019 15:39:59 drop   fw1    >elx0 mail proto tcp src
evil.example.org dst firewall service domain-tcp s_port
37401 len 44 rule 6

As you see, this email is alerting the fwadmin that someone has attempted to conduct an unauthorized zone transfer from the Firewall (which is also a Name Server for the domain).  However, the firewall denied the attempted, logged it, AND alerted the fwadmin.  In the email alert, we have parsed the critical information for easy reading, and also included the actual log entry for more data.  You can also create your own alerts, such as text pages or snmp traps.

To detect and monitor these scans, we will build a script that first logs whenever someone connects to a predetermined port and then emails us an alert.  To do this, we will be using the User Defined Alert.  The User Defined Alert will be our script, alert.sh.  Whenever the User Defined Alert is activated, it will launch our script,  which parses the data and does all the work for us.  We set up our Firewall rule base as follows…  First, we identify the most commonly scanned ports we want to monitor. Then we select two to three systems to listen on these ports.  When an intruder scans our network, the will most likely hit our systems listening on these ports. When these ports are scanned, the Firewall logs the attempts then executes the User Defined Alert…

The end result is you receive an email for each port scanned. If you have 3 systems, each listening on 4 ports, then you may get up to 12 emails from a single network port scan. However, this is normally not the case. If hackers are scanning an entire network, they are normally looking for a single vulnerability, such as imap (port 143). In this case, we would have received only three emails, one from each system. When they scan a single target, often they scan a range of ports, such as 1-1024. In that case, we would have received only 4 emails, one for each port on the system. Based on what emails you get, you can quickly determine what the intruder is interested in. Also, the script alert.sh is configurable to limit the total amount of emails as a result of a scan.  All scans are also logged to a database.

To implement this methodology, we first identify two to three systems to use for monitoring. Usually, select DNS servers and the Firewall as these are primary targets, many scanning tools start by scanning these systems. Once you have identified the systems you will monitor, create a workstation object in your Firewall 1 GUI for each of the systems.  The systems do not necessarily have to actually exist.  Create an object for a non-existing system, give it a valid IP that you are not using, and use this as one of your scannable objects.  Once you have created your workstation objects, then select the most commonly scanned ports. Ensure that your systems are not using these ports, or every time someone connects to it, you will be alerted. To identify commonly scanned ports, CERT alerts are a great place to start, you can find these alerts at http://www.cert.org. Some ports that you can use are:

imap (port 143)
SMB (port 139)
login (port 513)
http (port 80)
NetBus (12345)

These ports are commonly hackers look for them, but most of your systems will not be using them. Make sure these ports are not already blocked by a screening router. We will then set several systems to listen on these ports, alerting us when there is a connection.  Below is an example of a rule that monitors and detects the scans.  Notice that quite a few ports are being monitored for these systems.

As you can see from the image above, this system is monitoring quite a few ports.  Also, this rule is our User Defined Alert rule.  Whenever this rule receives a connection on one of these ports, the User Defined Alert will be activated.  Our User Defined Alert (alert.sh) will now parse the data for these connections, notify us via email, and archive the information.

That’s it, you are done.  All you have to do is download the script and run the installation script.  If you want to learn how the script works or modify it, read below.

The whole alerting process works as follows.  First, we define the User Defined Alert (shown in the image above) as our alert.sh script. The User Defined Alert produces a log entry for every connection that it monitors.  The log entry looks as follows:

02Dec2019 15:39:59 drop   fw1    >elx0 mail proto tcp src
evil.example.org dst firewall service domain-tcp s_port
37401 len 44 rule 6

The Firewall then pipes this log entry into our alert.sh script.  The script then parses the data and executes various functions we define.  The script alert.sh uses the following files to store and track the logs.

alert.uniq – A single, permanent log is added to this file whenever a system scans you.  Regardless of how many ports are scanned, there will only be one entry per unique source.  This way you can easily track who scanned you when.

alert.log – This file temporarily stores every port scanned for that day.  This is how the alert.sh script tracks how many times/ports have been scanned from a single source.  Every time the User Defined Alert executes alert.sh, the first thing the script does is count how many times the source ($src) has scanned us for that day by looking in alert.log.  When the script alert.sh is done, it adds an entry to the alert.log file.  So, the first time a source scans a port, alert.sh will not find anything logged in the alert.log.  The second time the sources scan a port, it will find one, and so on.  You set a limit on how many ports are scanned before the alert.sh stops reacting (this protects against DOS attacks).  At the end of the day, a cronjob (rotate.sh) will clear this file out and add all the entries to archive.log.  This way if the same source scans you again at a later date, alert.sh will log the attempt.

alert.archive – This file permanently archives every port scanned and logged.  This is an archival database that can be used for research.  This file stores the actual log entries, so it tends to be difficult to read.  I find the file alert.uniq to be much easier to read.

rotate.sh – This is a cronjob you need to run every day.  It clears the file alert.log, saving all the logged entries to archive.log.  This resets the counter for the alert.sh script.  If the same source scans you two days in a row, this allows you to log it.

Example of alert.sh as below :

#!/bin/ksh
#
# alert.sh: ver 1.4.2
# Lance Spitzner lance@spitzner.net
#
# Parse, log, and react to FW-1 User Defined Alerts
# For more information, please review the README
#

#################################################################
#               BEGIN CUSTOMIZING SCRIPT HERE                   #
#################################################################
# INSTALL DIRECTORY
# Define the directory that this script is in.
# Do NOT put a slash at the end.
# EXAMPLE: dir=/home/fwadmin/alert_1.4.2
dir=
# FW ADMIN
# Define the name of who gets the email alerts
# EXAMPLE: user=fwadmin@example.com
user=
# SCAN LIMIT
# Define maixmum number of scans/email alerts
limit=5
# EMAIL REMOTE SYSTEM
# Define as “true” if you want to automatically email
# the remote admin when you reach your scan limit.
email=false
# SAM
# Define as “true” if you want to autotmatically block
# the source if you reach your scan limit.
sam=false
# SAM TIMEOUT
# How long do you want the source blocked
# Default is 3600 seconds (1 hour).
timeout=3600
#################################################################
#               FINISH CUSTOMIZING SCRIPT HERE                  #
#################################################################
#################################################################
#                   DEFINE SYSTEM VARIABLES HERE                #
#################################################################
### Script variables
message=/tmp/.message_$$
send=/tmp/.send_$$
MAIL=mail
### Good code is secure code
umask=177
PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin
export PATH
if [ -f $message ]; then
rm $message
fi
if [ -f $send ]; then
rm $send
fi
### Set trap in case of abrupt exit
trap “rm $send $message ; exit 5” 1 2 15
### Grab User Defined Alert log, pipe to $message.
cat – | tail -1 > $message
### Determine number of scans.
ip=awk '{print $10}' $message
number=grep -c $ip $dir/alert.log
scan=expr $number + 1
### Check number of scans.  If we have reached our limit, lets bail
### now and save CPU cycles.
if [ $scan -gt $limit ];then
cat $message >> $dir/alert.log
rm $message
exit 10
fi
### Parse log file
date=awk '{print $1}' $message
time=awk '{print $2}' $message
dst=awk '{print $12}' $message
### Determine service (check some variables first)
#Determine if “Valid Address” is in log files for NAT
nat_check=grep -c "(Valid Address)" $message
#Determine if protocol is icmp
icmp_check=grep -c " icmp " $message
if [ “$nat_check” -eq 0 ];then
if [ “$icmp_check” -eq 0 ];then
service=awk '{print $14}' $message
else
service=awk '{print $15,$16,$17,$18}' $message
fi
else
if [ “$icmp_check” -eq 0 ];then
service=awk '{print $16}' $message
else
service=awk '{print $17,$18,$19,$20}' $message
fi
fi
#################################################################
#                           FUNCTIONS                           #
#                                                                           #
#   Build your own modules here, such as paging or snmp_trap    #
#   alerts, then add them to Phase 3 in the script.             #
#################################################################
## This function checks to see if the source is an IP or hostname.
## If the source is an IP, it is resolved to a domain name.

### This function determines who the admin is of the remote system
### and emails them about the scan.  Works only for .com, .edu, .net
### .mil, and .org.

TrackDown () {

# Define variables
dom=echo $src | awk -F"." '{print $NF }'
full_dom=echo $src | awk -F"." '{print $(NF-1) }'.$dom

# Start script, 1st check if it is a gTLD, if so, execute the following.
if [ $dom = net ] || [ $dom = com ] || [ $dom = edu ] || [ $dom = org ] || [ $dom = mil ] || [ $dom = gov ]; then
admin=whois $full_dom | grep "@" | awk '{print $NF}' | head -1
tech=whois $full_dom | grep "@" | awk '{print $NF}' | head -2 | tail -1

$MAIL $admin,$tech,$user < Subject: Your system $src may be scanning the Internet.

I logged your system $src scanning my network. It looks like they are scanning for the $service vulnerability. I recommend you research this, as $src may be scanning other networks as well.  Please respond to this message informing me what you have found.  If you require assistance from me, I would be more then happy to provide it.
Below is an example one of the multiple connections I received from $src.
— Logged Information —

Source:   $src
Date:     $date
Time:     $time
Service:  $service
— snip snip —

This notification has been sent to
$admin
$tech
abuse@$full_dom
Please respond to $user for any issues concerning this.
If you received this message in error, I apologize.
Thank you
EOF
fi
}

## This function blocks the source IP scanning/probing our
## network.  Edit ‘fw sam’ command to your taste.  For more
## info, typte ‘fw sam’.

#################################################################
#                          THE SCRIPT                           #
#################################################################
### Resolve the IP address of $src if it is still in IP format.
### You may want to disable this to improve performance.
Resolve
### Build Email Alert
### This builds our email alert.  This happens for
### every alert, until we exceed our $limit.  If we exceed
### our limit, the script never gets to this point, because
### it will have already exited (see around line 95).
cat < $send
You have received this message because someone is potentially
scanning your systems.  The information below is the packet
that was denied and logged by the Firewall. This is email alert
number $scan, with a limit of $limit from $src.

—– CRITICAL INFORMATION —–

Date:        $date
Time:        $time
Source:      $src
Destination: $dst
Service:     $service

—– ACTUAL FW-1 LOG ENTRY —–

cat $message
EOF
### PHASES START:
### What to do in addition to email alerts, depending on
### the number of scans.

# ##### PHASE 1 #####
# First unauthorized connection from the remote system.

if [ $scan -eq 1 ]; then

## Add this system to our scan database file alert.uniq
echo “$src      $date   $time   $service” >> $dir/alert.uniq

# ##### PHASE 2 #####
# Second to $limit connections from the remote system

elif [ $scan -gt 1 ] && [ $scan -lt $limit ]; then

## Add anything you would like for this.
:
# ##### PHASE 3 #####
# We are pretty sure this is a port scan or probe, since the
# same source has connected to us $limit number of times.
else
### Track down and email the admin of the remote system
if [ $email = true ]; then
TrackDown
fi
### Block source
if [ $sam = true ]; then
Block
fi
echo ” This is alert number $scan, you have reached your” >> $send
echo “maximum threshold. You will not receive anymore alerts”. >> $send
fi

### Send email alert and save log to alert.log
$MAIL -s “#### SCAN ALERT ####” $user < $send
cat $message >> $dir/alert.log
## All done, lets clean up after ourselves, just like Mom taught us 🙂
rm $message
rm $send
exit 0