Snort Rules Update

General questions.
IcyFire
Posts: 23
Joined: January 4th, 2016, 3:43 am

Re: Snort Rules Update

Post by IcyFire » January 2nd, 2018, 11:14 pm

dnl wrote:
January 1st, 2018, 10:33 am
...Is very disappointing that this basic requirement of the IDS won't be accepted for as a feature request. Especially after the community has shown much more interest in it than captive portal or RAID proposals. The evidence suggests that the core developers and the customers of Lightning Wire Labs don't actually use the IDS feature in IPFire.
Indeed. Back in 2016 I posted here the response from IPFire's Michael Treme:
...this does not automatically update at the moment. We have no plans to add this functionality at the moment.

The reason for that is that a custom selection of the rules is hard to preserve after the update.

Your Lightning Wire Labs Team,
Best regards,
-Michael Treme
So how do we get this as a package? A developed and managed addon? Is there a way to fund this? Or do we wait for the Suricata integration which is planed for IPFire-3.x? Hopefully Suricata will have an automatic update feature!
Image

dnl
Posts: 375
Joined: June 28th, 2013, 11:03 am

Re: Snort Rules Update

Post by dnl » January 7th, 2018, 3:11 am

IcyFire wrote:
January 2nd, 2018, 11:14 pm
So how do we get this as a package? A developed and managed addon? Is there a way to fund this? Or do we wait for the Suricata integration which is planed for IPFire-3.x? Hopefully Suricata will have an automatic update feature!
Normally the way to fund something, if like me you can't write it yourself, is to ask to have it on the wish list for sponsorship. However for no clear reason I've seen Michael has stubbornly refused this.

I hate to be negative, but as I understand it IPFire 3 has been in development for many years. (This project is going well but could really do some additional enthusiastic developers) So I wouldn't hold out for Suricata any time soon.

You'll need to get some basic update working with a shell script, for example try a few in this thread, or you'll have to do updates manually from the GUI for some time to come.
IPFire 2.x (Latest Update) on x86_64 Intel Bay Trail CPU, 4GiB RAM, RED + GREEN + BLUE + ORANGE

Edwin
Posts: 104
Joined: March 19th, 2016, 12:02 pm

Re: Snort Rules Update

Post by Edwin » January 7th, 2018, 12:35 pm

Yeah, would be nice if this was part of IPFire.
I use the scrips made by Kick@ss and H&M, posted by H&M in this thread on may 27th 2017.
I am happy to have daily snort updates ever since.
Image
Image

gitarman94
Posts: 7
Joined: March 17th, 2018, 1:37 pm
Location: US

Re: Snort Rules Update

Post by gitarman94 » March 31st, 2018, 1:34 am

Hey guys,

Like many of you, I’ve been on the search for an auto-updater for Snort ever since I joined the IPFire community, only to find that there really isn’t one. Was happy to find this thread and the scripts posted by kick@ss and later modified by H&M, but comments from dnl and others about feature enhancements made me think... why not?

So... I wrote an updated version that checks the inter-tubes and updates only if an update is available. It updates both Emerging threats and registered user rules, though (obviously), in order to update the registered user rules you need to have an oinkcode configured.

The script also does a couple other cool things besides this core update feature. First, is that you don’t need to modify the script in order to add the oinkcode, the script will automatically get the one you saved into the system (in the IDS page). If you don’t have an oinkcode configured it won’t attempt to check for those updates. Second thing it does, is checks which of your monitored network interfaces you have snort scanning on (red, blue, green, orange), and checks to see if those enabled services are actually running, and if any of your selected services are stopped, it’ll automatically re-start the service(s).

This basically means that we can put this update script in the hourly cron folder and it’ll do a quick internet check and if no update, moves on and does a super fast PID check, and if everything is updated and running, that’s it, it just exits. This prevents the restarting of services or downloading of unnecessary files or turning the services off to update (except when needed). It also ensures that the IDS services are always active, cause that’s kind of the whole point.

To install: All three files need to be copied over to the IPFire system and kept together in the same directory (the ‘/var/ipfire/snort’ directory is best). The ‘installer.sh’ file is what you’ll need to execute via ssh or terminal (it’s what moves the files into place and configures permissions, etc.)... And that’s pretty much it!

You can change the install.sh file so that it doesn’t run the script every hour (maybe you’d prefer daily?), simply comment/uncomment the appropriate lines. I’ve also heavily commented the primary ‘snortupdate.pl’ script file, so if you need to change something for it to fit your needs, it’s hopefully somewhat clear what I’m doing. Anyways, this was my first major foyer into the PERL realm (I typically program in C#, groovy, and others), so additions, enhancements, code consolidation suggestions, etc. are all welcome.

install.sh

Code: Select all

#!/bin/bash
#Install Script
echo "Starting install...\n"
echo "Copying files...\n"
cp snortupdate.pl /var/ipfire/snort/snortupdate.pl
cp update.sh /var/ipfire/snort/update.sh
echo "Files copied!"
echo "Creating hourly cron job..."

#change the interval here
#
#Daily
#ln -s /var/ipfire/snort/update.sh /etc/fcron.daily/snortupdate.sh
#
#Weekly
#ln -s /var/ipfire/snort/update.sh /etc/fcron.weekly/snortupdate.sh
#
#Monthly
#ln -s /var/ipfire/snort/update.sh /etc/fcron.monthly/snortupdate.sh
#
#Hourly
ln -s /var/ipfire/snort/update.sh /etc/fcron.hourly/snortupdate.sh

echo "Updating execution rights..."
chmod +x /var/ipfire/snort/snortupdate.pl
chmod +x /var/ipfire/snort/update.sh
chmod +x /etc/fcron.daily/snortupdate.sh
echo "Installation finished"
update.sh

Code: Select all

#!/bin/bash
logger -t SnortUpdate "SnortUpdate start..."
perl /var/ipfire/snort/snortupdate.pl
logger -t SnortUpdate "Process completed."
snortupdate.pl

Code: Select all

#!/usr/bin/perl
#################################
# Snort Rules Update for IPFire #
# ----- Contributions by: ----- #
# --------- Kick@ss ----------- #
# ----------- H&M ------------- #
# -------- gitarman94 --------- #
#################################

use autodie ':io';
use LWP::Simple;
use strict;
use warnings;
use HTTP::Request;
use LWP::UserAgent;


my $oinkcode='none';
my $Snort_Orange = 0;
my $Snort_BLUE = 0;
my $Snort_GREEN = 0;
my $Snort_Active = 0;
my $url = '';
getSnortSettings();
my $currentLocalSnortVersion = readFile("/var/tmp/SnortVersion.txt"); #itterate through local file to get currently installed Snort version
my $currentLocalETVersion = readFile("/var/tmp/ETVersion.txt"); #itterate through local file to get currently installed ET version
my $snortList = webGet('https://snort.org/downloads/registered/md5s.txt');
my $ETList = webGet('https://rules.emergingthreats.net/version.txt');
my $currentWebSnortVersion = webSnortParse();#iteration through list and sets this var to current sub-point release number from websit
my $currentWebETVersion = webETParse();
versionComparer();
verifySnortIsRunning();

sub getSnortSettings
{
  open my $fh, '<', "/var/ipfire/snort/settings"; #open file
  while (my $line = <$fh>) #itterate throught the file line by line
  {
      if ($line =~ m/ENABLE/ | $line =~ m/OINKCODE/) #only cares about lines that have the word 'ENABLE' in them
      {
          if ($line =~ m/ORANGE/) #looking for orange settings
          {
              my @lineItems = split('[=]',$line);
              if ($lineItems[1] =~ m/on/) #if the setting is "on"
              {
                  $Snort_Orange = 1; #default is 0 but if orange is enabled then make it so
              }
          }
          elsif ($line =~ m/BLUE/) #looking for blue settings
          {
              my @lineItems = split('[=]',$line);
              if ($lineItems[1] =~ m/on/) #if the setting is "on"
              {
                  $Snort_BLUE = 1; #default is 0 but if blue is enabled then make it so
              }
          }
          elsif ($line =~ m/GREEN/) #looking for green settings
          {
              my @lineItems = split('[=]',$line);
              if ($lineItems[1] =~ m/on/) #if the setting is "on"
              {
                  $Snort_GREEN = 1; #default is 0 but if green is enabled then make it so
              }
          }
          elsif ($line =~ m/OINKCODE/) #get oinkcode from within the settings file automatically
          {
              my @lineItems = split('[=]',$line);
              if (length($lineItems[1]) == 41) #get the length and match to the 41 character string it should be, if not proper length, then it's not a real oinkcode
              {
                  $oinkcode = $lineItems[1];
              }
              elsif ((length($lineItems[1]) > 2 && length($lineItems[1]) < 41) || length($lineItems[1]) > 41)
              {
                system("logger -t SnortUpdate 'WARNING: OINKCODE appears to be of incorrect length.'");
              }
          }
          else
          {
              my @lineItems = split('[=]',$line);
              if ($lineItems[1] =~ m/on/) #if the setting is "on"
              {
                  $Snort_Active = 1; #default is 0 but if red is enabled then make it so
              }
          }
      }
  }
  close $fh;
}

sub webGet
{ #used to http query snort for current version of version 2
    my $url = shift;
    my $ua = LWP::UserAgent->new( ssl_opts => { verify_hostname => 0 }, );
    my $request = HTTP::Request->new(GET => $url);
    my $response = $ua->request($request);
    if ($response->is_success)
    {
        return $response->content; #returns list found on above site
    }
}

sub webETParse
{
    chomp $ETList; #remove extra line/spaces
    return $ETList;
}

sub webSnortParse
{  #Itterator...
    my @array = split('[-]', $snortList); #splits array by '-'
    $currentWebSnortVersion = 0;
    foreach my $blob (@array)
    {
        if (index($blob, '.tar') != -1) #determines if array element has a '.tar', if not, we don't care about it
        {
            my ($number) = split('[.]', $blob); #further splits based on '.' to we can remove tar.gz
            if ( $number !~ /^30/) #skips any v3 entries in the list
            {
                my ($junk1, $junk2, $subVersion) = split(//, $number, 3); #splits the version number into 3, keeping only the non-29 (or 2.9) part
                if ($subVersion > $currentWebSnortVersion) { $currentWebSnortVersion = $subVersion; } #stores currentWebSnortVersion as only the highest number listed
            }
        }
    }
    return $currentWebSnortVersion;
}

sub readFile
{
    my $localFile = shift; #we're passing a variable into the subroutine, this sets it to a local variable
    my $currentLocalVersion = 0; #default value, in case we don't find the file
    if (-e $localFile)
    {
        open my $fh, '<', $localFile; #open file
        while (my $row = <$fh>) #itterate throught the file line by line (if multiple)
        {
            chomp $row; #removes empty lines and takes only good data
            $currentLocalVersion = $row; #should only be 1 line in the file so we'll take whatever it gives us
        }
        close $fh;
    }
    return $currentLocalVersion;
}


sub versionComparer
{ #now we test that above WebVersions against our local versions, if newer we'll update

      # VRT Community
          #$url=" https://www.snort.org/rules/community";
      #EmergingThreats Community
          #$url="http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz";
      # VRT Subscripted & VRT Community
          #$url="https://www.snort.org/rules/snortrules-snapshot-29111.tar.gz?oinkcode=$oinkcode\n https://www.snort.org/rules/community";
      # VRT Subscripted & EmergingThreats Community
          #$url="https://www.snort.org/rules/snortrules-snapshot-29111.tar.gz?oinkcode=$oinkcode\n http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz";

      my $performUpdate = 1;

      #We're testing for " !~ m/none/ " in the oinkcode section, as if we don't have a valid code, no point in running them
      if ($currentWebSnortVersion > $currentLocalSnortVersion && $currentWebETVersion > $currentLocalETVersion && $oinkcode !~ m/none/)
      { #updates both snort registered and EmergingThreats
          $url = "https://www.snort.org/rules/snortrules-snapshot-29111.tar.gz?oinkcode=$oinkcode\n http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz";
      }
      elsif ($currentWebSnortVersion > $currentLocalSnortVersion && $currentWebETVersion <= $currentLocalETVersion && $oinkcode !~ m/none/)
      { #updates only snort registered
          $url = "https://www.snort.org/rules/snortrules-snapshot-29111.tar.gz?oinkcode=$oinkcode";
      }
      elsif ($currentWebSnortVersion <= $currentLocalSnortVersion && $currentWebETVersion > $currentLocalETVersion)
      { #updates only emergingthreats
          $url = "http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz";
      }
      else
      { #no updates are performed as there were no newer versions available
        $performUpdate = 0;
      }

      if ($performUpdate == 1 && runUpdate() == 1)
      { #only runs if there are updates available and the updater itself ran without error
          writeFile(); #write the latest version to local files so we now know we have the latest
          #performing these system tasks from here, as we don't want to restart the snort service if we don't have to
          system("logger -t SnortUpdate 'SnortUpdate finished.'");
          system("logger -t SnortUpdate 'Changing ownership for rule files...'");
          system("chown -R nobody:nobody /etc/snort/rules");
          system("logger -t SnortUpdate 'Ownership permissions updated.'");
          system("logger -t SnortUpdate 'Restarting Snort...'");
          system("/usr/local/bin/snortctrl restart");
      }
      else
      {
        print "No updates found.\n";
      }
}

sub writeFile
{
  open my $fh, '>', "/var/tmp/SnortVersion.txt"; #open file for writing
  print $fh "$currentWebSnortVersion"; #actually write to file
  close $fh; #close file

  open my $fh2, '>', "/var/tmp/ETVersion.txt"; #open file for writing
  print $fh2 "$currentWebETVersion"; #actually write to file
  close $fh2; #close file
}

sub runUpdate
{
  my @df = `/bin/df /var`; #runs system command to get HD size and free disk space
  foreach my $line (@df) #multiple lines are returned, need to loop through them
  {
        my $errormessage = '';
        next if $line =~ m/^Filesystem/; #skips lines that have the word "Filesystem", meaning it's a header line
        my $return;

        if ($line =~ m/dev/) #only cares about lines that have the word 'dev' in them
        {
              my @temp = split(/\s+/,$line); #split the line at the spaces
              if ($temp[3]<300000) #Third value is free space, need 300MB
              {
                  $errormessage = "Not enough disk space, less then 300MB is available";
                  system("logger -t SnortUpdate 'Error encountered while update database: $errormessage'");
                  return 2;
              }
              else
              {
                  my @lns=split('\n',$url); #splitting at a new line and setting the URL to a variable
                  foreach my $l (@lns) #if more than 1 then we just loop
                  {
                      print "Update found!\n";
                      print "Downloading... $l \n";
                      sleep(1);
                      system("logger -t SnortUpdate 'Download start for: $l'");
                      system("wget -r -o /var/tmp/log --output-document=/var/tmp/snortrules.tar.gz $l");
                      sleep(3);
                      $return = `cat /var/tmp/log 2>/dev/null`;

                      if ($return =~ "ERROR")
                      {
                          $errormessage = $return;
                          system("logger -t SnortUpdate 'Error prevented the download: $errormessage'");
                          print $errormessage;
                          return 2;
                      }
                      else
                      {
                          system("logger -t SnortUpdate 'Processing rules downloaded...'");
                          system("/usr/local/bin/oinkmaster.pl -v -s -u file:///var/tmp/snortrules.tar.gz -C /var/ipfire/snort/oinkmaster.conf -o /etc/snort/rules >>/var/tmp/log 2>&1 &");
                          sleep(2);
                          system("logger -t SnortUpdate 'Update Successfull for $l'");
                          print "Update Successfull for $l\n";
                      }
                  }
              }
          }
      }
      return 1;
}

sub verifySnortIsRunning
{
    system("logger -t SnortUpdate 'Verifying Snort is running...'");

    if (($Snort_Orange == 1 && ! -e "/var/run/snort_orange0.pid") | ($Snort_BLUE == 1 && ! -e "/var/run/snort_blue0.pid") | ($Snort_GREEN == 1 && ! -e "/var/run/snort_green0.pid") | ($Snort_Active == 1 && ! -e "/var/run/snort_red0.pid"))
    { #if any of the services are enabled and are not started, go ahead and start them 'cause peeps be creepin'
        system("/usr/local/bin/snortctrl start"); #start command... it knows what to do
    }
}

User avatar
Deepcuts
Posts: 461
Joined: March 1st, 2016, 3:18 pm
Location: Romania

Re: Snort Rules Update

Post by Deepcuts » April 2nd, 2018, 6:20 am

Using the method posted by gitarman94 above, I get too much CPU load from snort after snortupdate.
All cores are hammered like crazy and throughput is limited to ~100 Mbps because if this.
Disabling the cron entry and a reboot fixes this.
Image
Image

gitarman94
Posts: 7
Joined: March 17th, 2018, 1:37 pm
Location: US

Re: Snort Rules Update

Post by gitarman94 » April 2nd, 2018, 9:47 am

That’s really odd behavior. I’m running the script on my production box and on a vm and neither have had that issue since I implemented it a couple weeks ago. There are no other issues with doing regular updates via ipfire directly? I’m wondering if the script is barfing at a certain part due to hardware differences?? Hard to tell for sure without being able to replicate the issue on my end. Apparently, I’m not allowed to PM so if you want to work on this off-line, send me a message.

TimF
Posts: 83
Joined: June 10th, 2017, 7:27 pm

Re: Snort Rules Update

Post by TimF » April 5th, 2018, 1:16 am

It looks like there are a couple of errors in the snortupdate.pl script:
  • The last few digits of the Snort update file name are the version of snort (currently 2.9.11.1); the MD5 at the beginning of the line is the nearest thing to a version number.
  • The red interface can be red0 or ppp0 - if it's the wrong one it tries to start snort when it's already running (possibly Deepcuts's problem)
I've generated an update version of the script that fixes these problems, and also gets the correct version of snort for the VRT rules by asking snort - unfortunately the ET rules don't work the same way, but they do seen to change the version in the file name less regularly.

So the updated snortupdate.pl:

Code: Select all

#################################
# Snort Rules Update for IPFire #
# ----- Contributions by: ----- #
# --------- Kick@ss ----------- #
# ----------- H&M ------------- #
# -------- gitarman94 --------- #
#################################

use autodie ':io';
use LWP::Simple;
use strict;
use warnings;
use HTTP::Request;
use LWP::UserAgent;

my $Version = `snort -V 2>&1 | grep 'Version'`;
my ($et_v) = $Version =~ m/(\d+\.[\d\.]*)/;
my $vrt_v = $et_v;
$vrt_v =~ s/\.//g;
$et_v = "2.9.0";


my $oinkcode='none';
my $Snort_Orange = 0;
my $Snort_BLUE = 0;
my $Snort_GREEN = 0;
my $Snort_Active = 0;
my $url = '';
getSnortSettings();
my $currentLocalSnortVersion = readFile("/var/tmp/SnortVersion.txt"); #itterate through local file to get currently installed Snort version
my $currentLocalETVersion = readFile("/var/tmp/ETVersion.txt"); #itterate through local file to get currently installed ET version
my $snortList = webGet('https://snort.org/downloads/registered/md5s.txt');
my $ETList = webGet('https://rules.emergingthreats.net/version.txt');
my $currentWebSnortVersion = webSnortParse();#iteration through list and sets this var to current sub-point release number from websit
my $currentWebETVersion = webETParse();
versionComparer();
verifySnortIsRunning();

sub getSnortSettings
{
  open my $fh, '<', "/var/ipfire/snort/settings"; #open file
  while (my $line = <$fh>) #itterate throught the file line by line
  {
      if ($line =~ m/ENABLE/ | $line =~ m/OINKCODE/) #only cares about lines that have the word 'ENABLE' in them
      {
          if ($line =~ m/ORANGE/) #looking for orange settings
          {
              my @lineItems = split('[=]',$line);
              if ($lineItems[1] =~ m/on/) #if the setting is "on"
              {
                  $Snort_Orange = 1; #default is 0 but if orange is enabled then make it so
              }
          }
          elsif ($line =~ m/BLUE/) #looking for blue settings
          {
              my @lineItems = split('[=]',$line);
              if ($lineItems[1] =~ m/on/) #if the setting is "on"
              {
                  $Snort_BLUE = 1; #default is 0 but if blue is enabled then make it so
              }
          }
          elsif ($line =~ m/GREEN/) #looking for green settings
          {
              my @lineItems = split('[=]',$line);
              if ($lineItems[1] =~ m/on/) #if the setting is "on"
              {
                  $Snort_GREEN = 1; #default is 0 but if green is enabled then make it so
              }
          }
          elsif ($line =~ m/OINKCODE/) #get oinkcode from within the settings file automatically
          {
              my @lineItems = split('[=]',$line);
              if (length($lineItems[1]) == 41) #get the length and match to the 41 character string it should be, if not proper length, then it's not a real oinkcode
              {
                  $oinkcode = $lineItems[1];
              }
              elsif ((length($lineItems[1]) > 2 && length($lineItems[1]) < 41) || length($lineItems[1]) > 41)
              {
                system("logger -t SnortUpdate 'WARNING: OINKCODE appears to be of incorrect length.'");
              }
          }
          else
          {
              my @lineItems = split('[=]',$line);
              if ($lineItems[1] =~ m/on/) #if the setting is "on"
              {
                  $Snort_Active = 1; #default is 0 but if red is enabled then make it so
              }
          }
      }
  }
  close $fh;
}

sub webGet
{ #used to http query snort for current version of version 2
    my $url = shift;
    my $ua = LWP::UserAgent->new( ssl_opts => { verify_hostname => 0 }, );
    my $request = HTTP::Request->new(GET => $url);
    my $response = $ua->request($request);
    if ($response->is_success)
    {
        return $response->content; #returns list found on above site
    }
}

sub webETParse
{
    chomp $ETList; #remove extra line/spaces
    return $ETList;
}

sub webSnortParse
{  #Itterator...
    my $current_md5;

    foreach my $line (split( '  ', $snortList))
    {
      chomp $line;
      next unless ($line);
      my ($md5, $file) = split( ' ', $line );
      $current_md5 = $md5 if ($file =~ m/$vrt_v/);
    }

    return "$current_md5";
}

sub readFile
{
    my $localFile = shift; #we're passing a variable into the subroutine, this sets it to a local variable
    my $currentLocalVersion = 0; #default value, in case we don't find the file
    if (-e $localFile)
    {
        open my $fh, '<', $localFile; #open file
        while (my $row = <$fh>) #itterate throught the file line by line (if multiple)
        {
            chomp $row; #removes empty lines and takes only good data
            $currentLocalVersion = $row; #should only be 1 line in the file so we'll take whatever it gives us
        }
        close $fh;
    }
    return $currentLocalVersion;
}


sub versionComparer
{ #now we test that above WebVersions against our local versions, if newer we'll update

      # VRT Community
          #$url=" https://www.snort.org/rules/community";
      #EmergingThreats Community
          #$url="http://rules.emergingthreats.net/open/snort-$et_v/emerging.rules.tar.gz";
      # VRT Subscripted & VRT Community
          #$url="https://www.snort.org/rules/snortrules-snapshot-$vrt_v.tar.gz?oinkcode=$oinkcode\n https://www.snort.org/rules/community";
      # VRT Subscripted & EmergingThreats Community
          #$url="https://www.snort.org/rules/snortrules-snapshot-$vrt_v.tar.gz?oinkcode=$oinkcode\n http://rules.emergingthreats.net/open/snort-$et_v/emerging.rules.tar.gz";

      my $performUpdate = 1;

      #We're testing for " !~ m/none/ " in the oinkcode section, as if we don't have a valid code, no point in running them
      if ($currentWebSnortVersion ne $currentLocalSnortVersion && $currentWebETVersion > $currentLocalETVersion && $oinkcode !~ m/none/)
      { #updates both snort registered and EmergingThreats
          $url = "https://www.snort.org/rules/snortrules-snapshot-$vrt_v.tar.gz?oinkcode=$oinkcode\n http://rules.emergingthreats.net/open/snort-$et_v/emerging.rules.tar.gz";
      }
      elsif ($currentWebSnortVersion ne $currentLocalSnortVersion && $currentWebETVersion <= $currentLocalETVersion && $oinkcode !~ m/none/)
      { #updates only snort registered
          $url = "https://www.snort.org/rules/snortrules-snapshot-$vrt_v.tar.gz?oinkcode=$oinkcode";
      }
      elsif ($currentWebSnortVersion eq $currentLocalSnortVersion && $currentWebETVersion > $currentLocalETVersion)
      { #updates only emergingthreats
          $url = "http://rules.emergingthreats.net/open/snort-$et_v/emerging.rules.tar.gz";
      }
      else
      { #no updates are performed as there were no newer versions available
        $performUpdate = 0;
      }

      if ($performUpdate == 1 && runUpdate() == 1)
      { #only runs if there are updates available and the updater itself ran without error
          writeFile(); #write the latest version to local files so we now know we have the latest
          #performing these system tasks from here, as we don't want to restart the snort service if we don't have to
          system("logger -t SnortUpdate 'SnortUpdate finished.'");
          system("logger -t SnortUpdate 'Changing ownership for rule files...'");
          system("chown -R nobody:nobody /etc/snort/rules");
          system("logger -t SnortUpdate 'Ownership permissions updated.'");
          system("logger -t SnortUpdate 'Restarting Snort...'");
          system("/usr/local/bin/snortctrl restart");
      }
      else
      {
        print "No updates found.\n";
      }
}

sub writeFile
{
  open my $fh, '>', "/var/tmp/SnortVersion.txt"; #open file for writing
  print $fh "$currentWebSnortVersion"; #actually write to file
  close $fh; #close file

  open my $fh2, '>', "/var/tmp/ETVersion.txt"; #open file for writing
  print $fh2 "$currentWebETVersion"; #actually write to file
  close $fh2; #close file
}

sub runUpdate
{
  my @df = `/bin/df /var`; #runs system command to get HD size and free disk space
  foreach my $line (@df) #multiple lines are returned, need to loop through them
  {
        my $errormessage = '';
        next if $line =~ m/^Filesystem/; #skips lines that have the word "Filesystem", meaning it's a header line
        my $return;

        if ($line =~ m/dev/) #only cares about lines that have the word 'dev' in them
        {
              my @temp = split(/\s+/,$line); #split the line at the spaces
              if ($temp[3]<300000) #Third value is free space, need 300MB
              {
                  $errormessage = "Not enough disk space, less then 300MB is available";
                  system("logger -t SnortUpdate 'Error encountered while update database: $errormessage'");
                  return 2;
              }
              else
              {
                  my @lns=split('\n',$url); #splitting at a new line and setting the URL to a variable
                  foreach my $l (@lns) #if more than 1 then we just loop
                  {
                      print "Update found!\n";
                      print "Downloading... $l \n";
                      sleep(1);
                      system("logger -t SnortUpdate 'Download start for: $l'");
                      system("wget -r -o /var/tmp/log --output-document=/var/tmp/snortrules.tar.gz $l");
                      sleep(3);
                      $return = `cat /var/tmp/log 2>/dev/null`;

                      if ($return =~ "ERROR")
                      {
                          $errormessage = $return;
                          system("logger -t SnortUpdate 'Error prevented the download: $errormessage'");
                          print $errormessage;
                          return 2;
                      }
                      else
                      {
                          system("logger -t SnortUpdate 'Processing rules downloaded...'");
                          system("/usr/local/bin/oinkmaster.pl -v -s -u file:///var/tmp/snortrules.tar.gz -C /var/ipfire/snort/oinkmaster.conf -o /etc/snort/rules >/var/tmp/log 2>&1 &");
                          sleep(2);
                          system("logger -t SnortUpdate 'Update Successfull for $l'");
                          print "Update Successfull for $l\n";
                      }
                  }
              }
          }
      }
      return 1;
}

sub verifySnortIsRunning
{
    system("logger -t SnortUpdate 'Verifying Snort is running...'");

    if (($Snort_Orange == 1 and not -e "/var/run/snort_orange0.pid") or
        ($Snort_BLUE   == 1 and not -e "/var/run/snort_blue0.pid")   or
        ($Snort_GREEN  == 1 and not -e "/var/run/snort_green0.pid")  or
        ($Snort_Active == 1 and not -e "/var/run/snort_ppp0.pid" and not -e "/var/run/snort_red0.pid" ))
    { #if any of the services are enabled and are not started, go ahead and start them 'cause peeps be creepin'
        system("/usr/local/bin/snortctrl restart"); #start command... it knows what to do
    }
}
I've not had had time to fully test this so use at your own risk.
Last edited by TimF on April 8th, 2018, 12:09 pm, edited 1 time in total.

Edwin
Posts: 104
Joined: March 19th, 2016, 12:02 pm

Re: Snort Rules Update

Post by Edwin » April 6th, 2018, 9:49 pm

Hi TimF,

Thanks for your effort in this!
First time I ran the script it detected an update for the VRT rules. I got a "wget" error message stating that an url was missing. Later it stopped snort on blue,, green and red as it should. Than started snort on blue, green and red, and started snort again causing IPFire to stop responding. After a reboot is seemed to have upgraded the vrt-rules. (according to the date after the "Download new ruleset" button on the IDS page.)

Second time there was no snort update to install. The script didn't stop snort, but did start it. This time IPFire kept running, but I had a lot CPU usage.

Regards,
Edwin.
Image
Image

TimF
Posts: 83
Joined: June 10th, 2017, 7:27 pm

Re: Snort Rules Update

Post by TimF » April 7th, 2018, 2:42 pm

It could be that it's not recognising the names of your interfaces. This is the problem I originally had since the script looked for red0 while I had ppp0. You could have yet another option.

Can you check the names of your interfaces:

Look for /var/run/snort_*.pid and also the contents of /var/ipfire/red/iface.

If you've got something that's not in the list red0, ppp0, green0, blue0 and orange0, then the tests at the end of snortupdate.pl will need to be changed.

Edwin
Posts: 104
Joined: March 19th, 2016, 12:02 pm

Re: Snort Rules Update

Post by Edwin » April 7th, 2018, 8:20 pm

The interfaces have standard names. So in /var/run the files snort_blue0.pid, snort_green0.pid and snort_red0.pid are present.
cat /var/ipfire/red/iface gives red0 (with no carriage return after the 0).
Don't know why it's not working.

So, this is what happens

Code: Select all

[root@ipfire snort]# ./update.sh 
No updates found.
Starting Intrusion Detection System on blue0...
Spawning daemon child...
My daemon child 23123 lives...
Daemon parent exiting (0)                                                                              [  OK  ]
chmod: cannot access '/var/run/snort_blue0.pid': No such file or directory
Starting Intrusion Detection System on green0...
Spawning daemon child...
My daemon child 23288 lives...
Daemon parent exiting (0)                                                                              [  OK  ]
Starting Intrusion Detection System on red0...
Snort is running and after the script determines there is no update, it starts snort on top of snort so to speak.
Image
Image

TimF
Posts: 83
Joined: June 10th, 2017, 7:27 pm

Re: Snort Rules Update

Post by TimF » April 8th, 2018, 12:13 pm

It's a silly little error: it was checking for redo rather than red0.pid; the box I have access to daily uses ppp0.

I've updated the code in my original post.

I've also changed it to 'restart' snort rather than 'start' so that it shuts down any existing instances, rather than just blindly starting them (even if only once instance isn't running).

Hopefully this will now fix all the problems.

Edwin
Posts: 104
Joined: March 19th, 2016, 12:02 pm

Re: Snort Rules Update

Post by Edwin » April 8th, 2018, 4:58 pm

Looking great now, thanks.
I will let it run, with an hourly update check, on my firewall I use for testing and will report back later.

One little thing; In your install-script you set a symbolic link in fcron.hourly and later you do a chmod +x on a link in frcon.daily, which isn't there. Minor thingy.

Tanks for your efforts!
Image
Image

Edwin
Posts: 104
Joined: March 19th, 2016, 12:02 pm

Re: Snort Rules Update

Post by Edwin » April 8th, 2018, 8:35 pm

Okay, here it is. All seems to work as expected, I only get some wget message I don't understand.

Code: Select all

[root@ipfire snort]# ./update.sh 
Update found!
Downloading... https://www.snort.org/rules/snortrules-snapshot-29111.tar.gz?oinkcode=xxxxxxxxxxxx 
Update Successfull for https://www.snort.org/rules/snortrules-snapshot-29111.tar.gz?oinkcode=xxxxxxxxxxxx 
Update found!
Downloading...  
wget: missing URL
Usage: wget [OPTION]... [URL]...

Try `wget --help' for more options.
Update Successfull for 
Update found!
Downloading...  http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz 
Update Successfull for  http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz
Stopping Intrusion Detection System on blue0...                        [  OK  ]
Stopping Intrusion Detection System on green0...                       [  OK  ]
Stopping Intrusion Detection System on red0...                         [  OK  ]
Starting Intrusion Detection System on blue0...
Spawning daemon child...
My daemon child 17616 lives...
Daemon parent exiting (0)                                              [  OK  ]
Starting Intrusion Detection System on green0...
Spawning daemon child...
My daemon child 17738 lives...
Daemon parent exiting (0)                                              [  OK  ]
Starting Intrusion Detection System on red0...
Spawning daemon child...
My daemon child 17966 lives...
Daemon parent exiting (0)                                              [  OK  ]
[root@ipfire snort]# 
When there are no updates:

Code: Select all

[root@ipfire snort]# ./update.sh 
No updates found.
[root@ipfire snort]#
I am very happy with the result!
Image
Image

Edwin
Posts: 104
Joined: March 19th, 2016, 12:02 pm

Re: Snort Rules Update

Post by Edwin » April 12th, 2018, 7:59 pm

I have been running this script for a few day's now without any problem. It's great!
The script checks every hour for updates and gets me the ET and VRT rules when updates are available.
Thanks so much.

Regards,
Edwin.
Image
Image

User avatar
H&M
Posts: 471
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Snort Rules Update

Post by H&M » April 13th, 2018, 11:57 am

Great Job!

One minor feature to be added: log in /var/log/messages if no updates are found - currently the perl script only print it on screen.

I saw above Michael's reason for not including an automated update job for the snort rules: lack of preserving user configuration (what rules gets activated in snort.conf).
To make thing worse, the ids.cgi has a bug: it does not put all downloaded rules files in snort.conf (user must manually add the missing ones)
See https://bugzilla.ipfire.org/show_bug.cgi?id=11263

If somebody can fix above 2 items then I don't see any reason why the automated update job should not be included as standard feature.

Again, great job!

Thank you,
H&M

Post Reply