Snort Rules Update

General questions.
Kick@ss
Posts: 8
Joined: November 28th, 2015, 8:54 pm

Re: Snort Rules Update

Post by Kick@ss » May 2nd, 2017, 1:52 am

Hi Every1,

Here is v0.1b for snort update.
https://drive.google.com/open?id=0B_CXl ... URqNFZXQm8

It's a script for automatically updating Snort rules for all four supplied rule sources ( two, whatever )

Right now it hit me I could just as well implement it into the WUI for the non-technical people.
So I'm working on that now
Stay tuned

EDIT: One thing I noticed is changing owner updates the last modified date, which means the date shown on the WUI will reflect the last time the update script ran, not the actual rule date. No easy way I see around that, if someone has an idea, let me know.
Not that it really matters I guess, seeing as it only checks community.rules for the date.

UPDATE: Bug fixes and minor improvements
v0.2r
https://drive.google.com/open?id=0B_CXl ... UYyR0Vtcmc
Last edited by Kick@ss on May 28th, 2017, 12:33 pm, edited 1 time in total.

steve_v
Posts: 4
Joined: May 13th, 2017, 5:59 pm

Re: Snort Rules Update

Post by steve_v » May 13th, 2017, 9:43 pm

Kick@ss wrote:Hi Every1,
Here is v0.1b for snort update.
https://drive.google.com/open?id=0B_CXl ... URqNFZXQm8
Well, your files are full of DOS style carriage returns...
And the install script has the wrong filename for the archive...
And expects it to extract to a subdirectory, which it doesn't...
But other than that, the script works fine, and saves me the hassle of writing my own.
Cheers :D

User avatar
H&M
Posts: 471
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Snort Rules Update

Post by H&M » May 25th, 2017, 7:44 pm

Hi,

Although I used vi to get rid of the DOS end of lines, the script does not start.
Any help will be appreciated

Code: Select all

No such class oinkcode at /var/ipfire/snort/snortupdate.pl line 7, near "my oinkcode"
syntax error at /var/ipfire/snort/snortupdate.pl line 7, near "my oinkcode="
Unmatched right curly bracket at /var/ipfire/snort/snortupdate.pl line 17, at end of line
syntax error at /var/ipfire/snort/snortupdate.pl line 17, near "}"
Execution of /var/ipfire/snort/snortupdate.pl aborted due to compilation errors.
Permissions Updated

After commenting line "my oinkode" still no result: no download

Code: Select all

 /var/ipfire/snort/update.sh
Downloading from 1
WARNING: combining -O with -r or -p will mean that all downloaded content
will be placed in the single file you specified.

Update Successfull for 1
Permissions Updated


Late edit: perl code is not functioning.lns should be an array ... split command uses regex so new line needs double \... or use "\n", missing () on split command?

Second: there is no snort restart after update. This is not ok... How the rules get applied after download without snort being restarted?



Thank you,
H&M

User avatar
Deepcuts
Posts: 461
Joined: March 1st, 2016, 3:18 pm
Location: Romania

Re: Snort Rules Update

Post by Deepcuts » May 26th, 2017, 3:54 am

@H&M
Check viewtopic.php?f=27&t=18750#p107603
Only the 1st two snippets.
Image
Image

User avatar
H&M
Posts: 471
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Snort Rules Update

Post by H&M » May 27th, 2017, 12:40 pm

Hi,

Improved: /var/ipfire/snort/update.sh

Code: Select all

#!/bin/bash
logger -t SnortUpdate "SnortUpdate start..."
perl /var/ipfire/snort/snortupdate.pl
logger -t SnortUpdate "SnortUpdate finished."
logger -t SnortUpdate "Changing ownership for rule files..."
chown -R nobody:nobody /etc/snort/rules
logger -t SnortUpdate "Ownership permissions updated."
logger -t SnortUpdate "Restarting Snort ..."
/usr/local/bin/snortctrl restart
logger -t SnortUpdate "Process completed succesfully."

Improved and fixed: /var/ipfire/snort/snortupdate.pl

Code: Select all

#!/usr/bin/perl
################################
# Snort Rules Update for IPFire#
# Author : Kick@ss   // corrected and improved by H&M          #
# Version : 0.1b               #
################################
my $oinkcode='_put_your_oinckcode_here_';
my $url='';
# VRT Community
        #$url=" https://www.snort.org/rules/community";
#EmergingThreats Community
        #$url="http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz";
# VRT Subscripted & VRT Community
        #$url="https://www.snort.org/rules/snortrules-snapshot-2990.tar.gz?oinkcode=$oinkcode\n https://www.snort.org/rules/community";
# VRT Subscripted & EmergingThreats Community
        $url="https://www.snort.org/rules/snortrules-snapshot-2990.tar.gz?oinkcode=$oinkcode\n http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz";
my @df = `/bin/df -B M /var`;
foreach my $line (@df) {
        next if $line =~ m/^Filesystem/;
        my $return;

        if ($line =~ m/dev/ ) {
                $line =~ m/^.* (\d+)M.*$/;
                my @temp = split(/ +/,$line);
                if ($1<300) {
                        $errormessage = "Not enough disk space, less then 300MiB is available";
                        system("logger -t SnortUpdate 'Error encountered while update database: $errormessage'");
                } else {
                        my @lns=split('\n',$url);
                        foreach my $l (@lns) {
                                print " Link - $l \n";
                                sleep(3);
                                system("logger -t SnortUpdate 'Download start for: $l'");
                                system("wget -r -o /var/tmp/log --output-document=/var/tmp/snortrules.tar.gz $l");
                                sleep(3);
                                $return = `cat /var/tmp/log 2>/dev/null`;

                                if ($return =~ "ERROR"){
                                        $errormessage = $return;
                                        system("logger -t SnortUpdate 'Error prevented the download: $errormessage'");
                                        print $errormessage;
                                } else {
                                        system("logger -t SnortUpdate 'Processing rules downloaded...'");
                                        system("/usr/local/bin/oinkmaster.pl -v -s -u file:///var/tmp/snortrules.tar.gz -C /var/ipfire/snort/oinkmaster.conf -o /etc/snort/rules >>/var/tmp/log 2>&1 &");
                                        sleep(2);
                                        system("logger -t SnortUpdate 'Update Successfull for $l'");
                                        print "Update Successfull for $l\n";
                                }
                        }
                }
        }
}
Best,
H&M

Kick@ss
Posts: 8
Joined: November 28th, 2015, 8:54 pm

Re: Snort Rules Update

Post by Kick@ss » May 28th, 2017, 11:06 am

Hi Guys,

Yeah sorry about the DOS style carriage returns, found that out myself last week.

Used Notepad++ to rewrite the scripts after I made it on a IPFire machine, set it to perl, don't know exactly why it used the DOS style CR.

Fixing it now, sorry for the non-technical people.

Like I said people, this was a beta...do you not know what that means???
If everyone prefers, I will keep my scripts to myself and you can write your own!
Working on v0.2r

PS: See original post

User avatar
H&M
Posts: 471
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Snort Rules Update

Post by H&M » May 28th, 2017, 11:13 am

Hi,

Besides the DOS carriage returns there are also some problems:
1. The variable lns should be an array if we want to use more than one link (update signatures from several providers). Yes, in your example might work because your example updates only one source but with multiple sources then variable must be an array
2. One split command is missing ()
3. Snort has to be restarted in order to take in consideration new rules
4. Nice to have: extensive logging to monitor the behavior of each step of the procedure.

I solved above and scripts (bash and perl) works fine.

Thank you!
Best regards,
H&M

Kick@ss
Posts: 8
Joined: November 28th, 2015, 8:54 pm

Re: Snort Rules Update

Post by Kick@ss » May 28th, 2017, 12:37 pm

Hi H&M,

Thanks used some of your code in the fixed v0.2r

New link is in my original post if you want to test it again.
I don't always have to time to check something like this as I created it on-the-fly and then later wrote the install scripts and didn't run extensive tests at that time.

But that's what a beta is for.
Thanks for the input

User avatar
H&M
Posts: 471
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Snort Rules Update

Post by H&M » June 3rd, 2017, 9:46 am

Hi,

I've updated logs.cgi/logs.dat in order to include SnortUpdate logging (I used log -t SnortUpdate to log all script actions):

Differences:

Code: Select all

--- /root/ipfire-2.x/html/cgi-bin/logs.cgi/log.dat      2016-11-03 01:24:13.716642691 +0200
+++ log.dat     2017-06-03 12:20:25.785835664 +0300
@@ -67,6 +67,7 @@
         'pakfire' => '(pakfire:)',
         'red' => '(red:|pppd\[.*\]: |chat\[.*\]|pppoe\[.*\]|pptp\[.*\]|pppoa\[.*\]|pppoa3\[.*\]|pppoeci\[.*\]|ipppd|ipppd\[.*\]|kernel: ippp\d|kernel: isdn.*|ibod\[.*\]|dhcpcd\[.*\]|modem_run\[.*\])',
         'snort' => '(snort\[.*\]: )',
+       'snortupdate' => '(SnortUpdate: )',
         'squid' => '(squid\[.*\]: |squid: )',
         'ssh' => '(sshd(?:\(.*\))?\[.*\]: )',
         'urlfilter bl' => '(installpackage\[urlfilter\]: )',
@@ -93,6 +94,7 @@
         'pakfire' => 'Pakfire',
         'red' => 'RED',
         'snort' => "$Lang::tr{'intrusion detection'}",
+       'snortupdate' => 'SnortUpdate',
         'squid' => "$Lang::tr{'web proxy'}",
         'ssh' => 'SSH',
         'urlfilter bl' => 'URLFilter Blacklist',

logs.dat-SnortUpdate.PNG
Best regards,
H&M

Edwin
Posts: 104
Joined: March 19th, 2016, 12:02 pm

Re: Snort Rules Update

Post by Edwin » July 2nd, 2017, 6:24 pm

Hi H&M,

I am no coder (at all) and still noob on linux. I try to get your IDS updatescripts working. Would be great!
Here is what I did.
I copied (cut&paste) your update scripts from the forum into nano onthe FW, saved the scripts, made them executable and inserted my oinkcode of course. When I run update.sh I get lots of errors.

Code: Select all

Bareword found where operator expected at /var/ipfire/snort/snortupdate.pl line 15, near "//www"
	(Missing operator before www?)
String found where operator expected at /var/ipfire/snort/snortupdate.pl line 17, near "$url=""
  (Might be a runaway multi-line "" string starting on line 15)
	(Missing semicolon on previous line?)
etc.

So I am definitely doing something wrong here. Guess something going wrong with the cut&paste(?)
Can you help me with this?

Regards,
Edwin.
Image
Image

dnl
Posts: 375
Joined: June 28th, 2013, 11:03 am

Re: Snort Rules Update

Post by dnl » July 4th, 2017, 11:41 am

Hi guys,

Thanks for the script! I've not yet tried it, but am very interested.

Can I suggest an improvement?

Instead of downloading the rulesets and trying to apply them, why not check to see if the rulesets are newer first?
This way the script can be set to run more often (say every 6 hours) so there's less of a delay between rulesets being updated and IPFire getting the rules.
It also prevents unnecessary restarts of Snort, preserving the state until an update is actually available.
Thanks!
IPFire 2.x (Latest Update) on x86_64 Intel Bay Trail CPU, 4GiB RAM, RED + GREEN + BLUE + ORANGE

User avatar
domsheldon1
Posts: 247
Joined: October 11th, 2011, 9:44 am
Location: Mons - Belgium

Re: Snort Rules Update

Post by domsheldon1 » July 4th, 2017, 3:13 pm

The option is installed on my latest IpFire.
Thank you for your advice.
Best regards,
Domsheldon1
(From Belgium)

http://fireinfo.ipfire.org/profile/edbd ... ffc0739326

Edwin
Posts: 104
Joined: March 19th, 2016, 12:02 pm

Re: Snort Rules Update

Post by Edwin » July 6th, 2017, 7:57 pm

Okay, fixed it. The updatescript is working now for VRT Subscripted & ET Community rules The copy-paste left me with some DOS carriage-returns I guess.

The script "says" that the update for the mentioned rules was successful.
It also writes:

Code: Select all

WARNING: combining -O with -r or -p will mean that all downloaded content
will be placed in the single file you specified.
Is that okay?

Tanks Kick@ss and H&M for this nice feature!

Regards,
Edwin.
Image
Image

IcyFire
Posts: 23
Joined: January 4th, 2016, 3:43 am

Re: Snort Rules Update

Post by IcyFire » December 31st, 2017, 10:49 pm

I wish this would become a package available in IPFire. How can we help make this happen? I still cant believe after all this time we still have to manually update Snort.

Any update on this? Working?
Image

dnl
Posts: 375
Joined: June 28th, 2013, 11:03 am

Re: Snort Rules Update

Post by dnl » January 1st, 2018, 10:33 am

IcyFire wrote:
December 31st, 2017, 10:49 pm
I wish this would become a package available in IPFire. How can we help make this happen? I still cant believe after all this time we still have to manually update Snort.
When I was notified of your reply to this thread, I re-read it all.

Is very disappointing that this basic requirement of the IDS won't be accepted for as a feature request. Especially after the community has shown much more interest in it than captive portal or RAID proposals. The evidence suggests that the core developers and the customers of Lightning Wire Labs don't actually use the IDS feature in IPFire.


Although I've wanted to try some of the other work-arounds in this thread, I'm still using a really basic 'fcron' job for EmergingThreats rules which does work reliably to this day. It might not be the best ruleset, but it does appear to work and I still get hundreds of hits a day (my connection isn't hosting any externally facing services at all).
IPFire 2.x (Latest Update) on x86_64 Intel Bay Trail CPU, 4GiB RAM, RED + GREEN + BLUE + ORANGE

Post Reply