IDS Rule updater - with rule state persistance

General questions.
Hellfire
Posts: 645
Joined: November 8th, 2015, 8:54 am

Re: IDS Rule updater - with rule state persistance

Post by Hellfire » December 14th, 2018, 3:10 pm

Tim,

I mentioned it briefly above: the settings file contains:

Code: Select all

DEBUG=0
APPLY_POLICY_CHANGE=on
LIVE_UPDATE=on
EMAIL=off
DOWNLOAD_LIMIT=21000
ENABLE=on
RATE=DAILY
POLICY=MAX-DETECT
VERSION=3
And the WebIF shows:
2018-12-14_160622.png
2018-12-14_160622.png (10.69 KiB) Viewed 1091 times
Perhaps you could have a look and fix this?

This happens when switching to MAX-DETECT and press the save button. The WebIF always changes this back to CONNECTIVITY although the settings-files was update correctly. IMO there is a further issue with the download limit, too, since this value changes to zero after hitting the save button. A refresh of the web page brings back the value 21000 but not the correct policy.

Thanks,
Michael
Image

Hellfire
Posts: 645
Joined: November 8th, 2015, 8:54 am

Re: IDS Rule updater - with rule state persistance

Post by Hellfire » December 14th, 2018, 3:42 pm

Maybe I'm too dumb to do some testing if the IDS updater works correctly, if all - at least on my side.

I've opened the rules file: browser-ie.rules and set all rules on comment by putting a #-char at the beginning of each line, e.g.

Code: Select all

#alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"BROWSER-IE Microsoft Internet Explorer image download spoofing attempt"; flow:to_server,established; content:".bat."; fast_pattern:only; http_uri; content:"MSIE "; http_header; pcre:"/^User-Agent:[^\n]*?MSIE\s[56]/Hmi"; metadata:service http; reference:bugtraq,11768; classtype:bad-unknown; sid:26937; rev:3;)
#alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"BROWSER-IE Microsoft Internet Explorer image download spoofing attempt"; flow:to_server,established; content:".html."; fast_pattern:only; http_uri; content:"MSIE "; http_header; pcre:"/^User-Agent:[^\n]*?MSIE\s[56]/Hmi"; metadata:service http; reference:bugtraq,11768; classtype:bad-unknown; sid:26936; rev:3;)
As a result the Intrusion Detection webpage lists all rules below the category browser-ie.rules as inactive.

I've then modified the status file of IDS updater like suggested and added a Z to each of those checksum lines and saved it again.
Afterwards, I've fired this command

Code: Select all

/usr/local/bin/ids-update.pl
and let the updater do its job. The policy is set to MAX-DETECT according to the settings file. The issues with the WebIF are posted above.

After the updater forced SNORT re-read its settings, I first had a look to the file modification date/time of the file browser-ie.rules - no changes made. Second, I checked the rules inside the file and in WebIF - no changes either.

I hope that the rule file I've used for those test does include at least one rule that the policy MAX-DETECT will detect and activate. If not how can I run some test to check if the updater does its job?

Michael
Image

TimF
Posts: 82
Joined: June 10th, 2017, 7:27 pm

Re: IDS Rule updater - with rule state persistance

Post by TimF » December 16th, 2018, 8:48 pm

I'll set up a test to have a look at it.

Hellfire
Posts: 645
Joined: November 8th, 2015, 8:54 am

Re: IDS Rule updater - with rule state persistance

Post by Hellfire » February 6th, 2019, 7:54 pm

Since weeks now, the updater does not log any changed rules now.
IDS Updater.png
How can I uninstall the IDS updater. Unfortunately no instructions exists on github or within this forum posting.

Thx,
Michael
Image

TimF
Posts: 82
Joined: June 10th, 2017, 7:27 pm

Re: IDS Rule updater - with rule state persistance

Post by TimF » February 8th, 2019, 8:37 am

Unfortunately I've lost my normal internet access which makes responding difficult.

Have a look from the command line and see if there's a copy of ids-update.pl running - if there is kill it and hopefully the next update attempt will work. It appears that one of the downloads from the internet can lock up occasionally and not timeout. I've not been able to track down what is locking up.

To uninstall (from memory):
  1. Use fcrontab -e and delete the lines for ids-update.pl (should be near the end).
  2. Check ids-update.pl isn't running. Wait for it to finish if it is.
  3. rm -R /var/ipfire/idsupdate
  4. rm /var/ipfire/addon-lang/ids-update.*.pl
  5. rm /usr/local/bin/ids-update.pl
  6. rm /srv/web/ipfire/cgi-bin/idsflowbits.cgi
  7. rm /srv/web/ipfire/cgi-bin/idsupdate.dat
  8. rm /var/ipfire/menu.d/EX-idsupdate.menu
  9. rm /usr/share/logwatch/scripts/services/ids-update
  10. rm /usr/share/logwatch/dist.conf/services/ids-update.conf
  11. rm /srv/web/ipfire/cgi-bin/idsupdate.cgi
I have written an uninstaller, but I can't upload it until I get my internet connection back.

Hellfire
Posts: 645
Joined: November 8th, 2015, 8:54 am

Re: IDS Rule updater - with rule state persistance

Post by Hellfire » February 8th, 2019, 1:50 pm

TimF wrote:
February 8th, 2019, 8:37 am
Have a look from the command line and see if there's a copy of ids-update.pl running - if there is kill it and hopefully the next update attempt will work. It appears that one of the downloads from the internet can lock up occasionally and not timeout. I've not been able to track down what is locking up.
Unfortunately no process running like this. I don't think this is the source of the issue I'm seeing for weeks now when looking at the IDS updater logs, 'cause according to the update date time, there is an actual download taking place, however the updater did not update any of the IDS rules so far.

Pls. see IDS updater settings:
IDS settings.png
Hence, I guess sthg. is wrong on my side, maybe missing some access right for important files?

Michael
Image

Hellfire
Posts: 645
Joined: November 8th, 2015, 8:54 am

Re: IDS Rule updater - with rule state persistance

Post by Hellfire » February 8th, 2019, 2:46 pm

FWIW, in case you did not see this posting: viewtopic.php?f=52&t=22266.

Michael
Image

TimF
Posts: 82
Joined: June 10th, 2017, 7:27 pm

Re: IDS Rule updater - with rule state persistance

Post by TimF » February 12th, 2019, 5:17 pm

You could try looking at the log file in /var/tmp.

Also check the permissions of the files in /etc/snort/rules - they should all be nobody.nobody (I think).

Finally the MANIFEST file on github gives the owner and permissions for all the updater files.

Hellfire
Posts: 645
Joined: November 8th, 2015, 8:54 am

Re: IDS Rule updater - with rule state persistance

Post by Hellfire » February 15th, 2019, 8:14 pm

All .rules files are set to owner nobody/nobody with exception of .config files those are set to root/root.
Guess this is OK.

OTH, /var/tmp/log shows:
/usr/local/bin/oinkmaster.pl: Error: no write permission on "/etc/snort/rules/EMERGING_THREATS_classification.config"
Write permission is required on all rules files inside the output directory.
But this seems to be caused by the above mentioned restricted permissions on .config files.
Image

Hellfire
Posts: 645
Joined: November 8th, 2015, 8:54 am

Re: IDS Rule updater - with rule state persistance

Post by Hellfire » February 15th, 2019, 8:40 pm

Some more feedback AND finally SUCCESS!

After setting permissions for those .config files to nobofy/nobody, to be precise for files
TALOS_VRT_classification.config, EMERGING_THREATS_classification.config and COMMUNITY_classification.config
and firing a manual

Code: Select all

./ids-update.pl
I now have some log enries telling me that some rules where deleted and others were updated. That's it!

This leaves one question open: why does the installer of IDS updater or whatever not set permissions as needed?

Michael

Edit: Tim, you should take notice of this posting, here viewtopic.php?f=52&t=22266, too as already mentioned above. Unless you do not fix ids-update-pl, the current Talos rules cannot be downloaded anymore.
Image

Stefan87
Posts: 73
Joined: July 20th, 2017, 11:55 pm

Re: IDS Rule updater - with rule state persistance

Post by Stefan87 » April 25th, 2019, 1:15 am

TimF wrote:
February 8th, 2019, 8:37 am


I have written an uninstaller, but I can't upload it until I get my internet connection back.
the uninstaller would be great with the new suricata, the update tool is not needed

TimF
Posts: 82
Joined: June 10th, 2017, 7:27 pm

Re: IDS Rule updater - with rule state persistance

Post by TimF » April 27th, 2019, 2:49 pm

I've uploaded the uninstaller. You should be able to do:

Code: Select all

wget https://github.com/timfprogs/ipfidsupdate/raw/master/uninstall-idsupdate.sh
chmod +x uninstall-idsupdate.sh
./uninstall-idsupdate.sh

Hellfire
Posts: 645
Joined: November 8th, 2015, 8:54 am

Re: IDS Rule updater - with rule state persistance

Post by Hellfire » April 27th, 2019, 3:01 pm

Thanks Tim!
Image

Stefan87
Posts: 73
Joined: July 20th, 2017, 11:55 pm

Re: IDS Rule updater - with rule state persistance

Post by Stefan87 » April 27th, 2019, 4:17 pm

Nice thanks

Post Reply