IDS Rule updater - with rule state persistance

[Hellfire]

Tim,

I mentioned it briefly above: the settings file contains:

Code: Select all

DEBUG=0
APPLY_POLICY_CHANGE=on
LIVE_UPDATE=on
EMAIL=off
DOWNLOAD_LIMIT=21000
ENABLE=on
RATE=DAILY
POLICY=MAX-DETECT
VERSION=3

And the WebIF shows:

2018-12-14_160622.png (10.69 KiB) Viewed 928 times

Perhaps you could have a look and fix this?

This happens when switching to MAX-DETECT and press the save button. The WebIF always changes this back to CONNECTIVITY although the settings-files was update correctly. IMO there is a further issue with the download limit, too, since this value changes to zero after hitting the save button. A refresh of the web page brings back the value 21000 but not the correct policy.

Thanks,
Michael

[Hellfire]

Maybe I'm too dumb to do some testing if the IDS updater works correctly, if all - at least on my side.

I've opened the rules file: browser-ie.rules and set all rules on comment by putting a #-char at the beginning of each line, e.g.

Code: Select all

#alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"BROWSER-IE Microsoft Internet Explorer image download spoofing attempt"; flow:to_server,established; content:".bat."; fast_pattern:only; http_uri; content:"MSIE "; http_header; pcre:"/^User-Agent:[^\n]*?MSIE\s[56]/Hmi"; metadata:service http; reference:bugtraq,11768; classtype:bad-unknown; sid:26937; rev:3;)
#alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"BROWSER-IE Microsoft Internet Explorer image download spoofing attempt"; flow:to_server,established; content:".html."; fast_pattern:only; http_uri; content:"MSIE "; http_header; pcre:"/^User-Agent:[^\n]*?MSIE\s[56]/Hmi"; metadata:service http; reference:bugtraq,11768; classtype:bad-unknown; sid:26936; rev:3;)

As a result the Intrusion Detection webpage lists all rules below the category browser-ie.rules as inactive.

I've then modified the status file of IDS updater like suggested and added a Z to each of those checksum lines and saved it again.
Afterwards, I've fired this command

Code: Select all

/usr/local/bin/ids-update.pl

and let the updater do its job. The policy is set to MAX-DETECT according to the settings file. The issues with the WebIF are posted above.

After the updater forced SNORT re-read its settings, I first had a look to the file modification date/time of the file browser-ie.rules - no changes made. Second, I checked the rules inside the file and in WebIF - no changes either.

I hope that the rule file I've used for those test does include at least one rule that the policy MAX-DETECT will detect and activate. If not how can I run some test to check if the updater does its job?

Michael

[TimF]

I'll set up a test to have a look at it.

[Hellfire]

Since weeks now, the updater does not log any changed rules now.

How can I uninstall the IDS updater. Unfortunately no instructions exists on github or within this forum posting.

Thx,
Michael

[TimF]

Unfortunately I've lost my normal internet access which makes responding difficult.

Have a look from the command line and see if there's a copy of ids-update.pl running - if there is kill it and hopefully the next update attempt will work. It appears that one of the downloads from the internet can lock up occasionally and not timeout. I've not been able to track down what is locking up.

To uninstall (from memory):

1. Use fcrontab -e and delete the lines for ids-update.pl (should be near the end).
2. Check ids-update.pl isn't running. Wait for it to finish if it is.
3. rm -R /var/ipfire/idsupdate
4. rm /var/ipfire/addon-lang/ids-update.*.pl
5. rm /usr/local/bin/ids-update.pl
6. rm /srv/web/ipfire/cgi-bin/idsflowbits.cgi
7. rm /srv/web/ipfire/cgi-bin/idsupdate.dat
8. rm /var/ipfire/menu.d/EX-idsupdate.menu
9. rm /usr/share/logwatch/scripts/services/ids-update
10. rm /usr/share/logwatch/dist.conf/services/ids-update.conf
11. rm /srv/web/ipfire/cgi-bin/idsupdate.cgi

I have written an uninstaller, but I can't upload it until I get my internet connection back.

[Hellfire]

Have a look from the command line and see if there's a copy of ids-update.pl running - if there is kill it and hopefully the next update attempt will work. It appears that one of the downloads from the internet can lock up occasionally and not timeout. I've not been able to track down what is locking up.

Unfortunately no process running like this. I don't think this is the source of the issue I'm seeing for weeks now when looking at the IDS updater logs, 'cause according to the update date time, there is an actual download taking place, however the updater did not update any of the IDS rules so far.

Pls. see IDS updater settings:

Hence, I guess sthg. is wrong on my side, maybe missing some access right for important files?

Michael

[Hellfire]

FWIW, in case you did not see this posting: viewtopic.php?f=52&t=22266.

Michael

[TimF]

You could try looking at the log file in /var/tmp.

Also check the permissions of the files in /etc/snort/rules - they should all be nobody.nobody (I think).

Finally the MANIFEST file on github gives the owner and permissions for all the updater files.

[Hellfire]

All .rules files are set to owner nobody/nobody with exception of .config files those are set to root/root.
Guess this is OK.

OTH, /var/tmp/log shows:

/usr/local/bin/oinkmaster.pl: Error: no write permission on "/etc/snort/rules/EMERGING_THREATS_classification.config"
Write permission is required on all rules files inside the output directory.

But this seems to be caused by the above mentioned restricted permissions on .config files.

[Hellfire]

Some more feedback AND finally SUCCESS!

After setting permissions for those .config files to nobofy/nobody, to be precise for files

TALOS_VRT_classification.config, EMERGING_THREATS_classification.config and COMMUNITY_classification.config

and firing a manual

Code: Select all

./ids-update.pl

I now have some log enries telling me that some rules where deleted and others were updated. That's it!

This leaves one question open: why does the installer of IDS updater or whatever not set permissions as needed?

Michael

Edit: Tim, you should take notice of this posting, here viewtopic.php?f=52&t=22266, too as already mentioned above. Unless you do not fix ids-update-pl, the current Talos rules cannot be downloaded anymore.

[Stefan87]

I have written an uninstaller, but I can't upload it until I get my internet connection back.

the uninstaller would be great with the new suricata, the update tool is not needed

[TimF]

I've uploaded the uninstaller. You should be able to do:

Code: Select all

wget https://github.com/timfprogs/ipfidsupdate/raw/master/uninstall-idsupdate.sh
chmod +x uninstall-idsupdate.sh
./uninstall-idsupdate.sh

[Hellfire]

Thanks Tim!

[Stefan87]

Nice thanks