Suricata much worse than guardian?.

Questions to IPFire Addons.
DJ-Melo
Posts: 672
Joined: July 8th, 2014, 7:12 am

Re: Suricata much worse than guardian?.

Post by DJ-Melo » April 28th, 2019, 2:57 pm

Would an actually ryzon have enough power? Cause i don't have a dedicated system only for ipfire?

User avatar
Roberto Peña
Posts: 761
Joined: July 16th, 2014, 3:56 pm
Location: Bilbao (España)
Contact:

Re: Suricata much worse than guardian?.

Post by Roberto Peña » April 28th, 2019, 3:35 pm

Good afternoon Michael.

I return my last post.
I have degraded the version to 130 and updated again to solve the status of the Service (viewtopic.php?f=6&t=22658#p124054) and now, doing the same tests with the Suricata running, "Suricata-Main" only uses 2% of the CPU.

Can be?.

In the Logs I have detections, but is there any other way to know if it really works?. What can be consulted? or do you know any web page where you can test ?.
Thanks for clarifying the doubts.
Image
Image

╔════════════════════════════════════════════════╗
Donate to improve IPFire: https://www.ipfire.org/donate
╚════════════════════════════════════════════════╝

ummeegge
Community Developer
Community Developer
Posts: 4990
Joined: October 9th, 2010, 10:00 am

Re: Suricata much worse than guardian?.

Post by ummeegge » April 28th, 2019, 6:11 pm

Hi Roberto,
some interesting infos can be found in here --> https://www.aldeid.com/wiki/Suricata-vs-snort in my opinion. For a fast test i´ve used now e.g. a Nmap full SYN Scan for example --> https://www.aldeid.com/wiki/Suricata-vs ... Test-rules whereby the 'emerging-scan.rules' includes the appropriate signatures (i use the 'Emergingthreats.net Community Rules') you can extend there the Nmap signatures to your needs. The suricata fast.log looks like this:

Code: Select all

$ tail -f /var/log/suricata/fast.log
04/28/2019-19:24:37.081093  [Drop] [**] [1:2010937:3] ET SCAN Suspicious inbound to mySQL port 3306 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:44814 -> 192.168.234.2:3306
04/28/2019-19:24:37.181583  [Drop] [**] [1:2010937:3] ET SCAN Suspicious inbound to mySQL port 3306 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:44836 -> 192.168.234.2:3306
04/28/2019-19:28:20.806106  [Drop] [**] [1:2010937:3] ET SCAN Suspicious inbound to mySQL port 3306 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:3306
04/28/2019-19:28:20.906565  [Drop] [**] [1:2010937:3] ET SCAN Suspicious inbound to mySQL port 3306 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:61320 -> 192.168.234.2:3306
04/28/2019-19:29:14.920219  [Drop] [**] [1:2002911:6] ET SCAN Potential VNC Scan 5900-5920 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:5919
04/28/2019-19:30:16.844413  [Drop] [**] [1:2002910:6] ET SCAN Potential VNC Scan 5800-5820 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:5818
04/28/2019-19:30:32.689910  [Drop] [**] [1:2010936:3] ET SCAN Suspicious inbound to Oracle SQL port 1521 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:58426 -> 192.168.234.2:1521
04/28/2019-19:30:32.790765  [Drop] [**] [1:2010936:3] ET SCAN Suspicious inbound to Oracle SQL port 1521 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:58442 -> 192.168.234.2:1521
04/28/2019-19:30:37.296643  [Drop] [**] [1:2010938:3] ET SCAN Suspicious inbound to mSQL port 4333 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:4333
04/28/2019-19:30:37.396789  [Drop] [**] [1:2010938:3] ET SCAN Suspicious inbound to mSQL port 4333 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:61320 -> 192.168.234.2:4333
04/28/2019-19:30:48.326169  [Drop] [**] [1:2002911:6] ET SCAN Potential VNC Scan 5900-5920 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:49166 -> 192.168.234.2:5911
04/28/2019-19:34:17.610651  [Drop] [**] [1:2010935:3] ET SCAN Suspicious inbound to MSSQL port 1433 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:59902 -> 192.168.234.2:1433
04/28/2019-19:34:17.710716  [Drop] [**] [1:2010935:3] ET SCAN Suspicious inbound to MSSQL port 1433 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:59922 -> 192.168.234.2:1433
04/28/2019-19:34:57.626548  [Drop] [**] [1:2002910:6] ET SCAN Potential VNC Scan 5800-5820 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:5815
04/28/2019-19:35:10.125533  [Drop] [**] [1:2002911:6] ET SCAN Potential VNC Scan 5900-5920 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:50138 -> 192.168.234.2:5913
04/28/2019-19:35:24.556865  [Drop] [**] [1:2010938:3] ET SCAN Suspicious inbound to mSQL port 4333 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:34786 -> 192.168.234.2:4333
04/28/2019-19:35:24.656951  [Drop] [**] [1:2010938:3] ET SCAN Suspicious inbound to mSQL port 4333 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:34806 -> 192.168.234.2:4333
04/28/2019-19:36:17.185843  [Drop] [**] [1:2002911:6] ET SCAN Potential VNC Scan 5900-5920 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:5915
04/28/2019-19:37:35.352564  [Drop] [**] [1:2002911:6] ET SCAN Potential VNC Scan 5900-5920 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:5906
04/28/2019-19:37:51.180214  [Drop] [**] [1:2010935:3] ET SCAN Suspicious inbound to MSSQL port 1433 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:1433
04/28/2019-19:37:51.280615  [Drop] [**] [1:2010935:3] ET SCAN Suspicious inbound to MSSQL port 1433 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:61320 -> 192.168.234.2:1433
04/28/2019-19:39:08.019905  [Drop] [**] [1:2002911:6] ET SCAN Potential VNC Scan 5900-5920 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:44594 -> 192.168.234.2:5912
04/28/2019-19:39:56.823824  [Drop] [**] [1:2002910:6] ET SCAN Potential VNC Scan 5800-5820 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:5807
04/28/2019-19:41:08.507580  [Drop] [**] [1:2010939:3] ET SCAN Suspicious inbound to PostgreSQL port 5432 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:49904 -> 192.168.234.2:5432
04/28/2019-19:41:08.607688  [Drop] [**] [1:2010939:3] ET SCAN Suspicious inbound to PostgreSQL port 5432 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:49924 -> 192.168.234.2:5432
04/28/2019-19:42:16.866023  [Drop] [**] [1:2002910:6] ET SCAN Potential VNC Scan 5800-5820 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:33490 -> 192.168.234.2:5810
04/28/2019-19:42:51.748971  [Drop] [**] [1:2002910:6] ET SCAN Potential VNC Scan 5800-5820 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:56290 -> 192.168.234.2:5813
04/28/2019-19:42:59.157461  [Drop] [**] [1:2010939:3] ET SCAN Suspicious inbound to PostgreSQL port 5432 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:5432
04/28/2019-19:42:59.257621  [Drop] [**] [1:2010939:3] ET SCAN Suspicious inbound to PostgreSQL port 5432 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:61320 -> 192.168.234.2:5432
04/28/2019-19:43:37.257884  [Drop] [**] [1:2002911:6] ET SCAN Potential VNC Scan 5900-5920 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:58816 -> 192.168.234.2:5908
04/28/2019-19:43:43.661114  [Drop] [**] [1:2002910:6] ET SCAN Potential VNC Scan 5800-5820 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:48320 -> 192.168.234.2:5819
04/28/2019-19:44:36.271049  [Drop] [**] [1:2002911:6] ET SCAN Potential VNC Scan 5900-5920 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:5901
04/28/2019-19:45:34.852384  [Drop] [**] [1:2002910:6] ET SCAN Potential VNC Scan 5800-5820 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:5812
04/28/2019-19:47:59.106395  [Drop] [**] [1:2010936:3] ET SCAN Suspicious inbound to Oracle SQL port 1521 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:61319 -> 192.168.234.2:1521
04/28/2019-19:47:59.206005  [Drop] [**] [1:2010936:3] ET SCAN Suspicious inbound to Oracle SQL port 1521 [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.234.3:61320 -> 192.168.234.2:1521

the test is not finished yet and there is also no prevention (FW DROPs) in it but a checkout if it is works.
Have checked also a little the needed resources with Netdata and made a very short video which compares Suricata, Squid, Ntopng and Ossec which are running on this machine --> https://fireinfo.ipfire.org/profile/90b ... f110293570 while the Nmap scan. For those how are interested --> https://people.ipfire.org/~ummeegge/vid ... _check.mp4 (Suricata settings are at the end findable).

Not sure if this is useful since the use cases can really really differs.

Best,

UE
Image
Image

Hellfire
Posts: 695
Joined: November 8th, 2015, 8:54 am

Re: Suricata much worse than guardian?.

Post by Hellfire » April 28th, 2019, 6:50 pm

Little OT: looking at the Suricata rules at the end of the video, it seems switching them on/off again is a PITA as it is currently for Snort, right?
Which means enabling, let's say a group on top level, does not enable *all* second level rules below.

Would be helpful if this could be changed while introducing Suricata...
Image

Stefan87
Posts: 74
Joined: July 20th, 2017, 11:55 pm

Re: Suricata much worse than guardian?.

Post by Stefan87 » April 28th, 2019, 9:24 pm

Nice how I access the suricata web interface

User avatar
Roberto Peña
Posts: 761
Joined: July 16th, 2014, 3:56 pm
Location: Bilbao (España)
Contact:

Re: Suricata much worse than guardian?.

Post by Roberto Peña » April 29th, 2019, 5:49 am

Thanks ummeegge for your reply.

Yes, it seems to work and only uses 2% of CPU. I find it amazing.

Code: Select all

...
04/29/2019-07:16:14.528442  [Drop] [**] [1:2403392:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 93 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 88.6.51.155:19230 -> 192.168.0.2:23
04/29/2019-07:17:06.086411  [Drop] [**] [1:2402000:5161] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 185.254.122.17:48653 -> 192.168.0.2:50102
04/29/2019-07:17:30.859977  [Drop] [**] [1:2402000:5161] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 185.176.27.26:42096 -> 192.168.0.2:3034
04/29/2019-07:17:50.132402  [Drop] [**] [1:2403397:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 98 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 91.237.40.203:51270 -> 192.168.0.2:23
04/29/2019-07:20:34.526824  [Drop] [**] [1:2402000:5161] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 185.176.27.242:45152 -> 192.168.0.2:39791
04/29/2019-07:20:34.971253  [Drop] [**] [1:2403337:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 38 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 46.232.112.18:55793 -> 192.168.0.2:7012
04/29/2019-07:21:32.181074  [Drop] [**] [1:2402000:5161] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 185.176.27.86:41639 -> 192.168.0.2:34891
04/29/2019-07:23:09.973605  [Drop] [**] [1:2402000:5161] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 185.176.27.14:41976 -> 192.168.0.2:3301
04/29/2019-07:23:40.492244  [Drop] [**] [1:2402000:5161] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 81.22.45.229:43023 -> 192.168.0.2:30503
04/29/2019-07:23:40.492244  [Drop] [**] [1:2403378:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 79 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 81.22.45.229:43023 -> 192.168.0.2:30503
04/29/2019-07:25:01.250939  [Drop] [**] [1:2403376:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 77 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 80.82.77.33:29011 -> 192.168.0.2:50050
04/29/2019-07:29:30.780633  [Drop] [**] [1:2403352:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 53 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 61.147.96.101:45240 -> 192.168.0.2:60001
04/29/2019-07:34:05.441913  [Drop] [**] [1:2403394:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 95 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 89.248.174.193:39854 -> 192.168.0.2:1070
04/29/2019-07:34:45.109243  [Drop] [**] [1:2402000:5161] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 81.22.45.231:43880 -> 192.168.0.2:441
04/29/2019-07:34:45.109243  [Drop] [**] [1:2403378:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 79 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 81.22.45.231:43880 -> 192.168.0.2:441
04/29/2019-07:36:38.520281  [Drop] [**] [1:2403398:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 99 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 92.118.37.84:44433 -> 192.168.0.2:806
04/29/2019-07:37:11.341610  [Drop] [**] [1:2403375:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 76 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 80.82.64.127:45287 -> 192.168.0.2:2210
04/29/2019-07:37:30.891501  [Drop] [**] [1:2403398:48799] ET CINS Active Threat Intelligence Poor Reputation IP group 99 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 92.118.37.86:44053 -> 192.168.0.2:21704
Only I have marked the following groups:

Code: Select all

%YAML 1.1
---

#Autogenerated file. Any custom changes will be overwritten!
 - whitelist.rules
 - emerging-p2p.rules
 - dshield.rules
 - compromised.rules
 - drop.rules
 - emerging-attack_response.rules
 - emerging-worm.rules
 - emerging-malware.rules
 - emerging-exploit.rules
 - emerging-trojan.rules
 - emerging-deleted.rules
 - botcc.rules
 - tor.rules
 - emerging-inappropriate.rules
 - ciarmy.rules
 - emerging-mobile_malware.rules
 - botcc.portgrouped.rules
 - emerging-dos.rules
 - emerging-current_events.rules
I will have to do more tests since it seems strange to me that after performing a downgrade and reapplying the 131 has gone from using 180% of the CPU to 2%.

Greetings.
Image
Image

╔════════════════════════════════════════════════╗
Donate to improve IPFire: https://www.ipfire.org/donate
╚════════════════════════════════════════════════╝

ummeegge
Community Developer
Community Developer
Posts: 4990
Joined: October 9th, 2010, 10:00 am

Re: Suricata much worse than guardian?.

Post by ummeegge » April 29th, 2019, 6:15 am

Hi all,
it seems a little like a problem carrousel at the moment which is in the testing phase the best time in my opinion :) . Haven´t encountered any problems with the system resources but have had the same problem Roberto in here --> viewtopic.php?f=50&p=124109#p124014 where i also replayed --> viewtopic.php?f=50&p=124109#p124109 to document this a little further. It seems that a change of the rules can delete the entire ruleset under /var/lib/suricata am currently not sure why that´s happened...
Hellfire wrote:
April 28th, 2019, 6:50 pm
Little OT: looking at the Suricata rules at the end of the video, it seems switching them on/off again is a PITA as it is currently for Snort, right?
Which means enabling, let's say a group on top level, does not enable *all* second level rules below.

Would be helpful if this could be changed while introducing Suricata...
i don´t think that this is what i personally want i think the activated/deactivated sets are a kind of best practice from the Suricata team and enabling all can be a PITA for the whole system (OOM) if it is an extensive ruleset and a weaker board.
What i think might be great if a rule update survives the individual selected rules, as far as i remember was this a problem with Snort ?! But as far as i remember it was a Oinkmaster thing which the current Suricata uses too...
Stefan87 wrote:
April 28th, 2019, 9:24 pm
Nice how I access the suricata web interface
It is located under the "Firewall" tab in the WUI.

Best,

UE
Image
Image

User avatar
Arne.F
Core Developer
Core Developer
Posts: 8516
Joined: May 7th, 2006, 8:57 am
Location: BS <-> NDH
Contact:

Re: Suricata much worse than guardian?.

Post by Arne.F » April 29th, 2019, 7:33 am

Little OT: looking at the Suricata rules at the end of the video, it seems switching them on/off again is a PITA as it is currently for Snort, right?
Which means enabling, let's say a group on top level, does not enable *all* second level rules below.
The default config of a rule group was delivered in the rule files itself. Some of them are for very specific use cases only and has much false positives so it is inteded that this rules are not enabled at default.
Arne

Support the project on the donation!

Image

Image

Image
PS: I will not answer support questions via email and ignore IPFire related messages on my non IPFire.org mail addresses.

Stefan87
Posts: 74
Joined: July 20th, 2017, 11:55 pm

Re: Suricata much worse than guardian?.

Post by Stefan87 » April 29th, 2019, 8:02 am

I want to activate everything have enough resources
I hope that not all my traffic is blocked ?

User avatar
Arne.F
Core Developer
Core Developer
Posts: 8516
Joined: May 7th, 2006, 8:57 am
Location: BS <-> NDH
Contact:

Re: Suricata much worse than guardian?.

Post by Arne.F » April 29th, 2019, 12:38 pm

Some rules like *_info and *_deleted for example should not enabled. They block normal traffic.

*_info are for testing purposes and contain many protocol decoders.
*_deleted are old rules that are removed from the other rulesets.

There are also other groups like *_policy must reviewed every rule before enabling or valid connections may blocked.
Arne

Support the project on the donation!

Image

Image

Image
PS: I will not answer support questions via email and ignore IPFire related messages on my non IPFire.org mail addresses.

DJ-Melo
Posts: 672
Joined: July 8th, 2014, 7:12 am

Re: Suricata much worse than guardian?.

Post by DJ-Melo » April 30th, 2019, 2:35 pm

google found this https://suricata.readthedocs.io/en/suri ... onfig.html is this an option for IPfire?

User avatar
Arne.F
Core Developer
Core Developer
Posts: 8516
Joined: May 7th, 2006, 8:57 am
Location: BS <-> NDH
Contact:

Re: Suricata much worse than guardian?.

Post by Arne.F » April 30th, 2019, 3:22 pm

Here the part of the IPFire config.

Code: Select all

detect:
    profile: custom
    custom-values:
        toclient-groups: 200
        toserver-groups: 200
    sgh-mpm-context: auto
    inspection-recursion-limit: 3000
Looks the same for me
Arne

Support the project on the donation!

Image

Image

Image
PS: I will not answer support questions via email and ignore IPFire related messages on my non IPFire.org mail addresses.

DJ-Melo
Posts: 672
Joined: July 8th, 2014, 7:12 am

Re: Suricata much worse than guardian?.

Post by DJ-Melo » April 30th, 2019, 3:28 pm

thanks. I have missed the Fire 16 GB Ram and the rules again optimized more than good 100 mbit are not in it that's a pity but ok in August T-Com delivers anyway only max 50 Mbit

User avatar
H&M
Posts: 467
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Suricata much worse than guardian?.

Post by H&M » July 21st, 2019, 4:03 pm

For what is worth,

I am loading in Suricata *all* rules from *both* ET and Talos (Suricata needs 1GB or RAM to do that) and still no problems with my speed on an APU2 machine. I am geting the same as I did before.
BUT!
Suricata seems to be Behing GeoIP chain and because I blocked all countries except 3, Suricata might get little from what FW already puts down?

Bottom line: you can go against all recommendation and activate all possible rules from many vendors, the box will continue to work more than OK.
I can sustain 100Mbit/s trafic with Interner destination while on Internal I am pushing Lan-to-Lan through IPFire another 130 Mbit/s.

Not a single problem for an APU2!
And this APU also has activated layer 2 switching (bridgectr) and is acting as layer 2 switch toward the NAS...
It should be no problems even after Suricata is loaded with so many rules (against all best practices)...

Hope it helps
H&M

Post Reply