Page 1 of 2

Ipfblocklist (IP Blocklists for IPFire)

Posted: November 7th, 2018, 10:17 am
by raffe
Hi!

I tried to install https://github.com/timfprogs/ipfblocklist and it seems to work. I think.

I also have ipfidsupdate (see https://github.com/timfprogs/ipfidsupdate and viewtopic.php?f=27&t=20965) and one strange thing I have seen is that I have:
  • IDS Update - Configuration - Update rate = Daily
  • IP Blocklists - Configuration - Update check rate = Slow
But if I do fcrontab -l I see

Code: Select all

# Snort rule update
%hourly,nice(1),random,serialonce(true) 33-43 /usr/local/bin/ids-update.pl

# Snort rule update
%hourly,nice(1),random,serialonce(true) 36-46 /usr/local/bin/blocklist.pl
(I *think* that before I installed ipfblocklist ids-update.pl said daily)

I first changed IP Blocklists - Configuration - Update check rate to "Fast", to "get it started". After one hour I tried to toggle the time settings to daily/slow, but it don't change in cron.

But I don't get any e-mails every hour yet, so...

Could it be that ipfblocklist also changes cron settings on ipfidsupdate?

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: November 7th, 2018, 9:04 pm
by TimF
The latest versions of both scripts are set to run hourly, but they check to see if they actually have to do anything in the current hour.

This is due to the WUI running as nobody, but needing to be root to change the fcrontab. It would be possible to use a setuid helper script to do the job, but in this case it's just simpler to run the script hourly and let it work out whether to do anything.

Note that when you start running the script, it won't download the blocklists immediately. It'll start downloading them the next time it runs, but it may be some hours before it's downloaded everything - it spreads downloads out to reduce the impact on network. This is not a problem when you reboot, because it saves the downloaded rules and reloads them all on startup.

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: November 8th, 2018, 9:16 am
by raffe
Thanks for your very fast answers! :)

Two more questions.

1. Do I have a problem with updates?
Today it looks like this in my settings
Image
"pkts", "bytes" and "Last updated" are all empty and I thought that there would be some info there by now.

2. How do I see that IPFire is blocking things with the help of the Blocklists?
Is there somewhere where I can see the might of the blocklists that I have enabled? Is there a log where I can see them doing their magic (like IDS logs) or a way that I can see that they really are enabled and are working? In System Logs I have checked, "URLFilter Blacklits", "Kernel", "RED" and so on.
The updates I guess I can see when the "Last updated" changes.

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: November 8th, 2018, 1:17 pm
by Saiyato
raffe wrote:
November 8th, 2018, 9:16 am
Thanks for your very fast answers! :)

Two more questions.

1. Do I have a problem with updates?
Did you download the latest script from github? I created a PR yesterday, there was a merge conflict, which prevented the lists to be downloaded. Removes lines 667-669 and you should be fine.

Additionally I found that after installation no NAT took place on the IPFire instance, but since I was configuring a lot of stuff, this might be unrelated. A reboot fixed the NAT problem btw.
raffe wrote:
November 8th, 2018, 9:16 am
2. How do I see that IPFire is blocking things with the help of the Blocklists?
Is there somewhere where I can see the might of the blocklists that I have enabled? Is there a log where I can see them doing their magic (like IDS logs) or a way that I can see that they really are enabled and are working? In System Logs I have checked, "URLFilter Blacklits", "Kernel", "RED" and so on.
The updates I guess I can see when the "Last updated" changes.
Apart from the overview, I think your only option is to check the values in iptables:

Code: Select all

iptables -L {CHAIN}
Exempli gratia

Code: Select all

iptables -L TOR_ALL

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: November 9th, 2018, 6:25 pm
by TimF
Thanks Saiyato, I've merged the change.

Raffe,

If you download the script again, you will hopefully see the 'pkts', bytes' and 'Last updated' fields start to fill with information; although this won't happen until the script starts to run (and it could take a few days for the last blocklists to be downloaded - they're not updated very often, so the script doesn't look for changes very often.

You should also see the added blocklists in the WUI under 'Firewall' > 'iptables' - look at the INPUT chain. 'Logs' > 'Firewall logs' will list the chain that's blocking the packet and 'Logs' > 'Summary' should have an added item listing the downloads on the previous day.

If it doesn't start downloading the blocklists, try running

Code: Select all

/usr/local/bin/blocklist.pl
from the command line - it should detect it's running on a terminal and output debugging information.

If you want to see what the blocked IP addresses are, you can either look at the files in /etc/ipset/blocklist or run

Code: Select all

ipset -n list
which lists the set names or

Code: Select all

ipset list [i]<name>[/i]
to list the contents of the set.

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: November 30th, 2018, 10:41 am
by Saiyato
I just came to the conclusion that using BOGON_FULL blocks 192.168.0.0/16, i.e. you can't access machines behind IPFire in that network as well ;) It's a two-way block.

Code: Select all

11:28:53	DROP_BOGON_FULL	red0	TCP	192.168.178.x	192.168.178.y	80(HTTP)
In my case I can't reach the Fritzbox modem, not too much of a problem, but thought I'd share my experience. I'm pretty sure I can manually fix this, but if I were to, I need a persistent solution, adding a firewall rule didn't help. I think the change is overwritten in a later chain.

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: December 4th, 2018, 8:35 pm
by PhilTheHill
Saiyato wrote:
November 30th, 2018, 10:41 am
I just came to the conclusion that using BOGON_FULL blocks 192.168.0.0/16, i.e. you can't access machines behind IPFire in that network as well ;) It's a two-way block.

Code: Select all

11:28:53	DROP_BOGON_FULL	red0	TCP	192.168.178.x	192.168.178.y	80(HTTP)
In my case I can't reach the Fritzbox modem, not too much of a problem, but thought I'd share my experience. I'm pretty sure I can manually fix this, but if I were to, I need a persistent solution, adding a firewall rule didn't help. I think the change is overwritten in a later chain.
Have come against the same problem with the 10.0.0.0/8 network as behind my home router I use
Green: 10.0.0.0/24
Red: 10.0.1.0/24
Orange: 10.0.2.0/24

It's not pretty (and I'm a complete novice with perl) but I made a small change to the blocklist.pl script as follows.

Code: Select all

[root@ipfire ~]# diff blocklist.pl.BACKUP /usr/local/bin/blocklist.pl
597c597,605
<       push @new_blocklist, $address;
---
>       if ( $address == "10.0.0.0/8")
>       {
>         log_message LOG_INFO, "Doing fixup on $chain to ignore 10.0.0.0/8 network";
>       }
>       else
>       {
>         push @new_blocklist, $address;
>       }
>
[root@ipfire ~]#
This will of course work for any blocklist, not just BOGON_FULL but hope it's useful.

And many thanks to TimF for creating ipfblocklist and ipfidsupdate which have both proved extemely useful addons for my IPFire setup. (I will be probably try ipfstatusmail in due course but that's for another day.)

PhilTheHill

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: December 4th, 2018, 10:07 pm
by TimF
The BOGON list blocks addresses that should never be seen on the internet. This includes the private address spaces (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), loopback (127.0.0.0/8), multicast (224.0.0.0/4), test (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24) and some others. It also includes address blocks that have not been assigned by the IANA (since all blocks have been assigned, this is now a null list).

The BOGON FULL list adds addresses that haven't been assigned by the Regional Internet Registries (there are still some of these).

If you're using an external modem and IPFire is handling PPP there should be no problems with using either of these lists, but if your connection to the internet is via a router then that will almost certainly have an address in one of the private ranges, and hence using BOGON or BOGON FULL will block access to the internet. To put it another way, if IPFIre is connecting via ppp0 you should be OK, if it's using red0 you probably won't.

The good news is that a router should block everything on the BOGON list anyway.

I'm thinking of adding code that will prevent the red/ppp interface address from being blocked, but this has to take account of the fact that it could have a single address or a network and also that the address could change periodically (if assigned by PPP or DHCP). PhilTheHill's fix to ignore a network looks OK, as long as you know that the address will remain fixed.

I'm afraid that the change to ignore the red/ppp address is not at the top of my todo list. However it's intended to add this functionality into IPFire itself; when that happens it will include the change. It'll probably be a few months so it can be thoroughly checked.

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: December 5th, 2018, 2:10 pm
by Saiyato
Thanks for the explanation. :) I thought it would block incoming connections only, but it's a two-way block. Which - when I come to think of it - actually makes sense, you don't want any incoming traffic from compromised systems, but you also don't want any outgoing connection (exfiltrating data or something).

I get that this isn't on the top of your todo list, you might even not want to allow the subnet red is in... Maybe an exclusion for just one IP or a subnet would be better, this is something the users can fill in themselves and it's their responsibility to maintain the setting. I'm going to see if I can get this to work, I suppose translating the IP to a range is hard part. The easy way would be to have users fill in a range that actually exists/makes sense when using those block lists.

So there's a few options:

1. omit the addition based on a subnet -> create a field in the addon
2. omit the addition based on a single IP -> create a field in the addon
3. change the addition based on a single IP; i.e. allow for a single IP but block the rest -> create a field in the addon
4. any of the above, but automatically; i.e. use the IP-address known for red and add a checkbox for allow the whole subnet or something like that

Obviously I'm kind of new to the world of both firewalls like IPFire and Perl, so any advice/pointers in terms of security is more than welcome.

And idd, thanks to TimF for providing an already great solution!

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: December 28th, 2018, 10:35 pm
by Mentalic
Just setup the Ipfblocklist without issues. After the rules updated I see that the order of operation is not what I had expected it would be. Specifically it seems the Ipfblocklist is taking place before the GeoIP block system. This results in an huge increase in entries in the firewall log for countries I had already blocked in the GeoIP block. So I disabled the Ipfblocklist as my goal was to reduce the noise in the firewall log.

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: December 29th, 2018, 4:34 pm
by TimF
The plan is to include this functionality into IPFire. As part of this I'll review the best place to put the blocklists.

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: December 29th, 2018, 10:25 pm
by Mentalic
Thanks TimF,

Adding this into IpFire would be great. Another thing I noticed is when I disabled the IPfblocklist it appears to not really disable it or broke the geoip block. Kept seeing entries for countries I had blocked appear in the firewall log. Ended up reinstalling Ipfire to clear it out.

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: December 30th, 2018, 3:19 am
by dnl
TimF wrote:
December 29th, 2018, 4:34 pm
The plan is to include this functionality into IPFire.
Awesome! 8)

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: February 10th, 2019, 10:44 pm
by cbrown
FWIW, in case you did not see this posting: viewtopic.php?f=52&t=22266

Charles

Re: Ipfblocklist (IP Blocklists for IPFire)

Posted: March 19th, 2019, 2:53 pm
by sardonico
The CINS Army List list can no longer be downloaded from http://cinsscore.com/list/ci-badguys.txt, now need to subscribe to the page:
https://cinsarmy.com/list-download/