some valid domains getting blocked

General questions.
axel2078
Posts: 294
Joined: January 30th, 2013, 3:53 am
Location: IL, USA

Re: some valid domains getting blocked

Post by axel2078 » September 19th, 2019, 2:19 pm

Alorotom wrote:
September 19th, 2019, 6:37 am
Hello axel2078,

since your IPFire is on VMWare, why not set up a second VM with a straight standard installation based on the current image. Just to check if the problems occure also on that or not. This could narrow it down to your productive VM and it's configuration or to an IPFire problem in general (maybe in a specific constellation).

If you query this bulletin board you'll find some more fellow sufferer. As far as I know, this has never been traced down to the clear reason. As I experienced this behaviour I could at least figure it out as a DNS problem. It did for some reason not resolve the target domain (combination of unbound / DNSSEC / german Telekom DNS). Just now all your provided links resolve fine for me but e.g. https://worldclassroom.webster.edu/login/ldap is not only worldclassroom.webster.de but also minimum four other domains: cloudfront.net, google-analytics.com, gstatic.com and instructure-uploads.s3.amazonaws.com. So if not all of them load correct, the site might time out although you can ping worldclassroom.webster.edu.

This said you could also give a try to configure other DNS Servers for IPFire.

Regards
Alorotom
Thank you for your response. I have actually done everything you suggested. The problem first started occurring when I upgraded from Core 133 to 134. I thought about restoring to a previous snapshot, but the last snapshot I took was running Core 129. I tried it anyway and restored the snapshot back to Core 129, but there was no change...I was still running into the same issues. So, since Core 135 was the latest version available, I downloaded the Core 135 iso image and built a new VM with it and then powered off the original one and moved the new one into place. Everything worked great for about a week and then the problem started up again. I have also tried changing my DNS servers multiple times. I was originally using Quad 9's DNS service (9.9.9.9) because it has always worked for me, but I also switched to 8.8.8.8 and 8.8.4.4 and the result was no different. Some web pages still won't load in a browser even if I can ping them and wget seems to indicate a successful connection.

IPfire has always been rock solid for me until the last couple months. Now, it's becoming more of a hindrance. I hate to say it, but I may have to abandon IPfire and go back to relying on my wireless router. I will lose a lot of functionality that IPfire offers, but at least I can get websites to load reliably.
Image

Alorotom
Posts: 429
Joined: March 30th, 2015, 6:56 am

Re: some valid domains getting blocked

Post by Alorotom » September 20th, 2019, 1:01 pm

axel2078 wrote:
September 19th, 2019, 2:19 pm
Now, it's becoming more of a hindrance. I hate to say it, but I may have to abandon IPfire and go back to relying on my wireless router. I will lose a lot of functionality that IPfire offers, but at least I can get websites to load reliably.
True. Also doubts on some recent changes with me. Hope, things improve.

My IPFire was on Core 131 and I just took the time to update to Core 135. So I can acknowledge that all your provided URLs still load fine. Core 135 does not seem to be the reason. But I'd still believe in a DNS related problem. What else to come into consideration? I'd bet unbound and DNSSEC will turn out finally.

Remember that unbound is a caching DNS. It forwards requests to the next node. All in the chain have to be DNSSEC validating. But some provider also intercept DNS traffic to redirect it to their own service. A Test that Arne has suggested many times:

Code: Select all

/etc/init.d/unbound test-name-server 81.3.27.46
The result has to be
Test failed for an unknown reason
Anything else would point to an DNS interception.

Also correct time should be checked. unbound / DNSSEC is dependent on correct time within very small divergence. If your system would run a minute ahead or behind, that could also be the culprit.

Hope that helps.
Regards
Alorotom
Image
Image

axel2078
Posts: 294
Joined: January 30th, 2013, 3:53 am
Location: IL, USA

Re: some valid domains getting blocked

Post by axel2078 » September 20th, 2019, 10:45 pm

Alorotom wrote:
September 20th, 2019, 1:01 pm
axel2078 wrote:
September 19th, 2019, 2:19 pm
Now, it's becoming more of a hindrance. I hate to say it, but I may have to abandon IPfire and go back to relying on my wireless router. I will lose a lot of functionality that IPfire offers, but at least I can get websites to load reliably.
True. Also doubts on some recent changes with me. Hope, things improve.

My IPFire was on Core 131 and I just took the time to update to Core 135. So I can acknowledge that all your provided URLs still load fine. Core 135 does not seem to be the reason. But I'd still believe in a DNS related problem. What else to come into consideration? I'd bet unbound and DNSSEC will turn out finally.

Remember that unbound is a caching DNS. It forwards requests to the next node. All in the chain have to be DNSSEC validating. But some provider also intercept DNS traffic to redirect it to their own service. A Test that Arne has suggested many times:

Code: Select all

/etc/init.d/unbound test-name-server 81.3.27.46
The result has to be
Test failed for an unknown reason
Anything else would point to an DNS interception.

Also correct time should be checked. unbound / DNSSEC is dependent on correct time within very small divergence. If your system would run a minute ahead or behind, that could also be the culprit.

Hope that helps.
Regards
Alorotom
This is what I get when I run that command:

root@ipfire-64 ~]# /etc/init.d/unbound test-name-server 81.3.27.46
Test failed for an unknown reason
[root@ipfire-64 ~]# /etc/init.d/unbound test-name-server 8.8.8.8
8.8.8.8 is validating
8.8.8.8 supports TCP fallback
EDNS buffer size for 8.8.8.8: 4096

That looks like expected behavior to me. I don't know what else to do. I still can't get to https://worldclassroom.webster.edu even though it responds to pings. I think I may have to try out a different firewall. :( It's really too bad. IPfire has been great for me until now.
Image

gpatel-fr
Posts: 51
Joined: July 24th, 2019, 7:59 am

Re: some valid domains getting blocked

Post by gpatel-fr » September 21st, 2019, 5:46 am

axel2078 wrote:
September 20th, 2019, 10:45 pm

root@ipfire-64 ~]# /etc/init.d/unbound test-name-server 81.3.27.46
Test failed for an unknown reason
[root@ipfire-64 ~]# /etc/init.d/unbound test-name-server 8.8.8.8
8.8.8.8 is validating
If you believe that DNS is the issue for you, just setup Google DNS and see what happens; Sounds extremely unlikely it will fix your problem, though - if the problem happens *after* connecting, DNS has nothing to do at this point, everything DNS linked happens before this step, if DNS don't work for you your system can't get the destination system address and so just can't even *attempt* to connect.

As of the 'unknown reason', what happens ? Running bash -x /etc/init.d/unbound test-name-server 81.3.27.46 reveals that it is running in fact

dig @81.3.27.46 A ipfire.org

it's testing that a computer that is NOT a dns server answers a dns query. Should not happen indeed.
If you use an open DNS server like 8.8.8.8, it works. Should happen if you run this from ipfire.
All this is not exactly news breaking stuff.

Anyway, your DNS queries being intercepted or not don't have anything to do with such a problem. If evil actors were targeting you, they would certainly avoid blocking you accessing your favourite web sites - if they were so clumsy, they would definitely not deserve your hard earned tax money.

Just try to set the MTU to 1300 on a station having the problem. On Linux, it's done with

ip link set <your-network-interface> mtu 1300

Alorotom
Posts: 429
Joined: March 30th, 2015, 6:56 am

Re: some valid domains getting blocked

Post by Alorotom » September 21st, 2019, 6:52 am

Hey guys, @axel2078 , the results show no DNS interception. Responses are as they should be.

@gpatel-fr , the 81.3.27.46 is on purpose NOT a DNS server. Search for posts of Arne (@Arne.F). It is a test he suggested several times. And of course you don't need to share my opinion that it could result somehow from DNS trouble. My attempt was to provide a possible reason, that could also cause such problems. I know that from own experience. It's good to sort that out though.

Since no one more familiar with the IPFire internals starts to investigate more into depth, especially the Devs not, we seem to run out of ideas here.

Regards
Alorotom
Image
Image

axel2078
Posts: 294
Joined: January 30th, 2013, 3:53 am
Location: IL, USA

Re: some valid domains getting blocked

Post by axel2078 » September 21st, 2019, 1:52 pm

Alorotom wrote:
September 21st, 2019, 6:52 am
Hey guys, @axel2078 , the results show no DNS interception. Responses are as they should be.

@gpatel-fr , the 81.3.27.46 is on purpose NOT a DNS server. Search for posts of Arne (@Arne.F). It is a test he suggested several times. And of course you don't need to share my opinion that it could result somehow from DNS trouble. My attempt was to provide a possible reason, that could also cause such problems. I know that from own experience. It's good to sort that out though.

Since no one more familiar with the IPFire internals starts to investigate more into depth, especially the Devs not, we seem to run out of ideas here.

Regards
Alorotom
Agreed. I am running out of ideas here because nothing seems to work. As far as DNS servers go, I've tried using all these:

9.9.9.9 (IBM's Quad 9)
8.8.8.8 (Google)
8.8.4.4 (Google)
208.67.222.222 (OpenDNS)

The result was the same in all of them. Some sites still time out in a browser even though I can ping them and do a wget on them. What's weird is that when I reinstalled IPfire using the Core 135 iso, everything worked fine for about a week and then the problem started again.
Image

axel2078
Posts: 294
Joined: January 30th, 2013, 3:53 am
Location: IL, USA

Re: some valid domains getting blocked

Post by axel2078 » September 22nd, 2019, 3:04 am

UPDATE:

Well, due to the intermittent web issues I've been having, I decided to give up on IPfire. I didn't really want to go back to relying on my wireless router though, so I decided to try out a different firewall. I am now running OPNsense virtually on my ESXi system. My installation is pretty vanilla so far since I just installed it, but I can say that I am not having any issues connecting to any websites. I am able to get to everything just fine. Now, the same thing happened the last time I rebuilt IPfire...it worked fine for a week...so only time will tell if I run into issues with OPNsense. I will post another update after using it for a while. Thanks to everyone who tried to help me troubleshoot.
Image

axel2078
Posts: 294
Joined: January 30th, 2013, 3:53 am
Location: IL, USA

Re: some valid domains getting blocked

Post by axel2078 » September 24th, 2019, 1:55 am

Well, after using OPNsense for a few days, I'm back to where I started. On the day I installed it, everything was working great. I could get to all my *webster.edu websites. The next day, I couldn't. They just time out in the browser again. Sometimes they respond to pings and sometimes they don't and sometimes I can get records for them via nslookup and sometimes I get a SERVFAIL error (no record found). However, access to every other website is blazing fast and without issue. As another test, I once again connected a computer straight to my cable modem and was able to browse all websites without issue. This rules out an ISP problem. The problem is inside my network. I thought that moving to a new firewall platform would solve the problem, but it didn't.
Image

JonM
Posts: 144
Joined: August 4th, 2017, 5:49 pm
Location: US

Re: some valid domains getting blocked

Post by JonM » September 24th, 2019, 6:45 am

from your signature it says Virtualized on VMWare... Is it possible to locate an old PC and build it up as new IPfire box (maybe without a restore) and see if the issues exist (or disappear) with a separate piece of hardware?
Production:
Image

Testing Raspi 3B+:
Image

jmpentney
Posts: 4
Joined: September 24th, 2019, 8:14 am

Re: some valid domains getting blocked

Post by jmpentney » September 24th, 2019, 8:25 am

I am having exactly the same problem, running IPFire Core 135 on a dedicated PC. I've tried turning off Intrusion Prevention, but it did not make any difference...

I use a Raspberry Pi running PiHole as my local DNS server and it seems to be working correctly. I find common websites blocked by some OS/browser configuration but not others e.g.

https://www.bbc.co.uk/news:

OK on Firefox on Arch Linux PC
Times out on Opera on same Arch Linux PC
Times out on Chrome on Windows 7 PC

Any further ideas about how to fix this issue?

axel2078
Posts: 294
Joined: January 30th, 2013, 3:53 am
Location: IL, USA

Re: some valid domains getting blocked

Post by axel2078 » September 24th, 2019, 11:24 am

jmpentney wrote:
September 24th, 2019, 8:25 am
I am having exactly the same problem, running IPFire Core 135 on a dedicated PC. I've tried turning off Intrusion Prevention, but it did not make any difference...

I use a Raspberry Pi running PiHole as my local DNS server and it seems to be working correctly. I find common websites blocked by some OS/browser configuration but not others e.g.

https://www.bbc.co.uk/news:

OK on Firefox on Arch Linux PC
Times out on Opera on same Arch Linux PC
Times out on Chrome on Windows 7 PC

Any further ideas about how to fix this issue?
Thank you for chiming in! It’s good to know I’m not just super unlucky. :). Perhaps if you find a fix, it will solve mine too.
Image

axel2078
Posts: 294
Joined: January 30th, 2013, 3:53 am
Location: IL, USA

Re: some valid domains getting blocked

Post by axel2078 » September 24th, 2019, 1:03 pm

JonM wrote:
September 24th, 2019, 6:45 am
from your signature it says Virtualized on VMWare... Is it possible to locate an old PC and build it up as new IPfire box (maybe without a restore) and see if the issues exist (or disappear) with a separate piece of hardware?
I don't have another physical machine to run this on. However, jmpentney mentioned above that he has a dedicated machine for his IPfire system and is running into the same issue. IPfire was rock solid for me running as a VM for years and I only ran into problems recently. I'm pretty sure it was after the Core 134 upgrade.
Image

jmpentney
Posts: 4
Joined: September 24th, 2019, 8:14 am

Re: some valid domains getting blocked

Post by jmpentney » September 24th, 2019, 10:29 pm

I have also only had this problem since upgrading to core135. I had been running core 131, and had not upgraded for a few months. I then allowed pakfire to bump me up to core 135 and immediately found this problem.

I've tried changing my NTP time server settings and update frequency, without success.

Curiously, I can sometime access "blocked" sites (such as bbc.co.uk/news) for a few seconds after restarting the web proxy service, but it is not reproducible and it is not long before they timeout again...

axel2078
Posts: 294
Joined: January 30th, 2013, 3:53 am
Location: IL, USA

Re: some valid domains getting blocked

Post by axel2078 » September 24th, 2019, 11:02 pm

I experienced times when the problematic sites would randomly start working again but it was only temporary. For some reason, I can’t get to the @webster.edu domains after Core 134. A complete rebuild solved it for a week but it came back.
Image

JonM
Posts: 144
Joined: August 4th, 2017, 5:49 pm
Location: US

Re: some valid domains getting blocked

Post by JonM » September 24th, 2019, 11:24 pm

jmpentney wrote:
September 24th, 2019, 10:29 pm
I've tried changing my NTP time server settings and update frequency, without success.
NTP always seemed "off" to me. To fix drift issues I added the servers to my ntp.conf:

Code: Select all

[root@ipfire ~]# cat /etc/ntp.conf
disable monitor
restrict default kod nomodify notrap nopeer
restrict 127.0.0.1
server  0.us.pool.ntp.org prefer
server  1.us.pool.ntp.org
server  127.127.1.0
fudge   127.127.1.0 stratum 10
driftfile /etc/ntp/drift
[root@ipfire ~]# 
Now all works as NTP should. Instead of the time being off by 12 to 20 seconds a day, now its less then 0.1 seconds. I was going to suggest this to @axel2078 since there are DNS (and pakfire) things that seem to be very time dependent.

see viewtopic.php?f=27&t=21861
Production:
Image

Testing Raspi 3B+:
Image

Post Reply