Ad blocking with DNS (unbound & dnsmasq)

Help on building IPFire & Feature Requests
bloater99
Posts: 403

Re: Ad blocking with DNS (unbound & dnsmasq)

Postby bloater99 » December 16th, 2016, 7:45 pm

mutley wrote:There are no rules with in this method, so no need to determine any rule that caused a block. But if there is a page or some content that's been blocked that you would prefer not to be blocked, the browser will give it to you the DNS / URL that's been blocked. Then just add that to your custom whitelist, and rebuild.
I actually do this with http://www.googleadservices.com, as my wife likes to click the first page of adds that comes back from a google search and just about every source lists http://www.googleadservices.com as ad-tracking (which it is).

I'm going to disagree with you here. Here is a specific example:
The current MVPS HOSTS file is running on my IPFIRE right now, via unbound. There is a website we use to view part spec sheets. One of the urls in MVPS HOSTS prevents these spec sheets from opening. The user gets no feedback from the browser as to which url is causing the spec sheet to not load. All they see is the page failed to load. They don't get any feedback about which url in MVPS HOSTS is causing the page to fail to load.

If IPFIRE's URL Filter was doing the blocking, then it would give more informative feedback via the IPFIRE Block Page. But URL Filter does not handle blocking ads well, because instead of just silently dropping the ads on the page while still showing the pages valid content, it pops up the IPFIRE block page, which confuses the user and slows down their workflow. In some cases, it completely stops their workflow because the user can't get to a page at all.

Blocking ads needs to be transparent in order to be effective. Hence, doing it via DNS makes more sense. Just with the caveat that it will be harder to determine which url is causing a false-positive.
Image

Image

mike-us
Posts: 48

Re: Ad blocking with DNS (unbound & dnsmasq)

Postby mike-us » December 17th, 2016, 2:33 am

In any case you can always use Browser Development Tools to display Ressource errors - this will show you the parts not loaded including the domain name or IP and even other errors.
At least for troubleshooting this is a valid option.

5p9
Mentor
Mentor
Posts: 1570

Re: Ad blocking with DNS (unbound & dnsmasq)

Postby 5p9 » March 18th, 2017, 1:10 am

Oh thats so pretty! Good job & very smooth to use this DNS blacklistblocker :D

But, got a few more questions. At which place can I set one log message for drop Siterequests in your Script?
Is this a part from unbound.conf in the line:

Code: Select all

        # Logging Options
        verbosity: 1


How i can block manually in /etc/unbound/local.d/blocklist.conf e.g. https://de-de.facebook.com

with http://www.heise.de is all okay and i see the timeoutsite, but not with facebook.com or de-de.facebook.com

e.g:

Code: Select all

local-data: "de-de.facebook.com A 127.0.0.1"


EDIT blacklist.conf:
#############
I have testing your nxdomain function (-r or --dns). It looks like good :D

Code: Select all

local-zone: "facebook.com" always_nxdomain


But how i can combine only my personaly blocklist.txt (-b myblocklist.txt) with the always_nxdomain?

I try to use disable the lists:

Code: Select all

#DEFAULT_SOURCES="1,2,3,12"
DEFAULT_SOURCES=""


and set this one:

Code: Select all

./dns_blocklist.sh -s black-test.txt -r always_nxdomain

or

Code: Select all

./dns_blocklist.sh -r facebook.com always_nxdomain


and nothing is set in my /etc/unbound/local.d/blocklist.conf.
############

And I'd wish ;D a Logscreen like http://forum.ipfire.org/viewtopic.php?t=11144#p72044 Is that possible with unbound?

Regards,
5p9

mutley
Posts: 30

Re: Ad blocking with DNS (unbound & dnsmasq)

Postby mutley » March 20th, 2017, 7:08 pm

5p9 wrote:EDIT blacklist.conf:
#############
I have testing your nxdomain function (-r or --dns). It looks like good :D

Code: Select all

local-zone: "facebook.com" always_nxdomain


But how i can combine only my personaly blocklist.txt (-b myblocklist.txt) with the always_nxdomain?

I try to use disable the lists:

Code: Select all

#DEFAULT_SOURCES="1,2,3,12"
DEFAULT_SOURCES=""


and set this one:

Code: Select all

./dns_blocklist.sh -s black-test.txt -r always_nxdomain

or

Code: Select all

./dns_blocklist.sh -r facebook.com always_nxdomain


and nothing is set in my /etc/unbound/local.d/blocklist.conf.
############

And I'd wish ;D a Logscreen like http://forum.ipfire.org/viewtopic.php?t=11144#p72044 Is that possible with unbound?

Regards,
5p9


I think you are mixing up the command line options.
For your own black list, create a txt file with domain names, and point to it with the -b options.
Example :- File /root/black-test.txt

Code: Select all

 
de-de.facebook.com
www.heise.de


Then run
./dns_blocklist.sh -b /root/black-test.txt

That should add de-de.facebook.com & www.heise.de to the blacklists.
Image

5p9
Mentor
Mentor
Posts: 1570

Re: Ad blocking with DNS (unbound & dnsmasq)

Postby 5p9 » March 21st, 2017, 7:09 am

Hey mutley,

thanks for replay. Okay i understand you, you meen i need two steps for the experimental nxdomain feature?
first step import my own blacklist (this point is clear for me) and then i must take the -r function?

I will block porn, warez, etc Lists from my own sourcefile (blacklistfile that i importing), but one more blocklistsite have one or more subdomains. e.g. facebook.com / de-de.facebook.com / login.facebook.com etc.
I will sort with the nxdoamins feature all those domains down to the minimum, and block everything under that.

What is the best way for this task?

Regards,
5p9

mutley
Posts: 30

Re: Ad blocking with DNS (unbound & dnsmasq)

Postby mutley » March 21st, 2017, 9:52 pm

Sorry, I wasn't clear. Every time the script runs it will re-do everything, it doesn't store any information between execution(s). So you need to pass everything you want to do on the command line. So to import a custom black list AND to do nxdomain you'll need to pass 2 command line parameters.

So create your custom blacklist, then run this

./dns_blocklist.sh -b /root/black-test.txt -r always_nxdomain

(obviously change /root/black-test.txt to your own blacklist)
Image

5p9
Mentor
Mentor
Posts: 1570

Re: Ad blocking with DNS (unbound & dnsmasq)

Postby 5p9 » March 23rd, 2017, 7:51 am

Hey,

its looks like great! :) Thx for your hint! But, unbound tell me not in the logfile (verbosity 5) anything he block sites from my blocklists. I see the timeoutsite in my browser but never more.

Is that normal?

My next step, i will importing my shella files (urls/domains) in the blocklist-file.txt and then i test it again.

EDIT:
I found one strange deviant behaviour! im edit my blocklist-file with 5 entriesites:

Code: Select all

pro-linux.de
golem.de
facebook.com
faq-o-matic.net
malwaremustdie.org


then i greate my nxdomainsite block:

Code: Select all

./dns_blocklist.sh -b black-test.txt -r always_nxdomain
Cleaning & Sorting list of 5 entries
Writing list of 4 entries to unbound nxdomain configuration
Stopping Unbound DNS Proxy...                                                                                                                                                                                                       [  OK  ]
Starting Unbound DNS Proxy...                                                                                                                                                                                                       [  OK  ]
Configuring upstream name server(s): 8.8.8.8 8.8.4.4                                                                                                                                                                                [  OK  ]
./dns_blocklist.sh: Blocked Hosts Update, 4 hosts blocked


and my last entry will never set it in the /etc/unbound/local.d/blocklist.conf

Code: Select all

server:
local-zone: "facebook.com" always_nxdomain
local-zone: "golem.de" always_nxdomain
local-zone: "pro-linux.de" always_nxdomain
local-zone: "faq-o-matic.net" always_nxdomain


I think there is a bug?! each additional entry is ignored in this list...

Regards,
5p9


Return to “Development”



Who is online

Users browsing this forum: Bing [Bot] and 2 guests