unbound - DoT

Help on building IPFire & Feature Requests
tikok974
Posts: 61
Joined: January 3rd, 2017, 9:53 am

Re: unbound - DoT

Post by tikok974 » May 15th, 2019, 9:30 am

Hi Ummeegge,

Thank you for your answers ;)

I reintegrated the patch of Ryan Jaeb found here: https://gitlab.com/snippets/1706804 after I deleted yours and it works !...all my DNS requests are filtered by DNSfilter.

For Unbound: /etc/unbound/unbound.conf

Code: Select all

#
# Unbound configuration file for IPFire
#
# The full documentation is available at:
# https://www.unbound.net/documentation/unbound.conf.html
#

server:
	# Common Server Options
	chroot: ""
	directory: "/etc/unbound"
	username: "nobody"
	port: 53
	do-ip4: yes
	do-ip6: no
	do-udp: yes
	do-tcp: yes
	so-reuseport: yes
	do-not-query-localhost: yes

	# System Tuning
	include: "/etc/unbound/tuning.conf"

	# Logging Options
	verbosity: 1
	use-syslog: yes
	log-time-ascii: yes
	log-queries: no

	# Unbound Statistics
	statistics-interval: 86400
	statistics-cumulative: yes
	extended-statistics: yes

	# Prefetching
	prefetch: yes
	prefetch-key: yes

	# Randomise any cached responses
	rrset-roundrobin: yes

	# Privacy Options
	hide-identity: yes
	hide-version: yes
	qname-minimisation: yes
	minimal-responses: yes

	# DNSSEC
	auto-trust-anchor-file: "/var/lib/unbound/root.key"
	val-permissive-mode: no
	val-clean-additional: yes
	val-log-level: 1

	# Hardening Options
	harden-glue: yes
	harden-short-bufsize: no
	harden-large-queries: yes
	harden-dnssec-stripped: yes
	harden-below-nxdomain: yes
	harden-referral-path: yes
	harden-algo-downgrade: no
	use-caps-for-id: yes
	aggressive-nsec: yes

	# Harden against DNS cache poisoning
	unwanted-reply-threshold: 1000000

	# Listen on all interfaces
	interface-automatic: yes
	interface: 0.0.0.0

	# Allow access from everywhere
	access-control: 0.0.0.0/0 allow

	# Bootstrap root servers
	root-hints: "/etc/unbound/root.hints"

	# Include DHCP leases
	include: "/etc/unbound/dhcp-leases.conf"

	# Include any forward zones
	include: "/etc/unbound/forward.conf"

remote-control:
	control-enable: yes
	control-use-cert: no
	control-interface: 127.0.0.1

# Import any local configurations
include: "/etc/unbound/local.d/*.conf"
For Unbound: /etc/sysconfig/unbound ...is empty.

For Unbound: /etc/init.d/unbound

Code: Select all

#!/bin/sh
# Begin $rc_base/init.d/unbound

# Description : Unbound DNS resolver boot script for IPfire
# Author      : Marcel Lorenz <marcel.lorenz@ipfire.org>

. /etc/sysconfig/rc
. ${rc_functions}

TEST_DOMAIN="ipfire.org"

# This domain will never validate
TEST_DOMAIN_FAIL="dnssec-failed.org"

INSECURE_ZONES=
USE_FORWARDERS=1
ALLOW_BROKEN_FORWARDERS=1

# Cache any local zones for 60 seconds
LOCAL_TTL=60

# EDNS buffer size
EDNS_DEFAULT_BUFFER_SIZE=4096

function cidr() {
    local cidr nbits IFS;
    IFS=. read -r i1 i2 i3 i4 <<< ${1}
    IFS=. read -r m1 m2 m3 m4 <<< ${2}
    cidr=$(printf "%d.%d.%d.%d\n" "$((i1 & m1))" "$((i2 & m2))" "$((i3 & m3))" "$((i4 & m4))")
    nbits=0
    IFS=.
    for dec in $2 ; do
        case $dec in
            255) let nbits+=8;;
            254) let nbits+=7;;
            252) let nbits+=6;;
            248) let nbits+=5;;
            240) let nbits+=4;;
            224) let nbits+=3;;
            192) let nbits+=2;;
            128) let nbits+=1;;
            0);;
            *) echo "Error: $dec is not recognised"; exit 1
        esac
    done
    echo "${cidr}/${nbits}"
}

ip_address_revptr() {
	local addr=${1}

	local a1 a2 a3 a4
	IFS=. read -r a1 a2 a3 a4 <<< ${addr}

	echo "${a4}.${a3}.${a2}.${a1}.in-addr.arpa"
}

read_name_servers() {
	local i
	for i in 1 2; do
		echo "$(</var/ipfire/red/dns${i})"
	done 2>/dev/null | xargs echo
}

config_header() {
	echo "# This file is automatically generated and any changes"
	echo "# will be overwritten. DO NOT EDIT!"
	echo
}

update_forwarders() {
	if [ "${USE_FORWARDERS}" = "1" -a -e "/var/ipfire/red/active" ]; then
		local forwarders
		local broken_forwarders

		local ns
		for ns in $(read_name_servers); do
			test_name_server ${ns} &>/dev/null
			case "$?" in
				# Only use DNSSEC-validating or DNSSEC-aware name servers
				0|2)
					forwarders="${forwarders} ${ns}"
					;;
				*)
					broken_forwarders="${broken_forwarders} ${ns}"
					;;
			esac
		done

		# Determine EDNS buffer size
		local new_edns_buffer_size=${EDNS_DEFAULT_BUFFER_SIZE}

		for ns in ${forwarders}; do
			local edns_buffer_size=$(ns_determine_edns_buffer_size ${ns})
			if [ -n "${edns_buffer_size}" ]; then
				if [ ${edns_buffer_size} -lt ${new_edns_buffer_size} ]; then
					new_edns_buffer_size=${edns_buffer_size}
				fi
			fi
		done

		if [ ${new_edns_buffer_size} -lt ${EDNS_DEFAULT_BUFFER_SIZE} ]; then
			boot_mesg "EDNS buffer size reduced to ${new_edns_buffer_size}" ${WARNING}
			echo_warning

			unbound-control -q set_option edns-buffer-size: ${new_edns_buffer_size}
		fi

		# Show warning for any broken upstream name servers
		if [ -n "${broken_forwarders}" ]; then
			boot_mesg "Ignoring broken upstream name server(s): ${broken_forwarders:1}" ${WARNING}
			echo_warning
		fi

		if [ -n "${forwarders}" ]; then
			boot_mesg "Configuring upstream name server(s): ${forwarders:1}" ${INFO}
			echo_ok

			# Make sure DNSSEC is activated
			enable_dnssec

			echo "${forwarders}" > /var/ipfire/red/dns
			unbound-control -q forward ${forwarders}
			return 0

		# In case we have found no working forwarders
		else
			# Test if the recursor mode is available
			if [ ${ALLOW_BROKEN_FORWARDERS} != "1" ] && can_resolve_root +bufsize=${new_edns_buffer_size}; then
				# Make sure DNSSEC is activated
				enable_dnssec

				boot_mesg "Falling back to recursor mode" ${WARNING}
				echo_warning

			# If not, we set DNSSEC in permissive mode and allow using all recursors
			elif [ -n "${broken_forwarders}" ]; then
				disable_dnssec

				if [ ${ALLOW_BROKEN_FORWARDERS} = "1" ]; then
					boot_mesg "DNSSEC has been set to permissive mode" ${WARNING}
					echo_warning

					boot_mesg "Unignoring broken upstream name server(s): ${broken_forwarders:1}" ${WARNING}
					echo_warning
				else
					boot_mesg "DNSSEC has been set to permissive mode" ${FAILURE}
					echo_failure
				fi

				echo "${broken_forwarders}" > /var/ipfire/red/dns
				unbound-control -q forward ${broken_forwarders}
				return 0
			fi
		fi
	fi

	# If forwarders cannot be used we run in recursor mode
	echo "local recursor" > /var/ipfire/red/dns
	unbound-control -q forward off
}

own_hostname() {
	local hostname=$(hostname -f)
	# 1.1.1.1 is reserved for unused green, skip this
	if [ -n "${GREEN_ADDRESS}" -a "${GREEN_ADDRESS}" != "1.1.1.1" ]; then
		unbound-control -q local_data "${hostname} ${LOCAL_TTL} IN A ${GREEN_ADDRESS}"
	fi

	local address
	for address in ${GREEN_ADDRESS} ${BLUE_ADDRESS} ${ORANGE_ADDRESS}; do
		[ -n "${address}" ] || continue
		[ "${address}" = "1.1.1.1" ] && continue

		address=$(ip_address_revptr ${address})
		unbound-control -q local_data "${address} ${LOCAL_TTL} IN PTR ${hostname}"
	done
}

update_hosts() {
	local enabled address hostname domainname

	while IFS="," read -r enabled address hostname domainname; do
		[ "${enabled}" = "on" ] || continue

		# Build FQDN
		local fqdn="${hostname}.${domainname}"

		unbound-control -q local_data "${fqdn} ${LOCAL_TTL} IN A ${address}"

		# Skip reverse resolution if the address equals the GREEN address
		[ "${address}" = "${GREEN_ADDRESS}" ] && continue

		# Add RDNS
		address=$(ip_address_revptr ${address})
		unbound-control -q local_data "${address} ${LOCAL_TTL} IN PTR ${fqdn}"
	done < /var/ipfire/main/hosts
}

write_forward_conf() {
	(
		config_header

		local insecure_zones="${INSECURE_ZONES}"

		local enabled zone server remark
		while IFS="," read -r enabled zone server remark; do
			# Line must be enabled.
			[ "${enabled}" = "on" ] || continue

			# Zones that end with .local are commonly used for internal
			# zones and therefore not signed
			case "${zone}" in
				*.local)
					insecure_zones="${insecure_zones} ${zone}"
					;;
			esac

			# Reverse-lookup zones must be stubs
			case "${zone}" in
				*.in-addr.arpa)
					echo "stub-zone:"
					echo "	name: ${zone}"
					echo "	stub-addr: ${server}"
					echo
					echo "server:"
					echo "	local-zone: \"${zone}\" transparent"
					echo
					;;
				*)
					echo "forward-zone:"
					echo "	name: ${zone}"
					echo "	forward-addr: ${server}"
					echo
					;;
			esac
		done < /var/ipfire/dnsforward/config

		if [ -n "${insecure_zones}" ]; then
			echo "server:"

			for zone in ${insecure_zones}; do
				echo "	domain-insecure: ${zone}"
			done
		fi
	) > /etc/unbound/forward.conf
}

write_tuning_conf() {
	# https://www.unbound.net/documentation/howto_optimise.html

	# Determine number of online processors
	local processors=$(getconf _NPROCESSORS_ONLN)

	# Determine number of slabs
	local slabs=1
	while [ ${slabs} -lt ${processors} ]; do
		slabs=$(( ${slabs} * 2 ))
	done

	# Determine amount of system memory
	local mem=$(get_memory_amount)

	# In the worst case scenario, unbound can use double the
	# amount of memory allocated to a cache due to malloc overhead

	# Even larger systems with more than 8GB of RAM
	if [ ${mem} -ge 8192 ]; then
		mem=1024

	# Extra large systems with more than 4GB of RAM
	elif [ ${mem} -ge 4096 ]; then
		mem=512

	# Large systems with more than 2GB of RAM
	elif [ ${mem} -ge 2048 ]; then
		mem=256

	# Medium systems with more than 1GB of RAM
	elif [ ${mem} -ge 1024 ]; then
		mem=128

	# Small systems with less than 256MB of RAM
	elif [ ${mem} -le 256 ]; then
		mem=16

	# Everything else
	else
		mem=64
	fi

	(
		config_header

		# We run one thread per processor
		echo "num-threads: ${processors}"
		echo "so-reuseport: yes"

		# Adjust number of slabs
		echo "infra-cache-slabs: ${slabs}"
		echo "key-cache-slabs: ${slabs}"
		echo "msg-cache-slabs: ${slabs}"
		echo "rrset-cache-slabs: ${slabs}"

		# Slice up the cache
		echo "rrset-cache-size: $(( ${mem} / 2 ))m"
		echo "msg-cache-size: $(( ${mem} / 4 ))m"
		echo "key-cache-size: $(( ${mem} / 4 ))m"

		# Increase parallel queries
		echo "outgoing-range: 8192"
		echo "num-queries-per-thread: 4096"

		# Use larger send/receive buffers
		echo "so-sndbuf: 4m"
		echo "so-rcvbuf: 4m"
	) > /etc/unbound/tuning.conf
}

get_memory_amount() {
	local key val unit

	while read -r key val unit; do
		case "${key}" in
			MemTotal:*)
				# Convert to MB
				echo "$(( ${val} / 1024 ))"
				break
				;;
		esac
	done < /proc/meminfo
}

test_name_server() {
	local ns=${1}
	local args

	# Return codes:
	# 0	DNSSEC validating
	# 1	Error: unreachable, etc.
	# 2	DNSSEC aware
	# 3	NOT DNSSEC-aware

	# Exit when the server is not reachable
	ns_is_online ${ns} || return 1

	# Determine the maximum edns buffer size that works
	local edns_buffer_size=$(ns_determine_edns_buffer_size ${ns})
	if [ -n "${edns_buffer_size}" ]; then
		args="${args} +bufsize=${edns_buffer_size}"
	fi

	local errors
	for rr in DNSKEY DS RRSIG; do
		if ! ns_forwards_${rr} ${ns} ${args}; then
			errors="${errors} ${rr}"
		fi
	done

	if [ -n "${errors}" ]; then
		echo >&2 "Unable to retrieve the following resource records from ${ns}: ${errors:1}"
		return 3
	fi

	if ns_is_validating ${ns} ${args}; then
		# Return 0 if validating
		return 0
	else
		# Is DNSSEC-aware
		return 2
	fi
}

# Sends an A query to the nameserver w/o DNSSEC
ns_is_online() {
	local ns=${1}
	shift

	dig @${ns} +nodnssec A ${TEST_DOMAIN} $@ >/dev/null
}

# Resolving ${TEST_DOMAIN_FAIL} will fail if the nameserver is validating
ns_is_validating() {
	local ns=${1}
	shift

	if ! dig @${ns} A ${TEST_DOMAIN_FAIL} $@ | grep -q SERVFAIL; then
		return 1
	else
		# Determine if NS replies with "ad" data flag if DNSSEC enabled
		dig @${ns} +dnssec SOA ${TEST_DOMAIN} $@ | awk -F: '/\;\;\ flags\:/ { s=1; if (/\ ad/) s=0; exit s }'
	fi
}

# Checks if we can retrieve the DNSKEY for this domain.
# dig will print the SOA if nothing was found
ns_forwards_DNSKEY() {
	local ns=${1}
	shift

	dig @${ns} DNSKEY ${TEST_DOMAIN} $@ | grep -qv SOA
}

ns_forwards_DS() {
	local ns=${1}
	shift

	dig @${ns} DS ${TEST_DOMAIN} $@ | grep -qv SOA
}

ns_forwards_RRSIG() {
	local ns=${1}
	shift

	dig @${ns} +dnssec A ${TEST_DOMAIN} $@ | grep -q RRSIG
}

ns_supports_tcp() {
	local ns=${1}
	shift

	dig @${ns} +tcp A ${TEST_DOMAIN} $@ >/dev/null || return 1
}

ns_determine_edns_buffer_size() {
	local ns=${1}
	shift

	local b
	for b in 4096 2048 1500 1480 1464 1400 1280 512; do
		if dig @${ns} +dnssec +bufsize=${b} A ${TEST_DOMAIN} $@ >/dev/null; then
			echo "${b}"
			return 0
		fi
	done

	return 1
}

get_root_nameservers() {
	while read -r hostname ttl record address; do
		# Searching for A records
		[ "${record}" = "A" ] || continue

		echo "${address}"
	done < /etc/unbound/root.hints
}

can_resolve_root() {
	local ns
	for ns in $(get_root_nameservers); do
		if dig @${ns} +dnssec SOA . $@ >/dev/null; then
			return 0
		fi
	done

	# none of the servers was reachable
	return 1
}

enable_dnssec() {
	local status=$(unbound-control get_option val-permissive-mode)

	# Log DNSSEC status
	echo "on" > /var/ipfire/red/dnssec-status

	# Don't do anything if DNSSEC is already activated
	[ "${status}" = "no" ] && return 0

	# Activate DNSSEC and flush cache with any stale and unvalidated data
	unbound-control -q set_option val-permissive-mode: no
	unbound-control -q flush_zone .
}

disable_dnssec() {
	# Log DNSSEC status
	echo "off" > /var/ipfire/red/dnssec-status

	unbound-control -q set_option val-permissive-mode: yes
}

fix_time_if_dns_fail() {
	# If DNS still not work try to init ntp with
	# hardcoded ntp.ipfire.org (81.3.27.46)
	if [ -e /var/ipfire/red/active ]; then
		host 0.ipfire.pool.ntp.org > /dev/null 2>&1
		if [ "${?}" != "0" ]; then
			boot_mesg "DNS still not functioning... Trying to sync time with ntp.ipfire.org (81.3.27.46)..."
			loadproc /usr/local/bin/settime 81.3.27.46
		fi
	fi
}

check_dnssec_validation() {
	echo -e "\nWill check for DNSSEC validation, this can take some seconds... "
	# Do a rudimentary DNSSEC check and inform user
	if  dig com. SOA +dnssec | grep -q ' ad' && dig +noall +comments ${TEST_DOMAIN_FAIL} | grep -q 'SERVFAIL'; then
		echo -e "\e[32mDNSSEC validation seems to work\e[0m"
	else
		echo -e "\e[31mThere is a problem with DNSSEC since it do NOT vaildating correctly!!! \e[0m"
	fi
}

use_custom_forwarders() {
	# Check if USE_FORWADDERS is '0'
	if grep -q "USE_FORWARDERS=0" /etc/sysconfig/unbound >/dev/null 2>&1; then
		# Check if forwarder entries are existing in local.d
		if grep -Rq "forward-zone:" /etc/unbound/local.d/* >/dev/null 2>&1; then
			echo "Use Custom Forwarders in local.d"
			unbound-control start >/dev/null
			# Update hosts
			write_tuning_conf
			update_hosts
			fix_time_if_dns_fail
			unbound-control list_forwards
			check_dnssec_validation
			$0 status
			exit 0
		fi
	fi
}


case "$1" in
	start)
		# Print a nicer messagen when unbound is already running
		if pidofproc -s unbound; then
			statusproc /usr/sbin/unbound
			exit 0
		fi

		eval $(/usr/local/bin/readhash /var/ipfire/ethernet/settings)

		# Create control keys at first run
		if [ ! -r "/etc/unbound/unbound_control.key" ]; then
			unbound-control-setup -d /etc/unbound &>/dev/null
		fi

		# If custom forwarders are presant, use it
		use_custom_forwarders

		# Update configuration files
		write_tuning_conf
		write_forward_conf

		boot_mesg "Starting Unbound DNS Proxy..."
		loadproc /usr/sbin/unbound || exit $?

		# Make own hostname resolveable
		own_hostname

		# Update any known forwarding name servers
		update_forwarders

		# Update hosts
		update_hosts

		fix_time_if_dns_fail
		;;

	stop)
		boot_mesg "Stopping Unbound DNS Proxy..."
		killproc /usr/sbin/unbound
		;;

	restart)
		$0 stop
		sleep 1
		$0 start
		;;

	status)
		statusproc /usr/sbin/unbound
		;;

	update-forwarders)
		# Do not try updating forwarders when unbound is not running
		if ! pgrep unbound &>/dev/null; then
			exit 0
		fi

		update_forwarders

		unbound-control flush_negative > /dev/null
		unbound-control flush_bogus > /dev/null

		fix_time_if_dns_fail
		;;

	test-name-server)
		ns=${2}

		test_name_server ${ns}
		ret=${?}

		case "${ret}" in
			0)
				echo "${ns} is validating"
				;;
			2)
				echo "${ns} is DNSSEC-aware"
				;;
			3)
				echo "${ns} is NOT DNSSEC-aware"
				;;
			*)
				echo "Test failed for an unknown reason"
				exit ${ret}
				;;
		esac

		if ns_supports_tcp ${ns}; then
			echo "${ns} supports TCP fallback"
		else
			echo "${ns} does not support TCP fallback"
		fi

		edns_buffer_size=$(ns_determine_edns_buffer_size ${ns})
		if [ -n "${edns_buffer_size}" ]; then
			echo "EDNS buffer size for ${ns}: ${edns_buffer_size}"
		fi

		exit ${ret}
		;;

	*)
		echo "Usage: $0 {start|stop|restart|status|update-forwarders|test-name-server}"
		exit 1
		;;
esac

# End $rc_base/init.d/unbound
For Squid: /var/ipfire/proxy/advanced/acls/include.acl

Code: Select all

# Force Squid tu use IPV4 first
dns_v4_first on

# Allow pass User-Agent from all computers
request_header_access User-Agent allow all

# Allow direct access to specific domain
acl dom_direct_access	dstdomain "/var/ipfire/proxy/advanced/acls/dom_direct_access.acl"
always_direct allow dom_direct_access

# Allow direct access from localhost
acl dst_localhost	dst 127.0.0.1/255.255.255.255
always_direct allow dst_localhost

# Allow direct access from internal network
acl dst_internal	dst 172.12.18.0/255.255.254.0
always_direct allow dst_internal

#DNS filter params (http://www.squid-cache.org/Doc/config/dns_nameservers/)
dns_nameservers 103.247.36.36 103.247.37.37

Ipfire web interface: Home page
Capture d’écran_2019-05-15_11-23-34.png
Ipfire web interface: Network > Assign DNS Server
Capture d’écran_2019-05-15_11-23-14.png

Here is also what happens when unbound restart:

Code: Select all

[root@myipfire opt]# /etc/init.d/unbound restart
Stopping Unbound DNS Proxy...                                                                                                                                                                             [  OK  ]
Starting Unbound DNS Proxy...                                                                                                                                                                             [  OK  ]
Ignoring broken upstream name server(s): 103.247.36.36 103.247.37.37                                                                                                                                      [ WARN ]
DNSSEC has been set to permissive mode                                                                                                                                                                    [ WARN ]
Unignoring broken upstream name server(s): 103.247.36.36 103.247.37.37                                                                                                                                    [ WARN ]
[root@myipfire opt]#
Also, here is an example of warnings that are in the logs...

Code: Select all

[root@myipfire opt]# tail -f /var/log/messages
May 15 11:33:35 myipfire unbound: [2996:1] info: validation failure cdn.embed.ly. A IN
May 15 11:33:35 myipfire unbound: [2996:0] info: validation failure cdn.embed.ly.cdn.cloudflare.net. A IN
May 15 11:33:35 myipfire unbound: [2996:0] info: validation failure cdn.embed.ly.cdn.cloudflare.net. AAAA IN
May 15 11:33:35 myipfire unbound: [2996:0] info: validation failure cdn-1.matterport.com. A IN
May 15 11:33:35 myipfire unbound: [2996:0] info: validation failure cdn.segment.com. A IN
May 15 11:33:35 myipfire unbound: [2996:0] info: validation failure my.matterport.com. A IN
May 15 11:33:36 myipfire unbound: [2996:0] info: validation failure api.segment.io. A IN
May 15 11:33:36 myipfire unbound: [2996:1] info: validation failure api.segment.io. AAAA IN
May 15 11:33:37 myipfire unbound: [2996:1] info: validation failure fresnel.vimeocdn.com. A IN
May 15 11:33:37 myipfire unbound: [2996:2] info: validation failure vimeo-video.map.fastly.net. A IN



I don't know if these alerts are serious or not and if this is a problem from a functional point of view ! (in addition to the fact that DNSSEC is disabled and not recommended !)

I gave you my configuration information above...maybe there's something that makes it work in there that could help us understand...I don't understand why...for now:(

Many thanks

ummeegge
Community Developer
Community Developer
Posts: 4899
Joined: October 9th, 2010, 10:00 am

Re: unbound - DoT

Post by ummeegge » May 16th, 2019, 10:21 am

Hi tikok974,
tikok974 wrote:
May 15th, 2019, 9:30 am
I reintegrated the patch of https://open.nlnetlabs.nl/pipermail/dns ... 00359.html found here: https://gitlab.com/snippets/1706804 after I deleted yours and it works !...all my DNS requests are filtered by DNSfilter.
what has been here helpful is the question why does DoT not work with DNSFilter ? What was the replay of the support ? In what kind did you debug this attempt ? It makes no sense that i try to debug your wanted settings without any response and digging from your side.
Regarding the patch which you are using now, it is really the best to ask there where it comes from if some unexpected things are happening since i do not use this patch it is here not only completely OT but also useless since i can not/won´t reproduce any problems with this patch.
tikok974 wrote:
May 15th, 2019, 9:30 am
I don't know if these alerts are serious or not and if this is a problem from a functional point of view ! (in addition to the fact that DNSSEC is disabled and not recommended !)
The DNSSEC validation failed what causes this can be different reasons may verb 2 can show you a bit more e.g. --> https://open.nlnetlabs.nl/pipermail/dns ... 00359.html but again it might be the best idea if you go for your own topic give a good description, ask Ryan Jaeb before to get as close as possible with own debugging and if needed open up a new topic.

Bets,

UE
Image
Image

tikok974
Posts: 61
Joined: January 3rd, 2017, 9:53 am

Re: unbound - DoT

Post by tikok974 » May 16th, 2019, 12:10 pm

Hi Ummeegge,

Thank you for your answers ;)
I understand your answer, but I have very little time to do my tests during the day. The network is very used and I do these tests directly on the firewall in production. Unfortunately, I do not have a 2nd internet line to carry out my tests with another machine.
So I have to try each time when it bothers me the least and then put back into a functional configuration...it's stressful.
I contacted DNSFilter to get their support...I am waiting for an answer.
I am not abandoning your patch because I find it very interesting but for the moment I have to deal with a functional configuration while trying to find a solution to the difficulties encountered.

thank you very much for your support
kind regards

tikok974
Posts: 61
Joined: January 3rd, 2017, 9:53 am

Re: unbound - DoT

Post by tikok974 » May 16th, 2019, 12:52 pm

Re,
The support of DNSfilter gave me an answer:
I’ve briefly reviewed the thread... there’s a lot going on.

I saw at least three issues:

1) unknown IP - possibly caused by transparent proxying by your isp - where you were getting our http://unknown.dnsfilter.com text
2) DNS over TLS functionality
3) DNSSEC enforcement

As for the last issue
It seems in the thread they pointed you to a resource which recommends:

val-log-level: 2

this prints a descriptive string into the log file, about why the
validation failure happened, ie. "validation failure name type class:
no RRSIG records from server 192.0.2.1".


However I have not seen any log output from you with this information.

Please note that we do not do DNSSEC enforcement on our server side.
We have also done minimal testing of customers enforcing DNSSEC by using another resolver in-line before us, as you’re trying to do.
I don’t yet know how it will go with filtering of domains, since in the course of filtering domains we will necessarily break DNSSEC for that domain....

I have increased the level of verbosity of the log (val-log-level: 2) and here is what I have for information:

Code: Select all

[root@myipfire opt]# tail -f /var/log/messages
May 16 14:44:50 myipfire unbound: [20028:3] info: validation failure <mail.infomaniak.com. A IN>: no signatures from 103.247.37.37 for DS infomaniak.com. while building chain of trust
May 16 14:44:50 myipfire unbound: [20028:0] info: validation failure <14.132.166.83.in-addr.arpa. PTR IN>: no signatures from 103.247.36.36 for DS arpa. while building chain of trust
May 16 14:44:51 myipfire unbound: [20028:1] info: validation failure <captive-cidr.origin-apple.com.akadns.net. A IN>: no DNSSEC records from 103.247.37.37 for DS aaplimg.com. while building chain of trust
May 16 14:44:51 myipfire unbound: [20028:0] info: validation failure <captive.g.aaplimg.com. AAAA IN>: key for validation aaplimg.com. is marked as invalid because of a previous validation failure <captive-cidr.origin-apple.com.akadns.net. A IN>: no DNSSEC records from 103.247.37.37 for DS aaplimg.com. while building chain of trust
May 16 14:44:51 myipfire unbound: [20028:0] info: validation failure <captive-cidr.origin-apple.com.akadns.net. AAAA IN>: no DNSSEC records from 103.247.36.36 for DS aaplimg.com. while building chain of trust
May 16 14:44:52 myipfire unbound: [20028:1] info: validation failure <settings-win.data.microsoft.com. A IN>: no DNSSEC records from 103.247.36.36 for DS microsoft.com. while building chain of trust
May 16 14:44:54 myipfire unbound: [20028:0] info: validation failure <settings.crashlytics.com. A IN>: no DNSSEC records from 103.247.36.36 for DS crashlytics.com. while building chain of trust
May 16 14:44:55 myipfire unbound: [20028:0] info: validation failure <graph.facebook.com. A IN>: no DNSSEC records from 103.247.36.36 for DS facebook.com. while building chain of trust
May 16 14:44:57 myipfire kernel: DROP_INPUT IN=red0 OUT= MAC=68:91:d0:60:4c:c2:00:25:84:b6:90:bf:08:00 SRC=85.5.250.35 DST=85.195.239.248 LEN=129 TOS=0x00 PREC=0x00 TTL=53 ID=0 DF PROTO=UDP SPT=51400 DPT=28170 LEN=109 
May 16 14:44:58 myipfire kernel: DROP_INPUT IN=red0 OUT= MAC=68:91:d0:60:4c:c2:00:25:84:b6:90:bf:08:00 SRC=112.169.183.209 DST=85.195.239.248 LEN=143 TOS=0x00 PREC=0x00 TTL=108 ID=34981 PROTO=UDP SPT=10044 DPT=28170 LEN=123 
May 16 14:44:59 myipfire unbound: [20028:1] info: validation failure <evoke-windowsservices-tas.msedge.net. A IN>: no DNSSEC records from 103.247.36.36 for DS msedge.net. while building chain of trust
May 16 14:45:00 myipfire unbound: [20028:0] info: validation failure <e12930.ksd.akamaiedge.net. A IN>: key for validation akamaiedge.net. is marked as invalid because of a previous validation failure <e9706.dscg.akamaiedge.net. A IN>: no DNSSEC records from 103.247.37.37 for DS akamaiedge.net. while building chain of trust
May 16 14:45:00 myipfire kernel: DROP_INPUT IN=red0 OUT= MAC=68:91:d0:60:4c:c2:00:25:84:b6:90:bf:08:00 SRC=72.66.51.87 DST=85.195.239.248 LEN=145 TOS=0x00 PREC=0x00 TTL=109 ID=8851 PROTO=UDP SPT=38866 DPT=28170 LEN=125 
May 16 14:45:01 myipfire unbound: [20028:3] info: validation failure <accounts.youtube.com. A IN>: no DNSSEC records from 103.247.37.37 for DS youtube.com. while building chain of trust
...
...
Thanks

ummeegge
Community Developer
Community Developer
Posts: 4899
Joined: October 9th, 2010, 10:00 am

Re: unbound - DoT

Post by ummeegge » May 16th, 2019, 1:32 pm

Hi tikok974,
tikok974 wrote:
May 16th, 2019, 12:10 pm
I understand your answer, but I have very little time to do my tests during the day. The network is very used and I do these tests directly on the firewall in production. Unfortunately, I do not have a 2nd internet line to carry out my tests with another machine.
So I have to try each time when it bothers me the least and then put back into a functional configuration...it's stressful.
this is no problem at all, just take the time you need to get to an appropriate result/answer to before discussed ideas we are at work not at the run ;) .
tikok974 wrote:
May 16th, 2019, 12:10 pm
I contacted DNSFilter to get their support...I am waiting for an answer.
Am tensed... Also because it seems to work here even they want me to connect to their dashboard/network which is clear since am not their customer. If you have a second working DoT working, you will may see something equal by the usage with Firefox ?! So it might be easier for them too to get a better idea what´s happening, the logs do say surely also something.
tikok974 wrote:
May 16th, 2019, 12:10 pm
I am not abandoning your patch because I find it very interesting but for the moment I have to deal with a functional configuration while trying to find a solution to the difficulties encountered.
Am totally fine if you use or abandon it, no problem at all what we need is a theme oriented workspace otherwise we get an unsatisfying result in here and other people needs to scroll their finger fret cause all that off topic stuff if they are interested in this topic...
tikok974 wrote:
May 16th, 2019, 12:10 pm
thank you very much for your support
You are welcome and am happy if it does help but away from that, am also happy about your support since we are here in the dev section :P .
tikok974 wrote:
May 16th, 2019, 12:52 pm
1) unknown IP - possibly caused by transparent proxying by your isp - where you were getting our http://unknown.dnsfilter.com text
Possible yes. Did you checked the DNSFilter issues site causing transparent proxying and give it a try --> https://docs.dnsfilter.com/docs/transparent-proxying ?
Beneath one. Since DNSFilter provides an idea how to bypass ISP DNS Proxying i modified the unbound init patch which should now also give the possibility to use other ports such 5353 for DoT, the patch will be updated in the next two days and will be available via installer.
tikok974 wrote:
May 16th, 2019, 12:52 pm
2) DNS over TLS functionality
Yes but not enforced since you can also use port 53 (TCP in that case).
tikok974 wrote:
May 16th, 2019, 12:52 pm
3) DNSSEC enforcement
No in here is no DNSSEC enforcement.
tikok974 wrote:
May 16th, 2019, 12:52 pm
As for the last issue
It seems in the thread they pointed you to a resource which recommends:

val-log-level: 2
Yes but this was an OffTopic idea to become a better overview of what happens and have nothing to do with DoT.
tikok974 wrote:
May 16th, 2019, 12:52 pm
this prints a descriptive string into the log file, about why the
validation failure happened, ie. "validation failure name type class:
no RRSIG records from server 192.0.2.1".
Exactly.
tikok974 wrote:
May 16th, 2019, 12:52 pm
However I have not seen any log output from you with this information.
Now it is here but again this is OffTopic :D DO NOT KIDNAP TOPICS !!!
tikok974 wrote:
May 16th, 2019, 12:52 pm
Please note that we do not do DNSSEC enforcement on our server side.
This patch do not uses DNSSEC enforcement.
tikok974 wrote:
May 16th, 2019, 12:52 pm
I have increased the level of verbosity of the log (val-log-level: 2) and here is what I have for information:
Looks informative doesn´t it ?

If the DNSFilter support reads here some question from my side:

For the first hello and welcome :) .
- Is TLSv1.3 available on your service ?
- What is needed to use TCP Fast Open with your service ?

Thanks and best regards,

UE
Image
Image

tikok974
Posts: 61
Joined: January 3rd, 2017, 9:53 am

Re: unbound - DoT

Post by tikok974 » May 16th, 2019, 2:18 pm

Re,
Regarding your question about the transparent proxy:
Possible yes. Did you checked the DNSFilter issues site causing transparent proxying and give it a try --> https://docs.dnsfilter.com/docs/transparent-proxying ?
Beneath one. Since DNSFilter provides an idea how to bypass ISP DNS Proxying i modified the unbound init patch which should now also give the possibility to use other ports such 5353 for DoT, the patch will be updated in the next two days and will be available via installer.
...I'm not in this situation.

unfortunately, I don't have actually a second DOT that establishes the same filtering as DNSFilter to be able to compare.
Could you send me your DOT configuration files so I can check when I activate your patch if I have the same things in my files ?
Also, a question comes to mind, should I keep the DNSFilter addresses in the Network > Assign DNS server part of the Ipfire GUI ?
...because I had also tried to remove them when I activated your patch but that didn't solve my problem...the DNS of DNSfilter were still not working :(

I'll try to check tomorrow with SafeDNS servers and your patch to see if I have better results ;)

I'll keep you informed as soon as I've done the test.
Many thanks for your support

ummeegge
Community Developer
Community Developer
Posts: 4899
Joined: October 9th, 2010, 10:00 am

Re: unbound - DoT

Post by ummeegge » May 16th, 2019, 4:00 pm

Just to keep it simple,
tikok974 wrote:
May 16th, 2019, 2:18 pm
...I'm not in this situation.
why then DoT does not work with DNSFilter in your environment but it works here ?
tikok974 wrote:
May 16th, 2019, 2:18 pm
unfortunately, I don't have actually a second DOT that establishes the same filtering as DNSFilter to be able to compare.
You can configure more then one DoT server in the WUI.
tikok974 wrote:
May 16th, 2019, 2:18 pm
Could you send me your DOT configuration files so I can check when I activate your patch if I have the same things in my files ?
You can find the current testing config in the starting topic --> viewtopic.php?f=50&t=21954&p=120691#p120691 some of them might not work as expected but i look currently what do works and what not...
tikok974 wrote:
May 16th, 2019, 2:18 pm
Also, a question comes to mind, should I keep the DNSFilter addresses in the Network > Assign DNS server part of the Ipfire GUI ?
...because I had also tried to remove them when I activated your patch but that didn't solve my problem...the DNS of DNSfilter were still not working :(
The patch does not overwrites your already configured DNS servers DoT will be used before the configured ones and if DoT do not works/is_off the old ones step in and resolve all that stuff as a fallback, that was the plan.
tikok974 wrote:
May 16th, 2019, 2:18 pm
I'll try to check tomorrow with SafeDNS servers and your patch to see if I have better results ;)
Try to keep it simple do not switch around until the open questions are resolved and DNSFilter works also you really need a DoT ready counterpart! Use the DNSFilter support (which you pay for right ?) and focus on DoT only, DNSSEC is not the question here for the first !!!
tikok974 wrote:
May 16th, 2019, 2:18 pm
I'll keep you informed as soon as I've done the test.
Alright.
tikok974 wrote:
May 16th, 2019, 2:18 pm
Many thanks for your support
You are welcome.

Best,

UE
Image
Image

ummeegge
Community Developer
Community Developer
Posts: 4899
Joined: October 9th, 2010, 10:00 am

Re: unbound - DoT

Post by ummeegge » May 17th, 2019, 9:11 am

Hi all,
ummeegge wrote:
March 13th, 2019, 5:17 pm
and it runs nice but also fast but a shady downside appears currently. kdig do not displays TLSv1.3 :(
short info. It seems that with the last update kdig shows now the correct cipher in the "TLS session" line so TLSv1.3 is now correctly displayed.
Used these two scripts --> https://gitlab.com/ummeegge/dot-for-ipf ... nection.sh and for a little more insides --> https://gitlab.com/ummeegge/dot-for-ipf ... est_tls.sh to debug the connections.

Best,

UE
Image
Image

ryan29
Posts: 5
Joined: March 16th, 2018, 10:52 pm

Re: unbound - DoT

Post by ryan29 » May 20th, 2019, 10:13 am

tikok974 wrote:
May 14th, 2019, 1:30 pm
Hi Ummeegge,

Thank you for your tests and your answers ;)
It doesn't seem easy to find out where the problem comes from :(
While waiting to find a solution, I will continue to use Ryan Jaeb patch found here: https://gitlab.com/snippets/1706804 ...works in all cases...except that I still have errors that appear in the "message" logs as mentioned HERE: viewtopic.php?f=27&t=20478&start=15#p122718
Also, I'll check with DNSfilter support to see if they can help me.

If I uninstall your patch, does it also delete the webuserinterface for DoT in the Ipfire interface ?
thank you
It's possible you're having an issue with the way IPFire falls back to local recursor mode. Maybe ummeegge can give some input on how the DoT stuff interacts with unbound. I didn't have a really good look at it, but the unbound init script seems to have unchanged logic for falling back to recursor mode.

When unbound starts with DNSFilter IPs as forwarders, it makes some queries to determine the capabilities of those upstream DNS servers. One query is for DNSSEC RRSIGs. Compare:

Code: Select all

dig @1.1.1.1 +dnssec ipfire.org

Code: Select all

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> @1.1.1.1 +dnssec ipfire.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42718
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1452
;; QUESTION SECTION:
;ipfire.org.                    IN      A

;; ANSWER SECTION:
ipfire.org.             3600    IN      RRSIG   A 8 2 3600 20190530000000 20190509000000 54142 ipfire.org. n+GlnxmpKWHLx31ZerLopJDzGA9JhX+T1z/hHxD2VfrsC4cZgT/W5EM3 HLYLzHXAIWmqcpiLaSDFQ+dNOrnoHY5J5WoyozSiBicHZtFvbW8KwjCg Ml/gdWcdhzpfRGVkcxsZ5oziCTGPm2MzYHT9sor2vs32+95fIJ/s8PuL RCIjp0CfQotrRycTsNCfJMtJGHlbTl78AhdPkmoUCET2VQJOqeut3Cyh hF1LYj9q/8N0so8/w1bevv46y25xnwsHjpmMHf49+dp6ripm8V4KLgqO OO8e4uTOFfYGCz9L9PMNO6gZM5SVIjNrnhpXEsuhfccJZijJp5swJQE9 EAlWug==
ipfire.org.             3600    IN      RRSIG   A 10 2 3600 20190530000000 20190509000000 6536 ipfire.org. g+Ns3HkJw84U5rZICtZs4nQvYCooksdgYiWEKwqaqHEPNkk5J+OQcRMu NOSxlAJY46hQ1kTmKsd1LpnseWhlr9Ye4jKViAle1pq/eh4C7UVKKC7n HUDct2qmgnF7922fhi4gx8DeyAprcC3RVaSB+kG5DCuU6wMpK6VBmY3e 8KPFs5AqMszb4AU4kh+zBoRReAUNeeTBENfG+c8hsTns1ZzstomsObld SSiCX7Xm6Sez6Zqro4vQIc9/2RJUo/L7KEMIzXA4MBgbbyzbyllON2kG auEsYYkMqwFvkaYPApVO1WFtKEUX4pX925fU3QeLPpJAA4AVTBmJeVUg 9But2w==
ipfire.org.             3600    IN      A       81.3.27.38

;; Query time: 115 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Mon May 20 03:22:47 CST 2019
;; MSG SIZE  rcvd: 651
with DNSFilter (I'm not sure why the IP is different here - probably a redirect because I'm not a customer):

Code: Select all

dig @103.247.36.36 +dnssec ipfire.org

Code: Select all

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> @103.247.36.36 +dnssec ipfire.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59096
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;ipfire.org.                    IN      A

;; ANSWER SECTION:
ipfire.org.             30      IN      A       45.32.196.109

;; Query time: 55 msec
;; SERVER: 103.247.36.36#53(103.247.36.36)
;; WHEN: Mon May 20 03:23:29 CST 2019
;; MSG SIZE  rcvd: 44
or OpenDNS:

Code: Select all

dig @208.67.222.222 +dnssec ipfire.org

Code: Select all

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> @208.67.222.222 +dnssec ipfire.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10385
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ipfire.org.                    IN      A

;; ANSWER SECTION:
ipfire.org.             3600    IN      A       81.3.27.38

;; Query time: 72 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Mon May 20 03:24:42 CST 2019
;; MSG SIZE  rcvd: 55
Note how the filtering services strip the RRSIG records. I'd be curious to hear DNSFilter's explanation for how they have customers that can do DNSSEC enforcement / validation by using an in-line resolver if they're stripping the RRSIGs. Maybe I need to read up on how DNSSEC validation happens.

When the unbound init script makes the above query and checks for RRSIGs, it flags the forwarder as broken (it is) and, if all the forwarders are broken, falls back to local recursor mode. In other words, it ignores the configured forwarders, uses it's own resolver, and sends DNS queries directly to the root DNS servers.

You can get a good idea of where your DNS queries are coming from by doing a DNS leak test. If you run that test and see your own IP address, you're (most likely) using a local recursor.

What my patch does, if ALLOW_BROKEN_FORWARDERS=1 is set, is ignore the checks that IPFire does to determine if the configured forwarders are DNSSEC aware. It basically says, "yep, I know it's broken, but just do it anyway." However, DNSSEC isn't actually disabled. The unbound init script sets unbound to permissive mode. It's still validating, but ignores the failures and replies with the bogus (as described by the unbound docs) query results. That's why you see all the log spam for failed DNSSEC validation.

Incidentally, there's a (not very well documented) setting to control that. This command won't survive a reboot, but you can use it to test:

Code: Select all

unbound-control set_option val-log-squelch: yes
I'll probably update my patch once the person I wrote it for tests that option for a while.

A few other things... I saw someone (maybe Michael) ask what the point of ignoring and then unignoring the broken forwarders was. Basically it was the easiest spot for me to change the existing init script. I didn't want to start changing the logic for testing DNS servers or disabling DNSSEC. Plus it had the benefit of letting me leave the existing warning(s) while adding a couple more to really drive home the point that it's a broken config.

Michael's right about it being an undesirable config. However, in our case, the forwarder(s) we MUST use are upstream, they strip RRSIGs, and we don't control them. They have a fair bit of split horizon DNS config and our traffic for certain hosts will get dropped if we come in via the wrong route. The upstream forwarders are trusted and they're accessed via a private network (ie: dedicated, physical infrastructure). Still, since the upstream forwarders appear to be unaware of DNSSEC (or explicitly disabling it for some reason), they're probably not doing any validation on their upstream queries and we basically have to trust the replies even though no one's actually validating them.

Hopefully that helps. I'll check this thread every couple days for a while in case you have follow up questions.

ummeegge
Community Developer
Community Developer
Posts: 4899
Joined: October 9th, 2010, 10:00 am

Re: unbound - DoT

Post by ummeegge » May 20th, 2019, 12:47 pm

Hi ryan29,
and thanks for this qualified informations but i really thinking about to move those posts here to another topic, have said it now several times that this is OffTopic. This is related to your patch so it might be great if you open up your own possibility for others to give feedback or ask questions or please tikok974 do this for Ryan.
ryan29 wrote:
May 20th, 2019, 10:13 am
I'll check this thread every couple days for a while in case you have follow up questions.
This is clearly a lovely offer from you but let me say simply NO not here :P .Please open up a new topic for this.

Thanks,

UE
Image
Image

ryan29
Posts: 5
Joined: March 16th, 2018, 10:52 pm

Re: unbound - DoT

Post by ryan29 » May 20th, 2019, 11:15 pm

ummeegge wrote:
May 20th, 2019, 12:47 pm
Hi ryan29,
and thanks for this qualified informations but i really thinking about to move those posts here to another topic, have said it now several times that this is OffTopic. This is related to your patch so it might be great if you open up your own possibility for others to give feedback or ask questions or please tikok974 do this for Ryan.
ryan29 wrote:
May 20th, 2019, 10:13 am
I'll check this thread every couple days for a while in case you have follow up questions.
This is clearly a lovely offer from you but let me say simply NO not here :P .Please open up a new topic for this.

Thanks,

UE
Hey ummeegge,

Sorry if I seemed to get a bit off topic. I thought that explaining why my patch works for tikok974 might help to show why this one doesn't work for them. I haven't tested a lot, but I think the issue tikok974 is running into is the DoT setup allows a user to supply a broken config that can be hard to diagnose without understanding what's happening. Based on my testing:
  • The normal IPFire config forbids broken forwarders and enables DNSSEC validation.
  • My patch allows broken forwarders and disables DNSSEC validation.
  • The DoT config allows broken forwards and enables DNSSEC validation.
I'm guessing the last one is what tikok974 is doing to get stuck. The DNSFilter servers are broken because they strip RRSIGs, the queries fail DNSSEC validation, and unbound doesn't reply to the client. It behaves like DNS is shut off if all the DoT servers are broken. It's even more difficult to diagnose if the user has several working DoT servers and one broken DoT server configured. The lookup failures could feel really random. You can test yourself by configuring DoT with a DNSFilter server (dns1.dnsfilter.com, 103.247.36.36), restarting unbound, and trying to make some queries with dig. Then restart unbound to clear caches and temporarily set unbound to permissive mode before trying to make more queries:

Code: Select all

unbound-control set_option val-permissive-mode: yes
The DNSSEC validation errors will still show up in /var/log/messages, but the non-validated replies should get returned to the client.

Based on my testing, all queries to broken forwarders like DNSFilter fail, so it might be better to give the user an error when they try to add a broken forwarder or to test the forwarders on startup like the unbound init script does. The only other option I can think of is to set unbound to permissive mode and that's a debate that you probably want to keep out of this thread.

ummeegge
Community Developer
Community Developer
Posts: 4899
Joined: October 9th, 2010, 10:00 am

Re: unbound - DoT

Post by ummeegge » May 21st, 2019, 10:04 am

Hi ryan29,
ryan29 wrote:
May 20th, 2019, 11:15 pm
Sorry if I seemed to get a bit off topic. I thought that explaining why my patch works for tikok974 might help to show why this one doesn't work for them.
in general no problem at all and again thanks for your support but what i really need here is a relation to the "Subject", the following from your last post includes also some reference to the issue here which i am happy about.
ryan29 wrote:
May 20th, 2019, 11:15 pm
I haven't tested a lot, but I think the issue tikok974 is running into is the DoT setup allows a user to supply a broken config that can be hard to diagnose without understanding what's happening. Based on my testing:
The DoT patch simply ignores the 'update_forwarders()' function if he finds something in forward.conf "# In case we have found no working forwarders" will be ignored but i am totally open to reintegrate those features i simply disabled it at that time since a lot of things has been broken (e.g. the local resolution) by the first tries to integrate DoT into IPFire, the plan was to leave the problematic code for the first out to find a good functional way to test the whole DoT thing and to integrate later all nice to haves again.
ryan29 wrote:
May 20th, 2019, 11:15 pm
[*]
The DoT config allows broken forwards and enables DNSSEC validation.[/list]
Indeed, both should be possible.
ryan29 wrote:
May 20th, 2019, 11:15 pm
I'm guessing the last one is what tikok974 is doing to get stuck. The DNSFilter servers are broken because they strip RRSIGs, the queries fail DNSSEC validation, and unbound doesn't reply to the client. It behaves like DNS is shut off if all the DoT servers are broken.
I can not confirm this here since i do got answer from DNSFilter --> viewtopic.php?f=50&t=21954&start=30#p124412 and they queried the detectportal which was clear since i am not their customer. From my understanding tikok974 have seen there nothing ?! But his answers wasn´t that clear to me in that manner.
ryan29 wrote:
May 20th, 2019, 11:15 pm
It behaves like DNS is shut off if all the DoT servers are broken.
If they are broken in the sense that DNSSEC does not work, they work nevertheless. Like you state above, this patch allows broken forwarders but it enables also DNSSEC validation if possible.
ryan29 wrote:
May 20th, 2019, 11:15 pm
It's even more difficult to diagnose if the user has several working DoT servers and one broken DoT server configured
Tried to keep it handy, that´s because i wrote some connection_checker scripts which checks DNSSEC but also the TLS session and if the certificates are trustworthy. Have offered the scripts in this topic. But away from that, the plan is to have some colour codes in the WUI for every entry like it is done in the DDNS CGI where the user should see the current state of this connection but that´s a future sound of music moreover a lot other work has been done until then but am not sure in general if this a way to go for me.
ryan29 wrote:
May 20th, 2019, 11:15 pm
The lookup failures could feel really random.
In fact it is random since 'rrset-roundrobin: yes' is set in unbound.conf --> https://www.monperrus.net/martin/random ... s-requests . Have currently ~15 DoT servers configured, some of them are always down since i use mainly testing server from DNSPrivacy --> https://dnsprivacy.org/wiki/display/DP/ ... st+Servers , there are always problems with the certificates or DNSSEC or they are simply offline but this makes no problems at all if some or minimum one works.
ryan29 wrote:
May 20th, 2019, 11:15 pm
You can test yourself by configuring DoT with a DNSFilter server (dns1.dnsfilter.com, 103.247.36.36), restarting unbound, and trying to make some queries with dig.
Have tested it, please see above.
ryan29 wrote:
May 20th, 2019, 11:15 pm
Based on my testing, all queries to broken forwarders like DNSFilter fail, so it might be better to give the user an error when they try to add a broken forwarder or to test the forwarders on startup like the unbound init script does.
Sure if this one will become a possible candidate for a release but we are here in the dev section and i have to presume that the involved people uses at the minimum the offered scripts to see/debug how it works for them.
ryan29 wrote:
May 20th, 2019, 11:15 pm
The only other option I can think of is to set unbound to permissive mode and that's a debate that you probably want to keep out of this thread.
Definitely not, if you want to extend the unbound init for DoT am more than happy with this. What i want to keep out of this thread are the regular unbound problems since they are discussed widely in the forum on other places, i only want a topic related discussion/development/feedback here.

Best and thanks for your feedback and your engagement to help others :) .

UE
Image
Image

ummeegge
Community Developer
Community Developer
Posts: 4899
Joined: October 9th, 2010, 10:00 am

Re: unbound - DoT

Post by ummeegge » May 23rd, 2019, 11:56 am

Hi all,
update to Core132 is up and can be updated via in- uninstaller from the starting topic --> viewtopic.php?f=50&t=21954#p120691 . Changes for unbound init and en.pl are integrated. Please only install this update if Core 132 is already installed otherwise DoT won´t work.

Greetings,

UE
Image
Image

ummeegge
Community Developer
Community Developer
Posts: 4899
Joined: October 9th, 2010, 10:00 am

Re: DoT - ESNI

Post by ummeegge » May 28th, 2019, 5:45 am

Hi all,
since ESNI --> https://datatracker.ietf.org/doc/draft-ietf-tls-esni/ is still an open question in here (or at least for me) i followed up the current results of the new partnership between Firefox and Cloudflare and their work to bring the "Encrypted Server Name Indication" (ESNI) to life --> https://bugzilla.mozilla.org/show_bug.cgi?id=1434852 --> https://blog.mozilla.org/security/2018/ ... x-nightly/ so i decided to give it a fast try.
Even you need to enable DoH (DNS-over-HTTPS which is here not supported) and the new TRR feature in Firefox i tried it out with the following steps --> https://wiki.mozilla.org/Trusted_Recurs ... .com/esni/ in Firefox , to my surprise 'network.trr.uri' in Firefox already includes the string 'https://mozilla.cloudflare-dns.com/dns-query' (which surprises me a little) so Cloudflare is already configured in Firefox as 'Trusted Recursive Resolver' (some alternative TTRs can also found in here --> https://miketabor.com/enable-dns-over-h ... n-firefox/ ) --> the privacy problem which arises here is worth to thinking about and some discussions causing that can also be found in here --> https://news.ycombinator.com/item?id=18251074 --> https://yro.slashdot.org/story/18/08/05 ... or-firefox
Image
<-- picture is from "https://ungleich.ch/en-us/cms/blog/2018 ... dangerous/" thanks by the way for an easy metaphoric explanation of your concerns ;) -->
but it seems there is also movement to bring on ESNI without DoH --> https://bugzilla.mozilla.org/show_bug.cgi?id=1500289 so i went nevertheless for the first back to some bloody testing.

Fast test:
1) Setting Firefox up like explained above.
2) Configure for the first only Cloudflare 1.1.1.1 in DNS-over-TLS WUI but tested afterwards also all TLSv1.3 ready servers in my ENV which where

Code: Select all

==============================================================================
From Host: rec1.dns.lightningwirelabs.com ---- With IP: 81.3.27.54 ---- Date: Tue May 28 07:10:40 CEST 2019
TLS1.3-ECDHE-SECP256R1-ECDSA-SECP384R1-SHA384-CHACHA20-POLY1305
==============================================================================
From Host: dot-jp.blahdns.com ---- With IP: 108.61.201.119 ---- Date: Tue May 28 07:10:41 CEST 2019
TLS1.3-ECDHE-SECP256R1-RSA-PSS-RSAE-SHA256-AES-256-GCM
==============================================================================
From Host: dot-de.blahdns.com ---- With IP: 159.69.198.101 ---- Date: Tue May 28 07:10:42 CEST 2019
TLS1.3-ECDHE-SECP256R1-RSA-PSS-RSAE-SHA256-AES-256-GCM
==============================================================================
From Host: cloudflare-dns.com ---- With IP: 1.1.1.1 ---- Date: Tue May 28 07:10:43 CEST 2019
TLS1.3-ECDHE-SECP256R1-ECDSA-SECP256R1-SHA256-AES-256-GCM
==============================================================================
From Host: dot.securedns.eu ---- With IP: 146.185.167.43 ---- Date: Tue May 28 07:10:43 CEST 2019
TLS1.3-ECDHE-SECP256R1-RSA-PSS-RSAE-SHA256-AES-256-GCM
==============================================================================
and also without Cloudflare but the main test was Cloudflare only!

Code: Select all

$ unbound-control list_forwards                               
. IN forward 1.1.1.1
3) Made then a test provided by Cloudflare --> https://www.cloudflare.com/ssl/encrypted-sni/

Result:
The Cloudflare test delivered an enabled ESNI and --> https://servo.org/cdn-cgi/trace -->

Code: Select all

fl=71f280
h=servo.org
ip=87.224.xxx.xxx
ts=1521043484.976
visit_scheme=https
uag=Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0
colo=FRA
http=http/2
loc=DE
tls=TLSv1.3
sni=encrypted
warp=off
too.

Looks good doesn´t it ?
OK let´s take a little a deeper look into it. Since Core 132 delivers tshark --> https://wiki.ipfire.org/addons/tshark/start and i have it already running here i was wondering if the 'Server Name Indication field' can somehow be checked.
Searching for an appropriate filter i found this topic here --> https://superuser.com/questions/538130/ ... tion-field which was a good starting help. Figuring out that the filter names has been changed a little (tls. instead of ssl.) i was coming up to this command:

Code: Select all

tshark -i red0 -p -Tfields -e tls.handshake.extensions_server_name -Y 'tls.handshake.extension.type == "server_name"'
which delivered:

Code: Select all

$ tshark -i red0 -p -Tfields -e tls.handshake.extensions_server_name -Y 'tls.handshake.extension.type == "server_name"'
Running as user "root" and group "root". This could be dangerous.
Capturing on 'red0'
cloudflare-dns.com
cloudflare-dns.com
cloudflare-dns.com
cloudflare-dns.com
cloudflare-dns.com
cloudflare-dns.com
cloudflare-dns.com
so even ESNI has been displayed as enabled it seems that the handling of the key does not work and it was not returned as part of the ESNI field like it should be e.g. -->

Code: Select all

$ kdig TXT _esni.servo.org +short
"/wHWKhptACQAHQAgb2vMSLCECX5VtKO1qHdgYu19CuyY0fjxRQlkYKj8DiwAAhMBAQQAAAAAXOi9wAAAAABc8KbAAAA="
(also explained in here --> https://blog.cloudflare.com/encrypt-tha ... x-edition/ ).

OK, smells a little fishy and will turn for the first all that back, also i will turn off the TRR mode in Firefox (value 5) and will delete the TRR URI string until this feature is better supported and free from DoH.

Just some news from here, if i have overseen something am happy for feedback/corrections :) .

Best,

UE
Image
Image

dnl
Posts: 370
Joined: June 28th, 2013, 11:03 am

Re: unbound - DoT

Post by dnl » June 30th, 2019, 9:57 am

Hi ummeegge,
Thanks for all your great work adding features to IPFire!

Do you think this DoT "DNS Privacy" support will be included in IPFire by default any time soon?
I appreciate the amount of effort you've done to investigate this.
IPFire 2.x (Latest Update) on x86_64 AMD CPU, 8GiB RAM, RED + GREEN + BLUE

Post Reply