Nfsen-Nfdump-fprobe{-softflowd} netflow

Help on building IPFire & Feature Requests
Post Reply
ummeegge
Community Developer
Community Developer
Posts: 4122
Joined: October 9th, 2010, 10:00 am

Nfsen-Nfdump-fprobe{-softflowd} netflow

Post by ummeegge » July 3rd, 2017, 6:44 pm

Hi all,
have compiled Nfdump --> http://nfdump.sourceforge.net/ <--> http://www.linuxscrew.com/2011/02/23/in ... -in-linux/ , fprobe --> http://fprobe.sourceforge.net/ and added Nfsen --> http://nfsen.sourceforge.net/ (an IPFire patched version with vhost and initscript and modified configuration) in one package and tried to bring them to live for IPFire systems (32 and 64 bit platforms are supported) .

In short, all together works in a process to collect (fprobe) and process (Nfdump tools) and display netflow data over a webinterface (Nfsen) under widely differentiable circumstances (if wanted).
It would sprinkle the frame here to list all possibilities but the docs are informative and a lot´s of info´s are findable in the web.

Made an in- and uninstaller for this for 32 and 64 bit IPFire systems which can be used via an

PLEASE USE TESTING SYSTEMS FOR THIS

Code: Select all

cd /tmp &&
curl -O https://raw.githubusercontent.com/ummeegge/scripts/master/nfsen-installer.sh &&
chmod +x nfsen-installer.sh &&
./nfsen-installer.sh
on your IPFire. During installation you will be ask

Code: Select all

Perl to use: [/usr/bin/perl] 
which you can return with an ENTER unless you´d installed Perl on another location ;-).
All packages are located in here -->http://people.ipfire.org/~ummeegge/Netf ... rob-nfsen/ .

- After installation the Nfsen webinterface are reachable under https://IPFires-green-IP:54321/nfsen.php . To change this, use /etc/httpd/conf/vhosts.d/nfsen.conf . It takes some minutes until the first data are collected and available over Nfsen.
- Nfsen will start also nfcapd, so in this version there are no initscript for nfcapd available.
- nfcapd and Nfsen works with lowered privileges (user: netflow ;group: netflow is member of the group: nobody), so new entries in /etc/passwd and /etc/group will be made via installation (uninstaller set´s all back again).
passwd:

Code: Select all

netflow:x:1004:1004::/home/netflow:/bin/false
group:

Code: Select all

nobody:x:99:netflow
netflow:!:1004:
- Nfsen have his own initscript (under /etc/rc.d/init.d/nfsen) whereby the following options are possible

Code: Select all

-> /etc/init.d/nfsen 
Usage: /etc/init.d/nfsen {start|stop|restart|status|state|reconfigure}
- Nfsen´s home is under /var/nfsen finable (libexec, bin, etc, and data) <-- take care that you have enough disk space (some Gigs should be there) since nfcapd can stores all collected data for a long time (long time history).
- fprobe deliver his flow probes only for localhost on port 65432 TCP and collect 'any' interface, to change this checkout /etc/rc.d/init.d/fprobe . Separate instances of fprobe can nevertheless be started.
- fprobe is chrooted to /var/empty but it currently do NOT work with lowered privileges to another user like e.g. 'nobody' if the option do exist.

Nfsen and fprobe can also be deactivated and also activated over IPFires webinterface under Status --> Services
Image
(since Nfsen locates it´s PID under his own directory structure, it won´t be shown in the WUI. Nfsen uses also more then one process so the Memory column is also empty)

Example screenshoots from my installation can also be found in here --> http://people.ipfire.org/~ummeegge/screenshoots/nfsen/

The possibilities are extensive and i do get currently not all of them but time will tell.

If you are interested in this, feel free to use it, test it, to give feedback or if you have questions just ask will try then to help you out.

Greetings,

UE

EDIT: The Nfdump has been compiled with the following options

Code: Select all

	cd $(DIR_APP) && ./configure		\
		--prefix=/usr			\
		--sysconfdir=/etc		\
		--localstatedir=/var/netflow	\
		--enable-nftrack		\
		--enable-sflow			\
		--enable-nfpcapd		\
		--enable-nsel			\
		--enable-nel			\
		--enable-ftconv			\
		--with-ftpath=/usr		\
		--enable-nfprofile
flow-tools and sflow support is available. If someone wants to make a try on the flow-tools --> https://code.google.com/archive/p/flow-tools/ say something.
Available binaries for the Nfdump package are:

Code: Select all

/usr/bin/ft2nfdump
/usr/bin/nfanon
/usr/bin/nfcapd
/usr/bin/nfdump
/usr/bin/nfexpire
/usr/bin/nfpcapd
/usr/bin/nfprofile
/usr/bin/nfreplay
/usr/bin/nftrack
/usr/bin/sfcapd
Image
Image
Image

User avatar
H&M
Posts: 375
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Nfsen-Nfdump-fprobe netflow

Post by H&M » July 4th, 2017, 7:11 am

Hi Ummeegge,

Will it work with interfaces in promiscuous mode (i.e Snort activated)?
Can it be activated only to some interfaces?

Very good job!
I'll test it if works on promiscuous mode interfaces.

Best regards,
H&M

ummeegge
Community Developer
Community Developer
Posts: 4122
Joined: October 9th, 2010, 10:00 am

Re: Nfsen-Nfdump-fprobe netflow

Post by ummeegge » July 4th, 2017, 8:33 am

Hi H&M,
thanks for your feedback.
H&M wrote:
July 4th, 2017, 7:11 am
Will it work with interfaces in promiscuous mode (i.e Snort activated)?
This should be no problem, fprobe is currently configured here to take probes from 'any' interfaces (IPFire specifics like green0 are possible), fprobe serves also a promiscuous mode which is deactivated in fprobe´s initscript. My Snort instance checks currently also the green interface and fprobe gatheres all interfaces ('any')

Code: Select all

root      1999  0.4  5.8 475636 118312 ?       Ssl  Jul03   6:18 /usr/sbin/snort -c /etc/snort/snort.conf -i green0 -D -l /var/log/snort --create-pidfile --nolock-pidfile --pid-path /var/run
netflow  12599  0.0  0.1  20032  2640 ?        S    Jul03   0:00 /usr/bin/nfcapd -w -D -p 65432 -u netflow -g nobody -B 200000 -S 1 -P /var/nfsen/var/run/p65432.pid -z -I ipfire -l /var/nfsen/profiles-data/live/ipfire
netflow  12601  0.3  0.9 117636 19984 ?        Ss   Jul03   2:29 /usr/bin/perl -w /var/nfsen/bin/nfsend
netflow  12602  0.0  0.6 112860 13824 ?        Ss   Jul03   0:02 /var/nfsen/bin/nfsend-comm
root     12576  0.2  0.2  47620  5812 ?        Ssl  Jul03   1:58 /usr/sbin/fprobe -i any 127.0.0.1:65432 -p -c /var/empty
and in here -->
Image
green is completely captured, so no problem with this.
H&M wrote:
July 4th, 2017, 7:11 am
Can it be activated only to some interfaces?
This should be also no problem. You can to configure fprobe on which interface(s) it should be listen/capture (softflowd meanwhile is the same). The interface is specified in fprobe´s initscript findable under /etc/rc.d/init.d/fprobe{softflowd} . The interfaces needs to be present which you want to configure during fprobes start (e.g. tun1 should be active) otherwise fprobe do not start with this parameter. You can also supplement different instances of fprobe, with different ARGS, you can use in that case different ports and point Nfsen via nfsen.conf to it (%sources section), this is also the way to create specific Nfsen profiles --> http://nfsen.sourceforge.net/#mozTocId623518 (tcpdump syntax for appropriate filters).

Greetings,

UE
Image
Image
Image

ummeegge
Community Developer
Community Developer
Posts: 4122
Joined: October 9th, 2010, 10:00 am

Re: Nfsen-Nfdump-fprobe netflow

Post by ummeegge » July 4th, 2017, 4:32 pm

Fix for the Nfsen initscript (under /etc/rc.d/init.d/nfsen ):

If more profiles (and more sources) will be configured, the checks in the beginning went into a loop which should not be the case.

All packages has been updated and includes the fix, SHA sums in the installer has also been adjusted --> https://github.com/ummeegge/scripts/com ... 0a1aa18c44 .

If you have collect already some data with an existing installation, you can also fix this manually with this changes:

Code: Select all

--- /etc/rc.d/init.d/nfsen.old	2017-07-03 18:50:25.607443677 +0200
+++ /etc/rc.d/init.d/nfsen	2017-07-04 18:26:30.752772067 +0200
@@ -16,22 +16,6 @@
 BIN="/var/${NAME}/bin/${NAME}";
 CONF="/var/${NAME}/etc/nfsen.conf";
 
-# Check for dependencies
-#Check if fprobe is running
-if ! pgrep fprobe > /dev/null; then
-	echo "Fprobe is not running, won´t get results. Will Quit now"; 
-fi
-# Check that fprobs port is equal to NFsens nfcapd configuration
-FPROBEPORTCHECK=$(ps aux | grep -v grep | grep fprobe | awk -F":" '{ print $4 }' | grep -o '[0-9]*');
-NFSENPORT=$(awk -F"'" '/ipfire/ { print $6 }' ${CONF});
-if [ "${FPROBEPORTCHECK}" != "${NFSENPORT}" ]; then
-	sed -i "s/PORT=\".*/PORT=\"${NFSENPORT}\"/" /etc/rc.d/init.d/fprobe;
-	echo "Have changed fprobes port to NFsens nfcapd port. Will restart fprobe now... ";
-	sleep 3;
-	/etc/init.d/fprobe restart;
-fi
-
-
 . /etc/sysconfig/rc
 . ${rc_functions}
 
or just delete the checks at the beginning.

Greetings,

UE
Image
Image
Image

ummeegge
Community Developer
Community Developer
Posts: 4122
Joined: October 9th, 2010, 10:00 am

Re: Nfsen-Nfdump-fprobe netflow

Post by ummeegge » July 5th, 2017, 7:59 am

Added now authentication possibility for the Nfsen web interface to the package. Authentication, if wanted, needs to be activated with the following steps
- Add a new user via

Code: Select all

htpasswd -c /var/ipfire/auth/netflow_users newusername
whereby "netflow_users" is the new file and "newusername" is the new user. This command leads you to the password settings.
After that you need to modify /etc/httpd/conf/vhosts.d/nfsen.conf. The actual package provides the new vhost conf for nfsen where you can find this section

Code: Select all

#	<Directory /srv/web/nfsen>
#		AllowOverride None
#		AuthName "NFsen - Restricted"
#		AuthType Basic
#		AuthUserFile /var/ipfire/auth/{YOUR_DEFINED_NETFLOW_USERSFILE}
#		Require user {YOUR_DEFINED_NETFLOW_USER}
#	</Directory>
Delete all '#' at the beginning of the line and add instead of "{YOUR_DEFINED_NETFLOW_USERSFILE}" the above mentioned name of the file and add also instead of "{YOUR_DEFINED_NETFLOW_USER}" the name of the new user which you want to authenticate to.
To stay in this example, this section should looks then like this:

Code: Select all

	<Directory /srv/web/nfsen>
		AllowOverride None
		AuthName "NFsen - Restricted"
		AuthType Basic
		AuthUserFile /var/ipfire/auth/netflow_users
		Require user newusername
	</Directory>
After that execute a

Code: Select all

apachectl graceful
so that the changes can take affect.

So from now you should see something like this
Image
if you want to login into Nfsen.

Greetings,

UE
Image
Image
Image

ummeegge
Community Developer
Community Developer
Posts: 4122
Joined: October 9th, 2010, 10:00 am

Re: Nfsen-Nfdump-fprobe(softflowd) netflow

Post by ummeegge » July 7th, 2017, 4:35 pm

Hi all,
the installer provides now also a choice between fprobe --> http://fprobe.sourceforge.net/ and softflowd --> https://code.google.com/archive/p/softflowd/ . Since softflowd provides some more options but reduces his privileges automatically to 'nobody' it might be an idea to make a choice between them both. softflowd has been compiled with "--with-chrootdir=/var/empty" , uses per default netflow version 9 and have a maxlife of 5 minutes.

For a standalone versions of softflowd or fprobe, 32 and 64 bit are available in here --> http://people.ipfire.org/~ummeegge/Netf ... _analyzer/ . Both packages includes initscripts and searches automatically for appropriate symlinks and set them while installation. uninstaller.sh deletes all again.

Greetings,

UE
Image
Image
Image

User avatar
H&M
Posts: 375
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Nfsen-Nfdump-fprobe netflow

Post by H&M » July 7th, 2017, 6:02 pm

Hi,

Besides modifying /etc/httpd/conf/vhosts.d/nfsen.conf what else should I change in order to give access to nfsen site (https://ipfire:54321/nfsen.php) from other networks?

I did added the IP range in /etc/httpd/conf/vhosts.d/nfsen.conf, restarted apache, but still no success

Code: Select all

Forbidden

You don't have permission to access /nfsen.php on this server.

Apache Server at a.b.c.d Port 54321
Thank you,
H&M

ummeegge
Community Developer
Community Developer
Posts: 4122
Joined: October 9th, 2010, 10:00 am

Re: Nfsen-Nfdump-fprobe netflow

Post by ummeegge » July 7th, 2017, 6:39 pm

Hi H&M,
you can open up the "<VirtualHost ..:..> line via an asterisk so the Nfsen WI won´t be binded to one interface (green0 per default installation) and then change both "Allow from ..." lines to the appropriate subnet or IP which should have access.
An example can looks like in this diff:

Code: Select all

--- /etc/httpd/conf/vhosts.d/nfsen.conf.restricted	2017-07-07 20:34:24.945402628 +0200
+++ /etc/httpd/conf/vhosts.d/nfsen.conf	2017-07-07 20:26:09.671419091 +0200
@@ -1,5 +1,5 @@
 Listen 54321
-<VirtualHost 192.168.0.1:54321>
+<VirtualHost *:54321>
     SSLEngine on
     SSLProtocol all -SSLv2
     SSLCipherSuite ALL:!ADH:!EXPORT56:!eNULL:!SSLv2:!RC4+RSA:+HIGH:+MEDIUM
@@ -15,7 +15,7 @@
 		Options +Indexes
 		AllowOverride All
 		Order Allow,Deny
-		Allow from 192.168.0.2
+		Allow from 192.168.0.0/255.255.255.0 10.23.32.0/24 192.168.9.2
 	</Directory>
 
 #	<Directory /srv/web/nfsen>
@@ -30,7 +30,7 @@
 
 		Order allow,deny
 
-		Allow from 192.168.0.2
+		Allow from 192.168.0.0/255.255.255.0 10.23.32.0/24 192.168.9.2
 
 
as you can see you can put several entries in those lines, subents and IPs should be possible. You can execute after your changes an

Code: Select all

apachectl graceful

so the changes takes affect
May this helps.

Apart from that, how does it works until now ?

Greetings,

UE
Image
Image
Image

User avatar
H&M
Posts: 375
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Nfsen-Nfdump-fprobe netflow

Post by H&M » July 7th, 2017, 7:02 pm

ummeegge wrote:
July 7th, 2017, 6:39 pm
as you can see you can put several entries in those lines, subents and IPs should be possible. You can execute after your changes an
That solved my problem. I added 2 lines with Allow...

ummeegge wrote:
July 7th, 2017, 6:39 pm
Apart from that, how does it works until now ?
Not that good as far as I see: nothing collected.
All graphs are empty.
Files also seems very low on space occupied:

Code: Select all

ls -la /var/nfsen/profiles-data/live/ipfire/2017/07/07/
total 68
drwxr-xr-x 2 netflow nobody 4096 Jul  7 21:55 .
drwxr-xr-x 3 netflow nobody 4096 Jul  7 20:45 ..
-rw-r--r-- 1 netflow nobody  276 Jul  7 20:45 nfcapd.201707072040
-rw-r--r-- 1 netflow nobody  276 Jul  7 20:50 nfcapd.201707072045
-rw-r--r-- 1 netflow nobody  276 Jul  7 20:55 nfcapd.201707072050
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:00 nfcapd.201707072055
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:05 nfcapd.201707072100
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:10 nfcapd.201707072105
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:15 nfcapd.201707072110
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:20 nfcapd.201707072115
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:25 nfcapd.201707072120
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:30 nfcapd.201707072125
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:35 nfcapd.201707072130
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:40 nfcapd.201707072135
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:45 nfcapd.201707072140
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:50 nfcapd.201707072145
-rw-r--r-- 1 netflow nobody  276 Jul  7 21:55 nfcapd.201707072150
I will investigate a bit...

Late edit: found cause - fprobe is missing

Code: Select all

 ps aux |grep fprobe
root     21063  0.0  0.0   4032   872 pts/1    S+   22:11   0:00 grep fprobe


cat /etc/rc.d/init.d/fprobe
cat: /etc/rc.d/init.d/fprobe: No such file or directory

No fprobe means no capture....
Need to check the install script...

Thank you for the help!
H&M

ummeegge
Community Developer
Community Developer
Posts: 4122
Joined: October 9th, 2010, 10:00 am

Re: Nfsen-Nfdump-fprobe netflow

Post by ummeegge » July 7th, 2017, 7:32 pm

OK, this seems strange ::) ...
If you use the current actual script from above (you can also delete the whole old installation via script) i would suggest to use softflowd instead of fprobe, after installation you can also check via the 'p' option if everything has been started. There is the need to wait some time after installation (10 min. should be enough) to get the first graphs.

Greetings,

UE
Image
Image
Image

User avatar
H&M
Posts: 375
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Nfsen-Nfdump-fprobe netflow

Post by H&M » July 7th, 2017, 7:36 pm

Ok.

Here is the status of the installation ('p' option from installer)

Code: Select all


                                                                - Process status of Nfsen installation -

------------------------------------------------------------------------------------------------------------------------------------------------------------------------

netflow  10116  0.0  0.1  13776  4648 ?        S    20:58   0:00 /usr/bin/nfcapd -w -D -p 65432 -u netflow -g nobody -B 200000 -S 1 -P /var/nfsen/var/run/p65432.pid -z -I ipfire -l /var/nfsen/profiles-data/live/ipfire
netflow  10118  0.3  0.3  21492 13108 ?        Ss   20:58   0:21 /usr/bin/perl -w /var/nfsen/bin/nfsend
netflow  10119  0.0  0.2  20232  8312 ?        Ss   20:58   0:01 /var/nfsen/bin/nfsend-comm
root     24511  0.6  0.0   5020  1716 pts/1    S+   22:35   0:00 /bin/bash - ./nfsen-installer.sh

------------------------------------------------------------------------------------------------------------------------------------------------------------------------


I will uninstall and reinstall the package.

Stay tuned! :D


And this is with softflowd - much better! ;)

Code: Select all

                 - Process status of Nfsen installation -

------------------------------------------------------------------------------------------------------------------------------------------------------------------------
nobody   25017  0.4  0.0   4416  2548 ?        Ss   22:37   0:00 /usr/sbin/softflowd -n 127.0.0.1:65432 -i any -t maxlife=5m -v 9

root     24837  0.2  0.0   5148  1812 pts/1    S+   22:37   0:00 /bin/bash - ./nfsen-installer.sh
netflow  25028  0.0  0.0  13780   336 ?        S    22:37   0:00 /usr/bin/nfcapd -w -D -p 65432 -u netflow -g nobody -B 200000 -S 1 -P /var/nfsen/var/run/p65432.pid -z -I ipfire -l /var/nfsen/profiles-data/live/ipfire
netflow  25035  0.0  0.2  21808  8944 ?        Ss   22:38   0:00 /usr/bin/perl -w /var/nfsen/bin/nfsend
netflow  25036  0.0  0.2  21808  8472 ?        Ss   22:38   0:00 /var/nfsen/bin/nfsend-comm

------------------------------------------------------------------------------------------------------------------------------------------------------------------------



ummeegge
Community Developer
Community Developer
Posts: 4122
Joined: October 9th, 2010, 10:00 am

Re: Nfsen-Nfdump-fprobe netflow

Post by ummeegge » July 7th, 2017, 7:41 pm

Alright, get now my beer o'clock :P ...

See you in a bit!

:)
Image
Image
Image

User avatar
H&M
Posts: 375
Joined: May 29th, 2014, 9:38 pm
Location: Europe

Re: Nfsen-Nfdump-fprobe netflow

Post by H&M » July 7th, 2017, 7:50 pm

It works!

Code: Select all

nfdump -M /var/nfsen/profiles-data/live/ipfire  -T  -r 2017/07/07/nfcapd.201707072240

This returns a tons of traffic captured! ^-^

I'll leave it to collect data over night...

Many, many thanks!

PS: if you need some additional test, do tell me!

Have a nice evening!
H&M

ummeegge
Community Developer
Community Developer
Posts: 4122
Joined: October 9th, 2010, 10:00 am

Re: Nfsen-Nfdump-fprobe netflow

Post by ummeegge » July 7th, 2017, 8:13 pm

Sounds lovely ^-^ ,
thanks for your testings and your offer, this might be great to get possibly more out of this solution. Am tensed what you get out of it...

Was hanging the last two days on another idea to keep IPFire more lightweight. Tried ELK (Elasticsearch, Logstash and Kibana) on an external machine and left softflowd in IPFire to deliver all probes to it but my first tries where not so delightful... Even Nfsen´s look and style is not that modern it delivers a lot of possibilities which i really appreciate.
Did you checked may the filter in "Details" section (tcpdump sytle syntax) ?
The Profiles are also a nice one to have, loved to get specific VPN and SSH, ..., connection tracking possibilities to get a better overview of who did what when but i need there also more insights... A nice idea for an automated profile setup can also be found in here --> http://www.linuxscrew.com/2012/03/15/nf ... breakdown/ but also a next one which i want to try out are the plugins which are avaliable for Nfsen e.g. --> https://github.com/mdjunior/nfsen-plugins .

Might be nice to get some of your ideas ;) ...

Have a nice evening.

Greetings,

Erik

EDIT: The 32bit version of softflowd misses the automatic symlink function in the package. This means that after a reboot softflowd won´t come up. Have fixed this and will add it after the rebuild (tomorrow morning) into the package... <-- Has been fixed at 08.juli-12:47h .
Image
Image
Image

Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests