2006-10-15 Netflow In Dynamic Environment

As you are maybe aware, I have a kind of interest in Netflow for some security monitoring projects. The past days, I was looking in the different methods to export Netflow on dedicated devices in a dynamic network connectivity. First, what do I mean by a "dynamic network connectivity" ? It's a connectivity where you have not that much control over the allowed protocols or communication mechanisms able to cross that network. The typical example is an Internet connectivity over a GPRS network or in a hotel room. You have for example only HTTP access and nothing more. It's often very difficult to know what will be the protocol in such environment. For a project in the area of emergency, this is very comment to have such "dynamic environment". We decided in a project to use Netflow version 5 and ipfix as a common method to monitor ip flows. The major issue in the project is that we cannot rely on a full tcp/ip connectivity to Internet or other IP networks. The idea was to use the common available protocols in such environment to "tunnel" netflow. The two main protocols often available are : DNS (name resolution) and HTTP. In the beginning, I was planning to use DNS-based transport mechanism. The DNS client (on the monitored device) is sending specific request to a specific domainname maintained by a specific nameserver. In order to overcome the DNS limitation (e.g. : the size of the UDP packet or the query method available), I was planning a method like the one used by the Malware database. The idea was to split each PDU of a Netflow datagram into a sequence of DNS queries. I quickly discovered with a small prototype using dpkt that was the wrong way. Two many DNS queries for a small amount of Netflow datagram in another word, too much overhead. But I left the approach due to the fact that a majority of DNS servers are discarding some requests using a "GPRS¨ provider (to limit DNS tunneling or use of bandwidth ?). So the remaining transport mechanism in a "dynamic network connectivity" is HTTP and very often proxified. As the first testing version is using Netflow version 5, the protocol used by the Netflow exporters, in that case, is UDP. I found a way to encapsulate UDP using HTTP with firepass (and it works quite well for non-interactive protocols like Netflow). I didn't want to go in the direction of a full-featured VPN/Tunnel/IP-over-HTTP as the cost of setup could be quite important and it's not very welcomed in such "dynamic environment". I'll use a similar Netflow-over-HTTP approach for the following weeks and keep you inform of any evolution.

Blog entry also available on the flowog site