= An introduction to network forensic - Courses notes 2016-2017 Alexandre Dulaunoy :Author: Alexandre Dulaunoy :icons: font :toc: :doctype: book :source-highlighter: coderay CAUTION: The courses notes are just for reference but don't forget to take your own notes during the classes. Network forensic is the art of discovering and analyzing security incidents relying on network evidences. == Network Evidences NOTE: A lot of security devices include packet capture capabilities including NIDS (Network Intrusion Detection System), network proxies or even servers. === Collecting Network Evidences Network evidences like packet captures are at the root of a good analysis. If your evidences are partial or incorrect, you'll produce incorrect or partial analysis. The collection of network evidences is often underestimated and seen as a simple process. Factors to check while performing a network capture: * Time reference (Is your system time synchronized? or What to do with network packet capture using an incorrect time-stamp?) * Is a passive network collection always passive? (e.g. Are you doing name resolution? Do you have internet access from your network collection system? Can my network collection system be exploited?) * Do you loose packets during the capture (or before?)? (e.g. Using netbeacon? Do you have any statistics from your network packet capture device?) * Do you capture the correct size? (e.g. jumbo frame?) * How large will be my packet capture? Should I have small file or large network capture files? Is my disk large enough to store the network capture? Is my disk fast enough to write capture files? * Which format should I use to store my network capture? (pcap, pcap-ng?) ==== Time References It's not uncommon to receive network capture files where the capture system was not time synchronized. To recover the original time, you can dig into the capture to find additional elements like HTTP headers, NTP packets or other time references. editcap -t can be used to adjust time difference. The value is expressed in seconds (with even the fractional seconds). If you know that you have 30 seconds of difference from a reference time, editcap can adjust the time in your network packet capture file: ---- editcap -t 30 original.pcap updated.pcap ---- WARNING: editcap will only change the time-stamp from the network capture header. Time information in the payload won't be updated (e.g. HTTP headers). ==== Passive Network Collection Usually, you don't want to tell everyone (especially the adversaries) that a network capture is performed. If the attacker knows where and how the network capture is performed, he will tend to evade the network capture process. If you want to ensure if your network collection is passive, external dependencies need to be reviewed. The most common example is the use of external name resolvers. If the external resolver is enabled, it will generate two major issues: * The resolver time will be added to the overall process (e.g. tcpdump -r myfile.cap versus tcpdump -n -r myfile.cap) * If the adversary controls a name server (especially reverse PTR records), the information will flow back to the adversary (e.g. leaking the presence of a network capture or NIDS, verifying which packets are intercepted...) ==== Size of Frame Captured When configuring a network capture software, you must take into consideration the length of packet to capture (also called the snaplen). If you need the whole packet (including payload), you have to capture the maximum transmission unit of the link layer. If you have specific constraints like space or privacy, you can limit the capture to the headers of the packets. An example of packet capture limited to 100 bytes (-s in tcpdump): ---- tcpdump -w mycapture.cap -s 100 ---- Another factor is the time to process captured packets. If the packets are bigger, it tends to take more time to process the packets and fill the buffers faster. You might lost some packets during the capture. tshark (part of Wireshark Network Analyzer) has the -s option but can be more granular regarding the buffer size to process the captured packets. -B option permits to increase the buffer size of the capture interface (as long as the capture library supports it). ---- tshark -B 4 -w mycapture.cap -s 100 ---- If you want to know the frame size capture from a pcap file, you can use the 'file' command: [source,shell] [adulau:/home/adulau/dess]↥ $ file capture.cap capture.cap: tcpdump capture file (little-endian) - version 2.4 (Ethernet, capture length 1500) [adulau:/home/adulau/dess]↥ $ file vibrowa.cap vibrowa.cap: tcpdump capture file (little-endian) - version 2.4 (Ethernet, capture length 65535) Before starting the analysis of network capture files, the capture length needs to be reviewed. ==== Network Capture Format Network capture format can be different depending of the capture devices. The most common format is the libpcap format but the pcap format declines in different format improved over time (from pcap original format to pcap-ng). So you have to be careful while performing a network capture to check which format is supported by your network capture tool and review the potential limitation for the analysis. [cols="2,3a,3a"] |=== |type of network capture file |description | | 5views | InfoVista 5View capture | | btsnoop | Symbian OS btsnoop | | commview | TamoSoft CommView | | dct2000 | Catapult DCT2000 trace (.out format) | | erf | Endace ERF capture | | eyesdn | EyeSDN USB S0/E1 ISDN trace format | | k12text | K12 text file | | lanalyzer | Novell LANalyzer | | modlibpcap | Modified tcpdump - libpcap | | netmon1 | Microsoft NetMon 1.x | | netmon2 | Microsoft NetMon 2.x | | nettl | HP-UX nettl trace | | ngsniffer | NA Sniffer (DOS) | | ngwsniffer_1_1 | NA Sniffer (Windows) 1.1 | | ngwsniffer_2_0 | NA Sniffer (Windows) 2.00x | | niobserver | Network Instruments Observer | | nokialibpcap | Nokia tcpdump - libpcap | | nseclibpcap | Wireshark - nanosecond libpcap | | nstrace10 | NetScaler Trace (Version 1.0) | | nstrace20 | NetScaler Trace (Version 2.0) | | pcap | Wireshark/tcpdump/... - pcap | Documentation: pcap-savefile(5) | pcapng | Wireshark/... - pcapng | | rf5 | Tektronix K12xx 32-bit .rf5 format | | rh6_1libpcap | RedHat 6.1 tcpdump - libpcap | | snoop | Sun snoop | | suse6_3libpcap | SuSE 6.3 tcpdump - libpcap | | visual | Visual Networks traffic capture | |=== == Analysing Network Capture Doing network analysis, you usually have more than one way to do it. The challenge when doing network analysis is to find the interesting packets in the haystack. There are various tools to analyse network captures. === Analysis Tools * capinfos (Wireshark) * mergecap (Wireshark) * editcap (Wireshar) * tcpdump * http://www.read.seas.harvard.edu/~kohler/ipsumdump/[ipsumdump] * tshark * tcpflow (1) and tcpflow (2) * ngrep * https://tools.netsa.cert.org/yaf/[yaf] ==== ipsumdump ipsumdump is a tool summarize network captures at network and transport level to provide output for programs. The tool helps to do quick analysis and statistics over a network captures include hashing payload output, list timestamp, specific IP options... The output can be easily parsable with other software or Unix tools (compared to tcpdump ascii output which is more painful to parse). An interesting usage of ipsumdump is to view each IP packet and hash the payload of each packets. By hashing each payload, the packets can be classified and sorted following their respective content (e.g. empty packets, same DDoS payload). ---- [adulau:~]$ ipsumdump -sd --payload-md5-hex -r capture.cap | head -50 158.64.60.71 192.168.1.2 c4d3690418c193f69c5df3563dd6ab57 192.168.1.2 194.154.192.1 50104a1f6262e15a57e81e62726d8231 194.154.192.1 192.168.1.2 fc11a18b16ba58d283314e64b8d54c48 192.168.1.2 158.64.60.71 240a1906d9ff84bf1412c3f1f8fa20cb 158.64.60.71 192.168.1.2 3636b23b1f0f9e57d15f58a93f3c9c6a 192.168.1.2 194.154.192.1 5c042de9d97775422977cb5edc365cdc 194.154.192.1 192.168.1.2 88e18c811089936073aa041f2bacaf38 205.218.249.167 192.168.1.2 331b8e512cf50ef273121c9b356383dc 192.168.1.2 205.218.249.167 400f2f74a5843fbd5a34b040f69a6c1c 61.184.221.42 192.168.1.2 331b8e512cf50ef273121c9b356383dc 192.168.1.2 61.184.221.42 ee56c9ed181b1c3034007ec722648cdb 200.207.190.33 192.168.1.2 331b8e512cf50ef273121c9b356383dc 192.168.1.2 200.207.190.33 de4a31a9cbfaca36e20b0054f9fbaadd 158.64.60.71 192.168.1.2 d41d8cd98f00b204e9800998ecf8427e 192.168.1.2 158.64.60.71 d41d8cd98f00b204e9800998ecf8427e 158.64.60.71 192.168.1.2 d41d8cd98f00b204e9800998ecf8427e 192.168.1.2 158.64.60.71 d41d8cd98f00b204e9800998ecf8427e 158.64.60.71 192.168.1.2 d41d8cd98f00b204e9800998ecf8427e 192.168.1.2 158.64.60.71 d41d8cd98f00b204e9800998ecf8427e 158.64.60.71 192.168.1.2 d41d8cd98f00b204e9800998ecf8427e 192.168.1.2 158.64.60.71 d41d8cd98f00b204e9800998ecf8427e 158.64.60.71 192.168.1.2 d41d8cd98f00b204e9800998ecf8427e ---- == Passive DNS analysis === Extracting and Decoding DNS Packets tshark provides various decoders including DNS decoding. If you want to have a look at all the fields decoded: ---- tshark -G fields ---- As an example, extracting the query name of all DNS requests: ---- tshark -Tfields -e dns.qry.name -r capture.cap ---- HTTP headers can also easily extract from IP packets like the User-Agent header: ---- [adulau:~]$ tshark -E header=yes -E separator=, -Tfields -e http.user_agent -r capture.cap | grep -v "^$" http.user_agent Wget/1.7 Wget/1.7 Wget/1.7 Wget/1.7 ----