Flow mode
Contents
Introduction
This tutorial details the use of the Flow mode being the default operational technique of the Anteater. As we discuss the enhanced aggregation and masking features in a different tutorial, we like to concentrate here on the flow timeout features often used in anomaly research, such as botnet detection and practical network monitoring applications.
Preparation
In order to do so, we need to prepare T2. If you did not complete the tutorials before, just follow the procedure described below.
First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin
folder ~/.tranalyzer/plugins and compile only
basicFlow,
basicStats and
txtSink.
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied
t2build tranalyzer2 basicFlow basicStats txtSink
...
BUILD SUCCESSFUL
If you did not create a separate data and results directory yet, please do it now in another cmd window, that facilitates your workflow:
mkdir ~/data ~/results
The anonymized sample PCAP being used here, can be downloaded here: faf-exercise.pcap. Please extract it under your ~/data folder. Now you are all set for T2 flow timeout experiments.
Flow timeout control
The Flow timeout controls of the Anteater is a core function, hence residing in the tranalyzer.h file:
tranalyzer2
vi src/tranalyzer.h
...
// Time mode
#define RELTIME 0 // 0: Absolute time, 1: Relative internal time
// Maximum lifetime of a flow
#define FDURLIMIT 0 // if > 0; forced flow life span of n +- 1 seconds
#define FDLSFINDEX 0 // if (FDURLIMIT) 0: Different findex; 1: Same findex for flows of a superflow
// The standard timeout for a flow in seconds
#define FLOW_TIMEOUT 182 // flow timeout after a packet is not seen after n seconds
...RELTIME defines the internal time base to be absolute or relative to the
pcap or interface ops start; default is absolute. For graphical applications,
it is sometimes convenient to have smaller time values relative to 0.
We used it in the Basic traffic volume and connection analysis tutorial
to produce a anomaly plot for the connStat plugin.
FLOW_TIMEOUT controls the timeout and releases flows from memory if no more packets are seen since FLOW_TIMEOUT seconds.
It is chosen a bit larger than three minutes, as most L7 protocols have lower
timeouts, so we capture most of them into one flow. Nevertheless, a large FLOW_TIMEOUT
saves flow allocation and output operations, but also increases the flow lifetime in memory.
So you need to optimize this value in high speed applications.
FDURLIMIT controls independent of FLOW_TIMEOUT the forced release of each flow in memory,
thus being a simulated Cisco flow mode. It is activated with a non zero value in seconds. Hence,
a value of 10 would terminate an active flow after 10 seconds if a new packet is encountered,
and start a new sub-flow with the next packet having the same flow hash. If there is no
new packet with the right hash value, then there is no flow release, until an appropriate packet
arrives.
This is very different from FLOW_TIMEOUT because any packet can trigger the flow termination.
It is a built in FORCE MODE as it does not use the T2_RM_FLOW(flowP) function, see
Force mode tutorial.
If FDURLIMIT is enabled FDLSFINDEX controls the enumeration of the sub-flows. Set it
to 1 and all sub-flows keep the findex of the original flow. Otherwise the findex
increments normally. This feature facilitates to extract sub-flows belonging to a flow.
================================================================================ Tranalyzer 0.8.14 (Anteater), Tarantula. PID: 45166 ================================================================================ [INF] Creating flows for L2, IPv4, IPv6 Active plugins: 01: basicFlow, 0.8.14 02: basicStats, 0.8.14 03: txtSink, 0.8.14 [INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406105 (406.11 K) [INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51345 (51.34 K) Processing file: /home/wurst/data/faf-exercise.pcap Link layer type: Ethernet [EN10MB/1] Dump start: 1258544215.037210 sec (Wed 18 Nov 2009 11:36:55 GMT) Dump stop : 1258594491.683288 sec (Thu 19 Nov 2009 01:34:51 GMT) Total dump duration: 50276.646078 sec (13h 57m 56s) Finished processing. Elapsed time: 0.037299 sec Finished unloading flow memory. Time: 0.037335 sec Percentage completed: 100.00% Number of processed packets: 5902 (5.90 K) Number of processed bytes: 4993414 (4.99 M) Number of raw bytes: 4993414 (4.99 M) Number of pcap bytes: 5087870 (5.09 M) Number of IPv4 packets: 5902 (5.90 K) [100.00%] Number of A packets: 1986 (1.99 K) [33.65%] Number of B packets: 3916 (3.92 K) [66.35%] Number of A bytes: 209315 (209.31 K) [4.19%] Number of B bytes: 4784099 (4.78 M) [95.81%] Average A packet load: 105.40 Average B packet load: 1221.68 (1.22 K) -------------------------------------------------------------------------------- basicStats: Biggest L3 talker: 143.166.11.10 (US): 3101 (3.10 K) [52.54%] packets basicStats: Biggest L3 talker: 143.166.11.10 (US): 4436320 (4.44 M) [88.84%] bytes -------------------------------------------------------------------------------- Headers count: min: 3, max: 3, average: 3.00 Number of TCP packets: 5902 (5.90 K) [100.00%] Number of TCP bytes: 4993414 (4.99 M) [100.00%] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Number of processed flows: 73 Number of processed A flows: 37 [50.68%] Number of processed B flows: 36 [49.32%] Number of request flows: 36 [49.32%] Number of reply flows: 37 [50.68%] Total A/B flow asymmetry: 0.01 Total req/rply flow asymmetry: -0.01 Number of processed packets/flows: 80.85 Number of processed A packets/flows: 53.68 Number of processed B packets/flows: 108.78 Number of processed total packets/s: 0.12 Number of processed A+B packets/s: 0.12 Number of processed A packets/s: 0.04 Number of processed B packets/s: 0.08 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Number of average processed flows/s: 0.00 Average full raw bandwidth: 795 b/s Average full bandwidth : 792 b/s Max number of flows in memory: 18 [0.01%] Memory usage: 0.01 GB [0.02%] Aggregated flowStat=0x0400000000004000 [INF] IPv4 flows
The end report states that there are 73 flows in total and 1986 A flow packets and 3916 B flow packets. Now select a flow which is long enough to be early terminated, such as the one with port 1379 involved.
tawk 'port(1379)' ~/results/faf-exercise_flows.txt | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm
A 10 0x0400000000004000 1258562467.749142 1258562509.633370 41.884228 1 3 eth:ipv4:tcp 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 14 18 1801 15606 0 455 128.6429 183.7502 0 22.97809 2.991731 6.053773 0.3342547 42.99948 -0.125 -0.7930717
B 10 0x0400000000004001 1258562467.754689 1258562509.653962 41.899273 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 18 14 15606 1801 0 1380 867 568.1608 0 22.97829 2.327737 5.515566 0.4296017 372.4647 0.125 0.7930717Now let us forcefully terminate flows above a certain length in the next chapter.
Flow duration limit: FDURLIMIT
Set FDURLIMIT to 3, recompile and rerun T2 on the pcap. Then t2 will terminate every
flow with the time frame >= 3 seconds and restart a new flow with the same hash but
different findex.
t2conf tranalyzer2 -D FDURLIMIT=3 && t2build tranalyzer2
t2 -r ~/data/faf-exercise.pcap -w ~/results/faf-exercise3================================================================================ Tranalyzer 0.8.14 (Anteater), Tarantula. PID: 46212 ================================================================================ [INF] Creating flows for L2, IPv4, IPv6 Active plugins: 01: basicFlow, 0.8.14 02: basicStats, 0.8.14 03: txtSink, 0.8.14 [INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406105 (406.11 K) [INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51345 (51.34 K) Processing file: /home/wurst/data/faf-exercise.pcap Link layer type: Ethernet [EN10MB/1] Dump start: 1258544215.037210 sec (Wed 18 Nov 2009 11:36:55 GMT) Dump stop : 1258594491.683288 sec (Thu 19 Nov 2009 01:34:51 GMT) Total dump duration: 50276.646078 sec (13h 57m 56s) Finished processing. Elapsed time: 0.005011 sec Finished unloading flow memory. Time: 0.005026 sec Percentage completed: 100.00% Number of processed packets: 5902 (5.90 K) Number of processed bytes: 4993414 (4.99 M) Number of raw bytes: 4993414 (4.99 M) Number of pcap bytes: 5087870 (5.09 M) Number of IPv4 packets: 5902 (5.90 K) [100.00%] Number of A packets: 1986 (1.99 K) [33.65%] Number of B packets: 3916 (3.92 K) [66.35%] Number of A bytes: 209315 (209.31 K) [4.19%] Number of B bytes: 4784099 (4.78 M) [95.81%] Average A packet load: 105.40 Average B packet load: 1221.68 (1.22 K) -------------------------------------------------------------------------------- basicStats: Biggest L3 talker: 143.166.11.10 (US): 558 [9.45%] packets basicStats: Biggest L3 talker: 143.166.11.10 (US): 799321 (799.32 K) [16.01%] bytes -------------------------------------------------------------------------------- Headers count: min: 3, max: 3, average: 3.00 Number of TCP packets: 5902 (5.90 K) [100.00%] Number of TCP bytes: 4993414 (4.99 M) [100.00%] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Number of processed flows: 104 Number of processed A flows: 53 [50.96%] Number of processed B flows: 51 [49.04%] Number of request flows: 52 [50.00%] Number of reply flows: 52 [50.00%] Total A/B flow asymmetry: 0.02 Total req/rply flow asymmetry: 0.00 Number of processed packets/flows: 56.75 Number of processed A packets/flows: 37.47 Number of processed B packets/flows: 76.78 Number of processed total packets/s: 0.12 Number of processed A+B packets/s: 0.12 Number of processed A packets/s: 0.04 Number of processed B packets/s: 0.08 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Number of average processed flows/s: 0.00 Average full raw bandwidth: 795 b/s Average full bandwidth : 792 b/s Max number of flows in memory: 18 [0.01%] Memory usage: 0.01 GB [0.02%] Aggregated flowStat=0x0400020000004001 [INF] IPv4 flows
As expected we have now more flows, but the same amount of packets, otherwise something would be
very wrong. The reporting of basicStats changed as the packets of an specific IP are now scattered
over many flows. And the end report has the RMFLOW bit set to indicate that flows are forcefully
terminated.
If you extract now the same flow as above you realize that the flows have a smaller length
as three seconds. Note the time difference between timeLast of flow 10 and timeFirst of flow 11 of
10 seconds. Hence, after 10 seconds a new packet came in and flow 10 terminated with its last packet
at 1258562467.900050 and restarted B-flow 11 with this very packet at 1258562477.772691.
This happens when you have bursty traffic. If the traffic would be regular, all flows would have
a duration of three seconds. The RMFLOW bit indicates the flow where the FDURLIMIT timeout
occurred.
tawk 'port(1379)' ~/results/faf-exercise3_flows.txt | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm
A 10 0x0400020000004000 1258562467.749142 1258562467.900050 0.150908 1 3 eth:ipv4:tcp 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 4 3 449 444 0 449 112.25 159.8441 0 0.144417 0.037727 0.05336916 26.50622 2975.323 0.1428571 0.005599104
B 10 0x0400000000004001 1258562467.754689 1258562467.761692 0.007003 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 3 4 444 449 0 444 148 170.8957 0 0.006751 0.002334333 0.002289841 428.3878 63401.4 -0.1428571 -0.005599104
A 11 0x0400000000004000 1258562477.713894 1258562478.454425 0.740531 1 3 eth:ipv4:tcp 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 8 13 1352 15162 0 455 169 185.5386 0 0.380024 0.09256638 0.09487616 10.80306 1825.717 -0.2380952 -0.8362601
B 11 0x0400020000004001 1258562477.772691 1258562478.454017 0.681326 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 13 8 15162 1352 10 1380 1166.308 353.6241 0 0.426916 0.05240969 0.0970576 19.08044 22253.67 0.2380952 0.8362601
A 13 0x0400020000004000 1258562501.432514 1258562501.432514 0.000000 1 3 eth:ipv4:tcp 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
B 13 0x0400000000004001 1258562501.432305 1258562501.432305 0.000000 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
A 16 0x0400000000004000 1258562509.633370 1258562509.633370 0.000000 1 3 eth:ipv4:tcp 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
B 16 0x0400000000004001 1258562509.653962 1258562509.653962 0.000000 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0Yes I picked this traffic on purpose to demonstrate that behavior of T2.
As you can see our flow 10 is now divided into four flows 10, 11, 13 and 16.
But if you want to select all the sub flows you need to select the
port or the full 6 tuple if there are many more flows this the same port
or IP pair. In order to facilitate that operation, since 0.8.8lmw2
the FDLSFINEX switch was added to label all subflows with the same
findex.
Flow duration, same index: FDLSFINDEX
In order to label all sub-flows with the same findex, the constant FDLSFINDEX
has to be enabled in conjunction with a non-zero value of FDURLIMIT
in tranalyzer.h or just use t2conf as indicated below.
t2conf tranalyzer2 -D FDLSFINDEX=1 && t2build tranalyzer2
t2 -r ~/data/faf-exercise.pcap -w ~/results/faf-exercise3f... -------------------------------------------------------------------------------- basicStats: Biggest L3 talker: 143.166.11.10 (US): 558 [9.45%] packets basicStats: Biggest L3 talker: 192.168.1.104: 7366431884 (7.37 G) [147522.95%] bytes -------------------------------------------------------------------------------- Headers count: min: 3, max: 3, average: 3.00 Number of GRE packets: 10 [0.17%] Number of TCP packets: 5902 (5.90 K) [100.00%] Number of TCP bytes: 4993414 (4.99 M) [100.00%] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ...
The end report looks the same as above with the exception that now
the FDLSIDX bit is set, indicating that all sub-flows belonging
to the original, un-terminated flow have now the same findex. Now select flow 10 again
and you discover that the flows 10, 11, 13 and 16 have now all findex 10
which is much easier to extract as indicated below.
tawk 'flow(10)' ~/results/faf-exercise3f_flows.txt | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm
A 10 0x0400020000004000 1258562467.749142 1258562467.900050 0.150908 1 0 fe:63:a3:b6:0d:22;46:d9:fe:63:a3:b6;ff:44:23:f6:80:ed 6d:59:a5:aa:46:d9;b1:e7:6d:59:a5:aa;0f:da:77:2f:6d:62 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 4 3 537 512 20 469 134.25 158.9655 0 0.144417 0.037727 0.05336916 26.50622 3558.46 0.1428571 0.02383222
B 10 0x0400000000004001 1258562467.754689 1258562467.761692 0.007003 1 0 a5:aa:46:d9:fe:63;68:75:6e:64:65:72;42:3e:be:0f:da:77 eb:f0:b1:e7:6d:59;47:45:54:20:2f:74;65:0d:0a:0d:0a:45 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 3 4 512 537 20 464 170.6667 169.3718 0 0.006751 0.002334333 0.002289841 428.3878 73111.52 -0.1428571 -0.02383222
A 10 0x4400000000004000 1258562477.713894 1258562478.454425 0.740531 1 0 53:92:60:01:48:7c;0a:53:65:72:76:65;0e:29:7b:63:67:d3 ff:44:23:f6:80:ed;30:30:20:4f:4b:0d;f4:37:fa:7b:fd:92 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 8 13 1512 15422 20 475 189 185.5386 0 0.380024 0.09256638 0.09487616 10.80306 2041.778 -0.2380952 -0.8214244
B 10 0x4400020000004001 1258562477.772691 1258562478.454017 0.681326 1 0 00:00:00:00:00:00;0a:53:65:72:76:65;ac:fc:04:db:61:d4 00:00:00:00:00:00;30:30:20:4f:4b:0d;56:29:14:c7:fc:a2 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 13 8 15422 1512 30 1400 1186.308 353.6241 0 0.426916 0.05240969 0.0970576 19.08044 22635.27 0.2380952 0.8214244
A 10 0x4400020000004000 1258562501.432514 1258562501.432514 0.000000 1 1 8d:26:6f:2f:b2:39 b2:87:64:65:6c:99 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 1 1 20 20 20 20 20 0 0 0 0 0 0 0 0 0
B 10 0x4400000000004001 1258562501.432305 1258562501.432305 0.000000 1 8 36:11:6e:37:b2:87 00:00:00:00:00:00 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 1 1 20 20 20 20 20 0 0 0 0 0 0 0 0 0
A 10 0x4400000000004000 1258562509.633370 1258562509.633370 0.000000 1 1 8d:26:6f:2f:b2:39 b2:87:64:65:6c:99 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 1 1 20 20 20 20 20 0 0 0 0 0 0 0 0 0
B 10 0x4400000000004001 1258562509.653962 1258562509.653962 0.000000 1 8 5e:ce:75:33:b2:87 00:00:00:00:00:00 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 1 1 20 20 20 20 20 0 0 0 0 0 0 0 0 0Now let’s explore the difference to the flow timeout.
No packets seen since: FLOW_TIMEOUT
As explained above the constant FLOW_TIMEOUT controls the lifetime of
a flow if a packet was not seen. Which is very different from the
FDURLIMIT which terminates a flow and starts a new flow after an
internal flow life time.
The timeout of flows can be very useful if you want to follow protocols and change the timeout in dependence of a protocol state, as implemented in tcpStates plugin.
FLOW_TIMEOUT is actually a float, so you can set a timeout Π seconds
if you like. We choose 3 as we like to compare it to the previous experiments.
So switch FDURLIMIT off, set FLOW_TIMEOUT to 3, recompile and rerun t2 on the same pcap.
t2conf tranalyzer2 -D FDURLIMIT=0 -D FLOW_TIMEOUT=3 && t2build tranalyzer2
t2 -r ~/data/faf-exercise.pcap -w ~/results/faf-exercise3t... -------------------------------------------------------------------------------- basicStats: Biggest L3 talker: 143.166.11.10 (US): 3101 (3.10 K) [52.54%] packets basicStats: Biggest L3 talker: 143.166.11.10 (US): 4436320 (4.44 M) [88.84%] bytes -------------------------------------------------------------------------------- Headers count: min: 3, max: 3, average: 3.00 Number of TCP packets: 5902 (5.90 K) [100.00%] Number of TCP bytes: 4993414 (4.99 M) [100.00%] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Number of processed flows: 88 Number of processed A flows: 45 [51.14%] Number of processed B flows: 43 [48.86%] Number of request flows: 44 [50.00%] Number of reply flows: 44 [50.00%] Total A/B flow asymmetry: 0.02 Total req/rply flow asymmetry: 0.00 Number of processed packets/flows: 67.07 Number of processed A packets/flows: 44.13 Number of processed B packets/flows: 91.07 Number of processed total packets/s: 0.12 Number of processed A+B packets/s: 0.12 Number of processed A packets/s: 0.04 Number of processed B packets/s: 0.08 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ...
Now we have 88 flows, not 73 as with 182 sec timeout or 104 with FDURLIMIT.
What is the reason? If you select the same port 1379 as before, you discover
that flow 10 shows the same behavior as with FDURLIMIT set to 3 sec,
because all flows did not see a packet for 3 seconds, hence they terminate
and the following packet landed in the next flow. Oups, bad example to
show the difference? Actually not, because it illustrates the need to
be observant, because the circumstances of the traffic produce for the
this very flow the same flow chunks.
tawk 'port(1379)' ~/results/faf-exercise3t_flows.txt | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm
A 10 0x0400000000004000 1258562467.749142 1258562467.900050 0.150908 1 3 eth:ipv4:tcp 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 4 3 449 444 0 449 112.25 159.8441 0 0.144417 0.037727 0.05336916 26.50622 2975.323 0.1428571 0.005599104
B 10 0x0400000000004001 1258562467.754689 1258562467.761692 0.007003 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 3 4 444 449 0 444 148 170.8957 0 0.006751 0.002334333 0.002289841 428.3878 63401.4 -0.1428571 -0.005599104
A 11 0x0400000000004000 1258562477.713894 1258562478.454425 0.740531 1 3 eth:ipv4:tcp 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 8 13 1352 15162 0 455 169 185.5386 0 0.380024 0.09256638 0.09487616 10.80306 1825.717 -0.2380952 -0.8362601
B 11 0x0400000000004001 1258562477.772691 1258562478.454017 0.681326 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 13 8 15162 1352 10 1380 1166.308 353.6241 0 0.426916 0.05240969 0.0970576 19.08044 22253.67 0.2380952 0.8362601
A 13 0x0400000000004000 1258562501.432514 1258562501.432514 0.000000 1 3 eth:ipv4:tcp 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
B 13 0x0400000000004001 1258562501.432305 1258562501.432305 0.000000 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
A 16 0x0400000000004000 1258562509.633370 1258562509.633370 0.000000 1 3 eth:ipv4:tcp 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 192.168.1.104 07 "Private network" 1379 63.245.221.11 us "Mozilla Corporation" 80 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
B 16 0x0400000000004001 1258562509.653962 1258562509.653962 0.000000 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 63.245.221.11 us "Mozilla Corporation" 80 192.168.1.104 07 "Private network" 1379 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0$Now look for port 49330, you will see two flows, one single A with one packet. The findex numbers differ from each other, therefore we select the port.
tawk 'port(49330)' ~/results/faf-exercise3t_flows.txt | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm
A 42 0x0400000000004000 1258594163.408285 1258594185.427968 22.019683 1 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 1513 3101 0 4268858 0 0 0 0 0 0.671332 0.01455366 0.0370894 68.71125 0 -0.3441699 -1
B 42 0x0400000000004001 1258594163.487027 1258594185.427506 21.940479 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 3101 1513 4268858 0 0 1380 1376.607 60.23097 0 0.67109 0.007075286 0.02745561 141.3369 194565.4 0.3441699 1
A 44 0x0400000000004000 1258594191.015208 1258594191.015208 0.000000 1 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0The FLOW_TIMEOUT produces one more flow 44, a packet which comes after the timeout. Most of the packets
are coming very regularly, in contrast to flow 10 in the example above.
The same flow from the previous chapter shows nine flows mostly evenly 3 seconds long. See the differences between those modes? Compare it to the same flow from the previous chapter listed below.
tawk 'port(49330)' ~/results/faf-exercise3f_flows.txt | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm
A 36 0x0400000000004000 1258594163.408285 1258594165.927703 2.519418 1 0 6f:64:65:20:28:31;73:69:76:65:20:4d;ff:00:74:41:8d:45 73:69:76:65:20:4d;04:02:13:6f:4b:cd;09:80:bd:e9:fe:ff 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 64 103 2132 140070 20 48 33.3125 11.21809 0 0.671332 0.03936591 0.1132588 25.40269 846.2272 -0.2335329 -0.9700145
B 36 0x0400020000004001 1258594163.487027 1258594165.927248 2.440221 1 0 4b:cd:73:69:76:65;20:b7:0d:5d:25:d7;6a:5d:b3:75:06:86 01:01:04:02:13:6f;40:36:58:4a:06:7c;00:00:00:00:00:1c 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 103 64 140070 2132 21 1400 1359.903 216.1209 0 0.67109 0.02369147 0.09572505 42.20929 57400.54 0.2335329 0.9700145
A 36 0x4400000000004000 1258594166.005912 1258594168.988792 2.982880 1 0 35:90:67:71:15:c1;4b:34:e8:e4:32:c2;f1:a7:67:71:15:c1 72:2e:6c:a9:4a:b6;67:71:15:c1:5c:92;bd:62:dc:62:7d:17 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 223 444 5256 620220 20 40 23.56951 5.735339 0 0.079202 0.01337614 0.02821018 74.75996 1762.055 -0.3313343 -0.9831936
B 36 0x4400020000004001 1258594166.005251 1258594168.988590 2.983339 1 0 20:b7:0d:5d:25:d7;64:8f:94:1e:39:1d;3d:a9:d9:9b:e3:70 40:36:58:4a:06:7c;86:94:0f:95:c7:74;d7:c2:b0:23:80:77 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 444 223 620220 5256 100 1400 1396.892 59.41461 0 0.079166 0.006719232 0.02111784 148.8265 207894.6 0.3313343 0.9831936
A 36 0x4400000000004000 1258594169.065239 1258594171.945675 2.880436 1 0 db:7c:58:f8:e7:c6;1f:88:a4:9d:18:76;ff:25:95:84:69:f2 e2:a9:31:cb:0a:5f;9c:d9:2b:f5:73:55;29:cb:97:0c:ac:60 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 194 402 4564 561943 20 32 23.52577 5.416288 0 0.128673 0.01484761 0.03038867 67.35091 1584.482 -0.3489933 -0.9838872
B 36 0x4400020000004001 1258594169.064789 1258594171.944501 2.879712 1 0 20:b7:0d:5d:25:d7;99:e1:f1:11:d1:9a 40:36:58:4a:06:7c;74:97:7e:53:6b:82 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 402 194 561943 4564 543 1400 1397.868 42.63219 0 0.128665 0.007163462 0.02259158 139.5973 195138.6 0.3489933 0.9838872
A 36 0x4400000000004000 1258594172.021878 1258594174.978021 2.956143 1 0 a8:11:a9:8d:21:5a;86:cf:16:c4:b1:28;01:37:9b:6a:46:36 8d:81:3b:ac:da:18;79:36:3f:ef:b6:e5;c7:17:6f:fe:9f:05 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 226 436 5636 610215 20 48 24.93805 6.821356 0 0.124433 0.01308028 0.02880465 76.45097 1906.538 -0.3172205 -0.9816968
B 36 0x4400020000004001 1258594172.021456 1258594174.977086 2.955630 1 0 20:b7:0d:5d:25:d7;29:2d:49:4b:5b:14 40:36:58:4a:06:7c;ae:d8:e2:0a:d8:1a 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 436 226 610215 5636 1215 1400 1399.576 8.828124 0 0.124927 0.006778968 0.02202431 147.5151 206458.5 0.3172205 0.9816968
A 36 0x4400000000004000 1258594175.055730 1258594177.968394 2.912664 1 0 1f:93:d9:2e:17:f1;34:cc:ea:ef:04:61;51:c6:08:2a:ae:a5 37:64:68:30:f6:78;44:a4:74:6f:5f:42;79:8e:65:d3:d1:f1 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 233 527 5664 737042 20 48 24.30901 7.982298 0 0.079708 0.0125007 0.02766376 79.9955 1944.612 -0.3868421 -0.9847476
B 36 0x4400020000004001 1258594175.055030 1258594177.966989 2.911959 1 0 20:b7:0d:5d:25:d7;bf:37:09:ad:90:41;35:51:96:70:6e:50 40:36:58:4a:06:7c;8f:cd:a5:bc:6d:02;97:e4:4a:25:76:c9 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 527 233 737042 5664 716 1400 1398.562 29.66002 0 0.07942 0.005525537 0.01959059 180.9778 253108.7 0.3868421 0.9847476
A 36 0x4400000000004000 1258594178.040853 1258594180.952029 2.911176 1 0 22:30:af:3d:9e:1c;2a:f7:81:67:e2:47;e0:da:4a:86:94:77 51:a3:5b:b4:68:c1;4e:8b:e3:fe:a5:b2;9d:cd:c8:01:28:7f 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 261 558 6380 780349 20 40 24.44444 5.955973 0 0.078208 0.01115393 0.02653476 89.65449 2191.554 -0.3626374 -0.9837809
B 36 0x4400020000004001 1258594178.040152 1258594180.951598 2.911446 1 0 20:b7:0d:5d:25:d7;ff:c9:2f:09:63:f2 40:36:58:4a:06:7c;37:43:6e:ad:d6:f6 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 558 261 780349 6380 549 1400 1398.475 35.87016 0 0.078343 0.005217645 0.01904833 191.6573 268028 0.3626374 0.9837809
A 36 0x4400000000004000 1258594181.028734 1258594183.937406 2.908672 1 0 f5:69:e6:87:99:1e;70:ce:95:5d:ca:97;79:c7:a1:8e:1b:3a 48:ad:11:b0:73:08;ca:b6:29:8e:58:8f;74:6c:eb:d8:d9:71 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 221 476 5408 666282 20 48 24.47059 6.61699 0 0.079713 0.01316141 0.02855756 75.97969 1859.268 -0.3658537 -0.9838973
B 36 0x4400020000004001 1258594181.028052 1258594183.937122 2.909070 1 0 20:b7:0d:5d:25:d7;c9:85:3a:55:c3:52 40:36:58:4a:06:7c;41:9b:2c:e9:b8:c9 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 476 221 666282 5408 1282 1400 1399.752 5.360048 0 0.079432 0.006111494 0.02068904 163.6262 229036.1 0.3658537 0.9838973
A 36 0x4400020000004000 1258594184.015600 1258594185.427968 1.412368 1 0 bb:7b:2c:4e:c5:26;3f:ff:ed:4c:fe:48;ac:50:de:51:ac:5e 54:51:0a:5d:0c:a1;fc:c7:f5:8a:e0:1d;a4:b4:6e:7c:d4:ef 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 91 155 2528 214765 20 48 27.78022 9.071207 0 0.079954 0.01552053 0.03033226 64.43079 1789.902 -0.2601626 -0.9767319
B 36 0x4400000000004001 1258594184.015154 1258594185.427506 1.412352 1 0 20:b7:0d:5d:25:d7;25:06:08:2b:06:01;2c:2d:49:d0:30:82 40:36:58:4a:06:7c;01:04:29:30:27:30;00:00:00:00:00:00 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 155 91 214765 2528 20 1400 1385.581 129.1754 0 0.079416 0.009111947 0.02447955 109.746 152062 0.2601626 0.9767319
A 36 0x4400000000004000 1258594191.015208 1258594191.015208 0.000000 1 5 70:6c:65:74:65:2e 65:72:20:63:6f:6d 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 1 0 20 0 20 20 20 0 0 0 0 0 0 0 1 1Conclusion
Do not forget to reset the config to default for the other tutorials.
t2conf --reset -a && t2build -R
Have fun experimenting!