Force mode
Introduction
This tutorial details the use of the Force mode of the Anteater, which enables the user to produce packet based plugin controlled flow release. Hence, any plugin can invoke an internal signal to release a specific flow at any time. An easy way to simulate L2-7 content based flow release which immediately produces a flow when a certain packet of interest is detected. The following plugins implement the force mode:
- basicStats (if 64 bit count registers overrun)
- dnsDecode (if arrays which hold names overrun)
- radiusDecode (when an access accept or reject is received)
Preparation
In order to do so, we need to prepare T2. If you did not complete the tutorials before, just follow the procedure described below.
First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin
folder ~/.tranalyzer/plugins and compile only
basicFlow,
basicStats,
tcpStates and
txtSink.
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied
t2build tranalyzer2 basicFlow basicStats tcpStates txtSink
...
BUILD SUCCESSFUL
If you did not create a separate data and results directory yet, please do it now in another cmd window, that facilitates your workflow:
mkdir ~/data ~/results
The anonymized sample PCAP being used here, can be downloaded here: faf-exercise.pcap. Please extract it under your data folder. Now you are all set for T2 force mode experiments.
Core force control
The Force mode controls of the Anteater are residing in the tranalyzer.h file:
tranalyzer2
vi src/tranalyzer.h
...
// Tranalyzer User Operational modes
// Operation modes, Plugins which use these modes have to be recompiled
#define ALARM_MODE 0 // only flow output if an Alarm based plugin fires
#define ALARM_AND 0 // if (ALARM_MODE == 1) then 1: AND, 0: OR
#define FORCE_MODE 0 // netflow mode: parameter induced flow termination, implemented by plugins
#define BLOCK_BUF 0 // block unnecessary buffer output when non tranalyzer format event-based plugins are active: e.g. Syslog, ArcSight crap
#define USE_T2BUS 0 // XXX experimental (not implemented yet)
...To enable the force mode edit tranalyzer.h and set FORCE_MODE=1 or
use t2conf as shown below, then rebuild all loaded plugins. Several plugins can trigger an alarm concurrently, as we do
not know what kind of plugin armada you might develop in future.
t2conf tranalyzer2 -D FORCE_MODE=1 && t2build -R
Now the force mode is activated in the core and all plugins which implement it.
Plugin force register and control
Plugins which implement the FORCE mode are basicStats, dnsDecode and radiusDecode.
Let’s look at basicStats first.
basicStats
ls src
basicStats.c basicStats.h Makefile.am
Open basicStats.c in an editor, move to the bl_claimInfo(...) function and search
for the FORCE_MODE macro. If numTBytes or numTPkts are about to be overrun
by the next packet the current flow is terminated and a new flow begins.
The macro T2_RM_FLOW(flowP) does all that for you, so you could now
add the force mode to other plugins to your liking in a heartbeat.
vi src/basicStats.c
...
static inline void bl_claimInfo(packet_t *packet, unsigned long flowIndex) {
bSFlow_t * const bSFlowP = &bSFlow[flowIndex];
...
#if FORCE_MODE == 1
if (UNLIKELY(UINT64_MAX - bSFlowP->numTBytes < ipLength || bSFlowP->numTPkts >= UINT64_MAX)) {
flow_t * const flowP = &flows[flowIndex];
T2_RM_FLOW(flowP);
}
#endif // FORCE_MODE == 1
...
}If you are interested to add the force mode to your own plugin please refer to the plugin force mode tutorial.
Now let’s see how it works by changing the threshold of numTPkts down to 1023. Just copy the original line for easier change back.
...
#if FORCE_MODE == 1
//if (UNLIKELY(UINT64_MAX - bSFlowP->numTBytes < ipLength || bSFlowP->numTPkts >= UINT64_MAX)) {
if (UNLIKELY(UINT64_MAX - bSFlowP->numTBytes < ipLength || bSFlowP->numTPkts >= 1023)) {
flow_t * const flowP = &flows[flowIndex];
T2_RM_FLOW(flowP);
}
#endif // FORCE_MODE == 1
...Now recompile basicStats and run t2:
t2build basicStats
t2 -r ~/data/faf-exercise.pcap -w ~/results
So 3 flows terminated early. The released flows are flagged in the flowStat column by the RMFLOW bit, as define in flow.h in the T2 core:
Note, the 4th line of the report, T2 informs about the [FORCE] mode.
The flowStat also contains the information that this flow was forcefully terminate.
The flowStat column with value 0x0400020000004000 is to be interpreted as follows: bit | flowStat | Description ============================================================================= 14 | 0x0000 0000 0000 4000 | IPv4 flow 41 | 0x0000 0200 0000 0000 | Force mode: remove this flow instantly 58 | 0x0400 0000 0000 0000 | IPv4 packet
So we can select all early terminated flows:
tawk 'bitsanyset($flowStat, 0x0000020000000000)' ~/results/faf-exercise_flows.txt | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm tcpStatesAFlags
B 36 0x0400020000004001 1258594163.487027 1258594172.495175 9.008148 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 1023 523 1405365 0 0 1380 1373.768 84.85606 0 0.67109 0.008805609 0.03729355 113.5639 156010.4 0.3234153 1 0x02
A 37 0x0400020000004000 1258594172.495183 1258594178.674264 6.179081 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 1023 478 1410797 0 696 1380 1379.078 22.22461 0 0.124927 0.006040155 0.02058982 165.5586 228318.2 0.3630913 1 0x03
A 38 0x0400020000004000 1258594178.674299 1258594184.879209 6.204910 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us "Dell" 64334 192.168.1.105 07 "Private network" 49330 6 1023 486 1410771 0 529 1380 1379.053 25.77088 0 0.079432 0.006065406 0.02059324 164.8694 227363.7 0.3558648 1 0x03So numPktsSnt does not exceed 1023 as requested. Please, DO NOT forget to switch back to the original condition for the next
tutorial by removing our changes to basicStats.c and uncomment the original line.
The plugin dnsDecode terminates flows when arrays which hold DNS names, so it acts on DNS_QRECMAX or DNS_ARECMAX:
dnsDecode
vi src/dnsDecode.h
...
/* ========================================================================== */
/* ------------------------ USER CONFIGURATION FLAGS ------------------------ */
/* ========================================================================== */
#define DNS_MODE 4 // 0: Only aggregated header info,
// 1: +Req Content Info,
// 2: +Answer Records,
// 3: +AUX records,
// 4: +Add records
#define DNS_HEXON 0 // 0: Hex Output flags off, 1: Hex output flags on
#define DNS_REQA 0 // 1: Request record aggregation mode
#define DNS_ANSA 0 // 1: Answer record aggregation mode
#define DNS_QRECMAX 15 // Max # of query records / flow
#define DNS_ARECMAX 20 // Max # of answer records / flow
#define DNS_WHO 0 // 1: Country and Organization of DNS reply address; 0: no
#define MAL_TEST 0 // 2: Mal test @ L4Callback, pcap ops; 1: Mal test @ flow terminated
#define MAL_TYPE 1 // 1: Type string; 0: Code
/* +++++++++++++++++++++ ENV / RUNTIME - conf Variables +++++++++++++++++++++ */
/* No env / runtime configuration flags available for dnsDecode */
/* ========================================================================== */
/* ------------------------- DO NOT EDIT BELOW HERE ------------------------- */
/* ========================================================================== */
...As an exercise, release DNS flows if there is more than one reply record in the flow, e.g., set DNS_ARECMAX to 2.
Unload basicStats to reduce complexity of the flow output and run T2 on this pcap 2015-05-08-traffic-analysis-exercise.pcap (Source: malware-traffic-analysis.net)
t2build -u basicStats
t2conf dnsDecode -D DNS_ARECMAX=2 && t2build dnsDecode
t2 -r ~/data/2015-05-08-traffic-analysis-exercise.pcap -w ~/results
So 8 flows are terminate forcefully.
Select the terminated flows:
tawk 'bitsanyset($flowStat, 0x0000020000000000)' ~/results/2015-05-08-traffic-analysis-exercise_flows.txt | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto tcpStatesAFlags dnsStat dnsHdrOPField dnsHFlg_OpC_RetC dnsCntQu_Asw_Aux_Add dnsAAAqF dnsQname dnsAname dnsAPname dns4Aaddress dns6Aaddress dnsQType dnsQClass dnsAType dnsAClass dnsATTL dnsMXpref dnsSRVprio dnsSRVwgt dnsSRVprt dnsOptStat
B 1 0x0400020000004001 1431031896.874326 1431031896.874326 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.2 07 "Private network" 53 192.168.138.158 07 "Private network" 60078 17 0x00 0x0801 0x8180 0x98_0x0001_0x0001 1_1_0_0 1.168421 "va872g.g90e1h.b8.642b63u.j985a2.v33e.37.pa269cc.e8mfzdgrf7g0.groupprograms.in" "va872g.g90e1h.b8.642b63u.j985a2.v33e.37.pa269cc.e8mfzdgrf7g0.groupprograms.in";"" "";"" 62.75.195.236;0.0.0.0 1 1 1;0 1;0 29;0 0;0 0;0 0;0 0;0 0x00000000;0x00000000
B 3 0x0400020000004001 1431031897.655926 1431031897.655926 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.2 07 "Private network" 53 192.168.138.158 07 "Private network" 65315 17 0x00 0x0801 0x8180 0x98_0x0001_0x0001 1_1_0_0 1.170213 "ubb67.3c147o.u806a4.w07d919.o5f.f1.b80w.r0faf9.e8mfzdgrf7g0.groupprograms.in" "ubb67.3c147o.u806a4.w07d919.o5f.f1.b80w.r0faf9.e8mfzdgrf7g0.groupprograms.in";"" "";"" 62.75.195.236;0.0.0.0 1 1 1;0 1;0 29;0 0;0 0;0 0;0 0;0 0x00000000;0x00000000
B 4 0x0400020000004001 1431031897.669844 1431031897.669844 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.2 07 "Private network" 53 192.168.138.158 07 "Private network" 50683 17 0x00 0x0801 0x8180 0x98_0x0001_0x0001 1_1_0_0 1.172043 "r03afd2.c3008e.xc07r.b0f.a39.h7f0fa5eu.vb8fbl.e8mfzdgrf7g0.groupprograms.in" "r03afd2.c3008e.xc07r.b0f.a39.h7f0fa5eu.vb8fbl.e8mfzdgrf7g0.groupprograms.in";"" "";"" 62.75.195.236;0.0.0.0 1 1 1;0 1;0 29;0 0;0 0;0 0;0 0;0 0x00000000;0x00000000
B 13 0x0400020000004001 1431031902.778136 1431031902.778136 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.2 07 "Private network" 53 192.168.138.158 07 "Private network" 53571 17 0x00 0x0801 0x8180 0x98_0x0001_0x0001 1_1_0_0 1.571429 "ip-addr.es" "ip-addr.es";"" "";"" 188.165.164.184;0.0.0.0 1 1 1;0 1;0 21599;0 0;0 0;0 0;0 0;0 0x00000000;0x00000000
B 17 0x0400020000004001 1431031903.089942 1431031903.089942 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.2 07 "Private network" 53 192.168.138.158 07 "Private network" 61720 17 0x00 0x0801 0x8180 0x98_0x0001_0x0001 1_1_0_0 1.571429 "runlove.us" "runlove.us";"" "";"" 204.152.254.221;0.0.0.0 1 1 1;0 1;0 14069;0 0;0 0;0 0;0 0;0 0x00000000;0x00000000
B 20 0x0400020000004001 1431031903.507883 1431031903.507883 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.2 07 "Private network" 53 192.168.138.158 07 "Private network" 56753 17 0x00 0x0801 0x8180 0x98_0x0001_0x0001 1_1_0_0 1.444444 "comarksecurity.com" "comarksecurity.com";"" "";"" 72.34.49.86;0.0.0.0 1 1 1;0 1;0 13888;0 0;0 0;0 0;0 0;0 0x00000000;0x00000000
B 28 0x0400020000004001 1431031941.364104 1431031941.364104 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.2 07 "Private network" 53 192.168.138.158 07 "Private network" 50329 17 0x00 0x0801 0x8180 0x98_0x0001_0x0001 1_1_0_0 1.326531 "7oqnsnzwwnm6zb7y.gigapaysun.com" "7oqnsnzwwnm6zb7y.gigapaysun.com";""You may also change DNS_QREMAX, this might be useful if you are interested in only one query/answer per flow at the arrival of a packet not on flow terminate.
This homework for the reader.
Conclusion
Do not forget revert our code change in basicStats.c.
So the code of basicStats should look like this again
...
#if FORCE_MODE == 1
if (UNLIKELY(UINT64_MAX - bSFlowP->numTBytes < ipLength || bSFlowP->numTPkts >= UINT64_MAX)) {
flow_t * const flowP = &flows[flowIndex];
T2_RM_FLOW(flowP);
}
#endif // FORCE_MODE == 1
...And reset the config to default for the other tutorials.
t2conf --reset -a && t2build -R
Have fun experimenting!