Plugin force mode

plugin development

Introduction
Getting started
Implementing the force mode capability
Conclusion
See also

Introduction

As detailed in force mode tutorial, T2 is capable to operate in a plugin controlled flow release. For example, the basicStats plugin can force the termination of a flow. In order to implement this feature in your own plugin, specialized macros are supplied, which will be detailed in this tutorial.

Getting started

Create folders for your data and results

If you have not created a separate data and results directory yet, please do it now. This will greatly facilitate your workflow:

mkdir ~/data ~/results

Reset tranalyzer2 and the plugins configuration

If you have followed the other tutorials, you may have modified some of the core and plugins configuration. To ensure your results match those in this tutorial, make sure to reset everything:

t2conf -a --reset

You can also clean all build files:

t2build -a -c

Empty the plugin folder

To ensure we are not left with some unneeded plugins or plugins which were built using different core configuration, it is safer to empty the plugins folder:

t2build -e -y


Are you sure you want to empty the plugin folder '/home/user/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied

Download the PCAP file

The PCAP file used in this tutorial can be downloaded here:

faf-exercise.pcap

Please save it in your ~/data folder:

wget --no-check-certificate -P ~/data https://tranalyzer.com/download/data/faf-exercise.pcap

Configure Tranalyzer2 core

For this tutorial, we need to activate the force mode. The required flag reside in tranalyzer.h:

tranalyzer2/src/tranalyzer.h
- FORCE_MODE=1 (force mode)

Change the configuration of Tranalyzer2 using the following t2conf command:

t2conf tranalyzer2 -D FORCE_MODE=1

Build tranalyzer2 and the required plugins

For this tutorial, we will need to build the core (tranalyzer2) and the following plugins:

As you may have modified some of the automatically generated files, it is safer to use the -r and -f options.

t2build -r -f tranalyzer2 basicFlow basicStats tcpStates txtSink


...

BUILDING SUCCESSFUL

Source code

In this tutorial, we will extend tcpWin11.tar.gz, the final version of the previous tutorial (Plugin alarm mode).

If you are impatient, you can download the final version of the tcpWin plugin we will develop in this tutorial.

To use one of those plugins, just unpack it in the plugins folder of your T2 installation.

tranpl

tar -xf ~/Downloads/tcpWin12.tar.gz

And let t2_aliases know about it:

source "$T2HOME/scripts/t2_aliases"

Implementing the force mode capability

If you want to implement the force mode, then the specific code related to the force mode, must be surrounded by #if FORCE_MODE == 1 and #endif // FORCE_MODE == 1 instructions and the T2_RM_FLOW(flowP) macro must be used.

So open tcpWin.c in an editor and add in the t2OnLayer4(...) callback the code surrounded by the FORCE_MODE pragmas. Look for the // <-- markers below.

tcpWin

vi src/tcpWin.c

...
void t2OnLayer4(packet_t *packet, unsigned long flowIndex) {

    ...

    if (tcpWin < TCPWIN_THRES) {              // is the window size below the threshold?
        tcpWinFlowP->winThCnt++;              // count the packet / flow
        winThCntG++;                          // increment global packet counter
        tcpWinFlowP->stat |= TCPWIN_STAT_THU; // set the status bit
        tcpWinStat |= tcpWinFlowP->stat;      // Aggregate all packet flags
#if ALARM_MODE == 1
        T2_SET_STATUS(flowP, FL_ALARM);       // Set the alarm bit in flow and global
#endif // ALARM_MODE == 1

#if FORCE_MODE == 1                                         // <--
        if (tcpWinFlowP->winThCnt > 1) T2_RM_FLOW(flowP);   // <--
#endif // FORCE_MODE == 1                                   // <--
    }

    ...
}

...

Just use the T2_RM_FLOW(...) macro and your flow will be terminated if your condition applies and all core and flow status bits will be set automatically. So save the file.

After you edited the skeleton code you should compare your implementation with tcpWin12.tar.gz.

Compile tcpWin and execute T2:

t2build tcpWin

t2 -r ~/data/faf-exercise.pcap -w ~/results/


================================================================================
Tranalyzer 0.9.0 (Anteater), Cobra. PID: 50672, SID: 666
================================================================================
[INF] Creating flows for L2, IPv4, IPv6 [FORCE]
Active plugins:
    01: basicFlow, 0.9.0
    02: basicStats, 0.9.0
    03: tcpStates, 0.9.0
    04: tcpWin, 0.9.0
    05: txtSink, 0.9.0
[INF] IPv4 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 481503 (481.50 K)
[INF] IPv6 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 41497 (41.50 K)
Processing file: /home/wurst/data/faf-exercise.pcap
Link layer type: Ethernet [EN10MB/1]
Snapshot length: 65535
Dump start: 1258544215.37210000 sec (Wed 18 Nov 2009 11:36:55 GMT)
Dump stop : 1258594491.683288000 sec (Thu 19 Nov 2009 01:34:51 GMT)
Total dump duration: 50276.646078000 sec (13h 57m 56s)
Finished processing. Elapsed time: 0.003186000 sec
Finished unloading flow memory. Time: 0.003198000 sec
Percentage completed: 100.00%
Number of processed packets: 5902 (5.90 K)
Number of processed bytes: 4993414 (4.99 M)
Number of raw bytes: 4993414 (4.99 M)
Number of pcap bytes: 5087870 (5.09 M)
Number of IPv4 packets: 5902 (5.90 K) [100.00%]
Number of A packets: 3533 (3.53 K) [59.86%]
Number of B packets: 2369 (2.37 K) [40.14%]
Number of A bytes: 4423178 (4.42 M) [88.58%]
Number of B bytes: 570236 (570.24 K) [11.42%]
Average A packet load: 1251.96 (1.25 K)
Average B packet load: 240.71
--------------------------------------------------------------------------------
basicStats: Biggest L3 flow talker: 143.166.11.10 (US): 3012 (3.01 K) [51.03%] packets
basicStats: Biggest L3 flow talker: 143.166.11.10 (US): 4148797 (4.15 M) [83.09%] bytes
tcpStates: Aggregated tcpStatesAFlags=0x4b
tcpWin: Aggregated tcpWinStat=0x01
tcpWin: Number of TCP winsize packets below threshold 1: 4 [0.07%]
tcpWin: IP: 192.168.1.105, country: 07, org: Private network
--------------------------------------------------------------------------------
Headers count: min: 3, max: 3, average: 3.00
Number of TCP packets: 5902 (5.90 K) [100.00%]
Number of TCP bytes: 4993414 (4.99 M) [100.00%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed      flows: 74
Number of processed IPv4 flows: 74 [100.00%]
Number of processed A    flows: 37 [50.00%]
Number of processed B    flows: 37 [50.00%]
Number of request        flows: 37 [50.00%]
Number of reply          flows: 37 [50.00%]
Total   A/B    flow asymmetry: 0.00
Total req/rply flow asymmetry: 0.00
Number of processed   packets/flows: 79.76
Number of processed A packets/flows: 95.49
Number of processed B packets/flows: 64.03
Number of processed total packets/s: 0.12
Number of processed A+B   packets/s: 0.12
Number of processed A     packets/s: 0.07
Number of processed   B   packets/s: 0.05
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 0.00
Average full raw bandwidth: 795 b/s
Average full bandwidth : 792 b/s
Max number of flows in memory: 18 [0.01%]
Memory usage: 0.01 GB [0.02%]
Aggregated flowStat=0x0400020000004000
[WRN] Number of flows terminated by force mode: 2 [2.70%]
[INF] IPv4 flows

Note the warning at the end of the T2 report. So we have total now 74 flows (originally 72), and two were released early. The following packets are deposited into a new flow having the same 6-tuple. Let’s select the early terminated flows by a logical AND using the RMFLOW bit in flowStat as defined in flow.h in T2 core:

tawk -V flowStat=0x0400020000004000


The flowStat column with value 0x0400020000004000 is to be interpreted as follows:

   bit | flowStat              | Description
   =============================================================================
    14 | 0x0000 0000 0000 4000 | IPv4 flow
    41 | 0x0000 0200 0000 0000 | Force mode: remove this flow instantly
    58 | 0x0400 0000 0000 0000 | IPv4 packet

The tawk command to select all forced flow is then:

tawk 'bitsanyset($flowStat, 0x0000020000000000)' ~/results/faf-exercise_flows.txt

%dir  flowInd  flowStat            timeFirst             timeLast              duration      numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  vlanID  srcIP          srcIPCC  srcIPOrg           srcPort  dstIP          dstIPCC  dstIPOrg             dstPort  l4Proto  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd  minPktSz  maxPktSz  avePktSize  stdPktSize  minIAT  maxIAT    aveIAT      stdIAT      pktps     bytps  pktAsm      bytAsm  tcpStatesAFlags  tcpWinStat  tcpWinIpTTL  tcpInitWinSz  tcpWinThCnt  tcpWinSzThRt
A     36       0x0400020000004000  1258594163.408285000  1258594164.647755000  1.239470000   1           3        eth:ipv4:tcp  00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800           192.168.1.105  07       "Private network"  49330    143.166.11.10  us       "Dell Technologies"  64334    6        49          89           0            120061        0         0         0           0           0       0.363767  0.02529531  0.05806321  39.53303  0      -0.2898551  -1      0x02             0x01        128          8192          2            0.04081633
B     37       0x0400020000004001  1258594165.319087000  1258594191.015208000  25.696121000  1           3        eth:ipv4:tcp  00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800           192.168.1.105  07       "Private network"  49330    143.166.11.10  us       "Dell Technologies"  64334    6        1465        3012         0            4148797       0         0         0           0           0       5.58724   0.01754002  0.1487483   57.0125   0      -0.3455439  -1      0x43             0x01        128          0             2            0.001365188

See? Play a bit around!

If you are interested in details, the macro T2_RM_FLOW(...) implements the following code in utils/t2Plugin.h:

utils

vi t2Plugin.h

...

#if FORCE_MODE == 1
#define T2_RM_FLOW(flowP) { \
    (flowP)->status |= RMFLOW; \
    globalWarn |= (RMFLOW); \
    rm_flows[num_rm_flows++] = flowP; \
    numForced++; \
}
#else // FORCE_MODE == 0
#define T2_RM_FLOW(flowP)
#endif // FORCE_MODE

...

And defines including the flow removal code are located in main.c:

tranalyzer2

vi src/main.c

...

#if (FORCE_MODE == 1 || FDURLIMIT > 0)
unsigned long num_rm_flows;
flow_t *rm_flows[10];
#endif // (FORCE_MODE == 1 || FDURLIMIT > 0)

...

The T2_RM_FLOW(...) macro sets the RMFLOW bit in flow status. The core will then act on that signal and releases all flows in the rm_flows[num_rm_flows] stack holding all flow pointers to be terminated next time the core is active. Moreover, it increments the numForced counter for the end report.

The dimension of rm_flows is defined also in main.c. Currently there are only 10 elements defined, so you can also terminate dependent flows, such as in ICMP. You normally require maximum two elements, but when we go parallel then we will need more. A very powerful concept, but always be sure of what you are doing.

Conclusion

If you have problems don’t hesitate to contact the Anteater, he’ll help

Note, as for the ALARM_MODE, if you change the FORCE_MODE in the core then you have to recompile all loaded plugins with t2build -R -r -f, otherwise you have inconsistent settings in core and plugins.

t2conf tranalyzer2 -D FORCE_MODE=0 && t2build -R -r -f

Or use the new command:

t2conf --reset tranalyzer2 && t2build -R -r -f

You can download the final version of the tcpWin plugin.

The next tutorial will teach you how to add support for pcap extraction.

Have fun writing plugins!