IP anonymization
ANONYM_IP anonymization IP IPv4 IPv6 layer 3
Contents
Introduction
Sometimes it is necessary to hide your personal or public addresses from anybody you share flow or packet files with. T2 implements a very simplistic mechanism for this purpose, it just blocks the output in the specific files.
Preparation
First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins:
t2build -e -y
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied
Then compile the core (tranalyzer2) and the following plugins:
t2build tranalyzer2 basicFlow basicStats connStat txtSink
...
BUILD SUCCESSFUL
If you did not create a separate data and results directory yet, please do it now in another bash window, that facilitates your workflow:
mkdir ~/data ~/results
The sample PCAP used in this tutorial can be downloaded here: faf-exercise.pcap.
Please save it in your ~/data folder.
Now you are all set!
ANONYM_IP
The control for IP anonymization is located in the core, tranalyzer.h. You can see the current value using t2conf -G option:
t2conf tranalyzer2 -G ANONYM_IP
ANONYM_IP = 0Set it to 1, recompile the core and all plugins and run t2 on the pcap:
t2conf tranalyzer2 -D ANONYM_IP=1 && t2build -R
t2 -r ~/data/faf-exercise.pcap -w ~/results/ -s
So you do not see IP addresses in the report summary anymore. IP addresses, including Teredo, L2TP and GRE entries, are also removed in the flow and packet files. Yes it is coarse, but if will be more elaborate in the future.
tcol ~/results/faf-exercise_flows.txt
%dir flowInd flowStat timeFirst timeLast duration srcMac dstMac ethType ethVlanID srcPort dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm connSip connDip connSipDip connSipDprt connF
A 1 0x0400000000004000 1258544215.037210 1258544215.372742 0.335532 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 1258 80 6 6 7 367 3547 0 367 61.16667 113.5846 0 0.166651 0.055922 0.06351066 17.88205 1093.785 -0.07692308 -0.8124681 2 1 10 18 9
B 1 0x0400000000004001 1258544215.202900 1258544215.537951 0.335051 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 80 1258 6 7 6 3547 367 0 1380 506.7143 474.1748 0 0.167371 0.04786443 0.06323512 20.89234 10586.45 0.07692308 0.8124681 1 2 9 17 17
A 2 0x0400000000004000 1258544216.385370 1258544216.723144 0.337774 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 1259 80 6 5 4 322 464 0 322 64.4 106.4982 0 0.16665 0.06755479 0.06471679 14.8028 953.3001 0.1111111 -0.1806616 2 1 8 16 8
B 2 0x0400000000004001 1258544216.551313 1258544216.888595 0.337282 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 80 1259 6 4 5 464 322 0 464 116 150.9982 0 0.169645 0.0843205 0.0658242 11.85951 1375.703 -0.1111111 0.1806616 1 2 7 15 15
A 3 0x0400000000004000 1258544216.908284 1258544217.008468 0.100184 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 1260 80 6 18 73 319 95603 0 319 17.72222 61.73868 0 0.013738 0.005565778 0.00363383 179.6694 3184.141 -0.6043956 -0.9933488 2 1 8 14 7
B 3 0x0400000000004001 1258544216.915576 1258544217.008019 0.092443 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 80 1260 6 73 18 95603 319 0 1380 1309.63 274.7284 0 0.021251 0.001266342 0.003059902 789.6758 1034183 0.6043956 0.9933488 1 2 7 13 13
A 4 0x0400000000004000 1258544217.003718 1258544217.348506 0.344788 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 1261 80 6 5 4 323 466 0 323 64.6 106.829 0 0.166899 0.0689576 0.06384765 14.50166 936.8075 0.1111111 -0.1812421 2 1 6 12 6
B 4 0x0400000000004001 1258544217.169421 1258544217.513942 0.344521 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 80 1261 6 4 5 466 323 0 466 116.5 151.649 0 0.177128 0.08613025 0.06695305 11.61032 1352.603 -0.1111111 0.1812421 1 2 5 11 11
A 5 0x0400000000004000 1258544217.349751 1258544217.413719 0.063968 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 1262 80 6 9 26 320 30820 0 320 35.55556 84.1992 0 0.009743 0.007107555 0.002012674 140.6953 5002.501 -0.4857143 -0.9794477 2 1 6 10 5
...Same applies to the packet file.
tcol ~/results/faf-exercise_packets.txt
%pktNo flowInd flowStat time pktIAT pktTrip flowDuration ethVlanID srcMac dstMac ethType srcPort dstPort l4Proto pktLen l7Len l7Content
1 1 0x0400000000004000 1258544215.037210 0.000000 0.000000 0.000000 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 1258 80 6 66 0
2 1 0x0400000000004001 1258544215.202900 0.000000 0.165690 0.000000 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 80 1258 6 62 0
3 1 0x0400000000004000 1258544215.203358 0.166148 0.000458 0.166148 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 1258 80 6 64 0
4 1 0x0400000000004000 1258544215.203850 0.000492 0.000950 0.166640 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 1258 80 6 425 367 GET /softw/90/update/avg9infoavi.ctf HTTP/1.1\r\nUser-Agent: AVGINET9-WXPPX86 90 AVI=270.14.71/2510 BUILD=707 LOC=1033 LIC=9I-ASXNN-X4WGW-M0XFR-T84VX-3VX02 DIAG=51E OPF=0 PCA=\r\nHost: backup.avg.cz\r\nAccept: */*\r\nAccept-Encoding: identity,deflate,gzip\r\nIf-Modified-Since: Tue, 17 Nov 2009 20:41:10 GMT\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nx-avg-id:78-175947826\r\n\r\n
5 1 0x0400000000004001 1258544215.370055 0.167155 0.166205 0.167155 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 80 1258 6 1434 1380 HTTP/1.1 200 OK\r\nDate: Wed, 18 Nov 2009 11:39:48 GMT\r\nServer: Apache\r\nLast-Modified: Wed, 18 Nov 2009 09:04:15 GMT\r\nETag: "15c007-cea-478a186e401c0"\r\nAccept-Ranges: bytes\r\nContent-Length: 3306\r\nConnection: close\r\nContent-Type: text/plain\r\n\r\nAVG CTF Index File;ver(10)\r\nbin(u7avi1777ff.bin)grp(avi:1777)tm(0911180855)pri(2)len(6637487)\r\nbin(u7avi1777u1323ff.bin)grp(avi:1777)dep(avi:1323)tm(0911180855)pri(2)len(590030)\r\nbin(u7avi1777u1705ff.bin)grp(avi:1777)dep(avi:1705)tm(0911180855)pri(2)len(95323)\r\nbin(u7iavi2511ff.bin)grp(iavi:2511)tm(0911180855)pri(2)len(45641416)\r\nbin(u7iavi2511u2251fo.bin)grp(iavi:2511)dep(iavi:2251)tm(0911180855)pri(2)len(6175342)\r\nbin(u7iavi2511u2353fn.bin)grp(iavi:2511)dep(iavi:2353)tm(0911180855)pri(2)len(4506824)\r\nbin(u7iavi2511u2404fm.bin)grp(iavi:2511)dep(iavi:2404)tm(0911180855)pri(2)len(3342546)\r\nbin(u7iavi2511u2431fl.bin)grp(iavi:2511)dep(iavi:2431)tm(0911180855)pri(2)len(2609643)\r\nbin(u7iavi2511u2458fk.bin)grp(iavi:2511)dep(iavi:2458)tm(0911180855)pri(2)len(1380674)\r\nbin(u7iavi2511u2480fj.bin)grp(iavi:2511)dep(iavi:2480)tm(0911180855)pri(2)len(681743)\r\nbin(u7iavi2511u2490fi.bin)grp(iavi:2511)dep(iavi:2490)tm(0911180855)pri(2)len(555496)\r\nbin(u7iavi2511u2500fh.bin)grp(iavi:2511)dep(iavi:2500)tm(0911180855)pri(2)len(332906)\r\nbin(u7iavi2511u2505fh.bin)grp(iavi:2511)dep(iavi:2505)tm(0911180855)pri(2)len(204942)\r\nbin(u7iavi2511u2508
6 1 0x0400000000004001 1258544215.370067 0.000012 0.166217 0.167167 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 80 1258 6 375 321 fg.bin)grp(iavi:2511)dep(iavi:2508)tm(0911180855)pri(2)len(72274)\r\nbin(u7iavi2511u2510ff.bin)grp(iavi:2511)dep(iavi:2510)tm(0911180855)pri(2)len(30540)\r\nbin(u9ichjw4qt.bin)grp(ichjw:4)pri(2)tm(0907131859)len(113488)\r\nbin(u9ichjw4u2ia.bin)grp(ichjw:4)dep(ichjw:2)pri(2)tm(0907131859)len(94470)\r\nbin(u9ichjw4u3gq.bin)grp(ic
7 1 0x0400000000004000 1258544215.370501 0.166651 0.000434 0.333291 00:0b:db:4f:6b:10 00:19:e3:e7:5d:23 0x0800 1258 80 6 64 0
8 1 0x0400000000004001 1258544215.370560 0.000493 0.000059 0.167660 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 80 1258 6 1434 1380 hjw:4)dep(ichjw:3)pri(2)tm(0907131859)len(13313)\r\nbin(u9idat249b243tc.bin)grp(idat:249)dif(idat:243)pri(2)tm(0911051401)len(621430)\r\nbin(u9idat249b245bp.bin)grp(idat:249)dif(idat:245)pri(2)tm(0911051401)len(197648)\r\nbin(u9idat249b246mj.bin)grp(idat:249)dif(idat:246)pri(2)tm(0911051400)len(159289)\r\nbin(u9idat249b247uh.bin)grp(idat:249)dif(idat:247)pri(2)tm(0911051400)len(95680)\r\nbin(u9idat249le.bin)grp(idat:249)pri(2)tm(0911051400)len(1927074)\r\nbin(u9ifw42jx.bin)grp(ifw:42)pri(2)tm(0911131400)len(541853)\r\nbin(u9ifw42u38we.bin)grp(ifw:42)dep(ifw:38)pri(2)tm(0911131400)len(123575)\r\nbin(u9ifw42u39ke.bin)grp(ifw:42)dep(ifw:39)pri(2)tm(0911131400)len(92745)\r\nbin(u9ifw42u40dc.bin)grp(ifw:42)dep(ifw:40)pri(2)tm(0911131400)len(67292)\r\nbin(u9ifw42u41tz.bin)grp(ifw:42)dep(ifw:41)pri(2)tm(0911131400)len(18393)\r\nbin(x8all234yc.bin)grp(xplph:0012;xplsb:0099;xplsb2:0118;xplsc:0149)tm(0911180700)pri(2)len(1146352)\r\nbin(x8xplph_12gj.bin)grp(xplph:12)tm(0910280700)pri(2)len(4551)\r\nbin(x8xplsb2_118c8.bin)grp(xplsb2:118)tm(0911180700)pri(2)len(4989)\r\nbin(x8xplsb_9946.bin)grp(xplsb:99)tm(0911160700)pri(2)len(985460)\r\nbin(x8xplsb_99d9546.bin)grp(xplsb:99)dif(xplsb:95)tm(0911160700)pri(2)len(32267)\r\nbin(x8xplsb_99d9646.bin)grp(xplsb:99)dif(xplsb:96)tm(0911160700)pri(2)len(3505)\r\nbin(x8xplsb_99d9746.bin)grp(xplsb:99)dif(xplsb:97)tm(0911160700)pri(2)len(644)\r\nbin(x8xplsb_99d9846.bin
9 1 0x0400000000004001 1258544215.370571 0.000011 0.000070 0.167671 00:19:e3:e7:5d:23 00:0b:db:4f:6b:10 0x0800 80 1258 6 520 466 )grp(xplsb:99)dif(xplsb:98)tm(0911160700)pri(2)len(581)\r\nbin(x8xplsc_149c8.bin)grp(xplsc:149)tm(0911180700)pri(2)len(151922)\r\nbin(x8xplsc_149d145c8.bin)grp(xplsc:149)dif(xplsc:145)tm(0911180700)pri(2)len(2561)\r\nbin(x8xplsc_149d146c8.bin)grp(xplsc:149)dif(xplsc:146)tm(0911180700)pri(2)len(1921)\r\nbin(x8xplsc_149d147c8.bin)grp(xplsc:149)dif(xplsc:147)tm(0911180700)pri(2)len(1767)\r\nbin(x8xplsc_149d148c8.bin)grp(xplsc:149)dif(xplsc:148)tm(0911180700)pri(2)len(1411)\r\n
...Conclusion
It is a very rudimentary anonymization and it does not currently include the other plugins providing IP addresses, e.g. dnsDecode. The Anteater knows that removing IP’s is not the solution, but it worked for us in an urgent job. And then there is content anonymization, a large challenge. Be patient he’s working on it.
Do not forget to reset the configuration for the next tutorials:
t2conf tranalyzer2 -D ANONYM_IP=1 && t2build -R
Or use the new command:
t2conf --reset && t2build -R
Have fun!