%include "./default.mgp"
%tab bullet icon arc "blue" 40
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
%nodefault
%bgrad 0 0 256 0 0 "cyan" "magenta"
%tfont "gn______.ttf"
%center, fore "black"
%size 20

cflowd
%size 8

%tfont "cour.ttf", size 5

Daniel McRobb
dwm@caida.org

http://www.caida.org/Tools/cflowd/

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
%bgrad 0 0 32 0 0 "white" "cyan"
Background
&bullet host software for collecting Cisco flow-export data
&bullet aggregates data into tables for continuous collection of\
 summary data (in time-series)
&bullet stores raw flow data in rotating log files
&bullet provides client/server collection of tabular data in time series

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Features
&bullet all tables are per input interface
&bullet support for flow-export version 1, 5 and 8
&bullet uses arts++ package for data storage and aggregation
&bullet real-time flow-matching API (example 'flowwatch' program included)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
cflowd

%center
%image "dataflow1.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
cflowd architecture

%center
%image "dataflow2.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
flow export version 5
&bullet sends UDP packets to a specified host address and port number
&bullet each UDP packet contains a number of flow entries

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
flow-export packets

%center
%image "flowPacket.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
flow header

%center
%image "flowHeader.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
flow entry

%center
%image "flowEntry.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
cflowd aggregation
&bullet cflowd primarily designed to aggregate flow data into\
 tabular data to be used for capacity planning
&bullet AS matrix, net matrix, port matrix, protocol table,\
 interface matrix and nexthop table aggregation across all flows

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Why so much aggregation?
&bullet data reduction
&bullet scale of intended use (backbone-wide)
&bullet disk and bandwidth requirements for raw flow data\
 in a backbone network.  We only transfer tabular data back to the\
 central collector.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
cfdcollect
&bullet collects cflowd data at regular intervals
&bullet produces time-series data for each of the tabular data\
 types (AS matrix, net matrix, port matrix, protocol table, interface\
 matrix, nexthop table)
&bullet stores data in ARTS files (1 file per router per day)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
centralized collection

%center
%image "centralCollection.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
arts++
&bullet C++ class library for subset of ARTS data
&bullet supports reading/writing of ARTS data via iostreams and\
 UNIX file descriptors
&bullet supports simple time-domain aggregation for several data types
&bullet simple command-line utilities included for viewing ARTS data\
 files and time domain aggregation

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
ARTS data files
&bullet efficient data archival (binary, simple size-reducing techniques)
&bullet data files are portable; arts++ class library is the interface
&bullet extensible for additional data types
&bullet versioning of data objects for different storage formats (typically\
 used for space/CPU tradeoff)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
ARTS data handled by arts++
&bullet AS matrix
&bullet net matrix
&bullet port matrix
&bullet selected port table
&bullet interface matrix
&bullet IP nexthop table
&bullet protocol table
&bullet TOS table
&bullet forward IP path
&bullet BGP4 route table

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
AS matrix
&bullet counters for traffic (packets and bytes) from\
 source ASes to destination ASes

%center
%image "asMatrixData.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
ARTS net matrix
&bullet counters for traffic (packets and bytes) from\
 source networks to destination networks
&bullet networks are identified by network number and netmask length (a.b.c.d/n)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
net matrix data

%center
%image "netMatrixData.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
ARTS port matrix
&bullet counters for traffic (packets and bytes) from source transport layer\
 port number to destination port number

%center
%image "portMatrixData.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
selected port table
%center
%image "selectedPorts.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
protocol table
&bullet counters (packets and bytes) versus IP protocol\
 (TCP, UDP, ICMP, IGMP, et. al.)

%center
%image "protocols.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
interface matrix
&bullet counters (packets and bytes) for traffic from input interfaces\
 to output interfaces

%center
%image "intfmatrix.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
nexthop table
&bullet counters (packets and bytes) for traffic from input interfaces\
 to each IP nexthop

%center
%image "nexthops.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
IP forward path
&bullet contains IP addresses of hops in forward path from a\
 source to a destination
&bullet contains an RTT value for the source to destination
&bullet may be extended in the future to hold more information

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
forward IP path data

%center
%image "ipPathData.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Time Domain Aggregation
Time domain aggregation reduces time granularity in time-series data.

Examples:

&bullet convert 5-minute data to 1-hour data for a summary bar chart
&bullet convert 1-hour data to one large aggregate for a pie chart

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Aggregation Utilities
&bullet artsasagg - AS matrix
&bullet artsnetagg - net matrix
&bullet artsportmagg - port matrix
&bullet artsprotoagg - protocol table
&bullet artsintfmagg - interface matrix
&bullet artsnexthopagg - nexthop table
&bullet artsagg - all of the above, plus TOS table

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Simple display utilities
&bullet artsdump
&bullet artsases
&bullet artsnets
&bullet artsportms
&bullet artsports
&bullet artsprotos
&bullet artsintfms
&bullet artsnexthops
&bullet artstos

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Supported platforms
&bullet FreeBSD
&bullet Linux
&bullet Sparc/Solaris

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Future
&bullet plotting utilities using XRT/PDS
&bullet Web reporting tools (java-based)
&bullet data keted by any combination of fields

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
v8 flow-export
Data aggregated on router:
&bullet  AS aggregation (AS matrix)
&bullet  destination prefix aggregation (like CEF accounting)
&bullet  prefix aggregation (net matrix and AS matrix)
&bullet  protocol and port aggregation
&bullet  source prefix aggregation (like CEF accounting)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Future v8 flow-export advantages: scalability
&bullet less state information on router
&bullet configure and export only the data in which you're interested
&bullet less bandwidth consumed for export
&bullet more amenable to export over reliable transport (TCP)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
More Information

http://www.caida.org/Tools/cflowd/

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Example chart (traffic by port)
%center
%image "portPie.gif"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Example chart (traffic vs. destination AS)
%center
%image "dstAsBar_weekly.gif"
