EtherSam (Y.1564) explained

This is the last article (at least for now) from the series about testing methodologies and testing standards. I will cover some bits and pieces in the region of testing in general but it won’t be as heavy on the theory as I want to write some “hands-on” scenarios for combined use of Wireshark and PackEth as well as about some multicast scenarios. Also I will be doing more Cisco and Juniper stuff so it is quite likely I will be blogging some configs and labs. Anyway enough about the future plans and let’s start with the topic at hand.

Introduction

The ITU-T Y.1564 also more commonly known as EtherSam (which originated in the old name of the standard ITU-T Y.156sam) is a service activation test suite whose goal is to allow for rapid link testing in deployment of services. The main advantage of this test is that it allows for testing of SLA (Service Level Agreements) while deploying new service and it can do that disregarding the Physical topology (i.e. it can verify end-to-end SLA even in live environment with live traffic flowing through the network).

There is few serious considerations in general that make this test suite bit awkward to use.

First one is that this is a very new standard (initiated in 2009, published 2011) and is still changing as new drafts are still being issued.

The next rather serious problem is that this test suite is for “service activation” which means in normal language that it is no good for lab testing as it doesn’t really stress the equipment. The reason is that the EtherSam is designed around the idea of rapid deployment of new links/services in Telcos (I will write about the disadvantages of the design in later).

The last issue is that as a new standard it is rather unknown among network engineers so it takes some education before it can be used.

Traffic parameters

The theory behind this test suite is somewhere half way through between the RFC2544 and BERT tests as it tried to get the best of both while achieving similar results to both. Lets start with definitions as they are the most important. In EtherSam you can configure multiple concurrent services and each service  can have following 4 parameters:

  • CIR – Committed Information rate
  • CBS – Committed Burst Size
  • EIR – Excess Information rate
  • EBS – Excess Burst Size 

This is not as complicated as it might seem at this point. These values are only used to set the SLA. The CIR defines the minimal amount of traffic within the available bandwidth and must be always fulfilled. If there is only CIR specified on the links/services it is a good practice to have some amount of bandwidth allocated to CBS as it will allow for a small overshoot in case of traffic burstiness. Obviously one might need more flexibility in how much traffic to pass through (like over-subscription) where some frame loss is acceptable in exchange for more data being delivered. That is the Excess Information Rate. As it is obvious that once EIR is in place the data from CBS would be calculated as part of EIR so CBS setting loses its meaning. If you want to get little more flexibility in case of having more bursty traffic you can specify EBS on top of the EIR.

Traffic coloring

In the paragraph above I have described the two out of three traffic types that exist in EtherSam which would be reffered to as a green traffic (CIR+CBS) and yellow (EIR+EBS). The standard also defines a red traffic which is a traffic non-conforming to either CIR or EIR. In effect based on the EtherSam methodology this traffic should never be passed and should be dropped. This look like a absolutely trivial and obvious thing but it has one very serious consequence in deployments with over-subscription in place – you must define the EIR as the “shared” part of your QoS with specific size allocated to it. So having a random amount of free-to-grab bandwidth for the tested service will result in failing the test as passing red traffic is a fail criteria on Y.1564.

Traffic profile for EtherSam - coloring

Bandwidth profile parameters – Coupling flag and Color mode

I am putting description of these two parameters at this place just for the sole reason that they are defined in the standard but I would like to stress out that I haven’t seen them implemented in any testing equipment so far so this section will be rather short and most people can just skip it as it has little to none practical use (at least at the time of writing). These two parameters allow for the metering algorithm to be adjusted and thus change the result. Also they are valid only in certain scenarios.

  • CF – Coupling flag – Could be only set as on or off. Is only useful for introducing new service in live environment with extremely bursty traffic. It allows for coupling unused green and yellow traffic thus allowing for higher throughput.
  • CM – Color mode – allows for two options color-aware and color-blind mode where the first one is requiring the tested equipment to re-mark/re-color the traffic streams to adhere to the existing network rules whereas the color-blind expect no interference with the coloring.

The Service Configuration test

This is the first test that you can run and is meant to test a individual service. The aim is to test the CIR/EIR (and optionally CBS/EBS) comply to the setup. It is a rather simple test but except the obvious CIR/EIR/policing it allows for some variability offering the following  options:

  • Fixed frame size or EMIX pattern (1518, 1518, 1024, 64, 64)
  • optional Step Load (25%,50%,75%100%)
  • optional Burst test for the CBS and EBS (defined in Bytes)

If you have multiple services configured each one will be done separately so be careful about the time-estimate as this test is not intended to run for long time. Especially with the ramped services it is important to realize that the total duration of this test will be number of services x number of steps x step time. Also the other thing is that CBS and EBS will be tested separately adding more time to the test. In total this should not take more than 10 minutes as this test is not supposed to be replacing a long term tests.

The Service Performance test

This test is the second (and last) test you can do in Y.1564 and is in place to test all services in one go in order to check that the sum of the CIRs is actually available on the path in question. It is also meant to be a long test with specified durations 15 min, 2 hrs and 24 hrs. The EMIX and ramped traffic in the services should be available as in previous test.

I think that this test due to its simplicity can replace the BERT in many cases while giving better results for service providing.

The results and pass/fail criteria

The pass/fail criteria are rather obvious

  • Fulfilling CIR (or CIR+CBS)
  • Fulfilling EIR (or EIR + EBS)
  • Policing overshoot of traffic > CIR+EIR+EBS
  • Conform to maximal acceptable delay variation (jitter)
  • Conform to maximal acceptable round-trip latency
  • Conform to SLA’s Frame loss (or availability)

These are solid criteria and there is not much you can say against these but as always there are some considerations that must be taken in account.

First one is something I have already mentioned – there is no way for the Y.1564 to consider a shared “best effort” overshoot above the defined CIR+EIR which might be problem in some scenarios but I think it could be avoided via some hacked configuration of EIR/EBS.

Second is the SLA frame loss or more known in the telco world as availability. So if you provide let’s say 99.99% availability it means that on a 100mbps stream it would be acceptable to lose  over 2000 frames single hour which I don’t think would be found acceptable in most environments. As far as I know there is no possibility to set the availability to 100% (also no SLA would ever have this number in it). I ma not currently aware of any possible workaround for this so the only advice is to go through the data in the results table very carefully and set this option to be as close to what you expect of the test as possible (i.e. in my opinion under normal circumstances there should be 0% packet loss on 2 hours test on most systems).

The last thing I would like to mention is that there is no built-in out-of sequence counting mechanism. This might sound as an unnecessary feature but in voice-enabled environment this is  actually a very important parameter to observe.

 Conclusion

 The EtherSam is rather interesting test suite but in my opinion cannot (and was never meant to) replace the RFC2544. In some ways it can partially replace BERT in some field operations. I have to say I do welcome this standard as it addresses the last bit of testing that was not properly included in any Ethernet/IP testing suite to my knowledge. It obviously has some drawbacks but I think it has its place in field service activation environment . Only time will tell if it will become as wide spread as the RFC2544 but I certainly hope so.

 

 

 

 

Bit Errror Rate Test (BERT) explained

This article will be rather short in comparison with the others in the mini-series about various Ethernet/IP testing methods but it is one that is necessary as Bit Error Tests have a long tradition in telco environment (circuit based networks) but are still quite valid even in nowadays packet networks – at least for some specific cases. So without further delay let start with some theory behind the testing and some practical use followed by some use cases and best practices.

BERT introduction

As you can guess from the name this test is really to test physical layer traffic for any anomalies. This is a result from the test origins where T1/E1 circuits have been tested and each bit in each time-slot mattered as the providers were using those up to the limit as bandwidth was scarce. Also as most of the data being transferred were voice calls any pattern alterations had quite serious implications on the quality of service. This also led to the (in)famous reliability of five nines or the 99.999% which basically states that the link/device must be available 99.999% throughout a specified SLA period (normally a month or a year). One must remember that redundancy was rather rare so the requirements for hardware reliability was really high. But by the move away from the circuit-based TDM networks towards the packet-based IP networks the requirements changed. The bandwidth is now in abundance in most places and the wide deployment of advanced Ethernet and IP feature rich devices provides with plenty options for redundancy and QoS with packet-switched voice traffic on rise – one would think it is not really necessary to consider BERT as something one should use as test method but that would be huge mistake.

Why BERT

There are few considerations that can make BERT an interesting choice. I will list some I think are the most interesting.

  1. It has been designed to run for extended period of time which makes it ideal for acceptance testing which is still often required
  2. BERT is ideal for testing jitter as it was one of the primary design goals
  3. The different patters used in BERT can be used for packet optimization testing (I will discuss this later in more detail)
  4. Most of the BERT tests are smarter than just counting bit errors so the test can be used for other testing

BERT Physical setup and considerations

On Ethernet network you cannot run a simple L1 test unless you test just a piece of cable or potentially a hub as all other devices would require some address processing. This makes the test being different on Ethernet network from unframed E1 as unlike on E1 we need to set framing to Ethernet with the source and destination  defined on the tester. Also as Ethernet must be looped on a logical level it is not possible to use simple RJ45 with pair of wires going from TX to RX as you could with E1 and either hardware or software loopback reflector is required. Most tester will actually allow you to specify even layer 3 and 4 for IP addresses and UDP ports. The reason is usually so the management traffic between tester and loopbacks can use this channel for internal communication.

Pattern selection options

As this test originates from the telco industry some interesting options are usually presented on the testers. The stream can generate these patterns:

  1. All zeros or all ones – which are specific patters originated from TDM environment
  2. 0101 pattern and 1010 pattern – patterns that can be easily compressed
  3. PRBS – Pseudo Random Bit Sequence – is an deterministic sequence that cannot be compressed/optimized the details and calculation can be found on wikipedia
  4. Inverted PRBS – the same as above but the calculation function is inversed to counter any “optimization” for the PRBS

The thing to remember is that PRBS will be applied to the payload of the frame/packet/datagram so it there is any sort of optimization present it will have no effect as PRBS is by design not compressible. There are various “strengths” of the pseudo-random pattern the higher the number the less repeating it will include. Normally it is possible to see two main variants: 2^15 which is  32,767 bits long and 2^23 which 8,388,607 bits long. Obviously the longer the pattern the better and more “random” behavior it emulates.

Error injecting Options

As this test originated in telco world injecting errors was a major thing but in Ethernet network it lost its importance. If you inject even a single bit error in an Ethernet frame the CRC should be incorrect and the whole frame should be dropped on first L2 equipment it will be passed through which should always result in alarm LoF(Loss of Frame)/LoP (Loss of Pattern).

Use cases, Best Practices and Conclusion

The most common use case for BERT in nowadays network would be in commissioning new links as you can run a fairly simple test for a long time that will give you a reasonable idea bout it’s quality in terms of frames drops and jitter.

The few recommendations about how to run this test would be as follows:

  • Use the largest pattern you can.
  • Remember that the line rate and L2 rates will be different because of the overheads.
  • Remember that 99.999% of availability results in 0.8s outage in 24 hours (which can be quite a lot of frames)
  • PRBS cannot be optimized

So as you can see BERT is rather simple and straight forward test that even though is in many ways deprecated by RFC2544 and others (like Y.156sam) it is still a very good test to know especially if you are in jitter sensitive environment e.g. where VoIP and IPTV is deployed.

PackEth tutorial part II – The Gen-B,Gen-S and PCAP options

PackEthThis is a Second part of an article I have written some time ago about the great tool called PackETH.  This article will be much shorted as it will be focused on the less complicated (but not useful!) modes of the tool.

In the previous par I have described how to build your own packet from L2 to L4 but what if you need something else ? maybe not a single packet but a burst of packets? or what is you need to send multiple streams of various frames ? Well then you need to use the Gen-S and Gen-B modes.

Gen-B

The name stands for Generate burst. The necessary prerequisite for using this mode is for you to have a ready and valid packet loaded in the Builder mode. Once you have that you can run the burst generation. This is how the GUI looks like:

Gen-B

The “Number of packets” field is I think rather self-explanatory. The second is more interesting and deserves our attention as it can be used for buffer testing. The “Delay between packets” field is actually referring to what is correctly called the Inter Frame Gap (IFG) which is a delay between end of one frame and beginning of the next one. The physical media has a limitation of how fast you can the frames be send. Then the “max speed” check box is in effect it means you will send all the frames with minimal IFG. This situation is also be known as back-to-back scenario.  Under normal circumstances the frames would not be played out from the buffer like this unless the buffer would be full. So you are asking what is this good for ? Well the traffic can be very bursty and the software on your network equipment might not be able to handle it very well so testing this behavior before you introduce new equipment into your network is highly recommended (and is actually standardized as part of RFC 2544 test suite).

The other interesting thing you can do in Gen-B mode is to change the frame’s content on fly (from the original you’ve build in the Builder mode). The things you can do with it are quite wild. let’s have a look at the options and what they do:

  • do nothing  – will result in sending the same frame you have in Builder mode with no changes
  • MAC set random source address – rather self explanatory – good for testing L2 paths and load balancing mechanisms
  • IP set random source address – rather self explanatory – good for testing L3 paths and load balancing mechanisms
  • RTP options – this allows to simulate a more real RTP stream
  • Change Byte X and Y values – probably quite good for protocol developers

Gen-S

This is a Stream generator as there is quite often a need to play different types of traffic. For example you want to try a nice voice traffic stream you have captured or build in combination with various bursty types of traffic and see how your network will be dealing with it? Well that is why you can select up to 10 packets in pcap format and give them some basic parameters (as seen below).

Gen-S

You can also run these stream in cycles so the traffic pattern will repeat itself up to infinity. You can also enable/disable the stream on the fly to alter the traffic mix.

The PCAP

The PCAP’s only function is to open a packet in pcap format (not pcapng!) and load is into the builder once it is selected. On the other hand this can be achieved using the load button from the Builder as well so I am bit unsure what is supposed to be the extra feature here.

The Summary & a small testimonial

Well what to say at the end – I hope I was able to describe PackETH’s features with few minor hints what they can be useful for. I will most likely include PackETH in some of my other articles as a method of testing thing while playing with various LAB scenarios. The truth is that it cannot replace a proper Ethernet tester but taking in account its flexibility,stability and the fact it is free  I must say I can only rate it as high as possible.

Before I will really end this article I wanted to write one more thing a small testimonial – I successfully used PackETH over period of about three years for various testing ranging from proving equipment behavior for L2 broadcasting/multicasting, faking ARP and ICMP messages to invoking network behaviors so as proving equipment’s dealing with QinQ and it has aways been a tool in my software toolbox I could and can completely rely on. There aren’t many pieces software like this one and I would recommend to anyone anytime for both  training and troubleshooting purposes.

PackEth tutorial part I – The Inetrface and The Packet Builder

PackEthIn my previous post I have mentioned ingenious piece of software called PackEth and I have also promised that will write up a separate article about it as I just think this amazing tool deserves as much attention as I can give it.  So what this software do?  Well let me quote the author ” PackETH is GUI and CLI packet generator tool for Ethernet…It allows you to create and send any possible packet or sequence of packets on the Ethernet link.” I would add that that it is the only tool I have found that actually allows you to assemble Ethernet frames and a IP packets that actually does what you would expect it to do while being multi-platform and incredibly stable. I think I have never seen it crashing which speaks for itself.  This article will focus around version 1.6 as that is the one that has both Linux and Windows versions available. The drawback is that the L3 IPv6 support is not included.

Introduction

Getting the package is quite simple as it is a a project hosted on sourceforge. There are versions available fro r Windows Linux and MacOS. But if you are using Linux then there is a good chance that the package will be in your repository (is present in Debian stable and most likely in Ubuntu). Installation is simple – in windows just unpack the .zip file and run packeth.exe – And yes it is completely standalone software so no installation no garbage in registry etc. the installation has all libraries included so the folder after unpacking has about 18MB which I think is very reasonable. I would recommend using Linux version as along with Packeth you will be most likely using WireShark as well and that has some issues on L2 in windows. If you are happy with L3 testing only then the OS choice is irrelevant.

The Gui will open in the builder mode which is probably the most useful and most interesting mode of all the ones that are offered. Switching between the mode you can on the top left part of the menu.

GUI-main-controlls

In the middle section you can save and load configuration you have made in the past for repeated testing. In the interface button you will have a selection of interface you can use for PackETH – in Linux it is simple ethX interface. Under windows it is bit more complicated as PackETH doesn’t use the “human readable” name a.k.a. local network 1 or similar but instead it uses the system name which looks like this

\Device\NPF_{653EFF5C-E308-4494-A7DC-1C65E8BCE92F}

If you are wondering where is that name coming from it is an ID from WinPcap library and there is no simple way how to find out which ID is which interface but as normally you want to use just one – you can just disable all the others and read the ID in PackETH. The most important thing is – if you are not a superuser (or have permitted access to network cards on the machine) the interface list will be empty.

The send button is simply sending the frame/stream from the interface. The stop button us useful only when you are running continuous streams as in all other cases.

Builder mode

Is the basic and most interesting for me personally as it allows for complete buildup of a L2 frame/ L3 packet / L4 datagram. It has multiple options and parts which make it incredibly handy.

Data Link Layer

Before going into Data Link Layer of the builder – just a small reminder of how ethernet frame looks like:

ethernet frame

In the Link layer section you can choose which standard of Ethernet you want to use for your frame. The majority of Ethernet traffic in networks nowadays is Ethernet Version II (Also known as DIX). In the last data I’ve read about this topic about 5 years ago were showing that Ethernet II is about 95% of all Ethernet traffic. This should be taken in account when you build your frames as the NIC in your PC might not even be able to build 802.3 frame due to driver restrictions. I have seen this on multiple PCs. Also the receiving party might be dropping this type of frames as despite the attempted compatibility not many vendors actually care about this at all.

ethertype

If you chose the Ethernet version II frame format the ethertype field (also known as DIX type) will become available. This field identifies what protocol is encapsulated in the frame. The current options are:

  • IPv4  – 0800
  • IPv6 – 86DD
  • ARP – 0806
  • User defined – whatever number you can fit in two octets

Be aware that there is no internal logic of PackETH stopping you from selecting let’s say IPv4 DIX type and then building and ARP packet on higher layer which is incredible advantage if you want to test behavior of equipment to invalid types of traffic, but is easy to overlook when you are not after such specialty.

When we’re in the Data Link layer the next after ethertype is 802.1Q and (in)famous QinQ.

As you can see in the  picture of the Ethernet frame the first field in the 802.1Q shim is the TPID which identifies the following data as part of the shim rather than Ethertype. This fields is followed by  PCP (Priority code point) which is defined in 802.1p and is used for CoS. The priority can be selected from the menu – it also tells the standard meaning of the p-bit in question. But be aware that the numeric value actually means nothing as long as the devices passing this traffic are p-bits aware. Also the mapping is based on standard’s recommendations so in every network it can be used as seen fit by the network admins.

802.1p

The CFI (canonical format identifier) has been deprecated and re-used drop eligibility but generally it can be just ignored as most equipment just ignores it anyway. The filed of interest is the VID which defines the VLAN ID. The problem with it is that it must be written in hexadecimal digits which is not exactly user friendly but should be no big problem.

The biggest topic in this part is the QinQ. Let’s start with the definition of what is QinQ. As the name suggest it is abbreviation for all sorts of nested VLANs (aka 802.1Q in 8021Q). This practice started as totally non-standard behavior and as a result it has been implemented in many different ways before a standard has been written. The major issue is that the standard is fairly new and mos of network vendors actually doesn’t support the standardized version. Fortunately PackETH support all versions that exist plus some more as you can define whatever you want. So what are our options in the TPID field for QinQ for the outer/SP tag ?

ethertype-QinQ

  • 0x8100 – the common vlan – as most vendors even nowadays support and do the original 802.1Q in 802.1Q – extremely common – default almost everywhere
  • 0x9100,0x9200 (and missing 0x9300) – proprietary outer/SP tags used by vendors like Cisco an Juniper and is fairly common on decent equipment
  • 0x8a88 – 802.1ad format that almost no-one supports

So the outer tag we must select from a drop-down list and the TPID for the inner/C tag is  always 0x8100 (that is why the filed is grayed out). So the only thing to do is fill in the VIDs for outer tag and the inner-one. The only next step is to select what is the next layer protocol as L2 configuration is finished.

Network Layer

In the network layer you can chose between ARP and IPv4 (ind IPv6 in the newer version) and user defined payload. All of these are quite simple.

This is how IPv4 setup looks like:

IPv4-header

Let’s go through the header fields:

  • Version – should be always set to 4 (that’s why it’s called IPv4)
  • Header length – IPv4 has a variable header length due to existence of “options” field at the least significant position (this causes a lot of issues with L3 aware devices and was important driver in IPv6 development) The header length is in increments of 4 Bytes so the most common value of 20 Bytes would be equal to the default value of 5. This field uses hexadecimal numbers so the allowed values are from 1 to F
  • ToS field as originally defined in RFC2474 Is no longer used as ToS (Type of Service) in about 99% of networks and is deprecated in favor of DSCP (Differentiated Services Code Points) and these are options available:

ToS  and DSCP options

As I have never really used ToS for anything I will not really dive into explaining the variables on this place. I might do that in article I am preparing about QoS theory and its implementation in some equipment types.

  • Total length – calculates the total length of the packet and unless you want to check behavior for runt packets keep it on auto so you will generate valid packets
  • Identification – this is completely useless field no 99% cases as it is only used for reassembly of fragmented packets
  • Flags – very important as it allows/disallows fragmentation of the frame – the best practice here is for the packet to be set to 2 (do not fragment), the other option is “more fragments”  anyway this seems to be broken in the 1.6 windows version and all three values are always zeroes

ipv4-flags

  • Fragment offset – again in my testing I have found no use for playing with fragmented packets
  • TTL – Time to live – number which is decreasing while the packet is being processed through L3 device might get very handy when you need to prove how many hops away your receiver is
  • Protocol – code informing about what higher-level protocol is encapsulated in the IP packet (options ate TCP,UDP,ICMP and IGMP) again this is just a number in the header of the packet and doesn’t prevent mixup with different protocol actually being configured in upper layer.
  • Header checksum – does exactly what you would expect and again – Unless you are testing runts there is no need to uncheck the tick-box that calculates the checksum for you
  • Source and destination addresses – These do not need any comment
  • Options – a barely used field – I don’t think I ever seen it used and I have never used it myself as far as I remember
ARP

If you think – that the IPv4 was pretty simple then ARP will look like a piece of cake. I have to say I like the possibility to send fake ARPs around the network as you can easily populate various tables (specifically APR and switching tables on L2/L3 devices) without having the real source in the network. This allows you to see behavior of elements of your network that would be difficult to observe otherwise. Well there is not much to say about it and here is the screen-shot:

ARP

As you probably know ARP has been deprecated in IPv6 and it is now a component of the IP protocol itself and is know as neighbor solicitation.

Session Layer

The session layer provides you with following options : UDP, TCP, ICMP ang IGMP. I will cover all of them briefly as I most of the time do not work with L4. Also IGMP and RTP will be covered in greater detail in an upcoming article about multicasting.

UDP

Is the most common protocol I am using as most of the data I am normally dealing with are various voice frames. The Protocol itself is minimalistic and so is the possible setup as you can see below:

UDP

The one thing I would like to point out is the option to apply some specific patter of your choice (so ti is not random. This is etremely useful if you have a suspicion that a specific frame with (or within) a specific patter inside is causing some troubles in your network.

TCP

TCP is unlike UDP statefull so it must have way more options included to accommodate the windowing mechanism and 3 way handshake and some other minor things. There is a very nice article on Wikipedia about TCP and as I normally do not care much about L4 in testing I will not elaborate on the details here. This is hw the GUI looks like with all the options it has:

TCP

ICMP

This is probably the most interesting of all the L4 protocols as you can actually invoke some actions from the network nodes. The main two options are echo request and echo reply which allows you to send ping to specific nodes (which is not that special) but also fake reply which I have found very useful in the past. The other option is to send network unreachable datagram with all the messages but unless you do testing of L4 aware network (like some firewalls) then it is not of  much interest.

ICMP

IGMP

The Internet Group Management Protocol is a predominantly last mile protocol used for membership in various multicast groups. IT is widely used for multimedia delivery specifically – IPTV. It exists in 3 versions and V2 is most widely used (at least to my knowledge).  IGMP is rather simple as it has basically only two types of messages – Query (from router) and Report (from client). As you can see all of those are available to you which is ideal when troubleshooting both ends of the multicast network as combined with Wireshark you can emulate the required response.

IGMP

Conclusion

The other three parts –  Gen-B mode, Gen-S mode and PCAP will be discussed separately in a follow up article as I must try keeping the length on a readable level.

How to capture, analyze, create and replay ethernet traffic

I have decided to write up new article after long time of silence as I think this is a topic that many engineers are facing on fairly regular basis but finding solutions to the simple and interconnected questions is rather time consuming and not exactly simple. Let me stress at the beginning that most of the tools mentioned below are free or cheap and all of them are easily available. So enough talk and let me start with the first chapter

Part 1 – Capturing the traffic – software

There are various situation in which you might need this and various ways how to achieve it. So let me start with software which will allow you to capture the traffic on a PC.

Wireshark LogoWireshark – the Alpha and Omega

I think the first software anyone always encounters is Wireshark – it is absolutely incredible and flexible piece of software with so many functions and features it is difficult to believe it is free. The software itself uses libpcap (or its windows variant Winpcap) and is multi-platform (Windows/LINUX/MacOS). I will not write about Wireshark’s functions and features on this spot as it has been done many times and I might do it in separate article anyway. But I will point out few thing that are a must-to-know when working with this amazing software.

  • Captures made in Windows OS will never have 802.1q tags as the drivers of most network cards will just strip them (might be also fault of libpcap – never found complete answer to this)
  • If capturing large amounts of data the default behavior is to display the captured data in real time on screen which is causing crashes (as the PC is usually unable to cope with this amount of calculation)
  • Alway use the latest stable version as the developers of this software make huge improvements every release

It is needles to say that Wireshark is great for small amount of traffic but generally for capturing of data it must be tweaked and is less stable than the second option I will talk about.

TcpdumpTCPdump

This is a Linux only tool – at least as far as I know which is exclusively used from command line interface. This might seem to be like quite a drawback as many people for different reasons don’t have or cannot have Linux PC. Well think again – there is huge amount of network equipment that is based on either Linux (Checkpoint SPLAT) or BSD (Juniper,Nokia IPSO) so you can still use it. The syntax is really simple :

$ tcpdump -i etho  – will set the interface on which we listen to eth0
$ tcpdump -w filename.pcap  –  will save the captured packets in file in pcap format so you can later read them either with tcpdump or Wireshark
$ tcpdump -d – display the captured frames on the active console

I have listed only these three as those you would normally use the most there is a whole bunch of commands this software can do out of which the interesting one is that it understands regular expressions so you can filter while capturing. For the complete list check this manpage.

Part 2 – Capturing the traffic – hardware

Now when we have something to capture the traffic with we must somehow direct the flow of the traffic to our endpoint. Most people go for the easy approach and just unplug whatever equipment is in the path of the traffic they want to intercept and run the analyzer. But this is fundamentally flawed method unless you are in control of the traffic transmitter (i.e. in lab environment) – in live networks or when investigating some protocol-related issues this is just not possible as the traffic will either die out on some timers or will just not arrive at all as the upper protocols will notice that their counterpart is no longer there. Well fortunately there are ways how to achieve the traffic diversion or transparent interception.

Hub

Hub is the first thing that will come to mind – it is a simple L1 device and the main thing is it sends everything everywhere. Of course this is a simplification but it does the job – or not – The problem here is that you will not be able to buy a hub nowadays as switches are so cheap that there is really no reason for hubs to be manufactured anymore. The other limitation is that there are no hub with gigabit Ethernet ports as the have never been built.

You might have luck on e-bay but even there most people are selling switches and even routers as “hubs” so it is unlikely to get one of those. Also there is a big limitation of a hub and that is that you can actually listen only to maximum of 1/2 speed of what its declared speed is. The reason is obvious – as you listen to both part of conversation on your link from the hub, it will be limited to max of 100mpbs or more typically 10mbps. So if you will  think about a composite, equal, bidirectional stream directed to your end point the maximum of the down link which is 100mbps (or 10mbps) which will be shared as 50Mbps conversation from A->B and 50Mbps from B->A.

Despite this limitation hubs are great especially for small lab environments but unfortunately they are almost impossible to get nowadays.

Port mirroring / Span port / Tap port

This is usually the most available way of diverting traffic from live network and is present on all equipment that has a cli (ok almost – but definitely on all decent, recent equipment). The configuration is usually very simple you just must identify the port to mirror and a port you want to direct your traffic to (in Cisco catalyst switches it can be also vlan). Here is a sample config:

Switch(config)# no monitor session 1
Switch(config)# monitor session 1 source interface fastEthernet1/0/1
Switch(config)# monitor session 1 destination interface fastEthernet1/0/24
and for verification
Switch# show monitor session 1

This is a very fast thing to configure but it has she same drawbacks

  • malformed frames will never be processed by the ASICS as they will be dropped on inbound of the source interface
  • even though most vendors swear that this is capturing fully transparent there will be protocols being filtered out so the trace is never complete

Port mirroring is excellent for protocol related problems but for lower layer problem it always must be combined with interrogation of the source interface and this might be rather tricky in busy networks.

tuxLinux bridge

I will add this only as a last resort thing as it has so many drawbacks and so narrow utilization that I’ve never seen it used in real life. In linux if you have two interface cards you can bind them in a bridge group using bridge control utilities and command brctl. This will create a L2 bridge on which you can listen to traffic but this is a classic bridge with all its disadvantages of being full L2 device. Also please note that the bridge would have spanning tree turned on by default ! The example below shows how to configure the bridge group and disables the stp.

$ brctl addbr “bridgename
$brctl addif bridgename (i.e. eth0)
$brctl stp bridgename off
$ brctl show  – shows tall bridges on system)
$ brctl showstp bridgename – shows current status of spanning tree on the bridge in question

Linux bridging is very useful thing to know and understand especially for building VPNs and when you do some virtualization. For configuration details see the tutorial on linux foundation.

dualcomm logoHardware tap

This is probably the most expensive option out of all the ones mentioned here but it is my personal favorite. The network tap is a L1 device that basically can work as a smart hub bun on gigabit speed. You can either sniff one direction or the other or both at once (with the limitation of the dow-nlink speed of course). I was looking for device that would be able to do this and wouldn’t cost thousands of dollars (and I was actually considering building one myself). But then I have found Dualcomm and specifically Etap-2306 . It is a very nice piece of equipment – very simple has two different modes for capturing and you can even inject some traffic if you want. But my favorite feature was that it has SFPs as the equipment I work with is 99% fiber only this was a huge bonus. The other great thing is that is is USB povered so you can run it off your laptop without any need of additional power supply.  The device costs about $670 so it is not the first choice if you have strained budget but after using it on several occasions I have to say it was worth of every penny spent.

There is one not so obvious drawback when you start capturing data on speeds of hundreds of megabits – it is the speed of the HDD you are capturing to. Normally you would do this in field with only your laptop – but the throughput of your laptops HDD is actually always below 400mbps (more likely to be close to 200mbps). This will obviously differ between older laptops and newer ones with SSD drives but it will be a bottleneck in most cases. There are ways how to improve the capturing so this will not be such a  big problem but eliminating this issue is very difficult in field conditions – in lab you just use PC with fast drives and plenty of RAM and it will do the trick.

Part 3 – Replaying captured traffic

This might seem like a bit of a useless thing to do – why would you want to do this if you can see the whole stream in the Wireshark analyzer ? Well this is extremely useful for replication of problem in lab where you can monitor the equipment more closely and also can change configuration while the problematic event is happening. Replaying the pcap files is very common feature on testing equipment but that cost thousands and tens of thousands dollars. There is also quite a few pieces of software that do it as one of their functions but I have found those usually have some problems either with timing or sending the packets in different way than they have been received originally. After quite some research I have found Playcap which ideally matched my requirements.

playcap Signal11 logoPlaycap

Playcap is the software of choice for replaying the pcap files from your PC – it is extremely simplistic and has Windows and Linux version. The most important thing about this software is that is rigorously follows the timing of the packets in the pcap files which most of the other pcap players don’t do.

Part 4 – Creating your own traffic

If you need to prove a theory there might raise a need to actually send a frame of specific type (i.e. multicast frame) or with specific payload without having an equipment that can do it. This is a difficult problem even for small testing equipment as it is not primarily ,meant to do these thing. With big testers you usually can do this but must pay a special license not talking about impossibility getting such a device anywhere close to field (or even other lab in the same building). After rather long and unfruitful search I have found project called packeth which actually does quite a few of the above mentioned.

PackEthPacketh

The project can be found on sourceforge and is just great. I am using it for about 3 years now and performed so many tricks with it ! Have no equipment to generate QinQ, igmp, L2 muticast, want to verify unknown unicast want to send fake ARPs test IPv6 ? Yes all of those (and more) you can actually do in Packeth – as with wireshark this is absolutely fabulous piece of software which runs on both Windows and Linux. As there are very specific thing you can do with this software I will most likely write a separate article just about it.

Part 5 Traffic analyzing

The whole exercise of getting the traffic (or even the creation of it) is being done so you one can easily troubleshoot and replicate issues in lab or in live network. but once the data are on a PC the only step must follow – the packet trace analysis. There is only one name I can say about this – Wireshark. IT is the ultimate tool for traffic analysis. I will not write how to use Wireshark as it is a topic for stand alone book. But there is one thing I will say – if you want to call yourself a network engineer understanding the basics of this software is just a must. If you can write decen exptressions and know where to look to find the flows or converstaions between endpoints – it will make your life much easier. As mentioned above multiple times – I will most likely write an article on “how to” for some scenarions that are interesting from my point of view – but the documentation and community around Wireshark is huge so if you want to know anything – just go to the project’s wikipage where you’ll find plenty of useful stuff (including some sample captures or protocols).

This is the end of this first article after a long time but I will try to write follow-ups soon as I should have more time and more interesting things to write about.