On March 3, 2013 CloudFlare experienced an hour-long outage as all of its edge routers failed or malfunctioned. The outage caused several CloudFlare customers’ online businesses to be unavailable when the network went offline.

According to CloudFlare, the company got hit by a DDoS attack. They say that the attack implemented data packets that were between 99,971 and 99,985 bytes long. Technically speaking, it is unlikely that a data packet can exceed the largest number 65535 calculated by 16 binary digits. Therefore it is very likely that CloudF suffered an IP fragment DDoS attack.

A fragment packet consists of an MF flag bit and a fragment offset. The offset value is an IPv4’s 13-bit unsigned binary number, which means the offset field of the payload on the IP datagram from the total IP payload and a value calculated as a multiple of eight bytes of fragment offset ; thus the largest number of the offset theoretically is:

(213 - 1)*8 =65528

In addition to the maximum length of the IP datagram (65535), the maximum length of the reconstructed packet is 131063, a value exceeding the packet size shown by CloudFlare.

Since a total length of a fragment packet is composed of:

a) the length of the fragment size. and
b) that of the fragment offset,

it cannot be known until the fragment packet is reassembled. Simply dropping the specified sized (total) fragments from routers, as CloudFlare did, will easily consume all RAM to reassemble fragments until all routers crash.

As for the routers’ bug issue, as noted by CloudFlare, it appears to be caused by the failure of processing extra large fragments. Normally, if the fragmented IP datagram is saved in the 16-bit binary IP header after reconstruction, the total length of the fragments won’t exceed 65515 bytes long (calculated by subtracting IP header size 20 from 65535). Improperly processing to the over-long fragments will cause routers to crash as well. In fact, such vulnerability was unveiled and exploited in Windows 95, Windows 98, NT and 2K software several years ago. In that vulnerability, a software tool named “jolt2” was a purpose-built attack tool to launch and initiate such kind of attack. We suspect this to be the same attack means implemented in the CloudFlare’s incident this time.

To defend against such DDoS attacks, it is not necessary to go to such great efforts to resemble fragment packets. A simple way to defend against this type of attack is to drop all fragment packets with fragment offset over a specified value. This method may miss some attack packets, but will at least ensure network continuity. In reality, fragment packets merely account for a very small percentage of all packets in most of network infrastructures. That is why it is a better solution to drop all fragment packets in attempt to keep networks going instead of taking the risk of causing all networks being taken down and unavailable.