Experiment - TCP Cubic vs TCP Vegas

PURPOSE: examine TCP responses to short and long haul 802.11n packet loss.

On a suggestion from one of the posters to jg’s blog, I took a look at tcp vegas. The results I got were puzzling.

With tcp cubic, I typically get 71Mbit/sec and the side effects of bufferbloat with a single stream.

With vegas turned on, a single stream peaks at around 20Mbit.

10 vegas streams did about 55Mbit in total.

Can I surmise that TCP cubic is like a dragster, able to go really fast in one direction down a straightaway, and TCP vegas more like an 80s model MR2, maneuverable, but underpowered?

The testbed network:

The first test path: laptop->nano-m->nano-m->openrd
(I note that this path almost never exhibits packet loss)

Most of the machines on the path are running with minimal txqueues and dma buffers running as low as they can go. (I’ll fully document this in a bit)

TEST 1 - raw throughput, maximum wireless speed

With cubic:

$ openrd: iperf -s 
$ laptop: iperf -t 60 -c openrd

With vegas (on both laptop and server)

modprobe tcp_vegas
echo vegas > /proc/sys/net/ipv4/tcp_congestion_control

openrd:$ iperf -s 
laptop:$ iperf -t 60 -c openrd &
laptop:$ ping openrd

On a failed hunch, I also re-ran the tests with a much larger
receive/transmit windows:

echo 8388608 > /proc/sys/net/core/rmem_max  # on both machines
echo 8388608 /proc/sys/net/core/wmem_max    # on both machines
openrd: iperf -w8m -s
laptop: iperf -t 60 -w8m -c openrd

To no net difference in effect.

Conclusion: In part, people like vegas because it is slower than cubic and thus has fewer side effects. More research into vegas’s response to latency is desirable, which we can do with series 2.

Also - some required background reading: http://neal.nu/uw/linux-vegas/ And potential test tools: http://psps.perfsonar.net/

Test 2 - testing with minimal wireless speed

TBD

Test 3 - testing with packet loss

TBD

Test 4 - testing with de-bufferbloated drivers

TBD

Test 5 - testing with traffic shaping

TBD