1. Virtual Networking Performance Stephen Hemminger stephen.hemminger@vyatta.com Linuxcon JapanVirtualization Summit

2. Bi-directional Virtualization Loss Test 100%75%Throughput50%25% 0% 0 500 10001500 Packet Size 3. Houston, we have a problem! 4. TopicsWhat is virtual router?Performance measurementsAnalysisCurrent workHow to help 5. Performance History20052006 20072008 2009201020111G line rateMultiqueue NIC10G line rate 1M ppsXen 3.0 KVM Hyper-Vdrivers 6. Performance measurementsForwarding RFC 2544 minimum packetThroughput TCP bulk transferLatency Request / Response 7. Router BenchmarkRFC2544RouterUnder TestSpirent 8. Virtualized Router Benchmark Router UnderGuestTest HypervisorBridgeBridgeSpirent 9. 1 Gigabit bi-directional forwarding 100% 75%Throughput 50% 25%0%0 200 400600 8001000 1200 1400 1600Packet Size 10. Physical NetworksRate limited by slowest componentAdding more layers does not slow down throughputSeparate resources 11. Virtual Network overheadHyper-Bridge outvisor InBridge In Hyper-visorRouterUnderTest 12. Network BaselineIntel i7 920Intel 10GLOM 1G networkLoopback 12G bits/secForwarding 1M packets/sec per core Your mileage may vary... 13. Emulated Network InterfaceFake PCI regionGuest Hypervisor To NetworkPacketBuffer 14. Legacy (emulated) NICRelative Performance vs Native% -25% -50% -75% Transmit -100% ReceiveHyper-V VmWareKVM 15. Virtual NICGuest Hypervisor To Network Packet Buffer 16. Virtual NIC performance0%-25%-50%-75%-100%Transmit Hyper-V VmWareReceive KVM 17. VNIC characteristics Hyper-V VmwareXenKVM vmxnet3 netfront virtio-netMTU 1500 900065521 65535Checksum YYYoffloadSegmentation YYYoffloadNAPI YYYLROYVLAN YMultiqueue Y?? 18. Guest to Guest Throughput TSO Tx csumNo offload 0 1000 2000 3000 40005000 Megabits/sec 19. VM to VM performance 2000Thorouhput (10^6 bits/sec)0100100010000 100000 MTU (bytes) 20. TCP performance2 Tier NATRouteBridge Bare 0 2 4 6 8 10 Gigabytes / sec 21. ObservationsSlowLegacy emulated NICNATFirewall connection trackingFastVirtual NICHardware passthroughBare Bridge 22. Multithread benchmark 20,000UP Guest SMP GuestTotal Transactions/sec 15,000 10,0005,000 0 Packet Steering (RPS)BaselineMulti Queue NIC 23. Multiple Queue HW or SWGuest VM ThreadDevicequeue ThreadFlowDevice VNICclasqueuesfier ThreadDevicequeue 24. TODOMore peformance measurement MACVTAP, VEPA, Configuration Pinning, IRQ assignment, Profiling 25. Future workVirtio-net optimizationMichael S. TsirkinMultiqueueKrishna Kumar KVM QEMUZero copyShirley Ma Skb lifetime SecurityBatchingRusty Russell Bufferbloat 26. How to helpAdministratorTuning and SizingTesterMeasure and TestDeveloper Multiqueue and Offload