Data Throughput Troubleshoot

7/30/2019 Data Throughput Troubleshoot

1/19

Data Throughput IssuesI have been being asked to troubleshot the throughput issue so many times. Unfortunately my experience says "There is noclear/logical/deterministic way to troubleshoot for throughput test".Then what are we supposed to do ? Are we supposed to rely on "Hit and Miss" strategy everytime we do the throughput test ? Isthis process totally random ?No at least we are not in such a worst case, fortunately. I think we can set some guidelines at least.

First thing to remember for throughput troubleshooting

What kind of tools we need for troubleshoot ?Test SetupsBasic and Common steps to check before floodWCDMA (R99)HSDPAHSPA+HSPA+ Dual Carrier

Ideal MAX throughput for UMTS UE

CQI vs Throughput for UMTS

LTELTE Physical Layer Throughput - PDSCH Decoding PerformanceThroughput in Live Network

Links on Throughput Test in the field

Factors influencing the throughput

Throughput Test Softwareiperf - UDP/TCP FloodingFilezilla - FTP

First thing to remember for throughput troubleshootingOne sentence. "Throughput troubleshooting is not simple at all.", "Don't expect it to be simple.". If I solved the problem with singleshot, I would say "I was just lucky, It is not because I am technically competent".Even troubleshooting with wired communication is not easy. Think about how many more factors would get involved in the datapath.That's all for the first thing. Now let's move to the second important thing for this issue. What is the second thing ?It's "Don't give up. You will eventually find the solution!" -:). It is just matter of time and depend on how much dedicated you areduring the troubleshoot.

Now the third things comes (Many people think this is the first thing since it sound more technical, but I don't think it is the case).What I want you to do as the third item is "list up all the nodes from the data transmitter to the reciever, and follow all the stepswithout skipping anything.". One example I can give you is (this is an example where you use a Network Emulator for the test).i) IP Application Software on PC (e.g, iperf, FileZilla)ii) TE port on PC (e.g, Ethernet Card).iii) TE port on throughput test equipment (e.g, Data packet port on Network Emulator)iv) PDCP layer on test equipmentv) RLC layer on test equipmentvi) MAC layer on test equipmentvii) L1 (Transport and PHY) layer on test equipmentviii) L1 (Transport and PHY) layer on UE (mobile phone or data card)ix) MAC layer on UEx) RLC Layer on UExi) PDCP layer on UExii) TE port on UE (e.g, Modem connector)

xiii) TE port on PC (e.g, USB port the UE is connected to)xiv) IP Application Software on PC to which the UE is connected.The more you understand on each of these items, the better position you are in for troubleshooting. (If you really enjoy your job asengineer, one of the topic I would recommend you is to try with throughput troubleshoot or optimization. To me it looks like an artat the same time being a technology).Now you would ask "Which component on the list is most important, most critical factor for the throughput ?". I wish I had asimple/clear answer to this, but my experience says "the answer varies depending on the situation". Especially it would differdepending on what kind of radio technoloty your device is using. (e.g, Is it R99 WCDMA Device, HSDPA, HSPA+, LTE ?)In addition to the major technical factors listed above, sometimes very simple things as follows make you spend several hours toseveral weeks for troubleshoot if you are in bad luck.i) LAN Cable type (Sometimes you have to use 'direct cable' and sometimes you have to use 'cross over' cable).

Technote http://www.sharetechnote.com/html/Throughput.html#CQI_vs_T

9 9/6/2012


2/19

ii) Category of LAN cable. (Is it Cat 5 cable or Cat 6 cable ?)iii) Ethernet Port Capability (Is it only for 10/100 M, or Gigabit ethernet ?)iv) Firewall setting on your PC (I will go back to this later in a separate section).I will try to go through each type of radio technology and try to point out the important factor for that specific technology. (Try tomemorize all the steps listed above sicne I will talk about the steps for each of the following sections).

What kind of tools we need for troubleshoot ?Since the throughput process get involved in the full protocol stack and one/two PCs, and in addition IP tools, you would need tohave tools to monitor each and every steps along the end-to-end data path. The logging tool should be able to show not onlyscheduling and event log, but also all the payload (contents of the data). Without these tools, you would end up saying "I havetested this device with many different test equipment and didn't see any problem before. This is the only equipment that I see thisproblem.. so the problem is on the equipment side." or "I have tested many different UE with this equipment, but I didn't see thiskind of problem before. So this is UE side problem". Both may be right or wrong at the same time. Our job as an engineer is to findthe root cause of the problem and fix it, not blaming the other party. (But to be honest, I have to admit I often blame the otherparty without even realizing it. Is this a kind of bad nature of engineers ? or my personal problem ?)Let's try to have proper tools and skills to fight against the problem, not fight against your counter part engineers. Myrecommendation about the tool is as follows :i) Ethernet, IP logging tool (e.g, Wireshark)ii) UE side logging tooliii) Network side logging tooliv) YOUR SKILLs to use and analyze the logging toolv) YOUR PATIENCE to step through the log for each and every transmission and reception

Test SetupsI saw a lot of persons trying to do IP throughput test in most complicated setup (e.g, Case 3 without IP monitoring tool) from thebeginning without preparing any troubleshoot tools. If you are lucky, you will get it working. But in most case especially when theuser tries the test for the first time with the specific test setup.But I always recommend user to prepare proper troubleshooting setup first and do some preparational test before you try with thefinal setup.For example, if you want to test with < Case 2 > configuration, first try with < Case 1 > Configuration and make it sure that thereare no PC related issues.If you want to try to test with < Case 3> configuration, first try with < Case 1> & < Case 2>, make it sure that there are norouter related issue.Have Wireshark both on UE PC and Server PC in Case 2/ Monitor PC in Case 3. You have to be able to trace each steps of the pathdescribed below for each cases.Does this sound too tedious ? Trust me ! This would be the fastest way to get the solution for any trouble.

< Case 1 >

< Case 2>


9 9/6/2012


3/19

Ping Path from UE PC to Server PC :Ping Request : UE PC -(1)-> UE IP Driver -(2)-> UE PDCP -(3)-> UE RLC -(4)-> UE MAC -(5)-> UE PHY -(6)-> RF Connection

-(7)-> Equipment PHY -(8)-> Equipment MAC -(9)-> Equipment RLC -(10)-> Equipment -(11)-> Equipment PDCP -(10)-> Equipment TE -(11)-> Network Interface on Server PC

Ping Reply : Server PC -(1)-> Equipment TE -(2)-> Equipment PDCP -(3)-> Equipment RLC -(4)-> Equipment MAC-(5)-> Equipment PHY -(6)-> RF Connection

-(7)-> UE PHY -(8)-> UE MAC -(9)-> UE RLC -(10)-> UE PDCP -(10)-> UE IP Driver-(11)-> Network Interface on UE PC

< Case 3 >

Ping Path from UE PC to Server PC :Ping Request : UE PC -(1)-> UE IP Driver -(2)-> UE PDCP -(3)-> UE RLC -(4)-> UE MAC -(5)-> UE PHY -(6)-> RF Connection

-(7)-> Equipment PHY -(8)-> Equipment MAC -(9)-> Equipment RLC -(10)-> Equipment -(11)-> Equipment PDCP -(10)-> Equipment TE -(11)-> Dummy Hub -(11)-> Router -(12)-> Network Interface on Server PC

Ping Reply : Server PC -(1)-> Router -(2)-> Dummy Hub -(3)-> Equipment TE -(4)-> Equipment PDCP -(5)-> Equipment RLC-(6)-> Equipment MAC -(7)-> Equipment PHY -(8)-> RF Connection


9 9/6/2012


4/19

-(9)-> UE PHY -(10)-> UE MAC -(11)-> UE RLC -(11)-> UE PDCP -(12)-> UE IP Driver-(13)-> Network Interface on UE PC

Basic and Common steps to check before floodEven before you try any throughput test (e.g, UDP, TCP), there are several critical steps to go through. In many cases, I had beenasked to be on-site saying "SOMETHING is NOT WORKING" (These two words are what we most frequently use, but most ambiguousword. What is the "SOMETHING" ? What do you mean by "NOT WORKING". .. Let's not get into this too much -:).First thing you have to check is to check if your device get assigned any IP address that a network or Network Emulator assigned. Ifyou have UE logging tool or special menu/tool on your UE to show the IP address, it will be great help.Another way to check the IP allocation for UE if the UE is a data card or connected to PC as a modem (tethered to PC), try ipconfigcommand as follows.C:\> ipconfigEthernet adapter Local Area Connection 9:

Connection-specific DNS Suffix . :IP Address. . . . . . . . . . . . : 192.168.1.1Subnet Mask . . . . . . . . . . . : 255.255.255.0Default Gateway . . . . . . . . . :

In some case, this result (i.e, ipcofig shows the IP address allocated to UE) is enough signal for you to go to next step like ping. Butsometimes just ipconfig result would not be a guarantee for next step. (I think it depends on UE driver implementation).A better insurance in this case would be as follows. Open up the Network Connection Menu and see if you see the Modem andNetwork interface card properly configured and connected.

And openup the LAN card property for the UE and set the IP allocated to UE.


9 9/6/2012


5/19

Now you are ready to take real first step for the test, which is PING test. I am using Ping for two purpose. One is to check if the endto end (from server to client) data path is established and the other is to figure out the latency (delay time) between server andclient.First I would do "ping to self". In my test, I allocated 192.168.1.1 to UE and 192.168.1.2 to server. All of these test was done onclient PC, the PC to which the UE is connected.This would always works.. but I did to give you some reference for latency for local loop. It is under 1 ms as you see.

C:\>ping 192.168.1.1 -l 1400 (WCDMA DL 384K / 64 K, Local Loopback)Pinging 192.168.1.1 with 1400 bytes of data:Reply from 192.168.1.1: bytes=1400 time


6/19

Pinging 64.233.183.99 with 32 bytes of data:Reply from 64.233.183.99: bytes=32 time=222ms TTL=43Reply from 64.233.183.99: bytes=32 time=221ms TTL=43Reply from 64.233.183.99: bytes=32 time=221ms TTL=43Reply from 64.233.183.99: bytes=32 time=222ms TTL=43Ping statistics for 64.233.183.99:

Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),Approximate round trip times in mil li-seconds:

Minimum = 221ms, Maximum = 222ms, Average = 221ms

Now let's get into the situation that we are really interested. I connected UE to my Network Emulator with WCDMA DL 384K /UL 64K radio bearer and got the following result. If you try with your device, you may have different number.. so the exact number forthe time delay would not be so important, but you see pretty long delay which is around 323 ms. I put "-l" option to send almostfull size IP packet. It is direct connection from UE to Network emulator. But if you see the delay value here, it is greater than thedelay between my PC and a remote server on wireline network which may have over 100 hops along the line.

C:\>ping 192.168.1.2 -l 1400 (WCDMA DL 384K / UL 64 K)Pinging 192.168.1.2 with 1400 bytes of data:Reply from 192.168.1.2: bytes=1400 time=332ms TTL=128Reply from 192.168.1.2: bytes=1400 time=326ms TTL=128Reply from 192.168.1.2: bytes=1400 time=324ms TTL=128Reply from 192.168.1.2: bytes=1400 time=323ms TTL=128Ping statistics for 192.168.1.2:



I used exactly same UE and same driver. Only changed radio bearer to HSDPA DL 3.6 M/UL 5.6 M. Pinged and got the result asfollows. Delay time decreased almost 3 times.

C:\>ping 192.168.1.2 -l 1400 (HSDPA DL 3.6 M/UL 5.6 M)Pinging 192.168.1.2 with 1400 bytes of data:Reply from 192.168.1.2: bytes=1400 time=112ms TTL=128Reply from 192.168.1.2: bytes=1400 time=107ms TTL=128Reply from 192.168.1.2: bytes=1400 time=116ms TTL=128Reply from 192.168.1.2: bytes=1400 time=115ms TTL=128

Ping statistics for 192.168.1.2:Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in mil li-seconds:Minimum = 107ms, Maximum = 116ms, Average = 112ms

And tried the ping with a little bit higher data rate HSPA Bearer (HSDPA DL 7.2 M/UL 5.6 M). It is almost same delay time as before(lower data rate HSPA). Definately you will see different throughput comparing to previous bearer, but in terms of ping delay wedon't see much difference here.

C:\>ping 192.168.1.2 -l 1400 (HSDPA DL 7.2 M/UL 5.6 M)Pinging 192.168.1.2 with 1400 bytes of data:Reply from 192.168.1.2: bytes=1400 time=116ms TTL=128Reply from 192.168.1.2: bytes=1400 time=111ms TTL=128Reply from 192.168.1.2: bytes=1400 time=110ms TTL=128

Reply from 192.168.1.2: bytes=1400 time=108ms TTL=128Ping statistics for 192.168.1.2:



From previous two test, I don't depect any big different with this test, but anyway I gave it another try with higher HSDPA bearer.

C:\>ping 192.168.1.2 -l 1400 (HSDPA DL 14.4 M/UL 5.6 M)Pinging 192.168.1.2 with 1400 bytes of data:Reply from 192.168.1.2: bytes=1400 time=126ms TTL=128Reply from 192.168.1.2: bytes=1400 time=113ms TTL=128


9 9/6/2012


7/19

Reply from 192.168.1.2: bytes=1400 time=112ms TTL=128Reply from 192.168.1.2: bytes=1400 time=110ms TTL=128Ping statistics for 192.168.1.2:



Now I pushed the same device one more step upward. I get it connected to HSPA+ Bearer (Category 14, 64 QAM). You see thedifference ? The delay time get halved comparing to conventional HSDPA.

C:\>ping 192.168.1.2 -l 1400 (HSPA+ Single Carrier)Pinging 192.168.1.2 with 32 bytes of data:Reply from 192.168.1.2: bytes=32 time=58ms TTL=128Reply from 192.168.1.2: bytes=32 time=54ms TTL=128Reply from 192.168.1.2: bytes=32 time=53ms TTL=128Reply from 192.168.1.2: bytes=32 time=52ms TTL=128Ping statistics for 192.168.1.2:



Now I upgraded the bearer one step further to HSPA+ Dual Carrier (Category 24) and I don't see much improvement comparing toprevious one.

C:\>ping 192.168.1.2 -l 1400 (HSPA+ Dual Carrier)Pinging 192.168.1.2 with 32 bytes of data:Reply from 192.168.1.2: bytes=32 time=53ms TTL=128Reply from 192.168.1.2: bytes=32 time=48ms TTL=128Reply from 192.168.1.2: bytes=32 time=47ms TTL=128Reply from 192.168.1.2: bytes=32 time=56ms TTL=128Ping statistics for 192.168.1.2:



Now let's get into each of different radio technologies and see if we can explain why we have different throughput and evendifferent ping delay.

I haven't completed the remaining part... but it would be good to give all of you a couple of days to think about this issue.

What kind of failure mode you see for each technology ? (Data rate lower than you expected ? Data rate no problem.. but allof the sudden call drop ? )The failure mode is same for all technology or different ?Recalling each of the steps along the data path, which one do you think would be the bottleneck for the throughput ?Would the bottleneck be the same for all technology ? or different ?Can you explain technically about the root cause of the failure ?

Don't expect that I would know all the answer and give you the clear answer. I also have to think a lot and will give you my opinionjust based on my experience and based on my shallow knowledge a couple of days later.

WCDMA (R99)

As I implied in questions listed above and as I experienced, the types of throughput test (I would call this "Failure Mode") differsdepending on what kind of radio technology you are using.In case of R99, I don't see many issue about the problem of "low throughput". The failure mode seems to be more like "All orNone", most common problem seems that it started working with full throughput (All) as specified in Radio Bearer Setup and all ofthe sudden the data path stop working (No throughput).Let's look into overall data path of R99. I i llustrated as follows but it may not be exact in terms of data packet size and number ofdata packets in each layer.. but I just wanted to give you overall path and relative packet size comparison across the multiplelayers.(This is transmitter side data path and the reciever side data path would be almost same, but just in reverse path).Normally I look at the first input and the last output at the first step. In most case, the first input is an "IP packet" from the serverPC (e.g, "UDP packet" generated by iperf). The size of the data would be around 1500 bytes or just a little less.Then what is the last output of the data path ? It is L1 output. What is the size of the data frame at L1 ? It gets different depending


9 9/6/2012


8/19

on the radio bearer setup and even with the same radio bearer each radio frame may carry different data size depending of TFCIselection for each transmission. So I cannot specify any fixed frame size, but I can say it is much less than one IP packet. It meansthat the input data size is much bigger than the final output data size. It implies that somewhere in the data path, there should besome steps where the input data (IP packet) is get splitted into multiple chunks each of which can be fit into final output data size.Then you would guess what should be done on reciever side. The reciever should recombined all those fragmented chunk into theorignal IP packet and push up to IP tools.One of the common problem that may happen in R99 case would bei) problems at the split and recombine processii) problems at selected proper frame size at L1 (selecting proper TFCI)If any problem happens in these step, it normally lead to total data throughput stop (not just a small throughput drop). But recentlyI don't see many of throughput problem in R99. It is now very mature technology.

I will put some real traffic example sometime later.

HSDPAWhenever I have inquiries about HSDPA related throughput problem, I am going through the following check list.

i) Does the Network (or Network Emulator) define TFRI table for max throughput ?ii) Does the TFRI Index has been selected at each transmission for max throughput ?iii) Does UE reflect the proper category information on RRC Connection Setup Complete ?iv) Does HARQ memory model is properly configured in Radio Bearer Setup ? (e.g, Implicit vs Explicit, Number HARQ, HARQMemory Size etc)v) Does PHY layer shows any HARQ Retransmission ?vi) Does RLC shows any retransmission ?vii) Does PC inject the packet which is big enough to fully uti lize the data pipe defined by theNetwork?viii) Does PC inject the data packet as frequently to fully utilize the data pipe ?Now you may understand why I put such a amphasis on having proper logging tools for throughput troubleshoot. Almost none ofthe list you can check without having proper logging tool. The best option is to have such a logging tool both on Network side andUE side, but if not.. you should have the tools at least on one side (UE or Network).Now let's look into overall data flow. In HSDPA case, the packet size at the input stage (IP packet size) is similar to the final L1

frame size, even though the final L1 frame size can be a little bit smaller and larger than the input packet size depending on HSDPACategory. But still you have MAC-d is involved in the data path and the MAC-d packet size is much smaller than IP packet size andL1 frame size. It means the IP packet should get splitted into multiple small chunks to go through MAC-d and have to bereassembled before it gets into L1. I don't think this is very efficient process but we would not be able to get rid of MAC-d becauseof current UMTS network architecture. Technically this kind of split/combine process can be a source of problems.


9 9/6/2012


9/19

In HSDPA case, there is another issue that make situation complicated. In R99 case, the most common L1 transmission timing is 10ms (1 TTI = 10 ms), but in HSDPA case the most common L1 transmission timing is 2 ms (1 TTI = 2 ms). It means that if L1 framesize is similar to one IP packet size, the PC tool should be able to create IP packet 500 times per second and Network's internallayer is operating fast enough to pass all those packets down to L1. It implies that PC performance or PC configuration can be a

bottle neck for throughput test (especially HSDPA Category 8, 10 case).For your reference, I created a table showing you a maximum (near maximum) throughput for most commonly used HSDPAcategories. Just for this throughput issues, let's just focus on TTI, TBS, PDU. TTI shows how often a network transmit a chunkthrough PHY layer. For example, TTI = 2 means the network transmit a PHY layer data chunk every 2 ms.TBS is Transmit Block Size. The unit in 3GPP table is in Bits, but I added another column showing TBS in Bytes just for you to easilycompare it with IP packet size which is normally expressed in Bytes.For example, if TTI = 2 and TBS = 3630, the network transmit a data chunk with the size of 3630 (about 453 bytes) bits every 2ms.PDU is the data chunk of MAC-d. So PDU size is the size of data chunk getting out of MAC-d. If you compare PDU size and TBS, youwill notice that TBS (PHY data chunk) is much bigger than PDU size. If you compare PDU size and common IP packet size (1500Bytes), you will notice IP packet size is much bigger than PDU size.Putting all these together, you will figure out that in this process an IP packet should split into many PDUs and those many PDUsshould be reassembled into a single Transport Block(TB) and then get transmitted through antenna. This is the meaning of diagramshown above.

Another important thing you can notice from the table above is that from Category 8, one transport block size gets bigger than oneIP packet. It means that PC has to tranmit one or more IP packets every 2 ms. If you see Category 10, you will notice that PC(DataServer) should be able to transmit more than 2 IP packets every 2 ms. So in this case, PC performance greatly influence the overallthroughput.So my recommendation, especially for high data rate category, is for you to check PC setting/performance and see if the PCperformance is good enough for this testing. (Connect the client and server PC directly with LAN cable and do the PC-to-PC wirelinethroughput test and make it sure that the throughput is well higher than the expected UE throughput.

HSPA+If you fully understood what was explained in HSDPA section, you can easily understand what would be the critical issue for HSPA+.Simply put, in HSPA+ case, you will have much bigger TBS comparing to conventional HSDPA. So you would guess, IP layerperformance (e.g, Data Server and Client PC) would be much more important comparing to HSDPA case.


9 9/6/2012


10/19

Just to give you clearer idea on this process. I put down the data flow at each layer. Go over these examples with special attentionto the data size that I put in ( ).< Example 1 > Cat 14, Ping Test

TE -> RLC : (60 Bytes)45 00 00 3c 00 e5 00 00 80 01 b6 88 c0 a8 01 02c0 a8 01 01 08 00 48 5c 03 00 02 00 61 62 63 6465 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71 72 73 74 75 76 77 61 62 63 64 65 66 67 68 69RLC -> MAC : (82 Bytes = 656 Bits)80 0d 79 fe 45 00 00 3c 00 e5 00 00 80 01 b6 88 c0 a8 01 02c0 a8 01 01 08 00 48 5c 03 00 02 0061 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71 72 73 74 75 76 77 61 62 63 64 65 66 67 68 6900 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00MAC->L1 : (3372 Bytes = 26976 Bits)00 40 1c 00 6b cf f2 28 00 01 e0 07 28 00 04 00 0d b4 46 05 40 08 16 05 40 08 08 40 02 42 e0 1800 10 03 0b 13 1b 23 2b 33 3b 43 4b 53 5b 63 6b 73 7b 83 8b 93 9b a3 ab b3 bb 0b 13 1b 23 2b 333b 43 48 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00.....

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000 00 00 00 00 00 00 00 00 00 00 00

HSPA+ Dual CarrierIn terms of throughput perspective, you can think of HSPA+ Dual Carrier as two HSPA+ running in parallel. So the IP layer toolperformance will be almost critical factor. In this case, Data Server should be able to transmit almost 9 IP packets every 2 ms. Inthe same token, this means Client PC to which UE is connected to is working fast enough to receive all of these data and process.So the client PC performance is important as well< Example 1 > Cat 24, Iperf UDPTE -> RLC : (1498 Bytes)45 00 05 da bf b5 00 00 80 11 f2 09 c0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 4c f0 00 00 00 41

4d cc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00 ff ff e8 90.....32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 3334 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39TE -> RLC : (1498 Bytes)45 00 05 da bf b8 00 00 80 11 f2 06 c0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 4c ed 00 00 00 444d cc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00 ff ff e8 90.....32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 3334 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39TE -> RLC : (1498 Bytes)45 00 05 da bf b8 00 00 80 11 f2 06 c0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 4c ed 00 00 00 444d cc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00 ff ff e8 90.....


19 9/6/2012


11/19

32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 3334 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39TE -> RLC : (1498 Bytes)45 00 05 da bf b8 00 00 80 11 f2 06 c0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 4c ed 00 00 00 444d cc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00 ff ff e8 90.....32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 3334 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39RLC -> MAC : (1500 Bytes)81 de 45 00 05 da bf af 00 00 80 11 f2 0fc0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 4c f6 00 0000 3b 4d cc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00 ff ff...30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 3132 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39RLC -> MAC : (1500 Bytes)81 de 45 00 05 da bf af 00 00 80 11 f2 0fc0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 4c f6 00 0000 3b 4d cc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00 ff ff...30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 3132 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39RLC -> MAC : (1500 Bytes)81 de 45 00 05 da bf af 00 00 80 11 f2 0fc0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 4c f6 00 0000 3b 4d cc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00 ff ff...

30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 3132 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39RLC -> MAC : (1500 Bytes)81 de 45 00 05 da bf af 00 00 80 11 f2 0fc0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 4c f6 00 0000 3b 4d cc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00 ff ff...30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 3132 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39MAC-> L1 : Primary Channel (5274 Bytes)eb b8 f4 eb b8 eb b8 e5 fb 81 1a 45 00 05 da bf 97 00 00 80 11 f2 27 c0 a8 01 02c0 a8 01 01 0461 13 89 05 c6 74 2d 00 00 00 23 4d cc 77 da 00 06 7e 63 00 00 00 00 00 00 00 01 00 00 13 89 00.....33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 3435 36 37 38 39 81 22 45 00 05 da bf 98 00 00 80 11 f2 26 c0 a8 01 02c0 a8 01 01 04 61 13 89 05

c6 74 2c 00 00 00 24 4d cc 77 da 00 06 7e 63 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02.....37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 3839 81 2a 45 00 05 da bf 99 00 00 80 11 f2 25 c0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 74 2b 0000 00 25 4d cc 77 da 00 06 7e 63 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00 ff.....31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 81 32 4500 05 da bf 9a 00 00 80 11 f2 24 c0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 4d 0b 00 00 00 26 4dcc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00 ff ff e8 90 3637 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38......37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 3839 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34MAC-> L1 : Secondary Channel (5274 Bytes)e5 be fa eb b8 eb b9 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39

30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31......32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 3334 35 36 37 38 39 81 3e 45 00 05 da bf 9b 00 00 80 11 f2 23 c0 a8 01 02c0 a8 01 01 04 61 13 8905 c6 4d 0a 00 00 00 27 4d cc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 0002 9f 63 00 ff ff e8 90 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39......36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 3738 39 81 42 45 00 05 da bf 9c 00 00 80 11 f2 22 c0 a8 01 02c0 a8 01 01 04 61 13 89 05 c6 4d 0900 00 00 28 4d cc 77 da 00 06 a5 82 00 00 00 00 00 00 00 01 00 00 13 89 00 00 00 00 02 9f 63 00......38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 3930 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 00 0000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00


19 9/6/2012


12/19

......00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Ideal MAX throughput for UMTS UE

CQI vs Throughput for UMTSIn live network for HSDPA, Network sends data with different transport block size depending on CQI value reported by UE. For thismechanism to work properly, there should be a certain level of agreement between UE and the network about "which CQI valuemeans which transport block size". These agreement is defined in the following tables of TS 25.214.

Table 7A: CQI mapping table A.

Table 7B: CQI mapping table B.

Table 7C: CQI mapping table C.

Table 7D: CQI mapping table D.Table 7E: CQI mapping table E.

Table 7F: CQI mapping table F.

Table 7G: CQI mapping table G

Then next question is which table do I have to use for which case ? The answer is in the following table from 24.214. As you see,we use different table depending on UE Category, Modulation Scheme, MIMO. For example, if a UE is Category 14 device and uses64 QAM and does not use MIMO, it use Table G for CQI-Transport Block Size Mapping as shown below.


19 9/6/2012


13/19

I put Table 7G as an example. As you see in the table, the range of CQI value is 0~30. 30 means the best channel quality andlower number indicates poorer channel quality. And Network has to send the data with the proper transport block size according tothe CQI values.For example,i) If UE report CQI value 15, it is expected for Network to send data with transport block size of 3328 bits/TTI which is equivalent toaround 1.6 Mbps.ii) If UE report CQI value 30, it is expected for Network to send data with transport block size of 38576 bits/TTI which is equivalentto around 19 Mbps.


19 9/6/2012


14/19

One thing you would notice that the transport block size for the highest CQI value is not amount to the ideal MAX throughputdefined in 25.306 Table 5.1a. It implies that you wouldn't get the ideal Max throughput in any case with live network conditionwhich may operate according to the CQI table defined in 3GPP. (It would not be any problem in real communication environmentsince your device would not report CQI 30 in most case).However, many UE manufacturer/developer wants to see if their device can really reach the ideal max throughput. In that case, wenormally use a special network simulator which allows us to set the largest transport block size for each UE category. It would beeven better if the network simulator allows us to define CQI-transport block mapping table arbitrarily. Fortunately I have access tothis kind of the equipment and I did an experiment as shown below using the network simulator and a HSDPA Category 10 UE.

First I defined a CQI-transport block size table very similar to Table 7D, but I changed the tranport block size for high end CQI (30,29, 28, 27) to allocate larger tranport block than the ones specified in Table 7D to push the ideal MAX throughput.I programmed Network Simulator so that I decrease the downlink power by a certain steps. As downlink power (Cell Power) getsdown, UE would report lower CQI and Network Simulator would transmit lower transport block size.The result is as follows.In the upper plot, you see three traces - Green, Red, Blue. Green trace means the everage CQI value within 500ms that UEreported. Red trace indicates the the amount of data in Kbps that the network emulator transmitted to UE within a second. Bluetrace indicates the amount of data in Kbps that UE successfully decoded. If the Red trace and Blue traces overlaps, it implies thatUE successfully decoded all the data transmitted by the network. If the Blue trace is lower than the Red Trace, UE failed to decodesome of the data transmitted by the network. The black line shown in section A, B, C is the data rate defined in Table 7D, but Iintentionally allocated the higher data rate for section A,B,C to push the data rate closer to the ideal Max throughput.In the lower plot, you see three traces - Green, Red, Blue. Green trace means the everage CQI value within 500ms that UE reported.Red trace indicates the amount of ACKs within 500 ms and Blue trace indicates the amount of NACKs within 500 ms.There are a couple of things you may notice (The notes here may be different from what you observed from your device and testsetting)i) Section A is the only region in which UE shows 100% data decoding without any failure. It means that you have to make it surethat your test equipment configuration, cable connection between the test equipment and UE is configured properly so that thechannel quality belongs to this area. (I would say "CQI should be much higher than 30". I know 30 is the max CQI value. What Imean is that the channel quality should be much better than the quality in which UE barely reports CQI 30).ii) In Section B, you see huge drops in terms of throughput and huge increase in terms of number of NACKs. Main reason would bethat I allocated too large transport block size for CQI 29, 28. There would also be some UE issues with this range.Section C,D,E shows a kind of normal trends, but ideally we should expect exact overlapping of rad trace and blue trace, but realitynever goes like ideal -:)


19 9/6/2012


15/19

LTEEven though there is huge difference between LTE and Non-LTE in terms of physical layer implementation, you can just think ofthis as just extention of HSPA. Which means that in LTE you will have even bigger TBS than HSPA+ Dual Carrier. So in this case,very high performance IP data server is the most important factor for throughput test. For the detailed data path for LTE, you canrefer to another pages of this site. Almost whole of this site is about LTE.For example, if you are testing a Category 3 Max throughput case (System BW 20 Mhz, Number of RB = 100), the idea maxthroughput is around 100 Mbps. Even with PC to PC direct connect, it is not easy to achieve this level of throughput. So for this

kind of extremely high throughput test, it would be the mandatory step to try the PC (both server and client) performance first byconnecting the two PCs directly with a crossover LAN cable and try the throughput test. In rare case, even the quality of LAN cablewould influence on the throughput. I recommend you to use Cat 6 LAN cable which support Gigabit data rate.In addition, using 64QAM in downlink would be very common in LTE. In this kind of very high modulation scheme, the throughput isinfluenced greatly by channel quality (RF signal quality). Small change in downlink power, Fading, AWGN can create large changesin throughput.One of the most frequent questions that I get on the throughput test is "What is the ideal throughput with this condition ?" In caseof R99 or HSPA, the ideal throughput is described in a neat page of table and a lot of people knows what kind of throughput theyhave to expect, but in LTE the situation got much more complicated since there can be several factors determines the throughputand each of the factors can have so many different values. So the number of all the possible combinations for defining thethroughput is so huge.The most important factors to determine the ideal throughput are as follows :

Number of RBs

MCS (Modulation Coding Scheme) which determines I_TBSMIMO or SISO

The way to calculate the ideal throughput using these factors are explained in "Throughput Calculation Example" in Quick Referencepage.I made several examples of resource allocation and its ideal throughput as follows. These conditions are the most common conditionfor maximum throughput test. The values marked in gray cell is the one going over Category 3 LTE device capability. In most case,if I try the condition marked in gray cell with most commercial UEs that I tested (Category 3 UEs), I got one of the following result.i) The throughput degrade a lot (normally much lower than 100 M)ii) It reaches almost 100 M, but does not go over.(Thank God ! Call drop didn't happen even in this case)

>


19 9/6/2012


16/19

No of RB MCS I_TBS TBS SISO(Mb) MIMO(Mb)

6 28 26 4392 4.19 8.38

15 28 26 11064 10.55 21.10

25 28 26 18336 17.49 34.97

50 28 26 36696 35.00 69.99

75 28 26 55056 52.51 105.01

75 27 25 46888 44.72 89.43

100 28 26 75376 71.88 143.77

100 23 21 50124 48.66 98.32

100 20 18 39232 37.41 74.83

>

No of RB MCS I_TBS TBS SISO MIMO

6 28 26 4392 4.19 N/A

6 20 19 2600 2.48 N/A

15 28 26 11064 10.55 N/A

15 20 19 6456 6.16 N/A

25 28 26 18336 17.49 N/A

25 20 19 10680 10.19 N/A

50 28 26 36696 35.00 N/A

50 20 19 21384 20.39 N/A

75 28 26 55056 52.51 N/A

75 20 19 32856 31.33 N/A

100 28 26 75376 71.88 N/A

100 20 19 43816 41.79 N/A

Now I got what to expect for the throughput. Do I get this value if I just plug in my device into the test equipment and do FTP/UDP?In most case, the answer is NO.Why not ?There are couple of factors you have to keep in mind as follows :i) The ideal throughput value in the table is on the basis of physical layer operation, not based on higher layer (e.g, IP layer)throughput.ii) The ideal throughput value in the table is based on the assumption that there is no physical layer overhead and we can allocatethese resource at every subframe

When a stream of data comes from IP layer to the physical layer, there are some overhead being added (e.g, PDCP header, RLCheader, MAC header etc). So the IP layer throughput gets lower than the physical layer.What kind of other overheads we can think of ? Followings are the most common overhead.i) SIB transmissionii) Symbols allocated for PCFICH and PDCCH.At the subframe where SIB is transmitted, you cannot allocate the full bandwidth for data transmission. If you can dynamicallyallocate a little smaller No of RBs in these slots, you only have to sacrifice the number of RBs for SIB transmission.. but in most testequipment the situation is even worse. The equipment does not allow such kind of dynamic resource allocation just to avoid theoverlapping of SIB and user data. In this case, the equipment does not tranmit any user data at the subframe where SIB is beingtranmitted. In such a case, SIB transmission can be pretty huge overhead for the max throughput test.Another overhead is the one by PCFICH and PDCCH. As you learned from Downlink Framestructure section, there is at least onesymbols (max 3 symbols of each subframe are allocated for PCFICH and PDCCH). If you allocate three symbols for PCFICH andPDCCH, which means that you set PCFICH, 3 out of 14 symbols are allocated for non-user data. However, speaking purely in ideal

sense this overhead would not influence the ideal throughput since the transport block size determined by 3GPP for eachcombination of resource allocation took this overhead into account. But in reali ty, if you allocate too large Transport block size (toohigh MCS and No of RBs) and allocate large PCFICH (e.g, 2 or 3), it normally reads to a lot of CRC error which in turn results inthroughput degradation.As far as I tried with commercial device, the MAX IP layer throughput (UDP) that I achieved was around 90 Mbps with 20 Mhzsystem bandwith and MIMO condition. Physical layer throughput approaches almost 100 Mbps (only a couple of Mbps lower than100 Mbps).

LTE Physical Layer Throughput - PDSCH Decoding PerformanceRefer to PHY/L1 Performance Test page.


19 9/6/2012


17/19

Throughput in Live NetworkAs you naturally guess, the throughput you would achieve in Live Network would be different in large degree from the one you getwith test equipment. It is because the radio channel condition and the network components (or performance of the components) willbe different and another big impact is that in live network usually multiple UEs shares a resource whereas in test equipment youcan allocate the full resources only to single UE (DUT).

Radio channel quality (Signal Strength, Noise Level, Interference)

Throughput of Network Backhaul

Number of UEs connected to the network at the time of the test

Amount of physical resources allocated to the DUT (It is hard to check this part especially in LTE case, since physical layer

resource allocation is not included in RRC messages. Unless you have low level logging tool either in UE or in Network, itwould be very difficult to figure out how much physical layer resources (e.g, Number of RB, MCS) are allocated).

Factors influencing ThroughputI have an excellent tips for troubleshooting for throughput or throughput optimization. It is "There is no short cut for it". I saw a lotof cases where people just are just trying to find the short cut and eventually spend more time and energy.I will give you my personal approach in this section.First, write down all the components in the data path (Really "write down" on the paper or in computer document software.).Following are a couple of examples for the description of the data path. You would have more Cases on your own and you woulddescribe it in more detail, meaning putting down more detailed components in the path. The more components you can write down,the sooner you would achieve your goal.Case 1 : UE Packet App on UE PC -(1)-> UE IP Driver -(2)-> UE PDCP -(3)-> UE RLC -(4)-> UE MAC -(5)-> UE PHY

-(6)-> RF Connection -(7)-> Equipment PHY -(8)-> Equipment MAC -(9)-> Equipment RLC -(10)-> Equipment-(11)-> Equipment PDCP -(10)-> Equipment TE -(11)-> Network Interface on Server PC -(12) -> Packet App on Server PC

Case 2 : UE Packet App on UE -(1)-> UE PDCP -(2)-> UE RLC -(3)-> UE MAC -(4)-> UE PHY

-(5)-> RF Connection -(6)-> Equipment PHY -(7)-> Equipment MAC -(8)-> Equipment RLC -(9)-> Equipment-(10)-> Equipment PDCP -(11)-> Equipment TE -(12)-> Network Interface on Server PC -(13) -> Packet App on Server PC

Case 3 : Client UE Packet App on UE -(1)-> WiFi Stack on Client UE -(2)-> WiFi Connection -(3)-> WiFi Stack on Mobile Hotspot UE

-(4)-> Hotspot UE PDCP -(5)-> Hotspot UE RLC -(6)-> Hotspot UE MAC -(7)-> Hotspot UE PHY-(8)-> RF Connection -(9)-> Equipment PHY -(10)-> Equipment MAC -(11)-> Equipment RLC -(12)-> Equipment-(13)-> Equipment PDCP -(14)-> Equipment TE -(15)-> Network Interface on Server PC -(16) -> Packet App on Server PC

Second, ask yourself "Do I have any measure/tools to see what's happening in each and every components ?". (Wireshark, UElogging tool, Network logging tool would be the minimum requirement).Third, ask yourself "Do you have knowledge and skills to analyze every and each components you wrote down at step 1?"

It would not be highly possible for you to be the one who knows everything. At least try to get other persons ready to help youanalyze those data.Fourth, try to indetify important parameters influencing the throughput. The more, the better. Following is an example list comingfrom my experience.

Factors Description

Transport Block Size

CQI Report Accuracy

Antenna Configuration

RLC Window Size

RLC Reordering Timer

TCP Window Size

IP Packet Latency

Data Buffer Size

USB Driver

Mobile Hot Spot Efficiency

Lastly, do the test and analysis as much as possible before the problem is find by somebody else. Normally if any problem happens,almost everybody including me wants to get it solved right away. But solving the throughput related problem right away is just amatter of luck, not the matter of engineering/science. I don't like any situation which would depend only on luck. The best way is toanalyze the device as in detail as possible and see how each of the factors listed above influence the throughput of the device. Eachof the factors influence in different ways to different device model/software. This is the only way to find the solution the soonestwhen the problem happens in the field.

Links on Throughput Test in the field


19 9/6/2012


18/19

Realistic LTE Performance (from Motorola)

Field Trial for LTE Network System (from Fujitsu)

iperf - UDP/TCP FloodingIperf would be one of the most common tools for data throughput test because of easy installation, easy operation, less overhead.You can download iperf from http://sourceforge.net/projects/iperf/I think you can get pretty good tutorial from http://openmaniak.com/iperf.phpTips :i) When you do UDP test, don't push too high data comparing to the expected data rate.ii) When you do TCP test, set window size (-w option value) very carefully. (There is no clear/direct way to figure out the optimumvalue. You would need some try and error).

Filezilla - FTPYou can download FileZil la Client and Server from following sites.

Filezilla Server Download from http://filezilla-project.org/download.php?type=server

Filezilla Client Download from http://filezilla-project.org/download.php

For general setup, refer to tutorials you can search from various source.For very high data rate application (e.g, LTE throughput test), downloading only one file would not push the enough data rate. Aworkaround for this case is to download multiple files simultaneously. For this kind of multiple download, you have to configureFileZil la server and Client to enable multiple download/upload. You can set this configuration as shown below.


19 9/6/2012


19/19

The maximum FTP throughput that I got was around 97 Mbps with FileZilla using a LTE data card, system bandwidth 20M, MIMO.


Documents

Data Throughput Troubleshoot