7/27/2019 generation Field Testing
1/16
FIELD TESTING
of2G 3G Devices
7/27/2019 generation Field Testing
2/16
Field Trials conducted on terminals ensure that the interoperability of
the terminal with the public networks will be robust and minimizes
returns from the market.
7/27/2019 generation Field Testing
3/16
Development Cycle
Product
Planning
Product validation
( Field Testing)
Product Validation
(LAB Testing)
Final QA
Commercial Sample
QA
Platform Design & Reference ArchitectureFull protocol stack and man machine interface softwareFunctional and Performance Specifications
Protocol Testing (RF/USIM/Protocols)Pre-Conformance-physical & signaling)Power consumption testing
Live Network TestingIOTInter-RATField TrialsKPIDATA performance testing
Check All the Reported issues should beclosed.
Carrier approval confirmation.Operator Acceptance.
A quick review of all the major issuesReported.Field Trial.OEM/Operator Specific requirement.Production Issues.
7/27/2019 generation Field Testing
4/16
Live Network Field Trials & IOT
Field Testing is carried out with the product connected to a live network in real
conditions. Field trials give confidence in end to-end product performance. This
highlights software quality and stability issues and how the product performs for the
customer. This testing can be carried out to customer-specific requirements, network
operator requirements or to the GCF field trial recommendations.
References :
DG 11Field Trial Guidelines
3GPP org
Test Case provided by:
OEM/ODM
Operator Acceptance User Acceptance (UAT)
7/27/2019 generation Field Testing
5/16
Features Tested in a Field Trials
Some of the features tested in a field trial are:
Cell selection and reselection Network selection
Location updating
Speech quality (FR / HR / EFR, AMR)
Roaming
Hand over (GSM/GPRS)
Signal strength
Routing area updates for GPRS
GPRS attach and detach
GPRS PDP activation and deactivation
Verification of different QOS Parameters.
TCP/IP traffic (UL/DL) with different Multi slot configurations GPRS Authentication
7/27/2019 generation Field Testing
6/16
Field Test Preparation
Field test objective definitions (Performance Test: RF Drive test, Call
Test )
Diagnostic monitor*
Laptop computer
Test and reference mobiles, chargers, extra batteries, etc.
Mass storage device GPS receiver ( For guided RF Drive Test , normally applicable to
typical topographies)
Data/RF cables
Dual UART for data testing
Extension cord (s)
USIM/ RUIM card reader/writer along with its software (for RUIM-
capable mobiles)
7/27/2019 generation Field Testing
7/16
Field Testing Tools
Phone Flashing Tool:
Generally these tools are used to download or upgrade the software in phone using desktop/laptop
with the help of data cables. Logging/Tracing Tool:
Tracing tool are used to capture the message of device, exchanged between the different layer and
with network while in operation .This help in debugging the errors and problem with the device or
network.
7/27/2019 generation Field Testing
8/16
Field Testing Tools
DUT and reference device:
During a testing phase of a device there should be a reference model for benchmarking and
comparing the performance of the DUT.
Engineering Mode:
Engineering Mode is designed to allow a field engineer to view and test the network
information received by a handset, when the handset is either in idle mode or dedicated mode
(that is: with a call active). In each mode, the engineer is able to view network interaction for
the serving cell (the cell the handset is currently registered with) or for the neighboring cells.Test cases:
Test Case of OEM/ODM (Open Market)
Test Case Operator Acceptance (operator specific)
QM Management tool:
Web based Defect tracking tool, Repository & Centrally Controlled Tool
Organize & Manage all Phases of Testing Process
Specifying Testing Requirements
Planning Tests
Executing Tests
Tracking Defects (e.g.. Test Director)
7/27/2019 generation Field Testing
9/16
Test Environment
Dense Urban
Dense urban environments consist of lots of multi paths, and 4-, 5-, and 6-way handoffs arehighly visible. Also, the network is usually loaded.
Urban
Urban environments also have enough multi paths to test 3-, 4-, 5-, and 6-way handoffs. The
network is usually loaded, as well.
Suburban
There are some multi paths present in suburban environments, but not a lot. Usually, 1- or 2-way
handoffs are normal, and the network is lightly loaded. The mobile may not be locked to one
active Pilot in traffic.
7/27/2019 generation Field Testing
10/16
Test Environment (Contd..)
Stationary Environment (MP)Stationary environments can be divided into near, middle, and farscenarios.
Good (near) coverage area, Rx -70 dBm or better
Medium (middle) coverage area, Rx -70 to -85 dBm
Poor (far) coverage area, Rx -85 to -103 dBm
Most mobile feature and conformance testing is done in a good
GSM/WCDMA coverage area.
Moving Environment (MR)
Moving environments are very dynamic, and most mobile field testing is carried out in
moving conditions.
2545 km/h
4565 km/h
65100 km/h
Variable speed
7/27/2019 generation Field Testing
11/16
Drive test Route Selection
Test Route
A test route is a prescribed traveling/driving route in a GSM/CDMA/WCDMA coverage area usedfor field testing in a real, dynamic RF environment.
Depending on the feature being tested and the test equipment involved, the test route can be walked.
The test route plays a key role in field testing.
Most field testing of network features can be performed in a stationary location in a goodGSM/CDMA/WCDMA coverage area.
A drive route may consist of, but is not limited to, these features:
Various received signal strengths along the drive route (coverage
areas)
Known cell site locations
Soft/softer handoff areas
Boundaries of hard handoff, idle handoff, SID/NID, Zone and IP
registrations, etc.
Various vehicle speeds along the drive route (stop signs, intersections,
freeways, boulevards, etc.)
Single- and dual-mode areas, CDMA ,GSM,WCDMA, etc.
GSM, CDMA,WCDMA boundaries and vice versa
7/27/2019 generation Field Testing
12/16
Quality of Service Testing
Network Services are considered end-to-end, this means from a Terminal Equipment (TE) to
another TE. An End-to-End Service may have a certain Quality of Service (QoS) which isprovided for the user of a network service. It is the user that decides whether he is satisfied
with the provided QoS or not.
To realize a certain network QoS a Bearer Service with clearly defined characteristics and
functionality is to be set up from the source to the destination of a service.
A bearer service includes all aspects to enable the provision of a contracted QoS. These
aspects are among others the control signaling, user plane transport and QoS management
functionality.
Measuring Voice and Data Service Quality
Traditionally, wireless operators have used a number of metrics that collectively provide a
measurement of network service quality from the user perspective. Various operators may
define and measure network performance somewhat differently, but overall service quality
will generally be determined by a combination of the following four key performance
indicators (KPIs).
System coverage
Call blockage
Voice quality
BER bit error rate
7/27/2019 generation Field Testing
13/16
Data throughput measurements
Downlink Performance under Mobility Conditions Description
Verify the downlink data performance of EGPRS implementation in live networks.
Reason for test
To ensure that the downlink GPRS/EGPRS/HSDPA data throughput of the ME under test is acceptable undermobility conditions.
Initial configuration
The MS is GPRS and IMSI attached.
Use the MS in tethered mode as a modem for laptop
Identify a route with 2-3 cell reselections and a Routing Area Update
Test procedure
1. Activate PDP Context
2. Start on the identified route
3. Use DOS FTP to download a 2-15MB file. If possible, utilize a server with minimal latency to the GGSN.
4. Repeat the test 5 times, record the average FTP throughput.
Expected behavior
1. The results of this test are highly dependent upon the number of cell reselections, RAU, and average signalstrength along the drive route. Average downlink throughput under mobility conditions for a two-slotdevice should exceed 40 kbps, three-slot devices should exceed 55 kbps, and four-slot devices shouldexceed 70 kbps averaged over five FTPs.
2. The data histogram for each test run should indicate relatively few data stalls in excess of 10 seconds.
3. The GPRS connection and file download shall be stable during cell reselection and RAU
7/27/2019 generation Field Testing
14/16
Call Performance Criterion
All The Short/Medium/Long Call Made during A CP Test Should be under following Test
Condition
Half Rate Channel
Full Rate Channel Enhance Full rate Channel
RX Sensitivity (RxLev and Rx_Qual)
Short/Medium/Long Call Made during A CP Test Should be under following Test
Condition
Attempt Call in Low Network Area and low RxQuality when
Rxlev
7/27/2019 generation Field Testing
15/16
7/27/2019 generation Field Testing
16/16
Application Testing
Application testing and signing programs perform testing at the application-acceptance-
testing level to ensure that your application meets the models requirements. This means
that Symbian Signed and Java Verified Program criteria documents must be used when
considering application requirements.