Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Teemu Rovanperä
RADIO TIMING MEASUREMENT AUTOMATION
RADIO TIMING MEASUREMENT AUTOMATION
Teemu Rovanperä Bachelor’s Thesis Fall 2021 Information Technology Oulu University of Applied Sciences
3
ABSTRACT
Oulu University of Applied Sciences Degree Programme in information Technology, Option of Software Development
Author: Teemu Rovanperä
Title of thesis: Timing Measurement Automation
Supervisor: Kari Jyrkkä
Term and year when the thesis was submitted: November, 2021
Number of pages: 29
The aim of this thesis work was to automate the Common Public Radio Interface
timing measurements for radio products, reduce the need to do this type of testing
manually and to remove a human error from the equation. Previously the testing
had been done manually with the help of a Python-based tool that takes the
measurements and provides the results. The Radio unit and most of the
measurement device configurations had to be done manually.
The automation was to be done with Robot Framework, using already existing
libraries, and creating new ones if needed. The tool to be automated would also
need some modifications to be robot compatible. The aim was to create a fully
automated testing process for the tester but to also add this automation to be a
part of the current CI framework.
As a result, the testing process was fully automated utilizing Robot Framework.
Adding more radio units to the test environment and further development of the
automation continues after the thesis.
Keywords: 5G, Robot Framework, Python, Automation
4
CONTENTS
ABSTRACT 3
CONTENTS 4
VOCABULARY 5
1 INTRODUCTION 6
2 COMMON PUBLIC RADIO INTERFACE 7
2.1 Common Public Radio Interface 8
2.2 Enhanced Common Public Radio Interface 8
2.3 Timing 9
3 CURRENT STATE AND FUTURE OF THE ECPRI TIMING MEASUREMENT
10
3.1 Current State of The Timing Measurement 10
3.2 Test Setup 12
3.3 Requirements for The Automation 14
3.4 How 14
4 ROBOTIZATION 16
4.1 Making Current Code Robot Compatible 16
4.2 Creating a Robot Keyword File 18
4.3 Packaging 19
5 AUTOMATION & ROBOT STRUCTURE 20
5.1 Jenkins 20
5.2 Jenkins Automation Structure 20
5.2.1 Trigger 21
5.2.2 Test Suite Pipeline 22
5.2.3 Testing Pipeline 24
5.3 Robot 26
6 MODULE TESTING 28
7 CONCLUSION 29
REFERENCES 30
5
VOCABULARY
4G .................................................................................................. 4th Generation
5G .................................................................................................. 5th Generation
BB ....................................................................................................... Base Band
BBU .............................................................................................. Base Band Unit
BW ....................................................................................................... Bandwidth
CC ....................................................................................... Carrier Configuration
CPRI ................................................................... Common Public Radio Interface
DL ........................................................................................................... Downlink
eCPRI ................................................ Enhanced Common Public Radio Interface
NR ....................................................................................................... New Radio
RE ............................................................................................. Radio Equipment
REC .............................................................................. Radio Equipment Control
RFSW ........................................................................................... Radio Software
RRU ....................................................................................... Remote Radio Unit
RU ........................................................................................................ Radio Unit
SSU ...................................................................................... Signal switching unit
UL ............................................................................................................... Uplink
6
1 INTRODUCTION
Test automation is the practice of running automatically tests that would be
performed otherwise manually. Software testing is an essential part of software
development, but when done manually, software testing can be a slow process
and there is always a possibility of human error. The major aim of automating
software testing is to reduce the time spent on testing to free resources from the
basic testing tasks.
Continuous Integration is a practice in software development where the members
of the team continuously merge commits to a shared trunk. Each of those
commits are verified by an automated qualitative testing to find faults in the
software as quickly as possible [1].
Robot Framework is a generic cross-platform open-source automation framework
developed originally by Nokia Networks. Being open-source, it has a growing
ecosystem of tools and libraries. Robot Framework’s code is intended to be
readable even to the untrained eye so someone with no programming expertise
can use it effectively. Robot Framework’s language is tabular which makes it easy
to read. Tasks can be created with simple keywords and phrases. Keywords and
libraries can be built-in or external. Also, custom keywords and libraries created
with Python or Java can be used. Robot Framework is a widely used tool for test
automation and test-driven development for various applications. It is also used
Robotic Process Automation, a technology that is used to create, deploy, and
manage software robot emulating human interactions with digital systems [2].
In this thesis the main objective was to create a fully automated testing process
for radio unit air interface timing utilizing Robot Framework to make the
automated process compatible with Nokia’s current continuous integration
framework.
7
2 COMMON PUBLIC RADIO INTERFACE
CPRI’s (Common Public Radio Interface) aim was to remove copper and coaxial
cables from between the RRU (Remote Radio Unit) and the BBU (Base Band
Unit) and replace them with an optical fiber. The RRU is located on the top of the
tower and the BBU is usually located at the base of the tower. The RRU receives
analog signal from mobile device and converts it to a digital form according to the
CPRI specification and sends it along the fiber to the BBU, vice versa to the other
data direction. (Figure 1.)
FIGURE 1. Radio base station
8
2.1 Common Public Radio Interface
The CPRI specification is defined in cooperation with such vendors as Nokia,
Ericsson, Huawei, and NEC [3]. The CPRI interface defines the key internal
interface specification between the RE (Radio Equipment Control) and the RE
(Radio Equipment) of radio base stations [4]. In other words, CPRI is the
fronthaul data transfer protocol that is used between RE and REC [5]. (Figure 2.)
FIGURE 2. CPRI
2.2 Enhanced Common Public Radio Interface
With the introduction of 5G and ever-growing data demands, a new fronthaul
protocol was needed to replace CPRI. To meet that demand eCPRI (Enhanced
Common Public Radio Interface) was introduced [6]. It provides a flexible packet
based fronthaul network. eCPRI can monitor, analyze, manage, and optimize the
data flow between the eRE and eREC. It provides an increased efficiency,
capability, and lower latencies in the network. eCPRI reduces the needed
bandwidth up to ten times when compared to CPRI [7]. The bandwidth is scalable
according to traffic to ensure a reliable and sustainable operation.
9
2.3 Timing
Like In all communication technology, 4G (4th Generation) and 5G (5th
Generation), there are certain specifications that manufactures need to meet for
their product to work correctly. In eCPRI the network elements require accurate
timing and synchronization to ensure that the data going over these devices is
placed on the RF air interface at an exact time. If the timing is not correct, the
throughput at the user end cannot be maintained when switching between
different radio units.
This thesis concentrates on DL (downlink) and UL (uplink) air interface timing
measurements. The air interface is the frequency portion between the base
station and the mobile device. The DL direction measurement measures when
the radio starts sending the data over the air interface relative to the RU’s slave
clock. The purpose of the measurement is to verify that the downlink data is sent
at the correct time. The uplink direction the measurement measures if the radio
has the correct timing for the incoming data over the air interface relative to the
RU’s slave clock. The purpose of the measurement is to verify that the RU
processes the UL data at the correct time.
10
3 CURRENT STATE AND FUTURE OF THE ECPRI TIMING
MEASUREMENT
3.1 Current State of The Timing Measurement
Currently, to perform eCPRI air interface timing measurements and setting up the
RU and the BBU emulator is done manually. The measurements are also
performed manually with the aid of the software. The software automates
measurement device configuration, takes the measurements, and calculates the
results taking every variable into account. Measurements in the DL direction are
in this scope fully automatic, requiring no user input during the measurement
process. However, during the measurements in the UL direction, the software
requires the user to take a capture of the UL data and place it into a specific folder
so that it can complete the measurement. When changing from DL to UL or vice
versa, or to a different antenna, the user is required to change antenna routing
from the SSU (Signal Switching Unit). In the figure below (Figure 3.), manual
steps are marked with an asterisk. In the automation these manual steps will be
automatized.
11
FIGURE 3. Current state diagram
12
3.2 Test Setup
The testing setup consist of a RU, a BB emulator, an SSU, an Oscilloscope, a
Signal generator, an RF circulator, and a test PC (Figure 4.). The PC is used to
control the devices in the test environment over Ethernet and also to run the
automation codes. RU and BB emulator are connected with an optical fiber.
RU’s every antenna is connected to the SSU with RF cables. With the SSU the
antenna to be measured can be routed to the wanted output without touching
the RF cables. The signal generator is connected to the SSU’s output 1 through
an RF circulator with RF cables. Using a circulator prevents the power from
RU’s antenna going to the signal generator. The oscilloscope is connected to
the SSU’s output 1 through the RF circulator and directly to the output 2. When
measuring DL timing, the output 2 is used and when measuring UL timing, the
output 1 is used. The Basic idea of the measurement in both data directions is
to measure at what time the data arrives at the oscilloscope in reference to the
time it should have arrived.
13
FIGURE 4. Test setup
14
3.3 Requirements for The Automation
Requirements for the 1st stage of automation are to use already existing Robot
Framework libraries to configure the RU and BBU emulator and, also to control
the SSU. The program used to perform the measurements needs to be modified
to be able to run it with Robot Framework. Keywords need to be created. They
will be used to run the program. The new Robot Framework compatible version
of the measurement software needs to be packaged as a Python module for easy
installation and useability. Jenkins must be used to trigger and run the
automation. Some sort of unit testing or at least testing the package installation
needs to be implemented and run when there is a commit or merge request in
Git.
3.4 How
The automation to be created, aims to create a fully automated testing process
requiring no user input to start the measurement, or to require any actions from
the user during the measurements. Automating the RU and BBU emulator
configuration and controlling the SSU would be done using existing automation
libraries. The measurement software needs to be fully automatic, needing no user
input during the measurements, i.e., taking the data capture of the UL data with
the BBU emulator needs to be automated. The software also needs modifications
to be able to run it with Robot Framework. Custom keywords need to be created.
They will be used to execute the program using Robot Framework.
15
At first, the test cases are simple, and they will be manually defined in a robot file.
In the future the test cases will be defined in a json file and the robot suite will
consist of loops that are utilized based on the JSON file provided. For example,
when there are multiple test cases to run for multi carrier configuration, i.e.,
different frequencies. It does not make sense to manually create a robot suite for
every possible test variation. It is more feasible to create one test suite that uses
a JSON file where the test cases and frequencies are defined. And based on it,
the test suite goes through all the frequencies in a loop and runs the test cases
for all the frequencies in the JSON file.
The automation will be run in Jenkins, so the trigger and the test case running
jobs will be created using pipelines. Module testing will also be done in Jenkins
pipeline job which will be triggered from GitLab using a webhook.
16
4 ROBOTIZATION
Robot Framework uses a keyword based tabular syntax where the test cases
are defined with keywords. Robot Framework has a lot of keyword libraries but
to utilize it for this specific purpose, custom keywords need to be created to run
custom Python programs. With Robot Framework, Python scripts should not be
executed using the script’s main function. If a script is run using main, it needs
to be modified to be run as a function or as a class method. Keywords needs to
be created to execute those functions or methods. The custom keyword file
needs to be created where those keywords are defined.
4.1 Making Current Code Robot Compatible
Since Robot Framework is a keyword driven framework, some changes were
needed to make the program run with Robot Framework.
The current program consists of multiple Python classes and “main.py”. The
timing measurement program was run using the “main.py” with the environment
and measurement settings and test cases needed to be defined in various
configuration files. The first step was to study the program and find out what
needed to be done to be able to run the program using Robot framework.
A new Python file and class were needed. They would be used to run the program
through methods instead of the previous “main.py”. The contents of the previous
main.py were modified to contain a class and only a method of that class. (Figure
5.) The program and all the test cases are run through a single method in the
class. That method was modified to accept the test case name as an argument
instead of the previous way of reading it from a file. This change was made so
that a keyword could be created for every test case.
17
FIGURE 5. Code Changes
18
4.2 Creating a Robot Keyword File
A robot keyword file was created to run the program with Robot framework. The
keywords were setup to create the Python class and run the test running method
with the wanted test case name as an argument and to return the result of the
measurement to a variable. (Figure 6.)
FIGURE 6. Keyword file
When forming the keyword file, the Python file must be imported as a Library in
the setting section. When creating the keywords, the keywords need to be named
according to their functions. For example, in the above picture (Picture 6.) the
keyword is formed by naming the keyword first, “Test dl air frame radio pps”.
Below the keyword name, the Python class and the method used are defined,
“Timing.Run”.
19
After the defining of the Python class and method, the test case name is written,
“dl_air_frame_radio_pps” to be supplied as a string argument running the “Run”
method. The method returns the measured delay value to the variable named
“delay_value”. Then the keyword is made to return the value to the robot suite
with the line “[return] ${delay_value}”.
4.3 Packaging
The new Robot Framework compatible version of the timing program was
packaged as a Python module which could be installed from a local copy with
Python’s PIP installation tool. When installed as a Python module, the keyword
file can be imported to the robot suite as a resource just like Python modules
are imported to Python scripts with the import command. This also ensures that
the program would always be installed correctly with all the Python libraries it
needs to work properly.
20
5 AUTOMATION & ROBOT STRUCTURE
5.1 Jenkins
Jenkins is an open-source automation server. It is generally used to automate
testing, building, and implementing CI (Continuous Integration). In this thesis it is
used to chain the automation components together to create an automation
framework [8]. Scripted pipeline jobs were chosen to run the robot scripts in
Jenkins. Scripted pipelines are written to a Jenkinsfile with the Groovy
programming language. The scripted pipeline contains the instructions for the job,
for example, running Python or robot scripts and other various tasks that are done
with those jobs.
5.2 Jenkins Automation Structure
In the automation framework created, Jenkins contains the trigger, test suite,
RFSW (Radio Software) and testing pipeline jobs. Jenkins takes care of triggering
the testing, managing the automation structure, and collecting artifacts from the
jobs. (Figure 7.). In an example case, the trigger is set to trigger the automation
for every hour. When the trigger is run, first it checks if the testing setup is free.
After that, the trigger checks the latest RFSW version available. If the setup is
free, the test suite is started with the latest RFSW version. The test suite launches
the RFSW update job and after the update has been done, the testing is run.
Lastly it collects results from the test pipeline’s run within the test suite. The test
suite’s main purpose is to manage and link together the jobs run from within it to
form an automation framework.
21
FIGURE 7. Automation structure
5.2.1 Trigger
Separate trigger pipelines were created to start the test suite pipelines since there
would be test suite pipelines for every radio unit tested. With the trigger, triggering
the testing can be set to specific times or time intervals. In this case the trigger
was to be set to run the automation during nighttime and weekends, when the
test environment was not in use.
Every time the trigger is run, it checks the latest software build available and if
the test environment is available. When the test environment is available, the
trigger starts the test suite and supplies it with information about the latest
software build available and what test cases to run.
22
5.2.2 Test Suite Pipeline
With the test suite pipeline, multiple different testing pipelines can be run with
different test cases without making the testing pipeline or the robot script itself too
long or complex. (Figure 7.) Test suite also keeps track of the latest RFSW build
that has passed the timing CI. The main reason for keeping track of the latest
working RFSW build is to quickly find the first RFSW build that has caused an
issue with the RU’s timing. The test suite runs a limited number of testing
pipelines and collects the results from the pipelines. Depending on the outcomes
of the pipelines, if all the tests are passed, it sets the tested RFSW as the latest
RFSW that has passed the timing testing, a.k.a. the “Latest passable”. If one or
more timing test cases fail in the test suite, the “Latest passable” is not updated.
The “Latest passable” means that the RFSW has passed the testing for a specific
configuration that is chosen for the CI. More testing would be needed to validate
that RFSW works with all the different configurations that the RU it is capable of.
Outcomes of the tests are also collected and set in the test suites job description.
Also, a hyperlink to the executed testing pipeline is added. (Figure 8.)
FIGURE 8. Test suite run with one 1xNR100 test case with passed results.
23
The trigger can be used to select what testing pipeline or pipelines are going to
be run with the test suite pipeline (Figure 9.). Also, the trigger can be used to
configure the test suite pipelines by passing down parameters from the trigger to
the test suite and from the test suite to the testing pipeline. For example, the
trigger can tell the test suite to test different test cases depending on the time of
day. For example, more extensive testing can be run during the night on the
RFSW build that the CI has marked as the latest working software.
FIGURE 9. Test suites
24
5.2.3 Testing Pipeline
Testing pipeline is where the timing test cases are run. (Figure 10.)
FIGURE 10. Content of the testing pipeline.
25
First, the pipeline parameters are defined. They are inherited from the test suite.
The pipeline accepts the following parameters: Node, CC, BW and Tech. Node
is the PC where the tests are run from. CC is the number of cell carriers. BW is
the carrier bandwidth and lastly, Tech is the mobile network technology, which in
this case is 5G NR. After the parameters are defined some more environment
variables are defined, for example, Git repository branches which to clone the Git
repositories.
After the parameters and environment variables are defined, repositories
containing the Python scripts and robot suites used in the testing are cloned from
Git.
Before launching the robot testing suite, the radio needs to be powered on. After
the radio is powered up, the robot testing suite is launched, radio and BBU
emulator are configured and then the tests are performed. After the
measurements are done, the results are visible in the pipelines job description.
(Figure 11.)
FIGURE 11. Measurement results.
Depending on the measurement results, passed or failed, the pipeline outcome
status is set as “success” or “failure”, which tells the testing suite if the
measurements have passed or failed in the testing pipeline.
26
5.3 Robot
Robot Framework portion of the automation consists of robot scripts and various
custom robot libraries that are used in these scripts. The first part of the robot
script is used to configure the radio unit, base band emulator and test
environment.
The second part of the robot script is where the robot compatible version of the
timing measurement program is used. In this robot script, the signal switching unit
is also controlled to select the desired antenna from the radio unit. (Figure 12.)
FIGURE 12. Example robot test case.
In the “*** Settings ***” tab the timing measurement program module is imported
as a resource. The SSU controller is imported as a Python script as it contains
only a class with methods which can be used as keywords by the Robot.
27
In the “*** Test Case ***”, first the IP address of the SSU is given. Next, the
antenna route is set for the downlink measurement. Then the timing
measurement is started with the keyword “Test dl air frame radio pps” which tests
radio’s downlink air frame timing. The keyword returns the result of the
measurement as nanoseconds, a csv result file is created with the Keyword
“Create Downlink Result Files”. The keyword accepts the RU name, result value,
measured antenna and frequency as input parameters. Then the uplink
measurements are run with the same principles.
28
6 MODULE TESTING
To guarantee that there is always a working version of the Robot framework
compatible timing program on the master branch of the Git repository, the
validity of the changes pushed or merged to master must be checked for errors.
The basic CI was setup using a GitLab webhook and Jenkins. The Jenkins
pipeline is triggered with a webhook from GitLab. (Figure 13.). At the time of
writing this thesis, only testing the installation of the timing program as a Python
module was implemented in the CI. In the future the plan is to add unit testing to
the CI to test the methods in the timing program.
FIGURE 13. Module testing flow
29
7 CONCLUSION
The aim of this thesis was to create a fully automated testing process for RU air
interface timing using Robot Framework to add the created automation to
Nokia’s existing continuous integration framework.
The automation created worked as planned, though there are still issues that
prevent taking the automation into use with the RU this automation was created
with. The main issue is configuring the RU and BB emulator correctly with the
automation libraries since the libraries were not developed directly to the RU
that was used in the development of the automation. Adding the support for the
RU was not in the scope of this thesis but development also continues on this
after the thesis. In the future, the plan is also to add to the test environment
more radios that are already supported by the automation libraries.
Most of the automation developed was done without the test setup shown in
figure 4. It slowed down the development when there was no possibility to test
the work done to the timing program. Later during the project, a test setup was
built for the purpose of developing the automation and it sped up the
development and issues with the code were debugged quickly.
30
REFERENCES
1. Fowler, Martin 2006. Continuous Integration. Date of retrieval: 1.9.2021
https://www.martinfowler.com/articles/continuousIntegration.html
2. Robot Framework 2021. Robot Framework. Date of retrieval: 18.8.2021.
https://www.robotframework.org
3. CPRI Info 2021. Common Public Radio Interface. Date of retrieval:
2.8.2021.
http://www.cpri.info/
4. Common Public Radio Interface Specification V7.0 2015. Common
Public Radio Interface (CPRI); Interface Specification.Date of retrieval:
2.8.2021.
http://www.cpri.info/downloads/CPRI_v_7_0_2015-10-09.pdf
5. Enhanced Common Public Radio Interface Specification V2.0 2019.
Common Public Radio Interface: eCPRI Interface Specification. Date of
retrieval: 2.8.2021.
http://www.cpri.info/downloads/eCPRI_v_2.0_2019_05_10c.pdf
6. RF Wireless world 2017. Difference between CPRI and eCPRI used in
5G. Date of retrieval: 3.8.2021.
https://www.rfwireless-world.com/Terminology/CPRI-vs-eCPRI.html
7. Carrier Tech 2020. eCPRI. Date of retrieval: 3.8.2021.
https://www.carritech.com/news/ecpri/
8. Jenkins 2021. Jenkins. Date of retrieval: 1.9.2021.
https://jenkins.io