Deliverable D2.1 Test Case Definition and Test Site ... · The large scale test area that has been selected is the EU 5G Large Scale Test Site in the border area between Luxembourg

Fifth Generation Cross-Border Control

Deliverable D2.1 Test Case Definition and Test Site

Description Part 1 Version: v1.0

2019-06-30

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825050. Any 5GCroCo results reflects only the authors’ view and the Commission is thereby not responsible for any use that may be made of the information it contains.

http://www.5g-ppp.eu

DISCLAIMER: This 5GCroCo D2.1 deliverable is not yet approved by the European Commission. The review process will take place within September/October 2019.

http://www.5g-ppp.eu/

Document: 5GCroCo/ D2.1 Version: v1.0 Date: 2019-06-30 Status: Final Dissemination Level: Public

2

Deliverable D2.1 Test Case Definition and Test Site

Description Part 1 Grant Agreement Number: 825050

Project Name: Fifth Generation Cross-Border Control

Project Acronym: 5GCroCo

Document Number: 5GCroCo/D2.1

Document Title: Test Case Definition and Test Site Description Part 1

Version: v1.0

Delivery Date: 2019-06-30

Editor(s): Eric Perraud (RSA), Kurt Eckert (BOSCH)

Authors: Andreas Schimpe (TUM), Israel Gonzalez Vazquez, Andreas Pfadler (Volkswagen AG), Karl Theo Floess (BOSCH), Johan Loefhede (Volvo), Eric Perraud, Stefania Sesia (RSA), Laurent Dizambourg (PSA), Daniel Rau, Fabien Coulet (HTW), Francisco Vázquez Gallego, Roshan Sedar, Juan Luis de la Cruz (CTTC), Miguel Catalán Cid (I2CAT), Maciej Muehleisen (ERI).

Keywords: Test case, KPI, ToD, HD mapping, ACCA

Status: Final

Dissemination level: Public


3

Version History Version Date of release Description of changes Last editor

v0.1 19-05-2019 First draft Eric Perraud V0.5 25-05-2019 Version including all inputs and first

internal review Kurt Eckert

V0.6 02-06-2019 Version with new figures for UC3 & UC2

Eric Perraud

V0.7 07-06-2019 Version for review by PMT Eric Perraud V1.0 26-06-2019 Version 1 Eric Perraud V1.0 30-06-2019 Final review by Project Manager Jesus Alonso-Zarate


4

Abstract 5GCroCo intends to specify, develop, trial and demonstrate future automotive use cases that require seamless availability of 5G telecommunication features at operator and country borders. Three use cases have been identified to be representative for the automated driving application domain and which pose high demands to the telecommunication network side. These three use cases are: 1) Tele-operated Driving (ToD), 2) High Definition (HD) map generation and distribution for autonomous driving (HD mapping), and 3) Anticipated Cooperative Collision Avoidance (ACCA). In this document, these three use cases are specified in detail together with the requirements that are imposed by them. Each use case is split in so-called user stories, which stand for special variants of each use case respectively. This detailed specification is serving as the base input for the work packages Architecture and Implementation, which are caring for the overall system architecture and the implementation of it, respectively.

For each user story, a set of key performance indicators (KPIs) are identified which represent the minimum needed performance of the underlying system consisting of vehicles, communication network and backend.

In addition, this document specifies the test cases that are identified for each user story of each use case. The test cases are defined to demonstrate the suitability of the new features of 5G core network for automotive use cases and to prove that critical system KPIs are met. For each user story, the test cases are mapped to one or several test sites. The test sites are selected on the one hand in the Metz-Merzig-Luxembourg large scale trial area and on the other hand amongst a set of small scale test areas. The selection has considered the requirements arising from the function itself (suitable road setup and safety considerations) but also the capability for generating cross-border, and cross-MNO scenarios. This document also defines the scope of each test case, the expected functional results and the monitored KPIs. A breakdown of the monitored KPIs into sub-KPIs is proposed when it is appropriate to root-cause the KPIs that are out of the expected range. Finally, constraints of the cellular network in each of the test sites are identified.


5

Contents 1 Introduction ......................................................................................................................... 8

1.1 Objective of the Document.......................................................................................... 8

1.2 Structure of the Document .......................................................................................... 8

2 Test Area Description.......................................................................................................... 9

2.1 Large Scale Test Area Metz-Merzig-Luxembourg ....................................................... 9

2.1.1 Areas around Schengen ........................................................................................11

2.1.2 Areas around Saarbrucken ....................................................................................12

2.2 Small scale Test Sites ...............................................................................................13

2.2.1 Small scale Test Site Germany Motorway A9 5G-ConnectedMobility ....................13

2.2.2 Small scale Test Site Barcelona ............................................................................13

2.2.3 Small scale Test Site UTAC-Montlhéry ..................................................................14

2.2.4 Small scale Test Site Munich .................................................................................15

2.2.5 Test Site AstaZero .................................................................................................16

3 Test Case Descriptions ......................................................................................................17

3.1 Use Case Tele-operated Driving ................................................................................17

3.1.1 Mapping of User Stories per Test Site ...................................................................17

3.1.2 Monitored KPIs for ToD .........................................................................................18

3.1.3 Test Case Locations in the Cross-Border Large Scale Test Site ............................20

3.1.4 Test Case Locations in Munich City Center ...........................................................27

3.1.5 User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity) ......28

3.1.6 User Story 2 – Remotely Controlled Trajectory-based Driving (indirect control, low velocity) .............................................................................................................................32

3.1.7 User Story 3 – Remotely Controlled Trajectory-based Driving on Rural Crossed Country Road (indirect control, high velocity) .....................................................................35

3.1.8 User Story 4 – Slim Uplink for ToD (indirect control, low velocity) ..........................38

3.2 Use Case HD mapping ..............................................................................................42


3.2.2 User Story 1 – Streaming the Map to the Car ........................................................42

3.2.3 User Story 2 – Streaming Sensor Data to the Cloud ..............................................44

3.2.4 User Story 3 – Closing the Loop ............................................................................45


6

3.2.5 User Story 4 – Upload Sensor Data and Download Map Updates in Non-real Time 47

3.3 Use Case ACCA ........................................................................................................49


3.3.2 ACCA System KPI Definition .................................................................................50

3.3.3 User Story 1 – Ego-detected Event ........................................................................54

3.3.4 User Story 2 – Ego and Remotely Detected Event .................................................59

3.3.5 User Story 3 – Traffic Jam Detection .....................................................................62

3.3.6 User Story 4 – Cross-border Scenario ...................................................................65

4 Summary ...........................................................................................................................74


7

List of Abbreviations and Acronyms 4G Fourth-generation 5G Fifth-generation 5GCroCo 5G Cross Border Communication ACCA Anticipated Cooperative Collision

Avoidance AD Automated Driving ADAS Advanced Driver-Assistance Systems ARQ Automatic Repeat Request CDF Cumulative Distribution Function CPM Cooperative Perception Message DENM Decentralized Environmental

Notification Messages EPC Evolved Packet Core EU European Union EV Ego Vehicle GPS Global Positioning System HD High Definition HMI Human Machine Interface HV Hazardous Vehicle HARQ Hybrid Automatic Repeat Request IMU Inertial Measurement Unit KPI Key Performance Indicator LTE Long Term Evolution MEC Mobile Edge Computing MNO Mobile Network Operator OBU On Board Unit PDF Probability Distribution Function PMT Project Management Team QoS Quality of Service RAN Radio Access Network ROI Region of Interest SLR Service Level Requirement TC Test Case ToD Tele-operated Driving UC Use Case US User Story V2X Vehicle to Everything VCoC Vehicle Control Center

vEPC Virtual Evolved Packet Core


8

1 Introduction 1.1 Objective of the Document The purpose of this document is tightly coupled with D4.11. It defines what is being tested and where it is being tested. It also allows to identify the requirements of each or the test sites. It will be used as the reference document for the trial execution of this program.

1.2 Structure of the Document The document firstly describes the test areas.

The rest of the document is organized based on the three use cases (Tele-operated Driving (ToD), High Definition (HD) mapping, and Anticipated Cooperative Collision Avoidance (ACCA)) that are defined in the project. For each use case, it describes:

• the different user stories,

• the conditions of the test cases which are associated to each user story,

• the expected functional outcomes per user story,

• the monitored Key Performance Indicators (KPIs) per user story,

• and the locations where the test cases shall be done.

1 5GCroCo D4.1: “Report Validating Development and Integration in Network, Vehicle and Backend on Small and Large Scale Trial Sites”, to be released in June 2020.


9

2 Test Area Description In this section of the document, all test areas that are considered for performing tests and trials are described. The test areas include the large scale trial area in Metz-Merzig-Luxembourg as well as each of the small scale test sites. The large scale area is meant to be used for the final trials and tests while the small scale test sites are mainly used to perform non-cross-border test cases. The small scale test sites are also used to assess some features or performances that can’t be tested at the large scale area (e.g. special 5G features of the network or special test setups that cannot easily be transferred to the large scale area).

The description contains information about the detailed geographical location of each site together with information about special conditions for road operation or road locking. In addition, there is a brief information given about the cellular network situation in each of the sites (operator, network setup). This information will be further detailed in further project deliverables.

2.1 Large Scale Test Area Metz-Merzig-Luxembourg

The large scale test area that has been selected is the EU 5G Large Scale Test Site in the border area between Luxembourg (LUX), France (FR) and Germany (GER) officially called “Digital Testfield Germany-France-Luxembourg” and later in this document referred to as “Metz-Merzig-Luxembourg”. Three telecom network operators representing partners of 5GCroCo are available in this area namely POST in Luxembourg, Orange in France, and DTAG in Germany. The following provides an overview of the geographical location (see Figure 2-1).


10

Figure 2-1: Geographical Location of Large scale Trial Area

During a workshop performed around the large scale Test Site, several test areas have been identified considering functional test setup but also information about network availability (as far as it is known at the date of writing this document), which proved to be fitting for performing the detailed tests. Those test areas are shown in the following section.

There happened already some discussions with local road authorities on possibilities for closing some roads temporarily.


11

2.1.1 Areas around Schengen

Figure 2-2: Test Locations around Schengen2

There are five potential test areas in this location (see Figure 2-2):

1) One is the highway A13 in Luxembourg, which provides on the one hand a good base for performing tests which need higher speeds and which need less restriction of the area and less special road conditions like sharp curves or special topographical setup. On the other hand, a tunnel might work as an obstacle and an emergency parking lot can be used for simulating accidents.

2) Next to the highway A13 in Luxemburg, in direction from Luxemburg towards Germany there is a “private” area (fenced plot of land) that has the qualities to provide a base to perform tests, which are conformed of slow speeds and maneuvering. The area can also provide access to the highway via a gate, which leads to the Markusbierg tunnels as well as offers access to a one lane rural road parallel to the highway.

3) Another potential area is on the German side of the border at the A8, where the rest area “Moseltal” is a suitable place to e.g. simulate a broken-down vehicle, as it will probably be possible to tune the networks in a way that there will be a handover (POST Luxembourg – DTAG) in this area.

2 Bundesamt für Kartographie und Geodäsie, Germany 2019, Sources: http://sg.geodatenzentrum.de/web_public/Datenquellen_TopPlus_Open.pdf.

http://sg.geodatenzentrum.de/web_public/Datenquellen_TopPlus_Open.pdf


12

4) In addition, there are two areas on rural roads which are equally fitting with respect to network conditions but provide more possibilities for locking or for generation of special traffic situation setup. One is in Luxembourg in the north of Schengen and another one is in parallel on the German side.

2.1.2 Areas around Saarbrucken

Figure 2-3: Test Locations around Saarbrucken3

There are another three potential test areas in this location (see Figure 2-3):

1) One is at the highway A20/A6 at the French-German border close to the rest area “Goldene Bremm”. In this area, there also is a bigger parking lot, which is completely locked and might be used for special setups like ToD maneuvering.

2) A second area is a rural road close to the highway in southwest direction. This area provides some steep curves and special conditions e.g. for the ACCA test cases.

3) Finally, there is a certain area in an industrial zone to the north of the rest area, which already is used for tests by the companies located in the industrial zone. This one has the advantage of a simple road-path combined with little traffic.




13

2.2 Small scale Test Sites 2.2.1 Small scale Test Site Germany Motorway A9 5G-

ConnectedMobility

This test area is part of the “Digital Motorway Test Bed” defined and set up by the German Federal Ministry of Transport and Digital Infrastructure. The test site consists of six cells, each with two sectors. They are deployed along approximately 30 km of German motorway A9 reaching from Nuremberg-Feucht to Greding. Furthermore, the rural road (Bundeststrasse) from Allersberg to Wendelstein is covered, allowing for trials outside of the motorway. All roads are public and according regulations apply. The motorway includes segments without speed limit.

2.2.2 Small scale Test Site Barcelona The test site is located in the city of Barcelona, in the 22@ district, and it is integrated within the 5GBarcelona end-to-end infrastructure. Actually, it has one cabinet and three small cells deployed along the “Ciutat de Granada” public street. In this test site, it is planned to emulate a cross-border scenario, with two different virtual MNOs being deployed at this place, having their own virtual core networks and the small cells.

Figure 2-5: Test Site Germany A9 Motorway Figure 2-4: Test Site Germany A9 Rural Road

The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.


14

Figure 2-6: Test Site Barcelona

2.2.3 Small scale Test Site UTAC-Montlhéry The test site at Montlhéry is a closed test track in France, south to Paris. It provides test area usage as a service and is used by various companies to perform tests of connected and automated vehicles. The site consists of 12 km testing tracks and associated facilities. The track is composed of three mobility environments (i.e. city, country road, and highway), to enable the support of various vehicular use cases (Figure 2-7).

1

2

3

1 Small Cell

Cabinet


15

Figure 2-7: Test Site UTAC-Montlhéry

2.2.4 Small scale Test Site Munich This site is located in the north west of Munich close to the facilities of Huawei at Riesstraße. It consists of a parking lot at a shopping area and a private parking lot at the Huawei premises. Huawei plans to set up a 5G New Radio enabled test network in this area, which can be used to test in a non-public communication network.

Figure 2-8: Test Site Riesstraße Munich


16

2.2.5 Test Site AstaZero AstaZero is a full-scale independent test environment for future road safety, outside Gothenburg, Sweden. AstaZero has testing capabilities within the following vehicle environments, super multi-lane, high speed area, rural road, multi-lane road, and city area. In addition, two communication networks are available: 1) a cellular test network that controls different objects during the tests, e.g., a balloon vehicle, a moose, and a pedestrian. 2) A second one with high flexibility, e.g., coverage, network load, and handover setups, including emulation of country borders with two separate MNOs. This network can be used to test different vehicle functions that are using V2X communication.

Figure 2-9: AstaZero Test Area


17

3 Test Case Descriptions In this section, the test cases that are planned to be implemented and performed during the project are described in detail. The section is organized on a per use case base. For each use case, there is a description of the use case itself followed by an overview on the mapping of each user story (sub-use case concentrating on certain specific features of the use case) to the different test areas as described in Section 2. For each user story, finally there is a description added about the details of the tests that are planned. This contains the test description itself, details about the KPIs that a certain test case is supposed to evaluate, the test areas that are considered, requirements for the telecom setup and a table summarizing the complete test case.

3.1 Use Case Tele-operated Driving In this section, detailed test case descriptions for each User Story will be given.

3.1.1 Mapping of User Stories per Test Site Table 3.1 shows how test cases of each User Story are mapped to the test sites.

Table 3.1: ToD User Story Mapping per Site

US 1 US 2 US 3 US 4

Munich City Center (Small scale)

X (X) (X)

Motorway A9 5G-ConnectedMobility (Large scale)

(X) (X) (X) (X)

Large Scale Test Site (Large scale)

LUX – GER X

GER – FR X X X

Munich City Center (small scale) will be the test site for User Story 1. If the development of the indirect control concept and the slim uplink is ready, User Stories 2 and 4 can also be tested on this test site. Metz-Merzig-Luxemburg (large scale) is planned to be the test site for all four user stories. User Stories 1, 2 and 4 will be tested in Germany at the border between Germany and France. User Story 3 will be tested in a cross-border scenario along the border between Luxembourg and Germany. Discussions defining, if to execute tests on the test site Motorway A9 5G-ConnectedMobility (large scale) and how to execute them, were not started yet and are planned to be finalized in deliverable D2.2. This test site may be considered for emulated cross-border testing before doing this in the corridors, Tele-operated Driving at higher speeds, and to assure interoperability at the different test sites in general. Overall, it is assumed that


18

interoperability and regulatory standardization is required at the application level for all User Stories and test sites.

The remainder of this section is structured as follows. Section 3.1.2 describes the KPIs that are monitored during the execution of the test cases. The potential test sites in the Cross-Border Large Scale Test Site and Munich City Center are described in the Sections 3.1.3 and 3.1.4. The detailed test case descriptions for each User Story are then given in Sections 3.1.5 to 3.1.8.

3.1.2 Monitored KPIs for ToD In this section, a more detailed explanation of the KPIs to be monitored during the tests is provided. The KPIs that are monitored during the tests may vary and are listed in the respective sections for each of the User Stories.

Range

ToD is applied as a fallback procedure to overcome an unexpected road blockage. The distance required to overcome this road blockage is what will be referred to as the Maneuvering Range.

The tele-operator is controlling the vehicle from the vehicle control center (VCoC), which is located at some distance from the vehicle, called the Service Range.

Information Exchanged and Estimated Payload

For ToD, information is exchanged over the mobile network from the vehicle to the vehicle control center and vice-versa. For instance, this data are sensor data, e.g. camera images, and state feedback from the vehicle, e.g. current speed or steering wheel angle. In the other direction, control commands, e.g. the desired speed or steering wheel angle for the direct control concept and the desired trajectory for the indirect control concept, need to be transmitted from the VCoC to the vehicle.

Service Level Reliability

Service Level Reliability4 (SLR) is of great importance for safe ToD. For critical information, e.g. keyframes in the encoded video feeds, there should be minimal to no loss during transmission.

During the test cases, the uplink SLR at service level for the transmission of the video feeds will be monitored by the ratio between the frame rates that are sent from the vehicle and the frame rates received at the VCoC. Coupled with the desired functional result that the tele-operator has sufficient situational awareness, there will be an objective to measure if the video stream in the VCoC is of sufficient quality.

For downlink transmission reliability, the SLR at service level can be understood as a packet error rate for the loss of control command data packets.

4 Reliability in a broad sense as e.g. described under “dependability” [IEC60050] and “functional safety” [IEC61508, ISO26262].


19

Service Level Latency

A low service level latency is critical for ToD. For this Use Case, the latency is referred to as the sum of the times required to process the data in the vehicle, transmit it from the vehicle to the VCoC and process it in the VCoC.

Maximum Age of Information

The maximum age of information, displayed in the VCoC, needs to facilitate an appropriate reaction time of the tele-operator to the current situation. Extending the KPI of the service level latency, the update rates of the sensors of the vehicle and state feedback need to be considered for the maximum age of the information. For instance, in the worst case, an update rate of 20 Hz can add another 50ms.

Deviation of Desired Trajectory

For the indirect control concept (user story 2), it is necessary to monitor the performance of the vehicle-internal trajectory tracking controller. On the one hand, this performance is quantified by the lateral deviation, which is the shortest distance from the position of the vehicle to the desired trajectory. On the other hand, another measure is the orientation deviation, which is the difference between the desired and the actual orientation of the vehicle.

Positioning Accuracy

The positioning accuracy specifies how precise the global position, obtained via a GPS/IMU or odometry is measured.


20

3.1.3 Test Case Locations in the Cross-Border Large Scale Test Site

Figure 3-1: Location of the French-German-Border (Saarbrücken) in the Cross-Border

Large scale Test Site5

Figure 3-1 shows the precise localization of the border crossing between Saarbrücken in Germany on the A6 motorway and Forbach/Stiring Wendel in France on the A320 motorway, the lilac line represents the political border.




21

Figure 3-2: Location of the Rest Area Goldene Bremm at the French-German-Border in the Cross-Border-Large Scale Test Site6

Figure 3-2 shows the rest area Goldene Bremm on the A6 exit 2 (Saarbrücken-Goldene Bremm) in Germany at the border to Forbach, France. A parking lot, marked in red, which is not open to public, may be used. The GPS coordinates are (49.207119, 6.96383).

6 IGN-FEDER-Préfecture de la region Grand-Est, France, 2019.


22

Figure 3-3: Location of the Industrie Gebiet Süd, Am Süd Ring/Untertürkheimerstrasse at

the French-German-Border in the Cross-Border-Large Scale Test Site7

Figure 3-3 shows the industrial area (Industrie Gebiet Süd), Am Süd Ring/Untertürkheimerstrasse in Saarbrücken, Germany, near to the French Border along the A6 Motorway. The GPS coordinates are (49.217751, 6.977434).

7 GeoPortal DOP20, Saarland, Germany, 2019.


23

Figure 3-4: Location of the Luxembourgish-German Border (Schengen/Merzig) in the

Cross-Border-Large Scale Test Site8

Figure 3-4 shows the location of the border crossing between Perl-Merzig in Germany on the A8 motorway and Schengen in Luxembourg on the A13 motorway, the purple line represents the border. The small highlighted area (A) shows the location of a private plot of land located next to the A13 motorway in direction to Germany, just before the entrance to the Markusbierg tunnels. The large highlighted area (B) are potential roads for the tests along the river Mosel.


A

B



24

Figure 3-5: Location of the private Plot of Land next to A13 Motorway9

Figure 3-5 shows the private plot of land (A). The GPS coordinates are 49.483271, 6.336812. The approximate dimensions are 87.12 meters X 50.29 meters.

9 Geoportal, Administration of the Cadastre and Topography, Luxembourg, 2019


25

Figure 3-6: Location of Rural roads along the Luxembourgish-German Border (Besh-Nennig) (Germany)10

Figure 3-6 shows the location of two rural roads along the border, the river Mosel flowing in the middle of the picture is the border. From Schengen to Remich on the Luxembourgish side there is the N10 (left side circle, GPS: 49.508157, 6.364024), from Besch to Nennig on the German side there is the B419 (right side circle; GPS: 49.508812, 6.372435)

10 Geoportal, Administration of the Cadastre and Topography, Luxembourg, 2019.


26

Figure 3-7: Location of Rural roads along the Luxembourgish-German Border (Perl-Besch) (Germany)11

Figure 3-7 shows the location of two rural roads along the border, the river Mosel flowing in the middle of the picture is the border. From Schengen to Remich on the Luxembourgish side there is the N10 (left side circle, GPS: 49.484576, 6.366835), from Perl to Besch on the German side there is the B419 (right side circle, GPS: 49.486695, 6.372499)



27

3.1.4 Test Case Locations in Munich City Center

Figure 3-8: Location of Parking Area and Street in Munich Riessstraße close to Huawei Premises

Figure 3-8 shows the location of the test area in Munich. It contains streets around two parking lots (yellow and red rectangles) and a part of Riesstraße shown as blue line in the lower part of the picture.


28

3.1.5 User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity)

This section contains the test case specifications for User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity). The goal of the test cases for this User Story is to demonstrate how Tele-operated Driving and 5G communication technology can be used to overcome scenarios in which an automated driving vehicle does not know how to handle an unexpected road blockage.

A/ Test case definition The vehicle is assumed to come to a safe stop on its own. Then, the tele-operator, located in the VCoC, gets informed and takes over the control with the direct control concept. He maneuvers the vehicle past the road blockage and brings it to a stop again. After this, the AD function takes over control again and the drive continues. The schematics and time schedule of the first user story are shown in Figure 3-9 and Figure 3-10.

A road blockage in these test cases is referred to as obstacles on the road, which constrain the safe driving Large Scale Test Site for the AD function, such that the vehicle cannot pass the obstacles. The obstacles could be two other vehicles that collided, a fallen tree or other larger objects. The reason the AD function cannot overcome the situation may be that traffic regulations need to be violated, e.g. driving on the bicycle path beside the road, or the AD function’s system limits need to be exceeded, e.g. driving on the soft shoulder.

The velocities during the remotely controlled maneuvering using the direct control concepts can be up to 15 km/h. Reverse maneuvers can also be in the scope of the test cases. In addition to video feeds, Lidar point clouds and audio signals may be transmitted to the VCoC and displayed to the tele-operator.

Figure 3-9: Use Case 1 User Story 1 Schematics


29

Figure 3-10: Use Case 1, User Story 1 Time Schedule

B/ Functional results

For ToD, it is of great importance that the tele-operator in the VCoC is aware of the situation. Situational awareness may be poor if the video images and other sensor data, as perceived by the operator, do not reproduce enough the vehicle surroundings. Furthermore, situational awareness can suffer if the video images, shown to the operator, do not have sufficient quality. This implies that the situational awareness is influenced significantly by the amount of data shown to the operator. Therefore, a functional result for the test case for User Story 1 is that the 5G network provides a sufficient bandwidth such that enough sensor data can be transmitted from the vehicle to the VCoC.

Furthermore, the tele-operator needs to be confident that he or she is in full control of the vehicle and it reliably executes what it is commanded. Network latency can have a great effect on this. On the one hand, from the vehicle’s perspective, the tele-operator has an increased reaction time. On the other, from the operator’s perspective, the vehicle also executes its commands with a delay. This yields the functional results that the 5G network’s latency is small enough, such that it does not negatively affect the tele-operation with the direct control concept used in User Story 1.

C/ Monitored KPIs

Range

In the test cases for User Story 1, the Maneuvering Range will be less than 100 m. The Service Range will be less than one kilometer. However, for reasonable application of ToD, the desired functional results should still be fulfilled at a range of several kilometers.

Information Exchanged and Estimated Payload

During the execution of the test case for US 1, the estimated offered data traffic for the network can be up to 30 Mbit/s in uplink. For safe ToD, it is important, that the network offers enough capacity to transmit all data throughout the whole course of the maneuver.


30

Service Level Reliability

The transmission reliability of the video feeds will be monitored as described in the previous section.

For the direct control concept, the control signals desired speed, steering wheel angle and gear position are sent from the VCoC to the vehicle. During the execution of the test case for User Story 1, it will be monitored, how reliable these commands are executed and tracked.

Service Level Latency

For the US 1 test case, the objective is to achieve a latency of 100 ms. This includes a maximum of 40 ms for the IP to IP transmission delay of the network with the transmission delay in the core network and over the 4G/5G radio layers. Furthermore, the data should be processed in both, the vehicle and the VCoC, in less than 30 ms.

Maximum Age of Information

During the tests for User Story 1, the maximum age of information for all data received from the vehicle and its sensors should be 140 ms. This is derived from the requirement of having a video feed with at least 25 frames per second.

D/ Test case Locations Small scale tests for User Story 1 will take place on the Munich City Center test site as described in the Sections 2.2.4 and 3.1.4. The test site is shown in Figure 3-8.

In the Sections 2.1 and 3.1.3, several potential test locations for large scale tests were identified. For User Story 1, the test area will probably be at the border between Germany and France around Saarbrücken. The potential test sites are shown in Figure 3-2 and Figure 3-3. The private plot of land at the border between Germany and Luxembourg, shown in Figure 3-5, may also be considered. It is part of ongoing work to decide on which of these sites will be selected.

E/ Telco requirements For Tele-operated Driving, it is required that there is a seamless network coverage along the route, where the vehicle is driving. Thus, the network shall provide means to predict the QoS changes and anticipate inter-MNO handovers.


31

F/ Summary Table 3.2: Test Case of ToD User Story 1

Test Property Description

Scope • Stopping of automated driving vehicle in front of unexpected blockage on desired route of vehicle

• Overcoming obstacle through Tele-operated Driving with direct control concept

• Continuation of journey with automated driving function

Test Goal Measurement of following KPIs:

• Range

• Information Exchanged and Estimated Payload

• Service Level Reliability

• Service Level Latency

• Maximum Age of Information

Test Type System and performance test

Test Setup • Blockage of road / parking lot by one or more obstacles

• Ego vehicle with Automated and ToD functions

Success Criteria • Overcoming of obstacle(s)

• Desired functional results achieved

• Vehicle control center

Test Deliverables Statistics of measured KPIs and associated breakdown

Test Dependencies None


32

3.1.6 User Story 2 – Remotely Controlled Trajectory-based Driving (indirect control, low velocity)

This section describes the test case specifications for User Story 2 – Remotely Controlled Trajectory-based Driving (indirect control, low velocity). Similar to User Story 1, the goals of the test cases for this User Story are to demonstrate how Tele-operated Driving and 5G communication technology can be used to overcome scenarios in which an AD vehicle does not know how to tackle an unexpected road blockage. In this User Story, the indirect control concept is applied.

A/ Test case definition The scope of the test cases for User Story 2 is similar to User Story 1. The difference is the application of the indirect, instead of the direct, control concept. Instead of a desired steering wheel angle and velocity, a trajectory is defined by the tele-operator in the VCoC. This trajectory is then transmitted and executed by the vehicle. The schematics and time schedule of the first user story are shown in Figure 3-11 and Figure 3-12.

Figure 3-11: Use Case 1, User Story 2


33

Figure 3-12: Use Case 1, User Story 2 Time Schedule

B/ Functional Results The functional results for the tests for User Story 2 are similar to tests for User Story 1. In summary, it is desired that the tele-operator successfully commands the vehicle around the road blockage while a sufficient situational awareness is achieved and the tele-operator feels comfortable that she or he is in full control of the vehicle. As described above, the difference of User Story 2 from User Story 1 is the application of the indirect control concept.

C/ Monitored KPI The monitored KPIs are the same as for User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity) . What differs is mentioned below.

Deviation of the Desired Trajectory

For User Story 2, it will be monitored if the vehicle fulfills the requirement to laterally deviate from the desired trajectory at a maximum of 0.3 m. The deviation of the vehicle’s orientation from the desired orientation should be less than 5°.

Positioning Accuracy

For well performing trajectory tracking control, positioning with an accuracy of 50 cm or even less is required.

D/ Test Case Locations Same as User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity).

E/ Telco requirements Same as User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity).


34




• Overcoming obstacle through ToD with indirect control concept


Test Goal Measurement of following KPIs

• Range





• Deviation of Desired Trajectory

• Positioning Accuracy





Success Criteria Overcoming of obstacle(s) Desired functional results achieved




35

3.1.7 User Story 3 – Remotely Controlled Trajectory-based Driving on Rural Crossed Country Road (indirect control, high velocity)

This section describes the test case specifications for User Story 3 – remotely controlled trajectory-based driving on rural crossed country road (indirect control, high velocity). The goals of the test cases for this User Story are to evaluate the limits of ToD with 5G technology, analyze the handover and velocity limits and check cross-country interoperability.

A/ Test case definition The vehicle is driving automated at a defined velocity vhandover and requests takeover or support from the tele-operator located in the VCoC. If a handover during driving is possible and on which velocity vhandover this handover is allowed, will be evaluated within this test case. According to the request, the tele-operator either can control the vehicle similar to [User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity)], or can approve/reject requests (e.g. route suggestions) from the AD vehicle. The test case is planned to pass a border of the communication network, while the tele-operator is connected to the vehicle. This test should check if the handover across telecommunication network providers is possible while keeping the teleoperation operational. The test case analyzes the maximal possible speed for tele-operated driving concepts and finds the limitations of ToD. The maximum speed is set to 80 km/h or to the maximum allowed velocity of the test vehicle on the test track. Reverse driving is not considered (already considered in (User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity))). In the same way the takeover from AD driving to tele-operated driving, the handover from tele-operated driving back to AD will be evaluated. The schematics and time schedule of the first user story are shown in Figure 3-13 and Figure 3-14.



36

Figure 3-14: Use Case 1 User Story 3 Time Schedule

B/ Functional Result In addition to the functional analysis of User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity) and User Story 2 – Remotely Controlled Trajectory-based Driving (indirect control, low velocity), this test case will evaluate the handover during driving, cross country interoperability and the limits of ToD. The main goal is to evaluate if the ToD control by a remote operator can be seamlessly kept during the change from one telecommunication network to another. Furthermore, the limits of operation like maximum velocity during ToD or maximum velocity for handover for ToD will be analyzed and described.

C/ Monitored KPIs The monitored KPIs are the same as for User Story 2 – Remotely Controlled Trajectory-based Driving (indirect control, low velocity). What differs is mentioned below.

Range

In User Story 3, the operational range is up to 5000 m. The range should be given as the maximum reasonable value (e.g. a warning about an accident should be distributed in a range of 5000 m).

Network Provider Handover Time

The handover from operator A to operator B should not affect the tele-operator and the time for handover has to be smaller than the defined maximum latency in User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity).

D/ Test Case Locations This User Story 3 is planned for the large scale test site (see Figure 2-1) and/or the Motorway A9 5G-ConnectedMobility Testbed (see Figure 2-5). According to the higher speed, the highway or rural road has to be blocked and kept free of other vehicles, pedestrians and obstacles.

It has already been agreed that the cross-border tests will be conducted on one test site around Schengen (Figure 3-4, Figure 3-6, Figure 3-7).


37

E/ Telco requirements Telco requirements are the same as for User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity). Furthermore, the handover between two network-providers has to be ensured in a cross-country area. The Tele-operation should continue without interruption due to the network provider handover.



Scope • Check cross country collaboration of network provider

• Get limits for tele-operated driving

• Analyze handover of tele-operator during driving


Test Goal Measurement of following KPIs:

• Range







• Network Provider Handover Time

• Evaluation of velocity restrictions for Tele-operated Driving


Test Setup • Free rural road or highway in cross country area



• The maximum velocity is limited to 80 km/h (22.2 m/s) and no reverse driving is planned for this test case. The maximum possible speed for handover and for tele-operated driving will be evaluated in this test case.

Success Criteria • Handover of telco network provider during tele-operator connection


38

• Velocity limits analyzed

• Handover during driving evaluated



3.1.8 User Story 4 – Slim Uplink for ToD (indirect control, low velocity)

This section describes the test case specification for User Story 4 – Slim Uplink for ToD (indirect control, low velocity). The goal of the test case is to demonstrate if a ToD session can be executed using a new approach. It implies to share periodically changing images instead of videos and to compensate this with additional vehicle perceived object information to the tele-operator. Furthermore, it has the goal to observe if this new approach allows the planning of future mechanisms that will be the key to calculate the overall resources needed across the vehicle and the VCoC. The main goal is to improve efficiency and scalability of the uplink data channel, while achieving comparable results as in previously described user stories, in which the main goal is to overcome scenarios where an AD vehicle does not know how to tackle an unexpected road blockage. Moreover, an interesting aspect of this test case is to compare the performance differences between remote sensing based on the slim uplink and the full video from User Story 2.

A/ Test case definition The vehicle is assumed to come to a safe stop on its own. Then the tele-operator, located in the VCoC, gets informed and checks the vehicle environment. The tele-operator based on the provided data from the vehicle camera setup and the list of perceived objects that are been recognized by its sensor setup proceeds to execute the indirect control concept. The tele-operator will perform surveillance of the automated vehicle until it passes the road blockage to terminate the ToD session. After this, the AD function takes over control again and the drive continues.

A road blockage in these test cases is referred to as obstacles on the road, which constrain the safe driving Large Scale test site for the AD function, such that the vehicle cannot pass the obstacles. The obstacles could be two other vehicles that collided, a fallen tree or other larger objects. The reason the AD function cannot overcome the situation may be that traffic regulations need to be violated, e.g. driving on the bicycle path beside the road, or the AD function’s system limits need to be exceeded, e.g. driving on the soft shoulder.

The velocities during this remotely controlled maneuvering using the indirect control concept can be up to 15 km/h. Reverse maneuvers can also be in the scope the test cases but are not mandatory.

The schematics and time schedule of this user story are shown in Figure 3-15 and Figure 3-16.


39


Figure 3-16: Use Case 1 User Story 4 Time Schedule


40

B/ Functional Results It has been highlighted that for Tele-operated Driving, it is of great importance that the tele-operator in the VCoC is aware of the situation. For this User Story and to not influence the “Situational awareness" perceived by the operator, it is considered to enhance the knowledge about the vehicle surroundings by delivering an object-based array of processed sensor data to the VCoC.

The frame rate in which the new visual information will be delivered to the tele-operator is also part of the research process in this user story. We want to obtain an appropriate rate in which the situational awareness is still sufficient for the ToD session.

Network capabilities can influence the amount of bandwidth or latency in undetermined periods. This approach also investigates if the slim uplink concept can be beneficial while the network capabilities are degraded.

C/ Monitored KPI The monitored KPIs are the same as for User Story 2 – Remotely Controlled Trajectory-based Driving (indirect control, low velocity).

D/ Test Case Locations Same as User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity).

E/ Telco requirements Same as User Story 1 – Remotely Controlled Maneuvering (direct control, low velocity).




• Overcoming obstacle through Tele-operated Driving with indirect control concept


Test Goal Measurement of following KPIs

• Range






41







Success Criteria Overcoming of obstacle(s) Desired functional results achieved




42

3.2 Use Case HD mapping 3.2.1 Mapping of User Stories per Test Site

Table 3.6: HD Map User Story Mapping per Site

US 1 US 2 US 3 US 4

A9 Motorway 5G-ConnectedMobility

UTAC Montlhéry

Barcelona

Munich City Center

Large Scale Test Site X X X X

AstaZero X X X X

3.2.2 User Story 1 – Streaming the Map to the Car A/ Test case definition

Figure 3-17: HD Map Streaming to the Car

Map data are streamed from the cloud to the car when driving continuously along the road, approaching a national border and continuing into a new country.


43

Map data are streamed from the MEC to the car when driving continuously along the road, approaching a national border and continuing into a new country.

Map data are streamed from the cloud to the car when approaching an area with low connectivity or with high cost connectivity which is due to roaming agreements among MNOs.


Map data for the planned path is continuously available in the car.

C/ Monitored KPI

• Download bandwidth

• Service availability

D/ Test case location

Development and initial tests will be performed at AstaZero in Göteborg, Sweden. Cross-border trials will be performed at the largescale test Large Scale Test Site between Germany, France, and Luxembourg.

E/ Telco requirements

Communication network from at least two different MNOs providing both 4G and 5G connectivity to public Internet access. MEC capability to cache the map data.

F/ Summary

Table 3.7: Test Case of HD Map User Story 1


Scope To measure the download bandwidth and quality of service for 4G and 5G

Test goal To know the download bandwidth and quality of service for the systems under test

Test type System test

Test prerequisites Network providing public Internet access to TomTom cloud available. Network with MEC capability to cache the HD map data

Test setup One car, connected to the network

Success criteria HD map continuously downloaded in time, with a 10 km horizon in front of the car.

Test deliverables Before the test: test plan submitted and driving permit acquired (Volvo internal procedure).


44

After the test: Test report filed, logged data stored in Volvo datastore.

Test dependencies TomTom for providing access to HD maps.

3.2.3 User Story 2 – Streaming Sensor Data to the Cloud A/ Test case definition

Figure 3-18: HD Map Streaming to the Cloud

Sensor data are streamed to the cloud from the car when driving continuously along the road, approaching a national border and continuing into a new country.

Sensor data are streamed to the MEC from the car when driving continuously along the road, approaching a national border and continuing into a new country.

Sensor data are streamed to the cloud from the car when approaching an area with low connectivity or with high cost connectivity which is due to roaming agreements among MNOs. B/ Functional results

Sensor data received by the cloud and MEC for further processing. C/ Monitored KPI

• Upload bandwidth



Development and initial tests will be performed at AstaZero in Göteborg, Sweden. Cross-border trials will be performed at the large scale test Large Scale Test Site between Germany, France, and Luxembourg.


45


Communication network from at least two different MNOs providing both 4G and 5G connectivity. MEC capability to evaluate localized processing and dissemination.

F/ Summary



Scope To measure the upload bandwidth and quality of service for 4G and 5G

Test goal To know the upload bandwidth and quality of service for the systems under test


Test prerequisites 4G and 5G network and enabled MEC available

Test setup One car, connected to the network

Success criteria Sensor data continuously uploaded to the cloud and MEC

Test deliverables Before the test: test plan submitted and driving permit acquired (Volvo internal procedure). After the test: Test report filed, logged data stored in Volvo datastore.

Test dependencies N/A

3.2.4 User Story 3 – Closing the Loop A/ Test case definition

Sensor data are streamed to the cloud from the lead vehicle (A) and then map data is streamed to the following vehicle (B), when driving continuously along the road, approaching a national border and continuing into a new country.

Sensor data are streamed to the MEC from the lead vehicle (A) and then map data are streamed to the following vehicle (B), when driving continuously along the road, approaching a national border and continuing into a new country.


46

Figure 3-19: Real-time HD Map Update


Sensor data sent from a car to the cloud and MEC, where it is used to update the map. The updated map is then sent to a following car. C/ Monitored KPI

End-to-end update duration in different permutations of roaming.


Development and initial tests will be performed at AstaZero in Göteborg, Sweden. Cross-border trials will be performed at the large scale test site between Germany, France, and Luxembourg.



F/ Summary



Scope To measure the end-to-end update duration

Test goal To know the end-to-end latency for the systems under test


Test prerequisites 5G network and MEC capability available

Test setup Two cars connected to the network


47

Success criteria Updated HD map continuously downloaded in time, with a 10 km horizon in front of the car

Test deliverables Before the test: test plan submitted and driving permit acquired (Volvo internal procedure). After the test: Test report filed, logged data stored in Volvo datastore.

Test dependencies TomTom, for delivering the HD map

3.2.5 User Story 4 – Upload Sensor Data and Download Map Updates in Non-real Time

A/ Test case definition

Sensor data are streamed to the cloud from the lead vehicle (A) and then map data is streamed to the following vehicle (B), when driving continuously along the road, approaching a national border and continuing into a new country.

Sensor data are streamed to the MEC from the lead vehicle (A) and then map data are streamed to the following vehicle (B), when driving continuously along the road, approaching a national border and continuing into a new country. We need to test different permutations of roaming when crossing national border where more than one MNO providing network services.

Figure 3-20: Non Real-time HD Map Update


Sensor data are uploaded from the vehicle to the cloud and MEC, the data is quantified at the cloud and MEC (i.e. amount of data sent to trigger a map update). Updated map tiles are sent to vehicles in the area.


48

C/ Monitored KPI


• Update duration


Test Large Scale Test Site between Germany, France, and Luxembourg



F/ Summary



Scope Measurement of data load for sending sensor data from the vehicle to the HD map server and MEC

Test goal The test is directed towards the data volume that is sent on the uplink and the downlink

Test type Service availability and update duration

Test prerequisites Network connectivity with access to the HD map provider cloud Network Exposure Function allowing communication between the network and the MEC application

Test setup Two cars connected to the network

Success criteria Test is successful if vehicle B fully downloads the updated tile of the map

Test deliverables output data with analysis for the measurement

Test dependencies TomTom HD map


49

3.3 Use Case ACCA 3.3.1 Mapping of User Stories per Test Site

The mapping of user stories is shown in Table 3.11.

Table 3.11: ACCA User Story Mapping per Site

US 1 US 2 US 3 US 4

UTAC Montlhéry x x x

Barcelona x x

Large Scale Test Site Luxembourg x

Germany

France

Luxembourg-Germany

x

Germany - France

Ideally

US2 and US3 are likely not tested on the large scale test site for the following reasons:

• Concrete barriers prevent on-board sensors of a vehicle driving in the opposite direction to detect a traffic jam or a stationary vehicle,

• The view angle and the range of the on-board sensors may not be large enough to detect a vehicle on the emergency lane.

Four regions of interest (ROI) are defined for the ego vehicle (EV). A ROI corresponds to a given road Horizon range and is possibly associated to a requested Quality of Service (QoS) level. To simplify and shrink the test combinations, the critical ROI and the Collision Warning ROI are merged in a single ROI called Critical ROI, while the Collision Awareness ROI and the information ROI are merged in a single ROI called Informational ROI.

The Geoservice is instantiated and hosted in a MEC server which covers a given geographical area. Each instantiation of the Geoservice collects the local road perception of the vehicles under its coverage and builds a consolidated view of the local road events. In the meantime, any vehicle can request the Geoservice to send back information about events of interest in a given ROI.

The ROI is defined as a geometric form (it can be an ellipsoid, a circle, or another form) which will be exactly defined in a next work package. The range of a Critical ROI depends on the vehicle


50

speed and is typically 200 m at 130 km/h; i.e. the validity of a Critical ROI is short (5 s at 130 km/h). The associated subscription is a one-shot subscription because the validity of this subscription is short. It means that the Geoservice responds immediately and its response carries the event information, which matches the subscription conditions when the request is received.

On the opposite, the typical range of the Informational ROI is 1 km - 2 km. It also depends on the vehicle speed and its validity is long (typically 30 s – 1 min at 130 km/h). The associated subscription is an upon-change subscription. It means that the Geoservice responds immediately if there are events, which match the subscription conditions when the request is received; but the Geoservice can also notify later the subscribing vehicle about a new event, an updated event (its confidence level has increased as an example) or a deleted event.

The ROI and hazard age are shown in Table 3.12.

Table 3.12: ROI and Hazard Age per User Story

US 1 US 2 US 3 US 4

ROI Critical Optional:

Informational (Barcelona)

Informational Critical

Informational Informational

Hazard is old/new

Old Old Old & New (being

updated)

Old

3.3.2 ACCA System KPI Definition • KPI1: Hazard information upload time

It is the time to upload the hazard information, starting from time when the vehicle detects a hazard (ego or remote) until the hazard information is received for the 1st time by the Geoservice and stored in its local database. The time when the hazard information is received may happen after multiple repetitions of the event notification messages.

KPI1: upper hazard upload time at 95% (meaning for 95% shortest upload times)

95% is written here as illustrative purposes and may be adapted when running real tests.

To determine this KPI, it might be needed to determine:

• Statistics of end-to-end upload time for the received hazard information, as a Probability Distribution Function (PDF) or a Cumulative Distribution Function (CDF),

• Percentage of successfully uploaded hazard information (a hazard information is lost when all the DENM repetitions have been exhausted)


51

The breakdown of this KPI is shown in Table 3.13.

Table 3.13: Breakdown of Hazard Upload Time

Transmission (RAN) time Time when the 1st DENM is fully received: including HARQ or ARQ or DENM repetitions, and radio channel access

Backbone & Geoservice delay

Time when the 1st DENM is fully received and processed by the local instantiation of the Geoservice (decoded at facility layer in the geoserver and e.g. processed by geoserver and stored in database when necessary)

• KPI2: Notification time of old hazard

It is assumed that there is a hazard in the ROI, which matches the subscription conditions, and that the information about the hazard is present in the memory of the local Geoservice (the hazard is old).

KPI2 is the time to get hazard information from the Geoservice, starting from the time when the vehicle initiates its subscription until it gets the response from the Geoservice to its request. It includes the session establishment time, the multiple messages between the vehicle and the Geoservice to set up the session prior to sending the request, as shown by Figure 3-21.

Figure 3-21: Notification of an Old Hazard

KPI2 (ROI type): upper response time at 95%


Since the ROI type may impact the QoS of the connectivity Ego Vehicle (EV)-Geoservice, it also may impact the KPI2.


52

To determine these KPI, it might be needed to determine:

• Statistics of end-to-end response time from the Geoservice, as a Probability Distribution Function (PDF) or a Cumulative Distribution Function (CDF),

• Percentage of successful received response from the Geoservice (normalized to the number of request and whatever the response time is): response may not be received either because a transaction EV/Geoservice fails,


Table 3.14: Breakdown of Notification of Old Hazard

Radio channel access + Session establishment time

Time up to when the subscription is received by the Geoservice

Geoprocessing time Time to prepare the response

Response time Time to successfully send the response to the vehicle

• KPI 3: Upon Change notification time

It is assumed that the upon-change subscription has succeeded. A new or updated event of interest occurs in the ROI occurs.

It is the time to be informed about a new, updated or removed event, starting from the time when the hazard is detected and sent to the Geoservice by a vehicle A until the subscribing vehicle B is notified about this new, updated or removed event, as shown by Figure 3-22.

Figure 3-22: Notification of a New Hazard


53

KPI3 (ROI type): upper notification time at 95%


Since the ROI type may impact the QoS of the connectivity EV-Geoservice, it also may impact the KPI3.


• Statistics of end-to-end received notification time from the Geoservice, as a Probability Distribution Function (PDF) or a Cumulative Distribution Function (CDF),

• Percentage of successful received notification from the Geoservice (whatever the notification time is): the notification may not be received because the updated event is not received by the Geoservice


Table 3.15: Breakdown of Notification of a New Hazard

Hazard upload time Similar as KP1

Geoprocessing time Processing the event information, storing it and prepare the response

Notification time Time to successfully send the new or updated event

• KPI4: Notification of a hazard beyond the border

The vehicle has successfully subscribed to the Geoservice, with ROI = Informational. But the ROI exceeds the coverage area of the current MEC of the EV and extends beyond the border. A hazard, which matches the subscription conditions, has occurred beyond the border for a long time.

KPI4 is the time to get information of the hazard beyond the border from the Geoservice, starting from the time when the vehicle initiates its subscription until it gets the response from the local Geoservice to the request. It includes the session establishment time and the inter-MEC (intersystem) communication time.

KPI4: upper response time at 95%



• Statistics of end-to-end received response time from the local Geoservice, as a Probability Distribution Function (PDF) or a Cumulative Distribution Function (CDF),


54

• Percentage of successful received response from the Geoservice (whatever the response time is),


Table 3.16: Breakdown of Cross-border Notification

Session establishment time Time up to when the subscription is received by the Geoservice

Geoprocessing time Time to process the request

MEC-Cloud time Time to get the information about the hazard which is out-of-coverage of the current Geoservice

Response time Time to successfully send the response to the vehicle

3.3.3 User Story 1 – Ego-detected Event A/ Test case definition

Figure 3-23: User Story of Ego Detected Hazard

Vehicle B has had an accident in the same lane as the vehicle A. It has sent an ego DENM to the local MEC. Vehicle A sends requests with ROI = critical. If the accident is within the ROI of the request, the MEC returns a response with information about the accident in a DENM.


55

The ROI is critical on UTAC Montlhéry site, meaning that the response is expected in a window shorter than 5 s, since vehicle A is driving at high speed. In the case of Barcelona test site, the ROI will be informational and the driving speeds lower, leading to longer time windows. The hazard is out of the field of view of the on-board sensors of EV.

RSA and PSA will provide two vehicles which are not equivalent from a technical perspective or which can be used for a different role:

• HV: Emitting car (with sensor capability) PSA_1 and RSA_1

• EV: Receiving car (with ADAS capability) PSA_2 and RSA_2

CTTC will provide two non-automated vehicles (CTTC_1 and CTTC_2) equipped with a custom On-Board Unit (OBU) and a Human Machine Interface (HMI). These vehicles will not be equipped with sensors. The idea is to emulate the detection of hazards from the HMI in order to manually trigger the transmission of DENMs.

Vehicle role is shown in Table 3.17.

Table 3.17: Vehicle Role

Vehicle type EV HV Detecting vehicle

Vehicle role (UTAC Montlhéry)

PSA_1 RSA_2 None

Vehicle role (Large Scale Test Site)

RSA_1 PSA_2 None

Vehicle role (Barcelona) CTTC_1 CTTC_2 None


The hazard information is successfully loaded in the MEC memory by the hazardous vehicle (HV) with an ego DENM.

When the ROI of the EV request covers the hazard, the EV is informed about the hazard and smoothly slows down and brakes to avoid collision.

C/ Monitored KPI

• KPI1

• KPI2 with geoprocessing being limited to fetching the information of interest in the local data base of the local MEC


56


Large Scale Test Site:

Figure 3-24: Exit of the Markusbierg Tunnel12

Figure 3-24 shows the exit of the Markusbierg Tunnel on the A13 from Schengen (Luxembourg) to Mondorf-les-Bains (Luxembourg), GPS Coordinates: 49.482157, 6.339562, the hazard vehicle is placed on the emergency lane on the right side of the road in direction to Mondorf-les-Bains

Hazard Vehicle



57

UTAC-Montlhéry:

Figure 3-25: Location of the Hazard on UTAC-Montlhéry Site

HV

EV

The highway portion is used (green ellipsoid). Both vehicles are put on the same lane (the right one), see Figure 3-25.

Barcelona test site:

In this user story, the lack of visibility can be based on the existence of an obstacle, a crossing or a curve in one of the streets covered by the small cells. The virtual EPC (vEPC) and the MEC with the Geoservice of the virtual MNO will be deployed in the cabinet, which is connected to the Small Cells using an optical link of 1 Gbps.


The hazard area shall be covered by both 4G and possibly 5G RAN. The EV and the HV shall be connected to the same Geoservice instantiation.

F/ Summary

Table 3.18: Test Case 1 of ACCA User Story 1

Test Property Description US1 – TC1

Scope The HV can upload its status in the local Geoservice instantiation The EV can get relevant information about HVs in its ROI at the time the request is sent ROI = Critical


58

Test goal Measure KPI1 Measure KPI2 (ROI=Critical)

Test type System & performance test

Test setup HV stopped on the same lane as the EV (UTAC Montlhéry) or at the neighbor lane (Large Scale Test Site), out of the field of the view of the onboard sensors of the EV Ego vehicle ROI = Critical

Success criteria Collision avoided Hazard upload: KP1 < 500 ms Notification Time: KPI2 < 1s

Test deliverables Statistics of KPI1 and KPI2 and associated breakdown

Test dependencies None

Table 3.19: Test Case 2 of ACCA User Story 1

Test Property Description US1 – TC2 (Barcelona)

Scope The HV can upload its status in the local Geoservice instantiation The ego can get relevant information about HVs in its ROI at the time the request is sent ROI = informational (A3)

Test goal Measure KPI1 (Connectivity is the default connectivity) Measure KPI2 (ROI=informational)

Test type Performance test

Test setup HV stopped on the same lane as the EV or at the neighbor lane, out of the field of the view of the onboard sensors of the EV Ego vehicle

Success criteria Hazard upload: KPI1 < 500 ms Notification Time: KPI2< 5 s

Test deliverables Statistics of KPI and KPI2 and associated breakdown

Test dependencies None


59

3.3.4 User Story 2 – Ego and Remotely Detected Event A/ Test case definition

Figure 3-26: User Story of Remotely Detected Hazard

Vehicle B has had an accident. It has sent an ego DENM to the local MEC. Vehicle A1 also has detected the hazard and has sent a remote DENM to the local MEC. The Geoservice merges both information into a single hazard information.

Vehicle A2 sends requests with ROI = critical. If the accident is within the ROI of the request, the MEC returns a response with information about the accident in a DENM. The ROI is critical, meaning that the response is expected in a window << 5 s.

Vehicle A3 sends requests with ROI = informational. If the accident is within the ROI of the request, the MEC returns responses with updated information about the hazard in a DENM.

The hazard is out of the field of view of the on-board sensors of EVs. Vehicle role is shown in Table 3.20.


60


Vehicle type Entering in dangerous area(A2)

Entering in dangerous area (A3)

HV Detecting vehicle (A1)

Vehicle role PSA_2 RSA_2 PSA_1 RSA_1


The hazard information is successfully loaded in the memory of the local instantiation of the Geoservice by the detecting vehicle with a DENM. The Geoservice successfully combines the information from both the HV and the detecting vehicle.

When the ROI of the EV A2 request covers the hazard, the EV A2 is informed about the hazard and smoothly slows down and brakes to avoid collision. When the ROI of the EV A3 request covers the hazard, the EV A3 is informed about the hazard and the hazard is displayed on an HMI.

C/ Monitored KPI

• KPI1

• KPI2 with geo-processing being limited to fetching the information of interest in the local data base


UTAC-Montlhéry:

The HV is stopped on the highway part, in the blue ellipsoid. For the sake of clarity, A3 vehicle is not shown.


61

Figure 3-27: Location of Hazard on UTAC-Montlhéry Site

B: HV

A1: Remote detecting vehicle

A2: EV vehicle coming in the dangerous area


The hazard area shall be covered by both 4G and possibly 5G RAN. The EV and the HV shall be connected to the same Geoservice.

F/ Summary

Table 3.21: Test Case of ACCA User Story 2


Scope The HV is detected by a remote vehicle and the remote vehicle uploads the detected event in the local Geoservice. The ego can get relevant information about HVs in its ROI at the time the request is sent ROI = informational (A3)

Test goal Verify that the Geoservice can merge DENM of the same hazard and being sent by different vehicles Measure KPI2 (ROI=informational)


Test setup HV stopped on the same lane as vehicle A2, out of the field of the view of the onboard sensors of the ego vehicle

B

A2

A1


62

Detecting vehicle Ego vehicle

Success criteria Hazard notified to A3 Notification Time: KPI2 < 2 s

Test deliverables Statistics of KPI2 and associated breakdown

Test dependencies TC 1, TC 2

3.3.5 User Story 3 – Traffic Jam Detection A/ Test case definition

Figure 3-28: User Story of Traffic Jam

Two vehicles (B1, B2) have been mimicking a traffic jam (driving very slowly or being stopped). They have been sending CPM messages to their current MEC. Additional vehicles, which are non-V2X equipped, can be added. Vehicle A1 has been driving in the opposite direction and sends CPM message.


63

Vehicle A2 sends requests with ROI = informational. If the traffic jam is within the ROI of the request, the MEC returns responses with updated information about the traffic in a DENM. The traffic jam is moving slowly so that the traffic jam attributes are regularly updated and so that the MEC sends regular notifications about traffic change updates in a DENM.

The hazard is out of the field of view of the on-board sensors of A2 vehicle. Vehicle role is shown in Table 3.22.


Vehicle type EV (A2) Traffic Jam vehicles (Bi)

Remote Detecting vehicle (A1)

Company RSA_2 PSA_2 and RSA_1

PSA_1


The CPM messages are successfully received by the Geoservice. The Geoservice has detected the traffic jam situation and has detected the traffic jam changes.

When the ROI of the EV A2 request covers the hazard, the EV A2 is informed about the traffic jam and about traffic jam changes. The traffic change is displayed and updated on an HMI.

C/ Monitored KPI

• KPI1 with CPM messages

• KPI3 with geoprocessing being the algorithm to update traffic jam information


This user story is tested on the same location at UTAC-Montlhéry site as the User Story 2.


64

Figure 3-29: Location of Hazard on UTAC-Montlhéry Site

B1, B2: Traffic jam vehicles

A1: Remote detecting vehicle

A2: EV coming in dangerous area


The hazard area shall be covered by both 4G and possibly 5G RAN. The EV and the HV shall be connected to the same Geoservice instantiation.

F/ Summary



Scope Detect Traffic Jam event in the Geoservice The traffic Jam information is old and has been in the MEC memory for a long period The traffic Jam information is updated: position of the traffic jam is updated The ego is notified by the traffic jam information updates ROI = informational

Test goal Verify that the Geoservice can detect traffic jam conditions from CPM information Measure KPI1 with CPM messages Measure KPI3 (ROI=informational)

B1

A2

A1

B2


65


Test setup Vehicles to mimic the traffic jam Detecting vehicles Ego vehicle

Success criteria Traffic jam detected and updated Hazard update: KP1 < 500ms Notification Time: KPI3 < 2 s

Test deliverables Statistics of KPI1 and KPI3 and associated breakdown

Test dependencies TC 1

3.3.6 User Story 4 – Cross-border Scenario A/ Test case definition

Figure 3-30: Story of Cross-border Scenario

Vehicle B has had an accident. It has sent ego DENM to the local MEC2.


66

Vehicle A2 is located beyond the border and sends requests with ROI = Informational. The local MEC is MEC1 and belongs to another MNO than the MNO of MEC2. The ROI range is quite large (>> 1 km) so that it the ROI extends beyond the MEC1 coverage. If the accident is within the ROI of the request, the MEC returns a response with information about the accident in a DENM. For this purpose, the MEC1 has to communicate with MEC2 to get the hazard information which is located under MEC2 coverage. The hazard is shown on the EV display.

As an alternative test case, the EV and the HV can be close together (on the same side of the border) and they can be under the coverage of two different MNOs, the EV being configured with H-PMN = MNO1 (aka with a SIM card of MNO1) and the HV being configured with H-PLMN = MNO2 (aka with a SIM card of MNO2).

The vehicle role is shown in Table 3.24.

Table 3.24 - Vehicle Role

Vehicle type EV HV

FR GER (optional) PSA_1 RSA_2

GER FR (optional) RSA_1 PSA_2

LUX GER PSA_1 RSA_2

GER LUX RSA_1 PSA_2

Barcelona CTTC_1 CTTC_2


The hazard information is successfully loaded in the MEC2 memory by the HV with an ego DENM.

When the ROI of the EV request covers the hazard, the EV is informed about the hazard and is shown on the EV display.

C/ Monitored KPI

KPI4


Barcelona test site:

In this user story, the lack of visibility can be based on an obstacle, a crossing or a curve in one of the streets covered by the small cells. Two virtual MNO networks are being deployed to simulate the cross-border test case.


67

Large Scale Test Site:

Figure 3-31: Location of the Cross-Border-Large Scale Test Site on the French-German

Border (Saarbrücken)13

Figure 3-31 shows the precise localization of the border crossing between Saarbrücken in Germany on the A6 motorway and Forbach/Stiring Wendel in France on the A320 motorway, the lilac line represents the political border.




68

Figure 3-32: Rest Area Goldene Bremm14

Figure 3-32 shows the rest area Goldene Bremm on the A6 exit 2 (Saarbrücken-Goldene Bremm) in Germany at the border to Forbach France, GPS Coordinates: 49.207119, 6.96383. It is planned to perform the user storyin both directions.

Hazard Vehicles

14 IGN-FEDER-Préfecture de la region Grand-Est, France, 2019.


69

Figure 3-33: Rural Road on the German-French border15

Figure 3-33 shows a rural road on the German-French border named L273 “zum Zollstock” in Germany and D32C in France from Saarbrücken to Spicheren, GPS Coordinates: 49.208808, 6.971469, the lilac line represents the political border.

Hazard Vehicles




70

Figure 3-34: Location of the Cross-Border-Large Scale Test Site on the Luxembourgish

German Border (Schengen/Merzig)16

Figure 3-34 shows the precise localization of the border crossing between Perl-Mezig in Germany on the A8 motorway and Schengen in Luxembourg on the A13 motorway, the lilac line represents the political border.




71

Figure 3-35: A8 from Merzig (Germany) to Schengen (Luxembourg)17

Figure 3-35 shows the motorway A8 from Merzig (Germany) to Schengen (Luxembourg), GPS Coordinates: 49.483886, 6.37412.

Hazard Vehicles



72

Figure 3-36: Rest Area Moseltal-Perl-Besch18

Figure 3-36 shows the Rest area Moseltal-Perl-Besch on the A8 between the exit 2 (Perl) and the exit 13 (Remich-Schengen) in Germany, GPS Coordinates: 49.483886, 6.37412. It is planned to perform the user story in both directions

Hazard Vehicles


The hazard area shall be covered by both 4G and possibly 5G RAN. The EV shall be connected to a MEC and the HV shall be connected to another MEC which does not belong to the same MNO.



73

F/ Summary



Scope Cross-border and cross-MNO scenario The ego is notified about the hazard beyond the border ROI = informational

Test goal Verify that the local Geoservice forwards the request for the ROI part which is not under the MEC coverage to the Cloud server or to the appropriate MEC Verify that local Geoservice receives the information of the hazard beyond the border or stored in a MEC of another MNO Measure KPI4


Test setup HV beyond the border and attached to another MNO than the EV MNO Ego vehicle

Success criteria Hazard beyond the border is notified to the EV Notification Time: KPI4 < 2 s

Test deliverables Statistics of KPI4 and associated breakdown

Test dependencies TC 1, TC 2


74

4 Summary This deliverable describes the three use cases and a set of user stories (so-called sub-use cases) that will be trialed in 5GCroCo. Each user story is associated to at least one test case. The test cases are specifying in particular what has to be tested and where it has to be tested.

The list of test cases offers a good coverage of the 5G core features, which might be pertinent for the automotive industry, as shown in table 4.1 below.

Table 4.1: Mapping of 5G Core Features vs Use Cases

ToD HD mapping ACCA

Throughput: vehicle offboard service X X

Reliability X X

Latency: vehicle offboard service X X X

MEC support X X X

Inter-MNO handover optimization X X X

The detailed test case definitions might be revisited when the whole architecture work is completed in WP3 and when the radio coverage measurements of the different test locations are finalized in WP4.

Documents

Deliverable D2.1 Test Case Definition and Test Site ... · The large scale test area that has been selected is the EU 5G Large Scale Test Site in the border area between Luxembourg