48

Approach to Evaluation Technology for Connected Product

1. Trend in Connected Product

Recently, a car has advanced in various ways. A business, which analyzes acquired various data such as vehicle conditions and surrounding road conditions on the network via Internet connection or others using mobile data communication and provides appropriate optimal services, has been growing along with the so-called “connected cars” that are the vehicles equipped with communication devices. For example, emergency call , vehicle management , monitoring/operation management, driver assistance/protection, telematics insurance, congestion elimination, entertainment, etc. start to be popularized commonly. In addition, automotive manufacturers are also expanding their service business (vehicle dispatch system/vehicle management, etc.) in collaboration with car sharing and rideshare operators. Changes in environmental surrounding the connected car society is shown in Fig. 1. We, DENSO TEN Limited, are also promoting

mass production of connected products such as a communication type dashboard camera, an emergency call system, TCU (Telematics Control Unit), and a taxi

Recently, a business which provides various services has been growing along with the popularization of so-called “connected cars” that are the vehicles equipped with communication devices.

Various devices are connected, leading to an increase in the amount of data to be handled and communication frequency in future. On the other hand, service for which reliability is required to function stably even in any environmental change and user usage is being expanded.

In order to meet such market demands, achievement for both short-term development and quality improvement of the connected products is an urgent issue. In addition to improvement of design quality, an approach to improvement of evaluation efficiency and evaluation quality are indispensable.

In this article, cases of evaluation technology development by establishment of a simulation environment and process improvement by front loading of evaluation are introduced.

Abstract

Approach to Evaluation Technology for Connected Product

Tatsuo WAKIMOTO

Shinichi SUGIURA

Takuya TSUJIMOTO

Shogo TANAKA

Eisuke HAYAKAWA

Environment Changes Conditions in 2030

Increased popularity of autonomous

driving vehicles

Emergence of sharing services

Powertrain diversification

Start of services using autonomous driving vehicles(From 2020: Level 3 or higher)

Shift in values from "Owning" to "Using"

International consensusregarding global warming

countermeasures (Paris Agreement)

30% of travelingdistance by

autonomous driving

Max. 50% are sharedvehicles

25% are xEV(Based on amount of

vehicles sold)

Increased popularity of

connected cars

Increased popularity of connected cars

(2020: 1/3 of the new cars sold in North America equipped with a remote diagnostics function,

launched virtual assist in the market, and increased popularity of music streaming)

Majority of passengervehicles (90% of newcars) are connected

carsConnected

Automated

Shared& Services

North America: Deregulation by government + Verification testing, businesstrials and practical implementation led by private industry

Europe: Standardization with the goal of changing from MaaS to autonomousdriving through connected technology led by the government

Global Mega-trends Changes in the Environment Surrounding the

Connected Car Society: "CASE"

Conversion to EV

Fig. 1　Changes in the Environment Surrounding the Connected Car Society 1)

49

DENSO TEN Technical Review Vol.3

dispatch system, and are developing various service businesses such as safe driving management along with hardware. Image of connected car society system is shown in Fig. 2.

2. Increase in Product Reliability Requirement

According to the expansion of services, various devices are connected, leading to an increase in the amount of data to be handled and communication frequency. Also, for data communications such as emergency notification, driving management, and dispatch service, high reliability is required to function stably in any site environment change and user usage. In order to ensure its quality, the expected value of evaluation for systems and products has been increasing.

In site environment, there are various state changes in mobile data communication, GPS reception, or the like depending on location and time. Therefore, implementing a comprehensive evaluation to combine the operating states of each device is necessary. Approaches from both evaluation technology development and evaluation process improvement are important in order to meet the requirements of short-term development and achieve quality improvement with limited resources.

3. Approach to Quality Improvement through Evaluation Technology

Figure 3 shows the V-shape process of product development. With the aim of improving efficiency of

evaluation and quality of evaluation, we have approaches to (1) the efficiency improvement of evaluation by establishing a simulation environment and to (2) front loading of issue extraction by evaluating specifications.

(1) Efficiency improvement of evaluation by establishing a simulation environment

The evaluation approach using two cases is introduced in this chapter.①Automatic evaluation of the emergency notification

systemThe emergency call system is a device which

automatically sends an emergency notification to the center about information such as the location where a collision occurs via TEL communication. Regardless of location and time, European legislation requires normal operation even under radio wave environment which changes with every moment.

Fig. 4 is a functional environment evaluation matrix that covers the functions (logic) to be tested and the radio wave environment (environment parameter). For each step from an alarm activation of emergency call to the call termination, it is necessary to test about 300 patterns and check the operation of about 600 items for confirmation of the function in the case of a change in the communication status such as connected, out of service, and reconnected states. Conventionally, all patterns were tested manually.

This time, an evaluation system which performs automatic execution and automatic judgment for this

AutomobileDomain

HMI

Telematics Unit

Sensor

ECUEV

(( ))

Non-Automobile

Domain

5GWiFi V2X

OEM Services

MAP OEM

National/PublicCenter

(( ))

OperatorIT

Overall System for Connected Cars

Autonomous Driving Support

Shared CarCommercial Vehicles Center

Open Services

Autonomous Driving

Control ECU

Entertainment System ECU

- Agent- Music/Video

Streaming- Mobility Services- Car Sharing

- Dynamic Maps- Traffic information

(Wide and narrow ranges)

- Maintenance- Engineering

Fig. 2　Image of Connected Car Society System 2)

Implementation procedure for system eligibility

confirmation test

System method design specifications

Systemrequirement

specifications

Development Process

System methoddesign

Software requirement

analysis

Software method design

Softwaredetail design

Coding

Systemverification and integration test

Software verification test

Softwareintegration test

Software unit test

Implementation confirmationof system

Implementation confirmation in a coupling state of

hardware/software

Implementation confirmation of software requirement

Implementation confirmation ofsoftware method design

Implementation confirmation of module

specifications

System requirement

analysisSystem

validation test

(1) Efficiency improvement of evaluation by establishing a simulation environment

Implementation procedurefor system coupling test

Software method design specifications

Software detail design specifications

Softwarerequirement

specifications

(2) Front loading of issue extraction by evaluating specifications

Fig. 3　V-shaped Process of Product Development

50

Approach to Evaluation Technology for Connected Product

test was established, utilizing a simulator (CRAMAS) developed by our company. Full automation enabled the system to operate 24 hours a day, which achieved the efficiency improvement of evaluation and the period shortening.

Fig. 5 shows the evaluation system structure. It consists mainly of a simulator (CRAMAS), which performs test pattern execution and automatic judgment of results, in addition to a base station simulator which simulates a radio wave environment and a call center, a microphone/speaker set for emergency call, and others. CAN information (vehicle speed and crew information), which is a vehicle signal, and an airbag signal at the time of vehicle collision secured the simplification of system and the synchronism of the test timing by generating a pseudo signal with the simulator (CRAMAS).

Furthermore, the confirmation of LED lighting state of the microphone/speaker kit serving as a human interface was performed by voltage monitoring, and the transmission/reception confirmation of voice was performed by judgment of transmission and reception of a specific frequency. Automatic judgement about whether voice transmission to the center and the center recording were success or failure was realized by this method, which led to the achievement of the fully

automated test. By establishing this evaluation system, the functional test period was shortened from 30 days to 10 days, and the evaluation man-hours were greatly reduced from 480H to 40H.

②Automatic Evaluation of Communication Type Dashboard Camera

The communication type dashboard camera has functions of recording data such as CAN communication, position information, and image in a built-in memory or SD card, and sending it to a server. Depending on the user usage, when the data processing timing overlaps with the time of startup or shutdown of the device, it may result in a problem that data is damaged or wrong data is sent. Since this kind of problem may occur only within a few milliseconds during specific data processing, it is extremely difficult to detect the defect by a normal power supply fluctuation test or a manual operation repetition test. Therefore, we worked on strengthening the evaluation of such power supply systems for establishing the automatic evaluation environment.

Fig. 6 shows the evaluation environment and Fig. 7 shows the evaluation time chart (case of evaluation patterns) of power supply ON/OFF.

電波圏外(無線基地局GSM)(-140dBm)

ネットワーク圏外(無線基地局GSM)(-140dBm)

キャリア圏線基地局(-140dBm

【9】自動緊急通報作動 No.1 No.40 No.91 No.1【10】手動緊急通報作動 No.2 No.41 No.92 No.1【11】圏外且つ自動緊急通報作動 - No.42 No.93 No.1【12】圏外且つ手動緊急通報作動 - No.43 No.94 No.1【13】圏外かつ2分経過 - No.44 No.95 No.1【14】圏内 No.3 No.45 No.96 No.1【15】PSAP側で受話 No.4 No.46 No.97 No.1【16】受話せず No.5 No.47 No.98 No.1【17】MSD送信完了(初回送信) No.6 No.48 No.99 No.1【18】T5タイムアウト(初回送信) No.7 No.49 No.100 No.1【19】T6タイムアウト(初回送信) No.8 No.50 No.101 No.1【20】T7タイムアウト(初回送信) No.9 No.51 No.102 No.1【21】MSD送信中にPSAPで終話(初回送信)

No.10 No.52 No.103 No.1

【22】MSD送信中に電波圏外(初回送信)

- No.53 No.104 No.1

【23】リダイヤル時間終了 - - - -【24】圏内 No.11 No.54 No.105 No.1【25】終話による切断 - - - -【26】CCFT(60分)による切断 - - - -【27】MSD要求 No.12 No.55 No.106 No.1【28】MSD送信完了 No 13 No 56 No 107 No 1

微弱電波GSM(-90dBm)

Func

tion（

Logi

c）

Radio Wave Environment (environment parameter)

Fig. 4　Emergency Call System Functional/ Environmental Evaluation Matrix

eCALL（TCU）

TEL

eCALL OUTPUT

Microphone

Speaker

DC Power Supplyfor TCU_BUB

LAN

LAN

Diagnostic Tool (CAN)

CAN

CAN

RELAY_BOX

Terminal Series 12V TCU

LAN

GPS Antenna

Base-station Simulator

KVASER

PowerSupply12Ｖ

【Simulator】

Simulator（CRAMAS)

CAN

Power/Vehicle Signal

Control PC

FailBOX

Fig. 5　Evaluation Environment of Emergency Notification System

51

DENSO TEN Technical Review Vol.3

Setting the time T1 to T4 shown in Fig. 7 in detail with a simulator program and executing a large number of patterns enable a defect with low occurrence frequency to be detected. In addition, the reproducible repeated patterns make it easy to analyze the defect and confirm improvement effects, and also enable the frequency of occurrence in the market to be calculated.

Data such as driving history is required for defect detection. Vehicle data such as vehicle speed and engine speed can be generated by the simulator program. Furthermore, pseudo rapid acceleration/deceleration and collision conditions were entered into the product by using a tilting device capable of program control. The operation check of a device was realized by automatically checking the LED lighting state with a sensor and leaving the timestamp when an image data was obtained as data. The storage of these driving

histories and the verification of data which is uploaded to the server made the quality confirmation of products more efficient.

(2) Front loading of issue extraction by evaluating specifications

Normally, all functions are evaluated after designing the application/viewer software. However, more than 80% of defects and improvements are operation screen transitions, wrong indications related to the display, and difficulties to see. These contents are incorporated at the time when the design specifications are created and can be confirmed before completing the software design. Therefore, front loading was performed to extract defects and improvements by “usability evaluation for specifications” in the design phase of application/viewer specifications. (Fig. 8)

Two new methods of “heuristic evaluation method” and “cognitive walkthrough” were adopted for “usability evaluation for specifications.”

Each function of the Web application was checked for 47 items from 17 viewpoints with “heuristic evaluation method,” and the operation/screen transition, etc. of all functions were confirmed with the “cognitive walkthrough” method.

As a result, improvement can be made in the design phase. Although the modified rate of the items which were pointed out at the time of release was 8% with the conventional method, 90% of the items which were pointed out in the overall evaluation could be improved by introducing “Usability Evaluation for Specifications,”

SD rewriting devicePowersupply12Ｖ

Control PC

Connected ProductPower Supply,Vehicle Signal

Simulator（CRAMAS)

G tilting device

LED monitoring(camera image)

G tilting deviceGPS Antenna

【Simulator】

Fig. 6　Evaluation Environment of Communication type Dashboard Camera

Power-supply voltage

13.5V

Vehiclespeed

ACCvoltage

8.0V

T4T1 0.5s

0km/h

60km/h10s 10s 20s 10s

T2

12.0V

3s T3

DrivingHistory

G Value0.６G

0.1s

Fig. 7　Evaluation Time Chart of Power Supply On/OFF

BeforeImprovement

Usability Evaluation

Confirmation of Basic Function

Release☆

EvaluationPlanning

Modified rate of items which were pointed out: 8％

AfterImprovement

Evaluation is conducted at the specification development stage before incorporating all functions

Release☆

Specifications Usability Evaluation

Development Evaluation

Development Evaluation

EvaluationPlanning

Confirmation of Basic Function

ProductsUsability Evaluation

Development of Specifications

Development of Specifications

Modified rate of items which were pointed out: 90％

Usability evaluation is conducted after incorporating all functions⇒Prioritizing the handling of 　a functional defect. 　Improvement is likely to be 　implemented in the next model 　or later.

Modified rate of items which were pointed out was greatly improved since it could be modified at thespecification stage and also the man-hours for modification were reduced.(Modified 132 items out of 147 items which were pointed out.)

Fig. 8　Front Loading of Usability Evaluation

52

Approach to Evaluation Technology for Connected Product

resulting in the improvement of the completion rate at the time of release.

Figure 9 shows the improvement case in the “Usability Evaluation for Specifications.”

4. Conclusion

In order to meet the high-quality requirements of connected products , technology development such as automatic evaluation which simulates actual environment of automobile and network is indispensable.

I n addit ion to the eva luat ion technology development described above, we would like to contribute to the development of connected products capable of being used safely and comfortably through an approach of developing evaluation technologies related to human interfaces such as gesture operation and voice recognition.

Lastly, we would like to express our sincere appreciation to the relevant people inside and outside the company for their cooperation and guidance in developing this evaluation technology.

・CRAMAS is a registered trademark of DENSO TEN Limited.

Reference1) Hiroyuki Watabe and the other, “Efforts toward

Realization of Connected Car Society” and DENSO TEN Technical Review Vol.1

2) How will the Automobile Industry be Different in 2030 (held February 8, 2017), Nikkei Automotive seminar

WithNotification

When detectingstrong G

Without Notification

Excessive Speed

Sudden Steering RapidDeceleration

Excessive Engine Speed Idling

Lane Departure

Rapid Acceleration

Dangerous Distance between Vehicles

Wandering

Expiry Notification

Violation items subject to notification

【Before Improvement】 Mail Notification Setting Screen

WithNotification

Without Notification

Driver’s license items

hr. min.

OFFON

☑ Excessive Speed

☑ Sudden Steering ☑ RapidDeceleration

☑ Excessive Engine Speed ☑ Idling

□ Lane Departure

☑ RapidAcceleration

□ Dangerous Distancebetween vehicles

□ Wandering

OFFON

Notification time

Changed to clearly legible characters

Changed to check box

When detectingstrong G

Violation items subject to a notification

Notification related to driver’s license Expiry notification

【After Improvement】 Mail Notification Setting Screen

hr. min.Changed to pulldown

Fig. 9　Improvement cases in Usability Evaluation for Specifications

TatsuoWAKIMOTO

ShinichiSUGIURA

TakuyaTSUJIMOTO

ShogoTANAKA

EisukeHAYAKAWA

Connected Business GroupQuality Assurance DivSystem Evaluation Dept

Connected Business GroupQuality Assurance Div

Connected Business GroupQuality Assurance DivSystem Evaluation Dept

Connected Business GroupQuality Assurance DivSystem Evaluation Dept

seconded toDENSO Corporation

Profiles of Writers