01

The testing of ECU software always takes up a significant portion of the overall development process. Each new soft-ware version must run through a defined release procedure that includes numerous individual tests with different focal points. At the same time, the results must be thoroughly documented as proof of the software’s degree of maturity and bug-free condition. The complexity of today’s ECUs makes this process unmanageable without test automa-tion. Still, test configurations and test scripts do not come into being on their own.To make this process more efficient, Daimler developed the CANoe application CANSpector years ago for network tests of their passenger car engine ECUs. This application

supported all tests for the network management (NM) of the Daimler network technology at the time with OSEK NM. Employees had to handle all other test scopes manu-ally with the CANoe software, requiring them to create scripts, configurations, and the like and to perform mea-surements, generate reports and so forth.In the current powertrain architecture, multiple engine ECUs and gateways are in development concurrently at Daimler. Because the automobile manufacturer uses plat-form software, the engine ECUs are similar in their soft-ware components for networking. The two development cycles based on the V-model per year with four software releases per V-cycle entail a considerable testing effort.

Network Tests for EveryoneWith Minimum Configuration Effort for Maximum Testing DepthEvery development department of automotive electrical control units (ECUs) naturally thinks about how they can test their units quickly and efficiently. Since 2012, Daimler has been successfully using a special network test generator to create test configurations for network tests of engine ECUs for passenger cars among others. The tool, which was devel-oped externally by Vector, is based on the simulation and test tool, CANoe. It covers comprehensive test cases from network management tests and gateway tests to monitoring of message counters and cycle times. Tedious adaptations and the programming or editing of test scripts are now a thing of the past.

02

Network Tests for Everyone, 08/2016

In the current topology, the engine ECU and the central powertrain controller are connected to one another via a CAN direct connection. Furthermore, the engine ECU has a CAN bus for connecting sensors as well as the powertrain CAN bus that leads to, among other things, the transmis-sion control module (TCM). The central powertrain control-ler (CPC) forms the interface between the powertrain and the vehicle and performs functions including gateway func-tions. Besides multiple CAN buses, there is also a FlexRay bus leading from the CPC. It serves, for example, the ESP, electric driving steering response, electric ignition switch, and radar ECUs for various assistance systems. The COM-Spector tests cover all seven current engine types of Mercedes Benz passenger cars.

Sophisticated tests with a mouse The basis process for creating a test case with the COM-Spector tool looks something like this: The first step is to select the communication matrix. That can be an AUTOSAR ECU extract, an AUTOSAR system description, or a CANdb database (network or routing database). Using selection trees, lists, and buttons, the test engineer can now easily select the test nodes, scopes, and options needed for his test case using a mouse. At the end, the program gener-ates an XML test module and a CAPL library. Up to this point, COMSpector works as a standalone tool fully inde-pendent of other software tools so that any employee can use it on his PC. The XML test module and CAPL library are then used on a computer with installed CANoe for creating the actual test configuration for the test system and can also be executed there (Figure 1).

The first software release is typically characterized by the most innovations so that the largest test scope also occurs then. If one considers the overall test effort for the net-working software, approximately 40 percent is allotted to the tests of the network changes. The rest is allotted to general software tests based on requirements. The time needed for testing is significantly less for subsequent soft-ware releases.

CANSpector becomes COMSpectorWith its new network topology, Daimler introduced an AUTOSAR-compliant network management. The lack of support for AUTOSAR standards, FlexRay, and new data-base formats were some of the reasons that the in-house tool underwent further development in 2011. In view of the extensive adaptation work and maintenance effort, the development department at the German Sindelfingen site decided to outsource the project and hired Vector to under-take the development and further development of the new tool. The new “COMSpector” network generator has a sig-nificantly expanded scope of functions and thus covers considerably more tests case and – like its predecessor tool – is based on CANoe. The latter provides numerous func-tions for automatic testing of ECUs via the TFS (Test Fea-ture Set). The tool also handles residual bus simulations, plays back recorded trace data, and has a report generator for creating detailed test reports (Figure 1). Automatic test sequences and network node simulations can be coded in the CAPL script language with C-like syntax. Because CANoe has long been used successfully within Daimler, COMSpector is integrated seamlessly into the existing tool chain.

Figure 1: Structure of the Daimler test execution: process from the network test generation by COMSpector up to execu-tion by CANoe

03

Network Tests for Everyone, 08/2016

For testing the routing behavior of the central powertrain controller, for example, the test system monitors the rout-ing times, data consistency, and data sequence for all cyclic and spontaneous messages. Every error that occurs trig-gers an entry in the test report, such as when messages are lost or the permitted routing times (MaxRoutingTime) are exceeded. The update bit routing test follows a similar pro-cess. The gateway must always set the update bit when a certain signal has been received on the source bus so that the destination ECU can detect that the original ECU is sending. Signal changes are thus detected correctly and not concealed by the gateway under any circumstances. The test system documents every violation of these rules in the error log.In further tests, message counters, checksums, and cycle times can be monitored in a similar way (Figure 2). The pa-rameters defined by the test engineer in COMSpector, such as MaxRoutingTime for gateway tests and MaxCycleTime/MinCycleTime for cycle time monitoring, are of vital impor-tance for the evaluations of the test run. The last two parameters mentioned define the upper and lower time- related tolerance limits within which the relevant cyclic message must arrive in each case. Each violation of the cycle time tolerance limits triggers an error entry in the test log. Passive tests also generally function with logging data in offline mode. This means that recorded network commu-nication from test drives can be subsequently tested at the workstation at any time.

COMSpector supports both passive and active tests. Active tests use the CANoe functions for the remaining bus simulation and are always intended for an individual ECU. Through a remaining bus simulation, it is possible to simu-late the missing environment almost completely for the isolated test object. This is the only way to realistically test the engine ECU or gateway in an early development stage before other network components exist. The simulation system simulates the send behavior of receive messages with the help of information from the communication matrices. Based on the interaction layer implemented in CANoe, the software is able to exactly reproduce the Daimler send model needed here.

Passive and active testingIn passive tests the test system assumes the role of an ob-server. In other words, it does not intervene in the events on the bus systems for the entire test period. Passive tests are thus typically well-suited for actual test drives. Because every network engineer at Daimler has an ECU at his desk, these tests are also possible directly at each workstation. There is constant testing there of the network traffic or certain messages for specific rules of behavior. In the first step, it is usually tested whether an ECU sends all messages in the correct time pattern and with the correct checksum and correct message counter. Missing messages may indi-cate an overload of the ECU itself or that the bus load is too high in general. Passive tests are possible both for indi-vidual buses and across buses.

figure_label_left: Figure 1: Lorem ip-sum dolor sit amet

Figure 2: COMSpector screen-shot with passive tests

04

Network Tests for Everyone, 08/2016

is then able to interactively select and manipulate particu-lar messages during operation. He can block messages, change signal values, falsify checksums, freeze message counters, and much more. Because the test system maps all messages in the form of system variables, these can be easily displayed using the Symbol Explorer. A click in the context menu of a system variable opens the related con-trol panel that permits the indicated manipulations.

Network management tests The network management (NM) controls the network traf-fic on the bus and is mainly responsible for the waking up and shutting down of ECUs. An ECU with faulty network management could, for example, keep the network traffic in the vehicle permanently awake causing the car battery to discharge after a short time. This test should discover any problems of this type well in advance. The NM is tested for the various Daimler network topologies in predefined test sequences. Besides the ECU to be tested, other nodes are simulated on the NM for this. The Daimler NM tests currently support the CAN and FlexRay bus systems.

Small test setup with major impactDaimler is especially pleased that the network tests can be implemented with a surprisingly small test setup. Besides a PC or laptop, a test engineer needs only a break-out box and one or two CAN or FlexRay interfaces from Vector. The break-out box with its handful of separated connections and cables represents, in effect, the minimum necessary hardware configuration. Among other things, it generates not only a 12 V steady plus but also a switchable voltage from a 12 V infeed for simulating an ignition switch. One of the requirements of the automobile manufacturer was

In active tests, CANoe simulates the network communica-tion of the network involved except for the ECU to be tested. The test system enters actively into the bus communica-tion in order to test the defined rules of behavior (Figure 3). The gateway routing tests take up a large test scope here. The message and signal contents are stimulated with dif-ferent values (minimum, average, and maximum values). Each stimulation is tested one after the other in a separate test case. The routing behavior of the ECU is monitored during each test case. The following functions are tested here in particular: routing times, data consistency, and data sequence. The gateway routing tests also include the simulation of message failures and the transport protocol routing.In the active tests of message counters, checksums, cycle times, and update bits, the corresponding signals of the test system are manipulated in such a way that an error entry is generated in the diagnostics error memory. The user then verifies the corresponding error entry via a diag-nostic tool.In addition to the predefined test sequences, interactive tests enable the engineer to select certain messages during operation and to manipulate them by interrupting the sending of messages or by automatic signal value changes, for example. The manipulations are made via control panels that are automatically available for each ECU.

Life tests in the vehicleA special use case that is just as important as tests at the workstation are the tests with signal manipulations direct-ly in the vehicle. CANoe is connected between the vehicle and the ECU to be tested and acts as a gateway by routing the data through one-to-one in both directions. The tester

Figure 3: COMSpector screenshot with active gateway tests

05

Network Tests for Everyone, 08/2016

that the Vector tool would work without expensive addi-tional hardware. In particular, Daimler wanted, if possible, to replace the very expensive and complex Hardware-in-the-Loop systems used for network testing, as the effort and costs associated with these were often disproportion-ate to the results achieved.In a given time period and with the available personnel, COMSpector allows the creation and run through of many more ECM and CPC test configurations. This increases the test depth and, above all, benefits product quality. For ex-ample, while the engineers previously had to determine the cycle times for each message type, a passive COMSpector test configuration now covers that completely for all mes-sages simultaneously. In general, numerous manual adap-tations were previously required, and complicated cases meant that constant changes in the CAPL code were inevi-table. Only well-trained testers were able to accomplish these tasks but even then only with significant preparation work. Today, even employees who are not proven test or net-work specialists can perform may simple testing tasks.Daimler is now using COMSpector worldwide, including in the USA and in China. The master distribution list of the Daimler employee in charge contains approximately 20 recipients. Recently, an American colleague reiterated that he found the tool very helpful for his work. So it is no won-der that the development department at Sindelfingen, Germany, is receiving a lot of requests for extending the tool. Ideas and suggestions for the next COMSpector ver-sion are being collected and reviewed there.

Development continuesThe network test generator COMSpector has become an indispensable component for testing worldwide at Daimler. It allows test configurations for passive and active network tests in the powertrain to be created quickly and without special expertise. In particular, the tool makes the laborious and time-consuming programming and adapting of test scripts unnecessary. The employees can concentrate their efforts instead on the actual development work and on quality improvements. It is now mandatory for many ECU and routing tests to be performed with COMSpector, according to the Daimler release procedure. The tester can easily store the finished report supplied by CANoe as proof of compliance with this requirement.Daimler and Vector are currently working on integrating test capabilities for CAN FD and Automotive Ethernet in COMSpector.

B.Eng. Viola Misch studied electrical engineering with emphasis on vehicle electron-ics and mechatronic systems at the German Ravensburg Univer-sity of Cooperative Education. Since 2014 she has worked in pas-senger car development at Daimler AG. She is active in the field of powertrain topology and networking and oversees the COM-Spector tool on the Daimler side.

Translation of a German publication in Elektronik automo-tive, issue September 2016

Image rights: Figures 0-3: Daimler AG

Dipl.-Ing. Katja Hahmannstudied electrical engineering at the German Chemnitz University of Technology. She joined Vector Informatik in Stuttgart in 1997 where she serves as Group Leader of customer-specific CANoe applications and test systems in the Networks and Distributed Systems product line.