Trento, March 13, 2008 Andrea Facco Info-telematics for Public Mobility and Logistics HMI Group - Speech Interfaces Centro Ricerche Fiat

Trento, March 13, 2008

Andrea Facco

Info-telematics for Public Mobility and Logistics

HMI Group - Speech Interfaces

Centro Ricerche Fiat

CRF Overview 2March 13, 2008Public

Il presente documento contiene informazioni di proprietà di CRF. Il documento e/o le informazioni in esso contenute non possono essere usate, riprodotte, comunicate a terzi, in tutto o in parte, senza il consenso scritto di CRF.

CRF the History

Centro Ricerche Fiat S.C.p.A. was constituted in 1976 as reference centre for the activities of innovation and research of Fiat Group. Then it became consortium (in the Italian legal form of “Società Consortile per Azioni”) participated by the Group’s Companies. Besides the headquarters in Orbassano (Torino), three branches Valenzano (Ba), Foggia, Trento and a controlled company (Udine) were set.

In approximately 30 years, CRF achieved the threshold of 750 partners from the Industry and 150 from the Academia, all around the world. This stable and tough network reinforces the global strategies of innovation, allows the execution of specific operative activities at local level, ensures the creation of competences and the endless monitoring of the business environment.

CRF in the 80ties



CENTRO RICERCHE FIATFactory of ideas for the competitiveness

Mission:

Centro Ricerche Fiat aims to use innovation as strategic lever to the business of Fiat Group and to give value to the results of its activities by the promotion, the development and the transfer of innovative contents which give competitiveness and distinctiveness to the product.

Date of establishment: 1976

Employees: 870

Average age: 40

Headquarters:Orbassano (TO)

Branches: Trento,Valenzano (BA), Catania,and Foggia



LEADERSHIP IN R&D: AREAS

Vehicle

Manufacturing and materials

TelematicsEnergy

Powertrain Research & Technologies



VEHICLES

Modular vehicle’s structures

Hybrid and Hydrogen vehicle architectures

Integrated chassis control

Electro /electronic architectures for the distribution of signal and power

Electromagnetic compatibility

Ergonomy

Methodologies and instruments to define and reach the product development objectives

Objective: to ensure the product competitiveness



POWERTRAIN RESEARCH & TECHNOLOGIES

Diesel Common Rail Injection Technology

Innovative Diesel technologies for the NOx reduction

Valve Electronic control Systems

Injection and after treatment systems for CNG engines

Dual Dry Clutch Transmission Technology

Hybrid and alternative propulsion systems

Electronic control innovative systems

New methodologies for the engine Design and analysis

Objective: to develop and use innovative technologies to improve the engine performance and to reduce the emissions and the fuel consumption of

engines and vehicles.



MANUFACTURING AND MATERIALS

Materials and technologies for interiors and for low weight vehicles

Materials and systems for the engine emissions abatment

End of life vehicle methodologies and treatment

Flexible and low cost production technologies and high efficiency machining

Cost engineering and factory logistics

Process diagnosys and on-line monitoring systems

Eco-efficient production processes and factory

Integrated systems based on micro-optics, micromechanics and functional materials

Objective: contributing to the technological development of new products taking into account the complete life cycle



telematics, mobility and intermodality systems

preventive safety systems and solutions

multimedia and HMI systems

High efficiency cogeneration systems

Telematics

Energy

Objective: promoting safety and improving people’s mobility

Objective: reducing the environmental impact

TELEMATICS AND ENERGY



COOPERATIONS WITHIN

THE EU FRAMEWORK PROGRAMMES

• BMW

• DC

• FORD

• JAGUAR

• OPEL

• PORSCHE

More than 700

partners in

Europe

• PSA

• RENAULT

• SAAB

• VW

• VOLVO

Several cooperative projects carried on with the major EU car manufacturers



CRF in numbers

More than 800 high qualified people

More than 760 inventions protected by more than 2000 patents: in 2006 CRF was the first italian companyin number of EU patent requests

Leadership in the EU Framework Programmes: 487 funded projects from 1989

More than 80 fellowships yearly

Strong emphasis on transfer of results

760 inventions

870 people

2000 patents

487 EU projects

80 fellowships



CRF and Elasis: Branches

Orbassano (TO)Promotes, develops and transfers the innovative contents which give competitiveness and distinctiveness to the product

CRF unità locale di TRENTOTechnical and technological solutions for special and low volumes vehicles;info-telematics for urban mobility and logistics1) Info-telematic Department2) Virtual manufacturing Department

CRP- Amaro (UD)Optics, plastics, micro and nanotechnologies

CRF unità locale di Foggia Diesels and methane engines

CRF Sede secondaria di Valenzano (BA)Injection systems for internal combustion engines

ELASIS sede secondaria di LecceAgricultural and building machines

Pomigliano (NA)Promotes the innovation and its application in projects, products and processes



TRENTO BRANCH

About 25 people in 2 departments:

1. Info-telematic department: Info-telematics for mobility management of public (single and collective)

transport systems Development of software applications based on a new on-board

telematic platform for automotive Technologies for logistics management Wireless sensor networks and “ad hoc” networks Electromagnetic Compatibility Simulation Analysis Preventive Safety for Vulnerable Road Users (Urban Mobility) Technology for Voice Recognition Systems

2. Virtual Manufacturing department



Fields of research:

1. DIALOGUE STRUCTURE DEFINITION

2. DATABASE COLLECTION, TRANSCRIPTION AND LABELLING

3. TEST AND VALIDATION OF SPEECH SYSTEMS

4. SOFTWARE DEVELOPING OF HIGH-LEVEL SPEECH APPLICATIONS

5. SIMULATION IN LABORATORY OF IN-CAR SPEECH INTERACTION

HMI GROUP: SPEECH ACTIVITIES

Facilities:

1. SPEECH TECHNOLOGIES LABORATORY of Orbassano



HMI GROUP: CRF Speech Technologies Laboratory

Main technical features

• Soundproof studio and professional equipments for the recording, playing and processing of audio signals both in laboratory and on moving vehicles

• Proprietary methodologies for the quantitative and objective analysis of the performances of speech recognition and voice synthesis systems, and particularly of the speech recognition performances in noisy environment.



HMI GROUP: CRF Speech Technologies Laboratory

Main applications

• evaluation tests of the performances of speech recognition and synthesis systems in real and simulated environment by means of objective and qualitative measurement of the following parameters:

• speech recognition quality;

• impact of the environmental noise on the recognition of voice commands;

• quality of the voice of the text-to-speech synthesis;

• usability of the systems;

• effectiveness of the grammars (and/or the lexicon associated to the grammars).

• testing of new technologies and new forms of dialogue;

• validation and comparative tests of systems already available or under development;

• creation of audio databases (speech and background noise) for the testing and tuning of speech recognition systems by means of professional recording and playing of audio signals;

• design, development, tune up and validation of multilingual voice interaction systems inclusive of definition and localization of speech dialogues in multiple languages.



1. FIAT Blue&MeTM Telematic Platform Born from the cooperation among FIAT, CRF and Microsoft Windows CE Operating System (customized in WMfA) Microsoft .NET Framework for developing high level speech applications (C#) HW platform built by Magneti Marelli NUANCE recognition and synthesis engines

3. Conversational Speech Demo Application Held by Magneti Marelli with the collaboration of IBM and CRF VxWorks operating system Developed on a high level Magneti Marelli radio system (named RT4, actually

mounted on Maserati cars) Integration of the IBM Free-Form command to manage Navigation, Music, Radio and

Hands-free Phone Dialing

2. AIDE European Project (Adaptive Integrated Driver-Vehicle Interface) Vocal interaction for Hands-free Phone Dialing, reading SMS, selecting Mp3 by artist,

song and album title, etc. NUANCE recognition and synthesis engines

CRF SPEECH PROJECTS



Blue&MeTM Telematic Platform

Hardware Features:

1. USB Port For Mp3 players, iPods, USB sticks.

2. Bluetooth connection For mobiles, PDA, Pocket PC (Hands-free feature)

3. GPS Antenna For the navigation system (based on pictograms)

4. GSM/GPRS Antenna Integrated SIM Card Slot

5. Radio speakers connection

6. Steering wheel buttons connection

7. CAN BUS connection

Blue&MeTM versions:

1. Blue&MeTM

2. Blue&MeTM NAV

3. Blue&MeTM MAP (coming soon…)



FIAT• 500• Grande Punto• Nuova Croma

Car models

LANCIA• Ypsilon• Nuova Delta

ALFA ROMEO• 159 (159 SW)• Brera• Spider• Junior

IVECO• Daily

• …and will be available on all new FIAT cars

Blue&MeTM Telematic Platform



Blue&MeTM Speech Characteristics

Application currently available on Blue&MeTM

1. Hands-free Phone Dialing Call name stored in the user Phonebook Dial number Recall and callback

2. Media Player Play, Stop, next and previous Artist, Album, Genre, Playlist selection (manual interaction is needed to choose the

specific item when the list is displayed) Shuffle and continuous play

3. SMS Reader SMS list, next and previous

4. On Board Navigation (Blue&MeTM NAV) Mixed mode destination entry (part by wheel buttons and then using voice) Personal and last destinations Route simulation Alternative route Show GPS/Car position



Languages currently available on Blue&MeTM:

1. ITALIAN

2. UK ENGLISH

3. FRENCH

4. GERMAN

5. SPANISH

6. PORTUGESE

7. BRASILIAN PORTUGESE

8. DUTCH

9. POLISH

10. TURKISH

Blue&MeTM Speech Characteristics



Objective:

To develop HMI concepts required for safe and efficient integration of ADAS and IVIS systems into the driving environment

Duration: 4 years Started: March 04

29 partners (~50/50 industry-academia division), coordinated by Volvo Technology.

Part of the EUCAR Integrated Safety Program

AIDE core group: VTEC (coordinator), BMW, Bosch, CRF, ICCS, JRC, PSA and TNO

The AIDE system monitors the Driver, the Vehicle and the Environment (DVE) state and adapts the driver-vehicle interface accordingly.

Examples:

• Lock-out/postpone non-critical information on demanding situations

• Adapt the timing/intensity of safety warnings (e.g. warn earlier when the driver is inattentive)

AIDE - Adaptive Integrated Driver-Vehicle Interface



Adaptive IntegratedDriver-vehicle Interface

In-vehicle information

systems (IVIS)

Driver / Vehicle /Environmentmonitoring

Advanced driver assistance systems

(ADAS)

visual haptic auditory

Nomad devices

AIDE demonstratorsMain Innovation items

1. Adaptive Driver Interface

2. Personalization

3. Real-time monitoring driver/car/environment

4. All traffic scenarios functionality

5. Common architecture

6. Dynamic information management

7. New input/output devices

8. Integrated management of all messages and i/o devices within the car interior





Speech Applications available:

1. Hands-free Phone Dialing Call name stored in the user Phonebook Dial number Read Phonebook (and call) Store voice tags Redial

2. Media Player Selection of music by artist’s name, album or song’s title or genre. Bilingual transcription of Mp3’s main tags (user language + English so far) Shuffle and continuous play

3. SMS Reader SMS list User controlled continuous reading of SMS

4. Supervised Adaptation Training of the acoustic models to adapt to the speaker’s voice



Adaptive HMI with fully reconfigurable instrument cluster and haptic input device

Wireless connection with Nomadic devices

Advanced speech interface for telephone, phone book, SMS reading, MP3 player

Read, please

AIDE Demonstrator vehicle



Conversational Demo Application

Application developed by Magneti Marelli together with CRF and IBM

1. Integration of the IBM “Free-form” commands to allow user to interact with the system using natural sentences (e.g. “I’d like to listen to some rock music please”)

2. Domains on which the Statistical Language Models have been build: Hands-free dialing, Music selection, Radio and Navigation

HW/SW main features:

1. IBM Speech Recognition Engine

2. IBM Speech Synthesis Engine

3. Magneti Marelli high level device (“RT4”) Radio CD and Mp3 player Integrated Hard Disk Color Display 7’’ Navigation Maps GSM module




Application available (FREE-FORM commands only in the submenus):

1. Hands-free Phone Dialing Call name and dial number Call back and Redial

2. Media Player Selection of music by artist’s name, album or song’s title or genre. Selection of music both by artist’s name and album title (and other combinations) Generic play, stop and replay, shuffle and continuous play

3. Radio Station and Band selection Seek and Scan Presets selection and storing

4. Navigation Italy map Selection of the destination city Selection of the POI (both near the current position or near the destination) Managing of the zoom (both near the current position or near the destination)




Technical features:

1. Speaker independent

2. Mixed Mode architecture (Grammar based + Statistical Language Models) Grammar based for the main menu Free-Form commands for each subdomain (using SLMs)

3. Dialogue manager to help user in wrong recognition cases

4. Push-to-talk steering wheel button to start vocal interaction

5. Barge-in enabled

6. Far-talk microphone integrated in the vehicle

7. Fully integrated in the car radio system: Radio loudspeakers used for Text-to-Speech and Hands-free talks Far-talk microphone






THANK YOU!

CONTACTS

Andrea Facco: [email protected]

Marco Darin: [email protected]

Head of Info-Telematics Department of Trento Branch

Enrica Deregibus: [email protected]

Head of HMI Group and Speech Technology Laboratory (Orbassano)

Centro Ricerche Fiat S.C.p.A. CRF Trento Branch (c/o BIC)

Strada Torino, 50 via Dei Solteri, 38

10043 Orbassano (TO) 38100 Trento

Web site: www.crf.it

Documents

Trento, March 13, 2008 Andrea Facco Info-telematics for Public Mobility and Logistics HMI Group - Speech Interfaces Centro Ricerche Fiat