Embed Size (px)
Citation preview
www.tulipp.eu
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 688403
H2020
TULIPP - Towards Ubiquitous Low-power Image Processing Platforms
Overall Project PresentationTobias Kalb
Ruhr-University [email protected]
2 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
Introduction
Overview• TULIPP: Towards Ubiquitous Low-power Image Processing Platforms• Horizon 2020: EU Framework Programme for Research and Innovation,
Grant Agreement 688403• Duration: February 2016 – January 2019 (36 Months)• Consortium: 8 Partners
• Thales S.A., France (Coordinator)• Ruhr-University Bochum (RUB), Germany• Sundance Multiprocessor Technology Ltd., UK• HIPPEROS S.A., Belgium• Fraunhofer IOSB, Germany• Norwegian University of Science and Technology (NTNU), Norway• Synective Labs AB, Sweden• Efficient Innovation, France
• Webpage: www.tulipp.eu
3 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
Introduction
ProblemA lot of modern industrial systems rely on embedded vision-based applications• Increasing number of sensors, amount of data, data bandwidth etc.• Demand for high-performance, low-power and heterogeneous computing• Build into small embedded devices where size, weight and power matter• Embedded systems for image processing are highly specialized systems
https://commons.wikimedia.org/wiki/File:IBM_Blue_Gene_P_supercomputer.jpg
http://edablog.com/2013/08/08/blackfin-bf609-pvp/
4 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
Introduction
General Goals of TULIPPTULIPP aims to push forward a reference platform for embedded image processing applications in order to• Define implementation rules for
vision-based applications• Provide guidelines for guaranteed
high performance and low power• Reduce development time and costs
With the focus on embedded vision-based applications TULIPP will• Set up an ecosystem• Work closely with standardization organizations• Propose new standards derived from the reference platform
5 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
TULIPP Reference Platform
Concepts of the TULIPP Reference PlatformThe goal is not to develop a fixed, generic embedded computing platform.Instead, the TULIPP project aims to define• Implementation rules and• Interfaces
This allows for a flexible platform for vision-based applications featuring• Hardware architecture:
a scalable low-power board• Low-power, real-time
operating system and image processing libraries
• Productivity-enhancing toolchain
6 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
Hardware Architecture
Hardware Architecture• Heterogeneous Systems-on-Chip (SoCs)
• Combinations of CPU, GPU and FPGA• Dedicated HW accelerators• Also dedicated real-times cores
• Appropriate hardware system• Selection of processing elements• Interconnections of on-chip components• Interconnections of several SoCs
• Support advanced features• Switch-off mechanisms• Dynamic Voltage and
Frequency Scaling (DVFS)• Dynamic Partial Reconfiguration (DPR)
http://www
.trenz-elec
tronic.de
7 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
Real-Time Operating System
Operating System and Low-Level Libraries• Real-time operating system with APIs to
• Support low power• Support image processing applications
• Run on instantiated processors• Support heterogeneous multi-core systems• Handle hardware resources (FPGAs)• Implementation of communication and synchronization
• Provide for the developer• Real-time guarantees• Reliability• Easy programmability
8 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
Toolchain
Toolchain• A TULIPP compliant platform may have different components from
different vendors• Expertise required for every vendor specific tool• Lack of expertise may prevent usage of fitting hardware components
• TULIPP toolchain is a set of Eclipse-based utilities to support the developer• STHEM - Supporting uTilities for Heterogeneous EMbedded
image processing• Wraps around, extends and connects existing vendor tools• Programmers use expert-written mechanisms for mapping and
optimizing applications
9 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
Use Cases
Use Cases• Instantiation of reference platform is use case driven• Verification of reference platform with image processing applications
• Medical imaging • Automotive imaging (ADAS)• Robotic imaging (UAV)
• Different fields of embedded applications, but similar constraints• Performance• Power consumption• Size, volume and cost
• Also often real-time constraints
10 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
https://commons.wikimedia.org/wiki/File:113abcd_Medical_Imaging_Techniques.jpg
Medical Imaging
Medical Imaging• Demand for high performance yet small devices
• Requires processing of large amount of data
• Mobile imaging equipment replacinghigh-end infrastructure devices
• Demand for real-time imaging• Fast processing of image data
during surgery• On mobile device
• TULIPP X-Ray use case aims to provide• Reduction of radiation dose of sensors• More powerful image processing • Low power since heat and other
RF emission could disturb sensors
11 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
Automotive Imaging
Automotive Driver Assistance System (ADAS)• Intelligent cars need more and efficient embedded devices
Image processing required for• Driving safety• Pedestrian safety
• More active safety systems• Vehicle, pedestrian and object detection• Traffic sign and lane recognition• Night vision and surround view• Driver monitoring
• TULIPP ADAS use case aims to provide• Real-time, low latency high-performance
image processing• Reliability and robustness
https://www.asdreports.com/news-10595/key-players-advanced-driver-assistance-systems-adas-market-north-america-20152019
http://www.rcs.ei.tum.de/forschung/driver-assistance/
12 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
Autonomous Unmanned Aerial Vehicles (UAVs)• UAVs more common for different applications
• Surveillance, search and rescue, logistics and research
• On-board real-time processing is key technology• Efficient and reliable automatic
collision avoidance needed• Opposing constraints
• The TULIPP UAV use case aims to provide• Optimized performance-to-weight and
power-consumption-to-weight figures• On-board stereo vision & depth estimation• Real-time and automatic detection of
obstacles for collision avoidance https://irevolutions.org/2014/03/24/launching-a-search-and-rescue-challenge-for-drone-uav-pilots/
Autonomous UAV Imaging
13 | 13
The
info
rmati
on c
onta
ined
in th
is d
ocum
ent a
nd a
ny a
ttach
men
ts a
re th
e pr
oper
ty o
f Tul
ipp
cons
ortiu
m. Y
ou a
re h
ereb
y no
tified
that
any
revi
ew, d
issem
inati
on,
dist
ributi
on, c
opyi
ng o
r oth
erw
ise u
se o
f thi
s doc
umen
t mus
t be
done
in a
ccor
danc
e w
ith th
e CA
of t
he p
roje
ct (T
RT/D
J/AC
U).
Tem
plat
e ve
rsio
n 1
.0
20.0
7.20
16 /
Sam
os X
VI /
Tobi
as K
alb
H202
0
Advisory Board
Everyone is welcome to join and participate actively!• Influence the technical directions in the project for your own exploitations• Use the TULIPP eco-system and partners to network with potential suppliers and customers• Free attendance to any TULIPP events, like workshops, tutorials and meetings• A free TULIPP ‘SoA’ development kit at the end of the project that will be provided after participating in
the final workshop. This will include a hardware board with the operating system and a development toolchain together with a sample image processing application
• Access to TULIPP Partners for help in using the TULIPP platform for your own products/developments• You get listed, with short profile and logo, on the TULIPP web-site – www.TULIPP.eu/abm
Information : http://tulipp.eu/advisory-board-letter-information
Forum: http://support.tulipp.eu
E-Mail: [email protected]
Web: www.tulipp.eu
Thank You!Questions?
