Embed Size (px)
DESCRIPTION
Any good IP core will conform to the specification it claims to support. But for video and image IP, matching a standard is not the same as producing the degree of image quality required for demanding applications. High quality output requires clever, high-quality development in all the crucial steps of a core's operation. In this talk from SoCIP China 2011, we look at key quality factors using JPEG 2000 compression and video deinterlacing as examples. Visit http://www.cast-inc.com/ip-cores/images/jpeg2k-e/ or http://www.cast-inc.com/ip-cores/video/vdint-ma/ to learn more.
Citation preview
Pretty as a Picture:Assessing Quality in
Image/Video IP
Nikos ZervasVice President, CAST, Inc.
2 SoCIP 2011
Quality can be crucial, especially in applications with high-resolution displays or analytics.
Customers automatically judge by the quality, they see, regardless of the end application.
Video and Image Quality
3
Compression and Quality Myths
Supporting an industry support defines a core’s quality.
A particular set of features automatically defines quality.
I can judge quality with vendor-provided tests.
RealitiesStandards typically describe a decoder, but quality is mainly determined by the encoder.
Within conformance to a standard, developers have great freedom in choosing algorithms and implementation details.
Image or video tests can be tuned to do well with particular encoders/decoders, and may not reflect your media content.
SoCIP 2011
4
Compression and Quality
Standards do not mandate quality.
There are potential pitfalls with every compression technology.
MPEG2 JPEG JPEG 2000H.264LJPEGJPEG-LS …
Let’s look at one: JPEG 2000.
SoCIP 2011
5
JPEG 2000 — Pitfalls
JPEG 2000 offers the best lossless and great lossy compression efficiency.
True only when all encoding switches are turned off.
JPEG 2000 offers the best lossless and great lossy compression.
True for Variable Bit Rate (VBR).
But under Constant Bit Rate (CBR), quality depends on Rate Control efficiency.
JPEG 2000 offers Region of Interest (ROI) for selective quality.
True, but the implementation of ROI may make ROI unusable.
JPEG 2000 minimizes blocking artifacts. True, but tiling artifacts can be present when you get more than one tile per image. SoCIP 2011
6
JPEG 2000 — Encoding SwitchesParallel Mode and Bypass
Simplify hardware design.
Destroy JPEG 2000 coding Efficiency.
SoCIP 2011
Image Resolution BppSize in Kbytes
Diff%No Switches Parallel & Bypass
bike 2048x2560 14 5,613 7,005 24.8
Boat 512x768 8 107 131 22.4
0 0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.2525
30
35
40
45
50
55Rate-Distortion for "Boat" Image
True J2k Parallel & Bypass JPEG
Rate (BPP)
Dis
torti
on (P
SNR)
-3dB
-1dB
JPEG 2000 — Region of Interest
slide 7 SoCIP 2011
Requires higher quality for the region of interest (ROI).
But you still care about the quality of the background.
Some encoders only do this: But others can do this:Example: Aerial photography with ROI compressed at 50:1
JPEG 2000 — Rate Control
Not defined by the standard, but needed for many applications.
Determines the quality you will get at a specific rate (compression ratio).
Its accuracy defines the buffering/storage requirements and latency on your system.
SoCIP 2011slide 8
Your SIP provider should be able to provide comparative data with respect to the de facto reference software (kakadu)
9
Image Processing and Quality
There are no standards for most image processing algorithms.
There are potential pitfalls with every image processing algorithm.
Image Scaling White BalanceBad Pixel Correction De-interlacing
…
Let’s look at one: De-interlacing.
SoCIP 2011
10
De-interlacing – Pitfalls
Quality is preserved by minimizing motion artifacts
True, but detail needs to be preserved too.
Motion detection is the best de-interlacing method
Not really. Motion detection algorithms can be easily fooled by small motion, image noise etc
SoCIP 2011
DSP techniques provide great quaiity
There is no continuous time signal in video, so DSP techniques just failTreats all pixels equally
11
De-interlacing - Quality
Detail without Motion Artifacts is preserved when filtering adopts not only to motion (temporal) but also to spatial variations
e.g. edges should be treated differently than flat areas
Content Adaptive algorithms deliver the highest quality results
SoCIP 2011
DSP Content Adaptive
Motion Detection
12
Considerations for SIP Selection
How easy is it to integrate my image/video Semiconductor IP?
Streaming versus SoC bus interfaces.
External memory bandwidth and tolerance to memory latencies.
Run-time programmability.
System-level buffering requirements.
How much external processing does it require?
For example: some H.264 SIPs work as accelerators rather than complete stand-alone solutions.
SoCIP 2011
13
Example: Camera SoC
SoCIP 2011
2 – ??? Months to integrate, High risk
3 – 4 weeks to integrate, Low risk
Good
Pixel data streamed from sensor to output; Host controls the SoC.
Easy SoC bus and memory arbitration, low software complexity.
Bad
Pixel data transferred over SoC bus; Host controls the SoC.
Challenging SoC bus and memory arbitration, moderate complexity.
Ugly
Pixel data transferred over SoC bus; Host performs some processing and controls the SoC.
SoC bus and memory arbitration becomes difficult, High complexity.
14
Takeaways
Evaluate your video and image IP for quality before making a purchase decision.
Ask for software model or reference design, or ask your vendors to encode your own clips using your settings.
Compare results using your perception, reference implementations, and stream analysis tools.
Consider integration complexity.Cores that are difficult to integrate can end up costing you several times more than the core license itself.
Failures in system-level integration can delay your product development, cripple your product and make it non-competitive.
SoCIP 2011
