Upload
arthur-harris
View
35
Download
0
Tags:
Embed Size (px)
DESCRIPTION
In the Name of God, the Compassionate, the Merciful. Reconfigurable Processor for Energy-Scalable Computational Photography. Class presentation for the course Custom Implementation of DSP Systems based on the ISSCC 2013 paper [1] :. M. R. Azarbad - PowerPoint PPT Presentation
Citation preview
Reconfigurable Processor for Energy-Reconfigurable Processor for Energy-Scalable Computational Photography Scalable Computational Photography
M. R. Azarbad
Department of Electrical and Computer Engineering, University of Tehran, Tehran, Iran
In the Name of God,In the Name of God,the Compassionate, the Mercifulthe Compassionate, the Merciful
Class presentation for the course Custom Implementation of DSP Systems based on the ISSCC 2013 paper [1]:
All the materials are copy rights of their respective authors as listed in references
R. Rithe, P. Raina, N. Ickes, S. V. Tenneti, and A. P. Chandrakasan
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
OutlineOutline
IntroductionIntroduction
Basic ConceptsBasic Concepts
This WorkThis Work
Chip FeaturesChip Features
Conclusions Conclusions
ReferencesReferences
22
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
IntroductionIntroduction
33
Computational photography enable capture and synthesis of images Computational photography enable capture and synthesis of images that could not be captured with a traditional camera [1]that could not be captured with a traditional camera [1]
Non-linear filtering techniques like bilateral filtering in computational Non-linear filtering techniques like bilateral filtering in computational photographyphotography [1] [1]
These techniques have a wide range of applications, including High-These techniques have a wide range of applications, including High-Dynamic Range (HDR) imaging, Low-Light Enhanced (LLE) imaging Dynamic Range (HDR) imaging, Low-Light Enhanced (LLE) imaging and video enhancementand video enhancement [1] [1]
The The real-time processingreal-time processing necessitates fast hardware necessitates fast hardware implementationsimplementations [1] [1]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
Basic ConceptsBasic Concepts
44
Filtering in image Filtering in image processingprocessing
Function of the values of the Function of the values of the input image in a small input image in a small neighborhood [2]neighborhood [2]
Gaussian low-pass filtering Gaussian low-pass filtering computes a weighted computes a weighted average of pixel values in the average of pixel values in the neighborhood (the weights neighborhood (the weights decrease with distance from decrease with distance from the neighborhood center) [2]the neighborhood center) [2] Fig. 1: The mechanism of spatial filtering [3]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
Basic ConceptsBasic Concepts
55
Images typically vary slowly over space, so near pixels are likely to Images typically vary slowly over space, so near pixels are likely to have similar values [2]have similar values [2]
Appropriate to average them together [2]Appropriate to average them together [2]
The noise values that corrupt these nearby pixels are mutually less The noise values that corrupt these nearby pixels are mutually less correlated than the signal values [2]correlated than the signal values [2]
Noise is averaged away while signal is preserved [2]Noise is averaged away while signal is preserved [2]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
Basic ConceptsBasic Concepts
66
How can we prevent averaging across edges, while still averaging within smooth regions?How can we prevent averaging across edges, while still averaging within smooth regions?
Iteration may raise issues of stability and, depending on the Iteration may raise issues of stability and, depending on the computational architecture, efficiency [2]computational architecture, efficiency [2]
bilateral filtering [2]bilateral filtering [2]
bilateral filteringbilateral filtering
The idea is to do in the range of an image (vs. traditional filters domain) The idea is to do in the range of an image (vs. traditional filters domain) [2][2]
Range filters are nonlinear (weight dependency on image intensity or Range filters are nonlinear (weight dependency on image intensity or color) [2]color) [2]
Combination of range and domain filtering [2]Combination of range and domain filtering [2]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
Basic ConceptsBasic Concepts
77
bilateral gridbilateral grid
The bilateral grid is a three dimensional array [3]The bilateral grid is a three dimensional array [3]
The first two dimensions (x, y) correspond to 2D position in the image The first two dimensions (x, y) correspond to 2D position in the image plane and form the spatial domain [3]plane and form the spatial domain [3]
The third dimension z corresponds to a reference range(typically, the The third dimension z corresponds to a reference range(typically, the range axis is image intensity) [3]range axis is image intensity) [3]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
This WorkThis Work
88
Implementation of bilateral filtering using a reconfigurable grid [1]Implementation of bilateral filtering using a reconfigurable grid [1]
scheduling the filtering engine so that only two grid rows scheduling the filtering engine so that only two grid rows need to be stored at a time [1]need to be stored at a time [1]
Reduces the storage requirement to 21.5kB (vs. 65MB for a Reduces the storage requirement to 21.5kB (vs. 65MB for a 10Mpixel image) [1]10Mpixel image) [1]
The implementation is flexible to allow varying grid sizes for The implementation is flexible to allow varying grid sizes for energy/resolution scalable image processing [1]energy/resolution scalable image processing [1]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
This WorkThis Work
99
Computes the bilateral grid [1]Computes the bilateral grid [1] The input image is partitioned into blocks of size σs×σs [1]The input image is partitioned into blocks of size σs×σs [1]
Grid Assignment EngineGrid Assignment Engine
Fig. 2: The Grid Assignment Engine [1]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
This WorkThis Work
1010
Convolves the grid intensities and weights with a 3×3×3 Gaussian Convolves the grid intensities and weights with a 3×3×3 Gaussian kernel, which is equivalent to bilateral filtering in the image domain kernel, which is equivalent to bilateral filtering in the image domain [6], and returns the normalized intensity [1][6], and returns the normalized intensity [1]
Convolution EngineConvolution Engine
Fig. 3: The Concolution Engine [1] Fig. 4: The Interpolation Engine [1]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
This WorkThis Work
1111
Fig. 5: Outputs of HDR imaging, LLE imaging and Glare reduction[1]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
This WorkThis Work
1212
Fig. 6: Energy vs. performance trade-off for a 10Mpixel image and the frequency of operation for a range of voltages. Run times for different image sizes
at 98MHz, 0.9V [1]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
Chip FeatureChip Feature
1313
The grid processing tasks are scheduled to minimize local storage The grid processing tasks are scheduled to minimize local storage requirements and memory traffic [1]requirements and memory traffic [1]
Fig. 7: Die photo of the testchip [1]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
ConclusionsConclusions
1414
bilateral filtering using a reconfigurable grid, which reduces the storage bilateral filtering using a reconfigurable grid, which reduces the storage requirement to 21.5kB [1]requirement to 21.5kB [1]
The implementation is flexible to allow varying grid sizes for The implementation is flexible to allow varying grid sizes for energy/resolution scalable image processing [1]energy/resolution scalable image processing [1]
A 10Mpixel image in 771ms with 17.8mW power consumption while A 10Mpixel image in 771ms with 17.8mW power consumption while operating at 98MHz, 0.9V [1]operating at 98MHz, 0.9V [1]
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation
ReferencesReferences
1515
[1] R. Rithe, P. Raina, N. Ickes, S. V. Tenneti, and A. P. Chandrakasan, “Reconfigurable Processor for Energy-Scalable Computational Photography,” ISSCC Dig. Tech. Papers, 2013.
[2] C. Tomasi, R. Manduchi, “Bilateral Filtering for Gray and Color Images,” IEEE International Conf. Computer Vision, pp. 839-846, 1998.
[3] R.C. Gonzalez, and R.E. Woods, “ Digital image processing,” 2nd edition, Prentice-Hall, 2002.
[4] F. Durand, J. Dorsey, “Fast Bilateral Filtering for the Display of High- Dynamic-Range Images,” ACM Trans. on Graphics, vol. 21, no. 3, pp. 257-266, 2002.
[5] G. Petschnigg, M. Agrawala, H. Hoppe, R. Szeliski, M. Cohen, K. Toyama, “Digital Photography with Flash and No-Flash Image Pairs,” ACM Trans. Graphics, vol. 23, no. 3, pp. 664-672, 2004.
[6] J. Oh, G. Kim, J. Park, I. Hong, H.-J. Yoo, “A 320mW 342GOPS Real-Time Moving Object Recognition Processor for HD 720p Video Streams,” ISSCC Dig. Tech. Papers, pp. 220-221, 2012.
[7] J. Chen, S. Paris, F. Durand, “Real Time Edge-Aware Image Processing With the Bilateral Grid,” ACM Trans. Graphics, vol. 26, no. 3, article 103, 2007.
[8] http://en.wikipedia.org
M. R. AzarbadM. R. Azarbad Custom Implementation of DSP Systems Course PresentationCustom Implementation of DSP Systems Course Presentation 1616
Thank Thank You!You!