Error-Aware Video Encoding to Extend EnergyQoS Tradeoﬀs ...kyoungwl/forge/ea/tecs/outline.pdf · Conference on Distributed and Parallel Embedded Systems ... para f2 f3 f0 Error−Aware

Error-Aware Video Encoding to Extend EnergyQoS

Tradeoffs for Mobile Embedded Systems1

KYOUNGWOO LEE

University of California, Irvine

NIKIL DUTT


and

NALINI VENKATASUBRAMANIAN


(1) Problem domain –

(2) Specific motivation and problem –

(3) Proposal –

(4) Contributions –

Categories and Subject Descriptors: []: —;

General Terms:

Additional Key Words and Phrases:

1. INTRODUCTION

(1) Emerging video applications on mobile devices and problem domain—video applications are becoming more popular.—energy efficiency is a key property to consider for desinging mobile embedded

systems.

1This is an expanded version of a paper published in the Proceedings of the IFIP WorkingConference on Distributed and Parallel Embedded Systems (DIPES) 2008. The currentmanuscript extends the previous paper by exploring the design space for the partially protectedinstruction caches and proposing the heuristic algorithms to efficiently find the best configurationwith minimal vulnerability under the performance penalty.

Author’s address: K. Lee, N. Dutt, and N. Venkatasubramanian, Department of ComputerScience, University of California, Irvine, CA 92697.

Permission to make digital/hard copy of all or part of this material without fee for personalor classroom use provided that the copies are not made or distributed for profit or commercialadvantage, the ACM copyright/server notice, the title of the publication, and its date appear, andnotice is given that copying is by permission of the ACM, Inc. To copy otherwise, to republish,to post on servers, or to redistribute to lists requires prior specific permission and/or a fee.c© 20YY ACM ????-????/20YY/0?00-0001 $5.00

(2) Previous Work

—Energy-aware video applications for mobile embedded systems

—Still, energy-efficient approaches are required to prolong the life of mobiledevices.

(3) Problem Definition

—wireless network is not stable.

—error-resilient video encoding is important.

—some error-resilient video encoding is energy-efficient as well.

—what if network is stable (i.e., no errors) but error-resilient video encodingwill work as a normal or standard video encoder.

(4) Our approach – Error-Aware Video Encoding (EA-VE)

—Intentional error injection (e.g., frame dropping) can be an effective knob totradeoff QoS for the energy efficiency.

—QoS-feedbacked mechanism can maintain the video quality in EA-VE schemes.

(5) Results and contributions

—Save the huge amount of energy consumption at the minimal cost of the videoquality.

—Our approaches can extend the design space significantly to tradeoff the videoquality for the energy reduction or vice versa.

2. THE BACKGROUND

2.1 Energy/QoS-aware Video Encoding

—Overview of quality aware viceo encoding

—Overview of energy aware video encoding

VLC : Variable Length CodingQ : QuantizationDCT : Discrete Cosine TransformME : Motion Estimation

ME DCT Q VLC

A

B

C

DB

C

A

D Error Exploitation(e.g., error injection)

Energy/QoS−awareVideo Encoding(Section 2.1)

Error−Resilient

(Section 2.2)

Standard Video Encoder

Constraint

Our Proposal

Quality of Service(e.g., PSNR)

Energy−Efficiency

Error−Resilience

Quality Control(e.g., Resolution, IP Ratio,

and Quantization [15])

Power Management

Resilience Control

i

i

ii

i ii iii

i ii iii

i

ii

iii

(e.g., Voltage Scaling [14,25])

(e.g., Intra−MB Threshold [10])

Video Encoding

(e.g., 10% packet loss rate)

(e.g., Residual Power)

[10,14,15,25]: References

i,ii,iiiA,B,C,D : Knobs

Knob

: Constraints

Energy−EfficientVideo Encoder

Error−ResilientVideo Encoder

Error−AwareVideo Encoder

Fig. 1. Constraints and Knobs considered by Previous Approaches and Our Pro-posal

2.2 Error-Resilient Video Encoding

—Overview of robust video encoding techniques

—Promising Intra Refresh encoding techniques

2.3 Error-Aware Video Encoding: Our Proposal

—passive error-aware video encoding

—error-aware energy reduction at network

—active error exploitation – our proposal

3. OUR APPROACH

3.1 System Model

(1) Our system model—Mobile Video Conferencing

IIFDT

WNI

Tx

WNI

Rx IIIFDT

Dec

CPU

IFDT

AP : Access PointWAN : Wide Area Network

: Video Data Flow

Rx : Receiver: TransmitterTx

FDT : Frame Drop TypeEnc : EncoderDec : Decoder

CPU : Central Processing UnitWNI : Wireless Network Interface

wiredwiredWAN

Network

AP 1 AP 2

wireless wireless

Mobile 1 Mobile 2

Transmission Errors

CPU

Enc

Fig. 2. System Model (Mobile Video Conferencing) and Frame Drop Types I/II/IIIfor Active Error Exploitation

—CPU and WNI are dominant contributors to power consumption—Why one path from an encoder to a decoder

3.2 Fundamentals of Active Error Exploitation

(1) Intentional error injection—Network is unreliable—ER and EC are designed against network errors—Error injection such as frame skipping can occur everywhere—Error injection can affect the following components in terms of EC and QoS—We study an active error exploitation at the Encoder since it is the most

effective

(2) Motivated examples—Error-aware video encoding techniques can present larger tradeoff spaces

compared to a normal video encoder (GOP-15) and an original PBPAIR—GOP-15 is a base video encoder for comparison, which indicates 15 P-frames

between two I-Frames.—PBPAIR is a previously proposed error-resilient video encoding technique—EA-PBPAIR presents a novel knob, error-injection rate (EIR), which is able

to tradeoff the video quality and the energy consumption

Table I. Energy Consumption Category

Type Description

Enc EC Energy consumed by CPU to encode a video streamTx EC Energy consumed by WNI to transmit an encoded video streamSrc EC Enc EC + Tx EC

Dec EC Energy consumed by CPU to decode a received video streamRx EC Energy consumed by WNI to receive a video streamDst EC Dec EC + Rx EC

EC = Energy Consumption

Encoding Energy Saving (FOREMAN 30 frames)

30

35

40

45

50

0 1000 2000 3000

All Possible Frame Loss Patterns (10% EIR)

En

erg

y S

av

ing

Ra

te (

%)

(a) Energy Reduction for En-coding

Decoding Energy Saving (FOREMAN 30 frames)

0

5

10

15

20

0 1000 2000 3000


En

erg

y S

av

ing

Ra

te (

%)

(b) Energy Reduction for De-coding

PSNR Degradation (FOREMAN 30 frames)

0

2.5

5

7.5

10

0 1000 2000 3000


PS

NR

De

gra

da

tio

n R

ate

(%

)

(c) Quality Degradation

Fig. 3. Energy Consumption Saving and Quality Degradation of Error-Aware VideoEncoder (EA-PBPAIR) compared to a standard video encoder (GOP-15)

3.3 EA-VE: Error-Aware Video Encoding

Error-InjectedVideo Data

OriginalDataVideo

Error-AwareVideo Data

Feedback(e.g., Frame Dropping)

Error Controller Error-ResilientVideo Encoder

(e.g., PBPAIR)Parameters

Error-Injection Unit Error-Canceling Unit

Constraints(e.g., quality requirement)

(e.g., quality feedback,packet loss rate)

(e.g., Error Rate,Intra Threshold)

Knob(e.g., error injection rate)

Error-Aware Video Encoder

Fig. 4. Error-Aware Video Encoder consists of Error-Injection Unit and Error-Canceling Unit.

(1) Introduction and wrap-up—Intentional error-injection can help VCS save the energy consumption

(2) Two-steps error-aware video encoder—Error-aware video encoding technique we present is composed of two units

such as error-injection unit and error-canceling unit—We present two simple approaches, which exploit the error-tolerance of video

data to increase the energy reduction for mobile video applications—(i) Error Controller

—Error-injection unit preprocesses the parameters for the following video en-coder, and inserts errors purposely to tradeoff between energy and qualityof service

—(ii) Error-Resilient Video Encoder—Error-canceling unit can be a normal video encoder or an error-resilient

video encoder.

—They reduce the effects of error-injection thanks to the nature of error-tolerance of video data and/or thanks to the error-resilience.

(3) In conclusion, EA-PBPAIR can save the energy consumption

—(i)—(ii)—(iii)—(iv)

3.3.1 Frame Dropping

(1) Simple error injection scheme is to skip the frames periodically

—Intelligent frame skipping can increase the quality—Different strategy from general frame skipping technique—Any frames can be dropped in EA-VE—For our approaches, we use “Frame Dropping Technique” at the error-injection

unit, and “Error Resilitne Video Encoder” at the error-canceling unit

(2) PFD

(3) MDFD

3.3.2 Error Canceling

(1) PBPAIR is energy efficient

—PBPAIR is a video encoding technique which is not only error-resilient butalso energy-efficient.

—PBPAIR has two parameters such as Intra Th and PLR (Packet Loss Rate)

(2) GOP

(3) PGOP

3.4 Adaptive EA-VE

QcQc

f1f2

f3

f0Original Video Data

f3

f0f2

Error−Injected Video Data

Increase EIR Decrease EIR

QfQc >

��

��

��

��

Lossy Video Dataf0

f2

EIRI

Qf

��

��

��

��

��

��

f3

f0f2

Error−Aware Video Data

para 2para 1 PFD

: input (constraint and parameter): feedback: control: video data

IEIR I

Error Injection (e.g. PFD)

Update EIR

NO

YES

Adaptive EIR

BEGIN

Error Exploiting Video Encoder

Error Controller

Dec

oder

f1 is dropped

f3 is lost

Error−Resilient Video Encoder

Lossy Networkpara = EIR + PLR + AER1 PLR

AER

: Periodic Frame Dropping

fn : n th video frame

EIR = EIR

CONSTRAINT)

INJECTION RATE)(Initial ERROR

(QUALITY

(QUALITYFEEDBACK)

(ADJUSTEDERROR RATE)

(PACKETLOSS RATE)

para : encoding parameter

Fig. 5. Flowchart of Error Controller for Adaptive EA-VE

(1) We introduce the EIR as a new knob to tradeoff energy and QoS—It opens and expands the operating points, which are newly discovered.

(2) First approach—This approach is to inject errors by dropping frames given an EIR—The sum of an actual network PLR and EIR is an input as Para1 for PBPAIR—For instance, the actual PLR is 10%, and EIR is 5%, then the parameter

PLR to PBPAIR is 15%

(3) Second approach—To adapt EA-PBPAIR for meeting QoS requirement,—EIR can be adjusted for dynamic network status—Change EIR based on the quality feedback from the decoding

(4) Third approach

—AER (Adjusted Error Rate) can be updated to increase the energy saving orto increase the video quality.

4. EXPERIMENTAL SETUP

Packet TrafficFrame Size

Analysis 1 Analysis 2

WNI Energy for Transmit

Analysis 3

CPU Energy for Decoding

Video Stream

CPU Power Parameters PLR

NS2 Simulator

Mobile 1

Encoder Tx

PrototypeSystem

Mobile 2

Decoder

SystemPrototype

Exe Time

Wireless Network Wireless Network

Quality Constraint

Encoding Parameters

Frame Dropping Policy

Error Injection Rate

Frame Dropping TrafficGenerator

Frame-Dropped Pattern

CPU Energy for Encoding(Enc EC)

WNI Energy for Receive(Tx EC)

(Rx EC)Video Quality(Dec EC)

(PSNR)

WNI Power Parameters

Network Topology

Packet LossesArrival Time

Wired NetworkAP 1 AP 2

: Control Flowof Simulation

: Video Data Flow

simulation output

simulation input

operation

temporary data

Rx

Fig. 6. Experimental Framework - System Prototype + NS2 Simulator

(1) Experimental Framework—Integration of System Prototype and NS2 using Perl

(2) System Prototype

(3) Power Values and Parameters for CPU and WNI

(4) Applications

(5) Test sequences such as Akiyo, Foreman, and Coastguard

(6) NS2 Simulator

Table II. Power Parameters

CPU WNI

Power Mode Active Idle Sleep Transmit Receive Idle Sleep

Power (W) 0.411 0.121 0.001 1.425 0.925 0.80 0.045

Transition (msec) 1.00 0.75

Energy Consumption at Source

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Enc Tx Enc + Tx

No

rma

lize

d R

ate

of

EA

-PB

PA

IR t

o G

OP

-8

EA-PBPAIR GOP-8

(a) Energy Consumption at the Source

Energy Consumption and QoS at Destination

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Dec Rx Dec + Rx Quality

No

rma

lize

d R

ate

of

EA

-PB

PA

IR t

o G

OP

-8

EA-PBPAIR GOP-8

(b) Energy Consumption at the Destination

Fig. 7. Energy Reduction and Quality Degradation of EA-PBPAIR compared toGOP-8 (EIR = 10%, PLR = 5%, FOREMAN 300 frames)

Energy Consumption and Quality

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Enc Tx Enc + Tx Dec Rx Dec + Rx Quality

No

rma

lize

d R

ate

of

EA

-PB

PA

IR t

o G

OP

-15

EA-PBPAIR GOP-15

(a) PLR = 0%


0

0.2

0.4

0.6

0.8

1

Enc EC Tx EC Enc EC +

Tx EC

Dec EC Rx EC Dec EC +

Rx EC

Quality in

PSNR

No

rma

lize

d R

ate

of

EA

-PB

PA

IR t

o G

OP

-3

GOP-3 EA-PBPAIR

(b) PLR = 10%

Fig. 8. Energy Reduction and Quality Degradation of EA-PBPAIR compared toGOP-K (EIR = 10%, FOREMAN 300 frames)

5. EXPERIMENTAL RESULTS

5.1 Energy Reduction from Active Error Exploitation

—A set of experiments to show the effects of our error-exploiting on energy reduc-tion without significant quality loss

(1) (A1) The first set of experiments : EA-PBPAIR vs. GOP300 with respect toenergy consumption and quality of service at actual PLR = 0, 5, and 10%,respectively—keep the transmitted data size the same—or include the energy consumption for larger data transmission in EA-PBPAIR

(2) (A2) EA-PGOP vs. PGOP w.r.t. energy consumption and quality of service—emphasize the effects of error-injection on energy consumption at PLR =

10%—clearly show that EA works for an error-resilient video encoding such as

PGOP


0

0.2

0.4

0.6

0.8

1

1.2


Tx EC


Rx EC

Quality in

PSNRN

orm

ali

ze

d R

ati

o o

f E

A-P

GO

P t

o P

GO

P-

PGOP-3 EAPGOP-4 EAPGOP-6

Fig. 9. Energy Reduction and Quality Degradation of EA-PGOP compared toPGOP (PLR = 10%, EIR = 10%, AER = 0% and -10%, FOREMAN 300 frames)

—Note that it shows the energy reduction at encoding and decoding whileshowing the energy overhead for transmission and receiving the video datasince PGOP does not have a mechanism to adjust the video quality andthe encoding file size as PBPAIR has (e.g., Intra Threshold). However, it isthe reason why we present AER, which may increase the amount of errorswe injected intentionaly to save the energy consumption, especially for thecommunication energy until the quality is satisfied with our feedback-basedmechanism.

—present the quality loss, which is why we propose the feedback-based errorcontrol approach in the next section

(3) (A3) EA-GOP vs. GOP w.r.t. energy consumption and QoS—we apply our active error exploitation for a standard video encoding such

as GOP, where we can see the positive impact of error-exploitation on theenergy saving at the slight loss of the video quality.

—Again, since there is no detailed mechanism to adjust the size of compressedvideo data at the encoing process without modifying the default quality-asware mechanisms such as the quantization scale value, we maintain theAER to keep the transmission energy minimal to save the overall energy atthe source and at the destination at the least cost of the video quality.

5.2 Sensitivity of Error-Injection Rate and Adjusted Error Rate

(1) The effectiveness of EIR on the energy consumption and the video quality(Fig. 12)

(2) The effectiveness of AER on the energy consumption and the video quality(Fig. 12)

5.3 Energy/QoS tradeoff

(1) Extended tradeoff space for mobile video embedded systems


0

0.2

0.4

0.6

0.8

1

1.2


Tx EC


Rx EC

Quality in

PSNRN

orm

ali

ze

d R

ati

o o

f E

A-G

OP

to

GO

P

GOP-15 EAGOP-10

Fig. 10. Energy Reduction and Quality Degradation of EA-GOP compared to GOP(PLR = 0%, EIR = 10%, AER = 0% and -10%, FOREMAN 300 frames)

Transmission Energy Consumption with increasing EIR

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

EIR (in %)

No

rma

lize

d E

ne

rgy

Co

ns

um

pti

on

EA-PBPAIR GOP-3 (baseline)

(a) Energy Consumption for TransmittingData

Video Quality with increasing EIR

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

EIR (in %)

No

rma

lize

d Q

ua

lity

to

GO

P-8


(b) Video Quality Evaluation

Encoding Energy Reduction with increasing EIR

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

EIR (in %)

No

rma

lize

d E

ne

rgy

Co

ns

um

pti

on


(c) Encoding Energy Consumption

Decoding Energy Reduction with increasing EIR

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

EIR (in %)

No

rma

lize

d E

ne

rgy

Co

ns

um

pti

on


(d) Decoding Energy Consumption

Fig. 11. Effects of Error Injection Rate on Energy Consumption and Video Quality(PLR = 10%, FOREMAN 300 frames, Each encoding is constrained with band-width)

(2) Effectively reduce the energy consumption of an encoding mobile device at thecost of the video quality

Video Quality vs. Energy Consumption at Source

0

5

10

15

20

25

30

15 20 25 30 35

PSNR (in dB)

En

erg

y C

on

su

mp

tio

n (

in J

ou

les

)

EA-PBPAIR PBPAIR

(a) Video Quality vs. Source Energy Con-sumption (Src EC)

Video Quality vs. Energy Consumption at Destination

0

5

10

15

20

25

30

15 20 25 30 35

PSNR (in dB)

En

erg

y C

on

su

mp

tio

n (

in J

ou

les

)

EA-PBPAIR PBPAIR

(b) Video Quality vs. Destination EnergyConsumption (Dst EC)

Fig. 12. Energy Consumption vs. Video Quality of EA-PBPAIR compared toPBPAIR (EIR = 1% to 50%, PLR = 5%, AER is set to Para1 = 15%, FOREMAN300 frames)

5.4 Feedback-based Error-Injection Control

(1) B. a set of experiments to show how our proposed feedback-based error-injectioncontrol works in order to keep the quality satisfied while minimizing the energyconsumption

(2) (B1) changing EIR can show how EIR works for adjusting quality of service fora fixed PLR—show how effectively our EA-PBPAIR can be used for adjusting quality by

adjusting EIR—base is a nominal EA-PBPAIR, i.e., a fixed EIR for error-injection

5.5 Adaptive EA-PBPAIR under Dynamic Network Status

(1) (B2) Also, adjusting EIR for varying PLR dynamically works well—Adapt EIR for EA-PBPAIR and Para1 for PBPAIR to satisfy the given

quality—base is a nominal EA-PBPAIR, i.e., a fixed EIR for error-injection

6. SUMMARY

REFERENCES

Video Quality vs. Source Energy Consumption

0

10

20

30

40

20 25 30 35 40

PSNR (dB)

En

erg

y C

on

su

mp

tio

n (

Jo

ule

s)

EA-PBPAIR PBPAIR GOP-8

(a) Video quality vs. Source Energy Consump-tion (AKIYO)

Video Quality vs. Destination Energy Consumption

0

2.5

5

7.5

10

20 25 30 35 40

PSNR (dB)

En

erg

y C

on

su

mp

tio

n (

Jo

ule

s)


(b) Video quality vs. Destination Energy Con-sumption (AKIYO)


0

10

20

30

40

15 20 25 30 35

PSNR (dB)

En

erg

y C

on

su

mp

tio

n (

Jo

ule

s)


(c) Video quality vs. Source Energy Consump-tion (FOREMAN)


0

2.5

5

7.5

10

15 20 25 30 35

PSNR (dB)

En

erg

y C

on

su

mp

tio

n (

Jo

ule

s)


(d) Video quality vs. Destination Energy Con-sumption (FOREMAN)


0

10

20

30

40

10 15 20 25 30

PSNR (dB)

En

erg

y C

on

su

mp

tio

n (

Jo

ule

s)


(e) Video quality vs. Source Energy Consump-tion (COASTGUARD)


0

2.5

5

7.5

10

10 15 20 25 30

PSNR (dB)

En

erg

y C

on

su

mp

tio

n (

Jo

ule

s)


(f) Video quality vs. Destination Energy Con-sumption (COASTGUARD)

Fig. 13. Video Quality vs. Energy Consumption (Each encoding is constrainedwith bandwidth, EIR = 1% to 50%, PLR = 5%, AER is set to Para1 = 15%, Para2is varying, FOREMAN 300 frames)

Energy Saving at the Cost of Video Quality - akiyo

0

20

40

60

80

100

Source EC Destination EC

En

erg

y S

av

ing

(%

)

No Quality Loss

5% Quality Loss

10% Quality Loss

(a) AKIYO

Energy Saving at the Cost of Video Quality - foreman

0

20

40

60

80

100


En

erg

y S

av

ing

(%

)

No Quality Loss

5% Quality Loss

10% Quality Loss

(b) FOREMAN

Energy Saving at the Cost of Video Quality - coastguard

0

20

40

60

80

100


En

erg

y S

av

ing

(%

)

No Quality Loss

5% Quality Loss

10% Quality Loss

(c) COASTGUARD

Fig. 14. Energy Saving at the Cost of Video Quality by EA-PBPAIR

Effectiveness of Adaptive EE-PBPAIR on Video Quality

10

15

20

25

30

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Run Number

PS

NR

(in

dB

)

Adaptive EE-PBPAIR with varying EIR

Original EE-PBPAIR with fixed EIR = 30%

(a) Delivered Video Quality

Adaptive EIR

0

5

10

15

20

25

30

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Run Number

EIR

(in

%)

(b) Error Injection Rate

Fig. 15. Adaptive EA-PBPAIR with a decrease of EIR vs. EA-PBPAIR with afixed EIR

Video Quality

25

26

27

28

29

30

31

32

20 15 10 5 10 15

Packet Loss Rate (%)

PS

NR

(d

B)

Adaptive EA-PBPAIR Static EA-PBPAIR

(a) Delivered Video Quality

Adaptive EIR based on Feedback

0

5

10

15

20

25

30

35

20 15 10 5 10 15

Packet Loss Rate (%)

Err

or

Inje

cti

on

Ra

te (

%)

Adaptive EA-PBPAIR Static EA-PBPAIR

(b) Adaptive Error Injection Rate

Fig. 16. Adaptive EA-PBPAIR Robust to Varying PLR under Dynamic NetworkStatus

Documents

Error-Aware Video Encoding to Extend EnergyQoS Tradeoﬀs ...kyoungwl/forge/ea/tecs/outline.pdf · Conference on Distributed and Parallel Embedded Systems ... para f2 f3 f0 Error−Aware