High Speed Networks Avish Shah - 12mict21
EvalvidIntroduction
EvalVid is a framework and tool-set for evaluation of the quality of video transmitted over a real or simulated communication network. It is targeted for researchers who want to evaluate their network designs or setups in terms of user perceived video quality. Besides measuring QoS parameters of the underlying network, like loss rates, delays, and jitter, a subjective video quality evaluation of the received video is provided. Currently H.263, MPEG-4 and H.264 are supported.
Framework and Design
In Figure ,the structure of the EvalVid framework is shown. The interactions between the implemented tools and data flows are also symbolized. In Section 3 it is explained what can be calculated and Section 4 shows how it is done and which results can be obtained.Also, in Figure , a complete transmission of a digital video is symbolized from the recording at the source over the encoding, packetization, transmission over the network, jitter reduction by the play-out buffer, decoding and display for the user. Furthermore the points, where data are tapped from the transmission flow are marked. This informa-tion is stored in various files. These files are used to gather the desired results, loss rates, jitter, and video quality. A lot of information is required to calculate these values.
The required data are (from the sender side):– raw uncompressed video– encoded video– time-stamp and type of every packet sentand from the receiver side:– time-stamp and type of every packet received– reassembled encoded video (possibly errorneous)– raw uncompressed video to be displayedSource: raw (uncoded) video files are usually stored in YUV format.
VS (Video Sender): generates two trace files necessary for the subsequent video quality evaluation- sender trace file- video trace file
Assignment – 1 : Evalvid Page 1
High Speed Networks Avish Shah - 12mict21
ET (Evaluate Trace): generates reconstructed erroneous video using also the receiver trace.
FV (Fix Video): is only needed if the used codec cannot provide lost frames (“empty” or the last decoded frames for lost frames)
PSNR (Peak Signal to Noise Ratio) is an objective quality measure.• It is a derivative of the well-known signal to noise ratio (SNR), which compares the signal energy to the error one.
Where
– MAXI is the maximum possible pixel value of the image
– MSE is the Mean Square Error: • I is the original image and K is the compressed one
Assignment – 1 : Evalvid Page 2
High Speed Networks Avish Shah - 12mict21
• MxN is the dimension of both images
MOS (Mean Opinion Score) is a subjective quality measure.• MOS ranges from 1 (worst) and 5 (best):
• PSNR approximated to the MOS scale:
Evalvid and NS2• To experiment video transmission using the NS2 simulator
• Simulation produces a receiver trace file containing information necessary to reconstruct the possibly corrupted video at the receiver side.
• To decode the video files obtaining a YUV (raw) video:
- ffmpeg -i video.264 video_raw.yuv
• To create a compressed raw video: MPEG-4
- ffmpeg -s cif -r 30 -b 64000 -bt 3200 -g 30 -i video_raw.yuv –vcodec mpeg4 video_cod.m4v
• To create a MP4 files containing the video samples (frames) and a hint track which describes how to packetize the frames for the transport with RTP:
-./mp4box -hint -mtu 1024 -fps 30 -add video_cod.m4v video_encaps.mp4
• To obtain the YUV file created by decoding the coded video
- ffmpeg -i video_encaps.mp4 video_ref_raw.yuv
• To compute PSNR that shows the codec impact on video quality - ./psnr 352 288 420 video_raw.yuv video_ref_raw.yuv > psnr_prima.txt
Assignment – 1 : Evalvid Page 3
High Speed Networks Avish Shah - 12mict21
Encoding the raw YUV video $mpeg4encoder.exe example.par
Assignment – 1 : Evalvid Page 4
//example.par…………………………………Source.Width = 176Source.Height = 144// set the first encoding frame to 0Source.FirstFrame = 0// set the last encoding frame to 399Source.LastFrame = 399//set the prefix file nameSource.FilePrefix = "foreman_qcif“
Scalability.Spatial.PredictionType = "PBB" // One of "PPP", "PBB"
High Speed Networks Avish Shah - 12mict21
After encoding, a compressed video file “foreman_qcif.cmp” can be found
Get the Video Traffic Trace File
• $MP4.exe –send 224.1.2.3 5555 1000 foreman_cif.cmp > st
• This MP4.exe command will read the compressed file. Fragment each frame
• into small packets with the maximum size of 1000 bytes. Then send this packets
• to IP:224.1.2.3 Port:5555. Here, the IP and Port is not important. Because we
• just need the frame information (stored in st file).
Run the simulation scriptset ns [new Simulator]
set nd [open out.tr w]
$ns trace-all $nd
set max_fragmented_size 1000
#add udp header(8 bytes) and IP header (20bytes)
set packetSize 1028
set s1 [$ns node]
set r1 [$ns node]
set r2 [$ns node]
set d1 [$ns node]
Assignment – 1 : Evalvid Page 5
0 H 29 1 segm at 33 ms 1 I 3036 4 segm at 66 ms 2 P 659 1 segm at 99 ms 3 B 357 1 segm at 132 ms 4 B 374 1 segm at 165 ms 5 P 693 1 segm at 198 ms 6 B 420 1 segm at 231 ms 7 B 460 1 segm at 264 ms
High Speed Networks Avish Shah - 12mict21
$ns duplex-link $s1 $r1 10Mb 1ms DropTail
$ns simplex-link $r1 $r2 0.18Mb 10ms DropTail
$ns simplex-link $r2 $r1 0.18Mb 10ms DropTail
$ns duplex-link $r2 $d1 10Mb 1ms DropTail
set qr1r2 [[$ns link $r1 $r2] queue]
$qr1r2 set limit_ 10
set udp1 [new Agent/myUDP]
$ns attach-agent $s1 $udp1
$udp1 set packetSize_ $packetSize
$udp1 set_filename sd_be
set null1 [new Agent/myUdpSink2]
$ns attach-agent $d1 $null1
$ns connect $udp1 $null1
$null1 set_trace_filename rd_be
set original_file_name st
set trace_file_name video1.dat
set original_file_id [open $original_file_name r]
set trace_file_id [open $trace_file_name w]
set frame_count 0
set last_time 0
while {[eof $original_file_id] == 0} { gets $original_file_id current_line
scan $current_line "%d%s%d%s%s%s%d%s" no_ frametype_ length_ tmp1_ tmp2_ tmp3_ tmp4_ tmp5_
# 30 frames/sec. (1 sec -> 1000000us)
set time [expr 1000 * 1000/30]
Assignment – 1 : Evalvid Page 6
High Speed Networks Avish Shah - 12mict21
if { $frametype_ == "I" } set type_v 1
if { $frametype_ == "P" } set type_v 2
if { $frametype_ == "B" } set type_v 3
if { $frametype_ == "H" } set type_v 1
puts $trace_file_id "$time $length_ $type_v $max_fragmented_size"
incr frame_count }
close $original_file_id
close $trace_file_id
set end_sim_time [expr 1.0 * 1000/30 * ($frame_count + 1) / 1000]
puts "$end_sim_time"
set trace_file [new Tracefile]
$trace_file filename $trace_file_name
set video1 [new Application/Traffic/myTrace2]
$video1 attach-agent $udp1
$video1 attach-tracefile $trace_file
proc finish {} { global ns nd
$ns flush-trace
close $nd
exit 0 }
$ns at 0.0 "$video1 start"
$ns at $end_sim_time "$video1 stop"
$ns at [expr $end_sim_time + 1.0] "$null1 closefile"
$ns at [expr $end_sim_time + 1.0] "finish"
$ns run
Assignment – 1 : Evalvid Page 7
High Speed Networks Avish Shah - 12mict21
sd_be
rd_be
Assignment – 1 : Evalvid Page 8
$ns be.tcl
0.033333 id 0 udp 29 0.066666 id 1 udp 1000 0.066666 id 2 udp 1000 0.066666 id 3 udp 1000 0.066666 id 4 udp 36 0.099999 id 5 udp 659 0.133332 id 6 udp 357 0.166665 id 7 udp 374 0.199998 id 8 udp 693 0.233331 id 9 udp 420 0.266664 id 10 udp 460
0.047958 id 0 udp 29 0.126000 id 1 udp 1000 0.171689 id 2 udp 1000 0.217377 id 3 udp 1000 0.219451 id 4 udp 36 0.250482 id 5 udp 659 0.267352 id 6 udp 357 0.285232 id 7 udp 374 0.317532 id 8 udp 693 0.337225 id 9 udp 420 0.358945 id 10 udp 460
High Speed Networks Avish Shah - 12mict21
Video Quality Evaluation• execute et program
• $et.exe sd_be rd_be st foreman_cif.cmp err_be.cmp 0
Packet/Frame lost report
• The first line indicates that the total number of packets sent (p->nA) is 1022.
It includes 350 I frame packets (p->nI); 222 P frame packets (p->nP);
and 449 B frame packets (p->nB).
• The second line indicates that the total number of packets lost (p->lA) is 161.
It includes 109 I frame packets (p->lI); 22 P frame packets (p->lP);
and 30 B frame packets (p->lB).
• Decode the video
• $mpeg4decoder.exe err_be.cmp err_be 176 144 > df_be
• Error concealment
• $myfixyuv.exe df_be qcif 400 err_be.yuv myfix_be.yuv
• Calculate the average PSNR
• $avgpsnr.exe 176 144 420 foreman_qcif.yuv myfix_be.yuv
• avgerage psnr:24.363058
Assignment – 1 : Evalvid Page 9
p->nA:1022, p->nI:350, p->nP:222, p->nB:449 p->lA:161, p->lI:109, p->lP:22, p->lB:30 f->nA:301, f->nI:34, f->nP:67, f->nB:199 f->lA:54, f->lI:20, f->lP:13, f->lB:21
High Speed Networks Avish Shah - 12mict21
Run the qos.tcl and compare the results from be.tcl
Assignment – 1 : Evalvid Page 10