Upload
zoie-standring
View
214
Download
0
Embed Size (px)
Citation preview
www.smartphonetech.org
Contents Forms of Motion Picture Multimedia products with Motion Picture Content Motion Video Technology Source for Motion Video Objects Playing Video on PC Video Formats : lines, frames, fields & aspect
Ratio Analog Video Signal Waveform T.V. Broadcast Standards Color of Video Analog Video Camera Principle
www.smartphonetech.org
Contents (cont.) Luma & Chroma Hue & Saturation Chroma Component Options Video Signals Composite Video Signal Wavwform :Color Digitisation Basics Effect of Low Pass Filters Magnitude Domain Discretisation Color Bitmap Encoding & CLUT Digitised Video Standards
www.smartphonetech.org
Contents (cont.) Video Compression Motion Compensation Artifacts due to digitisation & compression Video Compression Techniques & Standards Data Rates of Media Analog Video Recording : Tape Formats Digital Video Camera Digital Video Recording : Tape Formats Video File Formats Steps to PC Video
www.smartphonetech.org
Contents (cont.)
Video Capture : Basics & Platform Video Capture Cards Video Capture Considerations & Procedures Video editing : Primer & concepts Transitions & Superimpositions Non-linear editing Video Editing Tools Shooting Tips
www.smartphonetech.org
Motion Video Technology for Multimedia
Learning Objectives Appreciation of Technical Aspects of
Full Motion VideoDigital Video Storage and CompressionVideo EditingMultimedia Video Standards
Obtaining necessary background for acquiring and processing Full Motion Video Objects for embedding in Multimedia Contents.
www.smartphonetech.org
Many Forms of Motion Picture
Films 35 mm , 70mm, 16mm,8mm
Broadcast TV Ordinary DBS TV HDTV (coming up)
Laser Disk (Players) DVD (Players) Video in Cassettes and Tapes Video in computer
Multimedia component Stand alone CD Video Web Content
www.smartphonetech.org
Why use Motion Video in Multimedia?
Sin qua non in True Multimedia Most effective communication mode Entertains and educates with least audience
initiative Brings realism Multiple views (vista and zoom) Being there - presence People are already sensitised to movies Shows changes - as and when they happen
www.smartphonetech.org
The downside of Video in Multimedia
Takes two much disk space Needs large bandwidth when transmitted Costly to produce Quality is difficult to achieve and highly
dependent on target machine May easily become the weakest link in the
product
www.smartphonetech.org
Content is the King
Every king needs subjects - so does video what to show in video
Drama Dances interviews Large artwork or mural architecture Record of events (village fair) and social functions
(marriage) How to do things
Operation Installation Show How is know-how
Internals of a machine You get the idea
www.smartphonetech.org
Multimedia Products with Motion Picture content As embedded object.
Within an interactive multimedia product (say, web page, encyclopedia entry).
Short length snippets (one or many) Small format (1/4th- 1/16th).
As the primary content Within an interactive multimedia presentation. Good quality, with voice and music At least several minutes long. Rest of contents organised around it . Must be at least 1/4th screen
Stand-alone object Library/depository object Good quality
Stand-alone digital video product Good quality, Longer duration, Full screen preferred
www.smartphonetech.org
Before you begin
To incorporate video in multimedia, ask the following questions:
Could the video be substituted by still ? If not, why, what is the USP?
Could Animation be better than a video ? What would be the role of the video in the product? ( e.g.
embedded object, album ) Who would provide the footage ? What would be the duration of the video clip(s)? Delivery medium (CD-ROM , DVD ,Internet, LAN) Who would be editing and integrating it (skill level) and
where (facilities)? Does the producer know about the nuances of video
technology?www.smartphonetech.org
Studying Motion Video Technology
All skills are to be converged Many techniques of Multi Media borrowed
from full motion video production methodology.
Possibility of auxiliary services like Desktop Video Editing .
Close kin of Animation. Most complex and jargon ridden
www.smartphonetech.org
Principle of Motion Picture: Frame Rates
Successive images (frames) of a moving object are
captured in a movie camera at a fixed rate (frame rate)
The frames , when played (projected) at the same
frame rate gives an illusion of moving object.
Frame rate above 20/sec is adequate.
For slow moving objects even 15 frame/sec would do.
Below this rate, jerks are perceived.
Not only object motion but camera -optical motion
(pan, zoom ) is to be considered.
Cine uses 24 frames per second.
Higher frame rates used for special effects.
www.smartphonetech.org
Source for Motion Video Objects
Computer Files Only form usable in Multimedia. Many formats. Usually compressed.
Analog Video Camera Needs frame grabber for direct storage as a computer file. Equipment technology mature and low cost.
Analog Video Cassettes/Tapes Camera to tapes/cassettes. Different qualities and standards. Generation loss of quality. Needs frame grabber again.
www.smartphonetech.org
Source for Motion Video Objects (contd.)
Digital Video Camera May be stored in tapes/cassettes or directly stored
as computer files. Technology still evolving. Equipment Costly.
Digital Video Cassettes/Tapes/ disks Surprisingly predates digital video cameras. Old Digital Video Cassettes/Tapes/ disks are not
readable by computers. New Computer readable (compressed) formats are
now available. No generation loss. Best suited for editing.
www.smartphonetech.org
Playing Video on PC
Pentium and Power PC based personal computers are capable of playing full motion video of different qualities.
From Hard Disk (AV, SCSI-II F&W preferable) CD ROM (32x) Intranet/Internet DVD TV antenna or Cable TV (need special cards) VCP (you really do not need a PC)
Quality varies: Full speed , full screen , full colour Small quarter size (Fixed , variable and movable)
windows Smaller frame rates Video Overlay www.smartphonetech.o
rg
Playing Video on PC (Contd..)
Configuration: Pentium-II, 200 + MHz, 32MB Ram SVGA monitor and adapter with required resolution
and colour depth Sound Card and speakers Software Graphic accelerator ( desirable for full speed full
screen) CD-ROM
Configuration: additinally may require DVD (if source is DVD) Hardware assist (For playing MPEG-II) TV tuner Card
www.smartphonetech.org
Video Formats: Lines , Frames and Fields
Brodcast and VCR video are generally refreshed @ 25/30 frames /sec
Each frame is composed of a large number of horizontal lines.
Each frame is divided into two fields. The even field contains only even lines of a frame. The odd field contains only odd lines of a frame. The two filelds are displayed one after the other :
thus covering the whole frame in two installments. Each field takes half the time of a frame.
This is known as interlacing.
www.smartphonetech.org
Video Formats: Lines , Frames and Fields
www.smartphonetech.org
Video Formats: Lines , Frames and Fields (contd)
Interlacing Reduces Broadcast bandwidth (by transmitting only
half the lines per field) Reduces flicker as fields are updated 50/60 fields /sec.
To synchronise the frames and fields , the source supplies vertical synchronisation signal, once per field.
Loss of vertical sync results in jumpy vertical scroll . To synchronise the lines in a field , the source supplies
horizontal synchronisation signal, once per line. Loss of horizontal sync results in loss of picture . Computer Video is progressive i.e non-interlaced.
www.smartphonetech.org
Video Formats: Aspect Ratio
Films 35 mm 1.85:1(16:9) to 2.35:1 (~19:8) 70mm 2.35 : 1 16mm/8mm 1.85:1
Standard TV Monitors: 5:3 HDTV - 16:9 Computer Monitors : 5:3 (1.67:1) Video in computer (multimedia component)
height -h
width-w
AR=w:h
www.smartphonetech.org
Analog Video Signal Waveform: Monochrome
TBD
www.smartphonetech.org
www.smartphonetech.org
Television Broadcast Standards
Standard Frame rate Resolution Countries
NTS 29.97 fpsinterlaced
525 line USA, Japan, Canada,Mexico, part of Centraland South America,South Pacifics
PAL 25 fpsinterlaced
625 lines India,UK, Germany,Netherlands.
PAL-M 30 fpsinterlaced
525 lines Brazil.
SECAM 25 fpsinterlaced
625 lines France,Eastern Europe,Middle East
HDTV : Coming Up
www.smartphonetech.org
Color Components of Video
The information of a point in a colour picture may be described in terms of intensities of three colour signals RED , GREEN and BLUE.
Almost all perceivable colours can be described by varying the proportions of the three components.
If we keep the proportions same but change the absolute values of the RGB signals, the colour will remain the same but brightness will change.
However, If we keep the the absolute values of two signals same but change the proportions by changing the third signal, both the colour and brightness will change.
www.smartphonetech.org
Analog Video Camera Principle
Two basic types of video cameras Tube (e.g videocon, orthicon) CCD (semiconductor)
In a monochrome video camera the external image is focussed on the active surface of the tube or the CCD array.
An electron beam scans the active surface horizontally and vertically just like in a TV monitor.
The scanning action generates the video signal proportional to the luminosity of the points scanned.
www.smartphonetech.org
Analog Video Camera Principle
www.smartphonetech.org
Analog Video Camera Principle (monochrome)
The scans are synchronised with V and H sync
signals generated from a stable source.
The stable source may be the camera or another
camera or a studio master sync generator (gen-
locking).
The above video signal is composited with the V and
H sync signals to create the composite video signal.
www.smartphonetech.org
Analog Video Camera Principle (colour)
In a colour camera The scanning mechanism is identical to that of the
monochrome. An image is optically seperated into R G and B
components using filters and prisms. The three components are focussed into three
different active surfaces which generate corresponding (R,G,B) intensity signals.
In a single CCD colour camera There is only one active surface but containing
three different kinds of active elements, one each for R,G and B.
Each type of active element responds only to its corresponding colour lights.
www.smartphonetech.org
Analog Video Camera Principle(colour) (contd.)
Depending on the output type desired
The R,G,B signals and the sync signals are
separately ouput
The signals are composited and output.
www.smartphonetech.org
Luma and Chroma The Luminance or brightness of a small region in a
colour picture is given by:- Y=0.299R+.0587G+0.114B
In black and white TV, only the Luma component is required.
When the Luminance is known, we need two more quantities (say proportions) do describe the colour. These signals are known as chrominance components of the signal.
There are several ways to describe the Luma. Eye is most sensitive to spatial and temporal variation
of the Luma component.
www.smartphonetech.org
Hue and Saturation
If we (optically) add white colour to another pure colour say blue, what do we get ? Does the colour change (say to pink or yellow)?
Something (tint) remains constant and something (say shade) changes.
What remains constant is the tint or hue. What changes is technically called saturation. If a pure colour does not contain any white
component it is called fully saturated. An image with saturated components looks
bright. Unsaturated components create pastel shades.
www.smartphonetech.org
CHROMA component options Naive: Y and any two ( not used) CCIR 601
U= (B-Y): -yellow, V= (R-Y) : - cyan PAL (actual)
U= 0.492(B-Y), V=0.877(R-Y) Also known as Y-U-V
NTSC (Y-I-Q) I=0.596R-0.275G-0.321B (orange to blue) Q=.212R-.523G+.311B (purple to green) Eye is least sensitive to Q
JPEG/MPEG Cb=(B-Y)/2+0.5 Cr= (R-Y)/2+0.5 Range is always one.
www.smartphonetech.org
Video Signals
Component Video RGB and +Hsync (every line) +Vsync (every line)
: 3+2 or 3+1 (combined sync) R,G+ combined sync,B
TV signal and composite VHS Picture info +Hsync (every line) +Vsync (every
line) +chroma burst (every line) Picture info= Luma +chroma Luma + chroma on colour carrier as a combined
signal SVHS (camera and deck)
Luma and chroma on colour carrier as two separate signals
www.smartphonetech.org
Composite Video Signal Waveform: colour
www.smartphonetech.org
Digitisation Basics
Lines and frames are first steps of discretisation - already available, even in analog video.
Two dimensions for video data within a line : Time domain sampling, Magnitude domain discretisation.
Both these degrade picture quality to some extent.
Without sacrificing quality How often to sample ? How many bits to discretise the data?.
www.smartphonetech.org
Horizontal blanking interval
SYNC.WIDTH
BLANKINGWIDTH
BURSTSTART
FRONTPORCH
BURSTWIDTH
EQUALIZINGPULSES
VERTICAL BLANKING WIDTH
EQUALIZINGPULSES
VERTICALPULSES
EQUALIZINGPULSE WIDTH
VERT. SYNC.PULSE WIDTH
Vertical blanking intervalwww.smartphonetech.org
Spatial Resolution and Bandwidth
Consider the figure below which shows an alternate patterns of black and white and the corresponding video signal.
The number of pulses would roughly corresponding to the number of dark bands N.
If the horizontal display line time is Th, then the frequency is f= N/ Th
The frequency content (bandwidth) of the video signal is therefore directly proportional to the horizontal details (resolution) and vice versa.
Thus the bandwidth can be expressed as dots per line (spatially) or alternatively in MHz ( time equivalent) for a given horizontal clock.www.smartphonetech.org
Spatial Resolution and Bandwidth (contd)
If processing, transmission or display bandwidth of the video signal is much below the video signal f generated due to horizontal details, the video signal amplitude would decrease and hence the contrast would decrease and hence details ( bands ) cannot be
distinguished i.e resolution would be lost.
Similar effects may be seen after employing low pass spatial filter in still images.
Due to the limitation of transmission (and display) system, the bandwidth of video signal is (artificially) limited.
www.smartphonetech.org
Effect of Low-pass filters (spatial or temporal)
www.smartphonetech.org
Resolution and Bandwidth (contd)
Limiting bandwidth of video signal limits resolution of broadcast / VCR generated images but allows broadcast, reception and processing using
inexpensive equipment allows slower sampling rate.
The vertical resolution is limited by the number of lines. However, the actual resolution perceived by human eye is
a bit different than what the bandwidth calculation would predict.
Due to interlacing, a single pixel width line is displayed only in alternate frames and would tend to blink.
In SECAM a thin line may show two differnt colours (as Cr and Cb are sent alternately)
A projected film has much higher resolution in both vertical and horizontal directions.
www.smartphonetech.org
Sampling and Nyquist A theorem due to Shannon, attempted to answer the
question “How often to sample ?” Like many other question the answer is “it depends”. The best sampling rate depends on how quickly the signal itself is changing.
The agility of a signal or the quickness of its change is measured by its Bandwidth. For example in broadcast , the TV picture signal bandwidth is less than 6 MHz (6 million times/second).
The sampling rate should be at least twice the bandwidth or a part of the information would be lost and signal may not be reconstructed from its sampled version. This rate is called the Nyquist Frequency.
Nyquist frequency for TV picture signal is 12 MHz.
www.smartphonetech.org
Sampling and Nyquist (contd.)
www.smartphonetech.org
Sampling RGB or Composite? Sampling of composite signals are to be avoided
as : Resultant digital signal is difficult to manipulate. The colour subcarrier is also sampled. Artifacts (defects) are introduced.
Sampling of S-Video signal i.e. Luma and combined chroma is slightly better.
RGB is simplest and easiest to manipulate - but wastes bandwidth.
YUV (Luma and separate chroma) are the best as optimal sampling rates and compression are possible.
www.smartphonetech.org
Luma and Chroma Sampling
The Luma signal changes more rapidly and eye is most sensitive to it. It has a bandwidth of about 4 MHz in broadcast TV.
Luma signal needs to be known for every pixel. Chroma signal changes more slowly and can be
sampled less frequently. Good quality may be obtained by sampling the colour
components at half the frequency. (4:2:2). Reasonable colour rendering is obtained by 4:1:1
sampling. Colour signals may also be sampled in alternate lines
(4:2:0)
www.smartphonetech.org
Luma and Chroma Sampling(illustration)
www.smartphonetech.org
www.smartphonetech.org
Magnitude Domain Discretisation
For Digitisation the sampled magnitudes or values need be converted into binary integers.
For a black and white image, we would require only 1 bit (0 or 1) per pixel.
For grey scale image, the number of bits required would depend on the number of grey shades desired. For example, with 6-bits, 64 different shades may be
distinguished. with 8-bits the shade count would be 64
www.smartphonetech.org
Magnitude Domain Discretisation (contd.)
The total number of bits employed in RGB component video would determine the varieties of shades.
For RGB component video, the number of bits employed are usually identical for each channel.
For Chroma-Luma component video,the Luma channel may use more bits than the Chroma
Employing more number of bits per pixel would increase the fidelity of the picture, but increase file size and play banndwidth requirement.
Some lossy compression techniques may reduce the bit depth
www.smartphonetech.org
Colour Bitmap Encoding
Colour of a point (pixel ) may be described by the quantities of the R-G-B components.For computer ,the quantities would be described by digital numbers- say integers.
So we need three integers for every point to describe the image .
For the image, the sets of numbers for all points is called the Colour Bit Map.
For true colour representation , each such numbers should be of 8-bit size. So we need 24 bits , giving Millions of shades.
We may not need such huge varieties of shades for every single picture.
www.smartphonetech.org
Bitmap Encoding and CLUT (contd.)
If our requirement for shades is modest , say 200 shades only, we may reduce the number of bits required. Example: night scene, sky, buildings...
This is done by using a palette or a colour look-up table (CLUT).
A colour look-up table contains entries of RGB components against shade numbers. (we may throw- in a colour sample strip along, just like the shade cards available in paint shops)
As there are less than 256 numbers, an 8-bit integer would be adequate.
We may then represent each point with just one 8-bit number. Of course we knew this while studying still image and graphics. What we did not know is that for a video, the 256 shade
CLUT limitation may be applicable for the whole clip.www.smartphonetech.org
Digitised Video Standards Studio Quality
NTSC PAL CIFpels/line 720 720 360Active lines 485 576 288LumaSampling
485 576 288
Chroma SubSampling
4:2:2 4:2:2 4:2:0
Fields/s 60 50 30Interlace 2:1 2:1 1:1Aspect Ratio 4:3 4:3 4:3
www.smartphonetech.org
Pixel Arrays
120
240
483
600
720
900
1080
160 352 720 800 1152 1280 1920
QSIF(19kp)
SIF (82 kp)
601 (300kp)
SVGA (500kp)
ATP (1Mp)
Workstation (1 Mp)
HDTV (2Mp)
www.smartphonetech.org
Rationale for Compression
Example : 640x240 (full screen) , 24 bit full colour, 30fps 221 M bits/s (~ Indian Internet BW) 28 M Bytes/s 15 s file takes 420 MByte space (about 1 CD)
Storage Streaming rates Distribution Transmission
www.smartphonetech.org
Data rates/BW of Medium
Medium Bitrate/BWOrdy Telephone 0.3-56 Kb/sDS-0 Telephone 56 Kb/s
ISDN px64 Kb/s
T1 1.5 Mb/s
Ethernet 10, 100 Mb/sBISDN 100-200 Mb/s
www.smartphonetech.org
Compression Basics
Redundancies Exist Spatial (subsampling, subband)
neighbouring pixels tend to have very similar brightness and colour, except for the edges.
Temporal (predictive) Successive frames have many pixels identical to
those of previous. Intraframe Vs. Interframe
Spectral (subband) Not all shades of colour appear in a clip
lossy Compression may destroy information
Also called source encoding !www.smartphonetech.org
Lossless Encoding & Compression No loss of information Compression depends on content. Low compression fast encoding and decoding Run length encoding (RLE)
Simplest form of compression Usually intraframe, also used in fax, zip etc. Example: aaaaabbbccccd=5a3b4cd
Huffman Encoding Determine the most frequently occurring pattern,
give it a short code and so on. Dictionary is required.
Transform Encoding (some)
www.smartphonetech.org
Lossy compression Enforces ‘almost similar ‘ things to be similar and
then compress. Large compression ratios may be obtained. Quality may suffer Trade-off may be done between compression ratio
and quality. When applied spatially , reduces levels of
contrast Hue
Almost similar successive frames are treated as similar
May be carried out without notice (Backdoor Compression) When you reduce size When you reduce colour depth When you reduce frame rates
www.smartphonetech.org
Transform Encoding: The basic idea
Similarity is not always obvious. By transforming the original data, the
similarities are revealed.
In the above picture, the horizontal lines are not similar.
If we rotate the thick line (i.e. mathematically transform it) to be horizontal, similarity is revealed.
www.smartphonetech.org
Transform Encoding: The basic idea (continued)
A picture may be transformed into components which
may be easily and compactly expressed.
A picture may produce very large number of components.
One may discard the insignificant components.
Discarding component implies loss of information- but
this may not be perceptible.
By varying the number of components retained, one may
trade off between quality and compactness.
The trick is to find the right transformation which would
produce a small number of significant components.
www.smartphonetech.org
Transform Encoding(contd.)
Available Transforms Direct Cosine Transform (DCT) Fractal transform
Fractal Transform Highly asymmetrical Needs huge time during compression proprietary
Discrete Cosine Transform Popular (MPEG, JPEG) Well understood Good trade off 1:200 compression feasible
www.smartphonetech.org
1-D Cosine Transform Example
The following waveform represents the luminosity distribution along a 180 pixel line.
A cosine transform of this would need ~ 180 components
sigma
-2
0
2
4
6
8
10
12
0
10
20
30
40
50
60
70
80
90
10
0
11
0
12
0
13
0
14
0
15
0
16
0
17
0
18
0
sigma
www.smartphonetech.org
1-D Cosine Transform Example (contd): total Vs 30s ig m a
- 2
0
2
4
6
8
1 0
1 2
0 10
20
30
40
50
60
70
80
90
10
0
11
0
12
0
13
0
14
0
15
0
16
0
17
0
18
0
s ig m a
-1
0
1
2
3
4
5
6
7
8
9
01 8 0
3 0 C o m p o n e n t A p p ro x im a t io n
2 0 C o m p o n e n t A p p r o x i m a t i o n
0
1
2
3
4
5
6
7
8
0 1 8 0
www.smartphonetech.org
1-D Cosine Transform Example (contd): 40 Vs 30
-1
0
1
2
3
4
5
6
7
8
9
0180
30 Component Approximation
40 component Approximation
-2
0
2
4
6
8
10
0 180
www.smartphonetech.org
1-D Cosine Transform Example (contd): total Vs 20
s ig m a
- 2
0
2
4
6
8
1 0
1 2
0 10
20
30
40
50
60
70
80
90
10
0
11
0
12
0
13
0
14
0
15
0
16
0
17
0
18
0
s ig m a
2 0 C o m p o n e n t A p p r o x im a t io n
0
1
2
3
4
5
6
7
8
0 1 8 0
www.smartphonetech.org
1-D Cosine Transform Example (contd): 30 Vs 20
20 Component Approximation
0
1
2
3
4
5
6
7
8
0 180
-1
0
1
2
3
4
5
6
7
8
9
0180
30 Component Approximation
www.smartphonetech.org
1-D Cosine Transform Example (conclusion)
If we retain just 50 components there is almost no loss of information, except some noise .
If we retain just 40 components, we get a reasonable replica
We may try going down to 30 or even 20 components. One may observe the rapid deterioration of details.
We see the trade-off between compactness and loss of detail.
The above concept may be extended to 2-D as well.
www.smartphonetech.org
8x8 Discrete Cosine Transform
Popular 2-D transform with 64 coefficients
www.smartphonetech.org
Interframe Compression
Temporal redundancy is eliminated by interframe compression.
Consider a title page, which have to be shown for 3 seconds (i.e 75-90 frames) Each such frame is absolutely identical Sending one frame and N-1 ditto’s are good enough and
non-lossy. In general, successive frames may be very similer. Compression could be obtained by sending a
reference frame and then sending only the changes or differneces. In order that the differences do not become too big,
reference frames may be sent periodicallywww.smartphonetech.org
Motion Compensation
A special case of interframe compression Applicable with camera pan or moving subject A part of the picture remain same but move
as a group (translate) By supplying only the motion vector (direction
and amount) the pixels concerned may be defined.
Macroblocks (16x16) between successive frames are compared for close matches.
Differences are computed after considering the motion vector.
www.smartphonetech.org
Why filtering is required
With spatial filters we may limit the details and levels of contrast
Limiting the details reduces bandwith and sampling at lower frequency is possible without violating Nyquist limits.
Bandwidth and contrast limited pictures compress well.
Lossy compression will reduce bandwidth and contrast in any case
Artifacts may be introduced if filtering is not performed.
www.smartphonetech.org
Artifacts due to Digitisation and Compression
Pixelisation Jagged edges Moiré fringe Contouring Colour seepage and smear Palette Flash Jerky movement Salt and pepper
www.smartphonetech.org
Video Compression Techniques and Standards
JPEG & M-JPEG (intra frame) H.261 (intra frame) MPEG-1 MPEG-2 MPEG-4
www.smartphonetech.org
JPEG and M-JPEG
JPEG (Joint Photography Expert Group) Recommended standard for digitisation and compression
of still images May be used for frame by frame compression of MP.
JPEG Modes: lossless sequential progressive hierarchical
JPEG lossless mode : pixel-wise predictor encoding using neighbours
Sequential Mode each image component encoded in a single left to right,
top to bottom fashion.www.smartphonetech.org
JPEG and M-JPEG (continued)
M-JPEG JPEG extended for motion picture and animation,
still intraframe Sequential JPEG applied to each frame
JPEG progressive : image quality improves as more data arrives Suitable for web images
JPEG hierarchical Code and compress a down-sampled image Upsample it and take the difference with original, send the down sampled and difference may be repaeated multiple times
www.smartphonetech.org
MPEG
MPEG (Motion Picture Expert Group) Recommended encoding and compression
standard. Accepted by ITU and ISO
Three different standards MPEG-1 (for PC Video, near VHS) MPEG-2 (for HDTV and Broadcast studio) MPEG-4 (primarily for video telephony)
MPEG-1 and MPEG-2 are already available and popular
www.smartphonetech.org
MPEG standards Overview Three layers or streams
System Audio Video
System = Info regarding synchronisation random access, management of buffers time stamp
State of the art compression Flexibility for optimisation Software Decoders available
Quicktime Xing Mediamatics with windows Media player
www.smartphonetech.org
MPEG-1
1991 Vintage For CD-ROM video and Audio Audio is near studio quality Video is near VHS Targeted frame rates ~ 1.5 Mbits/s lossy & asymmetric compression
Intra frame spatial redundancy crunch using DCT (6:1)
Interframe ,temporal redundancy elimination, using P (predictive) difference
A number of RLE and compaction steps Software coding and decoding possible.
www.smartphonetech.org
MPEG-2
For higher data rates (4 to 9 Mbits/s)
For better picture quality and HDTV (US and Europe)
Would play MPEG-1 objects
Used in DVD video and DBS (hughes)
Interframe ,temporal redundancy elimination, using
P (predictive) and B (bi-directional) frames
Seq: I-P-P-P-B-I-P-P-B-I
4:2:2 Chroma subsampling
www.smartphonetech.org
MPEG-2 (continued)
10-bit DCT DC precision non linear qualtisation Other compression steps Optional time stamp error and transport control Hardware codec recommended Don’t expect MPEG-2 to be better than MPEG-
1 at lower bit rate.
www.smartphonetech.org
MPEG-2Standards and levels
Level Sample xLine
Framesper sec
Bit rateMbps
Quality
Low 352 x258
30 4 VHS
Main 720 x576
30 15 CCIR 601,Studio
High1440
1440 x1152
60 HDTVStudio
High 1920 x1152
60 80 Filmproduction
www.smartphonetech.org
MPEG-4
Expected to be formalized in Nov 1998 Targeted bit rate 4.8 to px 64 Kbps (p <80) Suitable for
Video phones Video conference Internet video Mobile video Computer generated animation
Object based encoding
www.smartphonetech.org
Data Rates of Media
Media Data Rates (KBytes/sec)
CDROM-2X 300 (theor)CDROM-8x 1,200 (theor)CDROM-32(SCSI)x
4,800 (theor)3,500 (typical)
F & W SCSI &AVI HardDisk
6MBps- 10MBps
SCSI & AVIHardDisk
2.5MBps- 6MBps
IDE HardDisk 1.5MBps- 2MBps
DVD-R 2.5MBps- 6MBps
Striped Disks 10MBps- 20MBpswww.smartphonetech.org
Analog Video Recording:Tape Formats
VHS (1/2”: Consumer, composite) 8 mm ( Consumer camcorder, composite signal) S-VHS ( improved VHS, S-Video, midrange) Hi-8 ( improved 8 mm , S-Video/RGB,midrange) Betacam 1/2”(high end, studio, RGB and other) Betacam-sp (high end, studio, RGB and other) U-matic-3/4’’ - now obsolete Tapes (1’’) : Broadcast and studio
Type -B Type-C
www.smartphonetech.org
Digital Video Cameras
Type Price Note
SonyDSR-200
$6,400 DVCAM tapeFireWire Conn.
SonyDSR-130
$19,400 Multiple backs(tape formats)
PanasonicAJ-D700
$22,000 DVCAMAnalog feed-in
IkegamiHC-5900
11,106
JVCENG-1910
$11,000
www.smartphonetech.org
Media For Storage and Distribution
Considerations Capacity Streaming rates erasability transportability
Types Hard Disk Zip disks (between stations or for back up) Tape (between stations or for back up) CD-ROM
Writable for small volume Mass produced
DVD www.smartphonetech.org
Digital Video Recording:Tape Formats
Type ColourDepth
Sub-Sampling Compression
SonyDVCAM
8 bitcomp
4:1:1 5:1Intra-
Pana VisionDVC Pro
,, ,, ,,
Pana VisionDVC Pro-50
,, 4:2:2 3.3:1
JVCDigital-S
,, ,, ,,
SonyBetacam-SX
10 bit ,, 10:1interfram
D1 10 bit ,, Uncompressedwww.smartphonetech.org
Media For Storage:DVD
DVD = Digital Video/Versatile Disks The Latest Storage Medium, capable of
storing multiple full length feature films. Songs (multiple albums) Data (4 GB- 16 GB)
Standards still evolving Different Capacities
Single/double layer Single/double side(may require manual flip) Typical capacity of single side-single layer 4.7 GB
www.smartphonetech.org
DVD(contd.) Different Versions
DVD-Video DVD -Audio DVD-ROM DVD-R (write once-by user) DVD-RAM (rewritable)
DVD -Video and DVD -Audio may be played in dedicated players.
Computer may play all. DVD-R : Best suited for small volume multimedia content
distribution. DVD-RAM : Useful for object capture and edit Data rates 2-5 Mbytes /s (video)
Supports MPEG-II
www.smartphonetech.org
Video File Formats
Video For Windows AVI (Audio- Video interleaved) QUICKTIME
A very popular PC-Video format from Apple for Mac ,PC, Internet
Mature and open standard. Play backs most forms of files (CODECs). Suitable for virtually all forms of multimedia objects like
video, sound animation stills, MIDI,VR. Scaleable - quality adjusted automatically to suit
platform. Latest :- real-time support and Vector graphics in QT3.
Special Animation files (FLI, FLC)
www.smartphonetech.org
CODECS
Cinepack Most widely used and supported. Software and hardware versions available. Best for 320x240 @ 15 fps,180 KBps 10:1 achievable compression Shortcomings : pixelisation, chunky, poor colour
saturation. Indeo
From Intel (Hardware and software) 3x version good for talking heads, poor colour
saturation. 4x and 5x versions comparable to MPEG-1 Realtime adjustment of brightness and contrast
www.smartphonetech.org
Steps to PC Video
Plan Content Decide Target platform , run length and quality. Shoot and Grab or otherwise Acquire (vide
Video tips) Prepare Edit Decision List (EDL) Edit and Compose Dub Clean and Compress Store in CD or Distribute via Net
www.smartphonetech.org
Video Capture: Basics
Required for analog sources like . VCP VTP Laser-Disk players Camera
Requires Special Hardware (frame Grabber Cards) which Samples the video signal Digitise the chroma and Luma components Stores them in disk files Optionally compresses the data Displays while grabbing
www.smartphonetech.org
Video Capture Platforms.
Should be the highest performance machine affordable.
Would serve as video editing and compressing stations as well.
Windows NT Pentium-II, 128 MB memory, SCSI-Disk 4GB
AV, RAID is better. Separate disk for software and data storage
recommended. Zip drive for transportation of large files or 100 Mbps
fast Eathernet and of course the frame grabber card .... ...
www.smartphonetech.org
Video Capture Cards
Also called frame grabbers Some popular cards
Megamotion (JPEG) Broadway (MPEG-1) Video Blaster (Creative) Video Spigot (Creative) Matrox Illuminator
www.smartphonetech.org
Video Capture Cards :check list
Input Signal types Composite, S-Video, RGB Whether Audio channels supported NTSC/PAL/SECAM
Sampling rates supported Colour resolution (8/16/24) Whether full screen frame grab possible. Maximum frame rates Video Overlay, whether needs special video
cards. Whether saves in raw or compressed mode. Software Provided
www.smartphonetech.org
Video Capture Cards :Typical Specs Data Translation- Broadway
32 bit PCI master NTSC or PAL capture Frame Size: 352x288 (240) Colour: 16bit,YUV 4:2:2 SVideo and Composite Preview (15 fps typical ) Overlay (full fps) Capture to AVI (uncompressed) Capture to MPEG (compressed) 16 bit audio Bundled
Edit Software (Ulead Media Studio VE) and Compress Software (MPEG-1)
www.smartphonetech.org
Video Capture Cards :Connection
RGB or Composite orS-Video connectors
Feature connector
I/O bus connector
To mother board
To video source
To video Card
www.smartphonetech.org
Video Capture- considerations.
In house or outsource? Budget Type of capture CODEC Quality
Equipment availability High end camera/VTP/VCP Video capture platform , card with appropriate CODEC. Example
QUICKTIME Time required to perform the job.
Asymmetric Encoding Operator skill Do not use lossy compression if editing is envisaged(MJPEG-ok)
www.smartphonetech.org
Video Capture- considerations(contd.).
Two Approaches Best Capture Just Capture
Best Capture Capture with the highest screen size, colour depth and frame
rates Same for audio Better safe than sorry : you can always reduce information Huge file sizes Requires the most expensive boards
Just Capture Why bother for full frame and true colour when the target
machines cannot display them? Use the target screen size, colour depth and frame rates Save time and money
www.smartphonetech.org
Video Capture procedures
Estimate the file size. Create a blank file and defragment the disk. Check the settings Trial run
Electrical Settings Palette
www.smartphonetech.org
Video Editing Primer
Contents: Concepts Tools Adobe Premiere Story board
www.smartphonetech.org
Video Editing Concepts
Edit Decision List Linear and non linear editing
Edit like serial tape or Edit Like disks File edit
Just cut and paste True Edit
File edit and more Transition Conversion Superposition Picture edit
Filters, Colour correction &cwww.smartphonetech.org
Video Editing: Conversions (continued)
Cropping Resizing Pan & Scan
For 70 mm format Performed often at telecine stage.
Frame Rate Conversion More : duplicate frames Less: discard frames CAUTION !
Pace Slow motion Fast Play
Colour Depth www.smartphonetech.org
Transitions Dissolve
Gradual superposition of the incoming clip on the outgoing
May ustilise other subtechniques also. Fade-out
The ongoing clip gradually turns into black Fade-in
The incoming clip gradually turns from black to normal Wipe
A line divides the frame into the outgoing and incoming clip areas.
The line moves to increase the incoming clip area . Many more
www.smartphonetech.org
Superposition
Whatisit Many layers of picture with transparent and opaque
regions, superposed together Kinds
Picture +Text Matte + Text Video +Video Picture + Animation Video +Animation (Spilberg)
Keying Chroma Keying Alpha Channels: pixel by pixel transparency control
www.smartphonetech.org
Traditional Non-linear Editing! Tradition goes back to
Music Channels. “Jurassic Park” “Who framed Roger Rabbit”
Used Silicon Graphics High End Work stations Composite or RGB digitised Random Access Abekas Disks and Exabyte digital
tape drives Digital Edit and special effects including animation Tape to film
Not been a cup of tea for the PC’s But stay tuned
www.smartphonetech.org
Non Linear Editing using Model
Instead of an “edit decision list” on paper, one or more story boards may be created using compressed (lower quality) clips.
Such “dummy” products are called models of the final. We would call them prototypes.
Models may be iteratively improved. Compressed, smaller screen , slower frame
rate and lower resolution content allows quick prototyping and improvement.
www.smartphonetech.org
Video Editing Tools
VidEdit Comes Free with Video for Windows and many
frame grabber cards. From the stable of Microsoft.
Adobe Premiere Most popular
www.smartphonetech.org
VidEdit Features
Basic Video File Edit Tool. Input may be .AVI file, Autodesk Animator .FLI
or .FLC file. Cut , Paste and Insert of Video and Audio
components. Frame rate convert Colour Depth convert Crop and Resizing Palette building compression
RLE, Indeo, Microsoft video-1 Custom settings based on target (CD-ROM etc.)
www.smartphonetech.org
Adobe Premiere: Features
Fairly Comprehensive Video Clip Editing Wide selection of picture edit Conversion applied on whole clip Drag and Drop Wide selection of Transitions Film Strip metaphor Super Audio edit Compression
www.smartphonetech.org
Adobe Premiere Windows
Preview
Clip
A
B
Super
Videos
Transitions
Audios3 of
Construct
ProjectSources of all clips
Info
TransitionPalettes
Premier
www.smartphonetech.org
Adobe Premiere Construction Window
www.smartphonetech.org
Media Merge
Simpler video editor Uses story board metaphor Sequence of clip and transition boxes Text superposition Drag and drop Three components
Story board editor Scene editor Audio Editor
www.smartphonetech.org
Video Compression Techniques and Standards
JPEG & M-JPEG (intra frame) H.261 (intra frame) MPEG-1 MPEG-2 MPEG-4
www.smartphonetech.org
Tips for Video in multimedia
Shoot or Acquire Lights more Lights Shooting Ancillaries More tips
www.smartphonetech.org
Shoot or Acquire
Don’t shoot if good quality “footage” is available.
Employ a professional for shoots if fund permits.
Choose the best available camera (SVHS and High-8
recommended).
Tripod is equally important.
Lighting can make or break.
Choose a simple background.
Check if direct storage to PC is feasible.
See ‘tips’ at the end.
www.smartphonetech.org
Lights... more lights
Eye is more forgiving than camera, which needs better lighting.
At least three light sources (including natural light). The main light from front but a bit left (or right). The key light from up and behind. A compensatory light to reduce stark shadows
from the main Don’t neglect the background:
The background and the foreground should have difference in illumination.
For bright background , exposure should be adjusted or the subject would look too dark.
www.smartphonetech.org
Shooting Ancillaries
Reflectors Thermocol on masonite backing Aluminium foil White board
Halogen main light Halogen “table lamps” and stand Turn table (for machine parts and small
objects) Back screen
Remember chroma key Prompting stand Mike boom
www.smartphonetech.org
More Tips for Video in multimedia
While shooting landscapes and large buildings, use slow pans. It would look good, compress well.
Use shooting notes - detailing location, date, event , shot number, identity of persons etc.. Do not rely on memory.
Always capture audio along with video. It would back up your shooting notes.
Take shots that may be used as the back drop of titles for long clips.
www.smartphonetech.org
More Tips
For indoor objects experiment using short clips with back ground lighting shooting techniques filtering rate conversions compression.
This extra trouble would be rewarded. Cuts take less disk space than transition . Use
transitions judiciously. Use fonts specially designed for video (e.g.
sans serif faunts) in titles.www.smartphonetech.org
The Future :
DVD video’s at Indian homes and libraries. CD-ROM social function video a must for
Indian middle class. DVD-video of museums Indian Cartoon films (Amar Chitrakatha) on
CD-ROM’s and DVD. Amature film directors. Spielberg from India Desktop Editing of Broadcast Quality Software Video on demand
www.smartphonetech.org
Desktop Editing of Broadcast Quality Content
Favourable Factors: Digital Camera, DVD Players and HDTV Increased Processing Power at the Desktop at
affordable price. Cost 1/100 th of present analog edit station cost Emergence of MPEG-II
Special Equipment would still be required Media
Tape .. Computer .. Tape Analog Tape or Digital ? DVD.. or Tape ? Camera to Computer ?
India- the destination ? Labour intensive Special Care, Closer control
www.smartphonetech.org
Thank You
www.smartphonetech.org