View
2.768
Download
1
Category
Tags:
Preview:
Citation preview
MPEG-VMedia Context and Control
Christian Timmerer
Klagenfurt University (UNIKLU) Faculty of Technical Sciences (TEWI)Department of Information Technology (ITEC) Multimedia Communication (MMC)
http://research.timmerer.com http://blog.timmerer.com mailto:christian.timmerer@itec.uni-klu.ac.at
N107{82,83,85,86} – Text of ISO/IEC 23006-[1-4] CD
Christian Timmerer, Klagenfurt University, Austria 2
MPEG-V System Architecture
2009/07/03
Media context and control
Pt. 1: Architecture
Pt. 3: Sensory Information
Pt. 4: Avatar Information
Pt. 2: Control Information
Christian Timmerer, Klagenfurt University, Austria 3
Control Information
2009/07/03
Sensory Device Capabilities asext. of dia:TerminalCapability• unit, max/minIntensity, numOfLevels, delay, position• light (color, flash), heating, cooling, wind, vibration• scent, fog, water sprayer, color correction• kinestetic, tactile
User Sensory Preferences asext. of dia:UserCharacteristics• adaptability, max/minIntensity• light (color, flash), heating, cooling, wind, vibration• scent, fog, water sprayer, color correction• kinestetic, tactile
Fundamental Input to any Control
Device (aka Adaptation Engine)
Christian Timmerer, Klagenfurt University, Austria 4
Avatar Characteristics• Appearance
– Contains the high level description of the appearance and may refer a media containing the exact geometry and texture
• Haptics Properties– Contains the high level description of the haptics properties
• Animation– Contains the description of a set of animation sequences that the avatar is able to
perform and may refer to several medias containing the exact (geometric transformations) animation parameters
• Communication Skills– Contains a set of descriptors providing information on the different modalities an avatar
is able to communicate• Personality
– Contains a set of descriptors defining the personality of the avatar• Control
– Contains a set of descriptors defining possible place-holders for sensors on body skeleton and face feature points
2009/07/03
Christian Timmerer, Klagenfurt University, Austria 5
What is MPEG Sensory Information?
• UMA = Universal Multimedia Access– Access to content anywhere, anytime, any device == addressing QoS issues– Technical feasible thanks to standards ➙ MPEG-21 DIA
• UME = Universal Multimedia Experience– User-centric approach – takes into account the user– Quality of Multimedia Experience
• Sensorial, e.g., sharpness, brightness• Perceptual, e.g., what/where is the content• Emotional, e.g., feeling, sensation
• MPEG-V Sensory Information– Consumption of multimedia assets may stimulate also other senses, e.g., olfaction,
mechanoreception, or thermoception– Achieved by annotating the media resources with sensory effects
➙enhanced, worthwhile, and informative user experience, giving the user the sensation of being part of the actual media
2009/07/03
N9113
N10785
Christian Timmerer, Klagenfurt University, Austria 6
Concept of MPEG-V Sensory Information
2009/07/03
Christian Timmerer, Klagenfurt University, Austria 7
MPEG-V Sensory Information
System Architecture
2009/07/03
Scope of Standardization
Standardized representation device capabilities &➙
user preferences
Adaptation
Native representation ➙ standardized representation
Delivery
Authoring
Packaging
Christian Timmerer, Klagenfurt University, Austria 8
Committee Draft Structure
• Sensory Effect Description Language– Basic building blocks: GeneralInformation,
Declaration, Effect, GroupOfEffects, Parameter, ReferenceEffect
– Common attributes: XSI (timing), activate, duration, fade, alt, priority, intensity, adaptType, adaptRange
• Sensory Effect Vocabulary– Light (colored, flash), temperature, wind, vibration– Water sprayer, scent, fog, shadow, color correction– Kinesthetic, tactile
2009/07/03
Christian Timmerer, Klagenfurt University, Austria 9
Demo Setup
2009/07/03
Display
FanFan
Vibration Bar
Subwoofer
Wall-washer / Light
Light Light
Speaker Speaker
Christian Timmerer, Klagenfurt University, Austria 10
Demo Video with Effects
2009/07/03
Christian Timmerer, Klagenfurt University, Austria 11
Kinesthetic / Tactile Effects
2009/07/03
Viewers Television
Touch or exploration
Manipulation
Following movements
Feeling haptic effects
Haptic interaction
♬♩♪♫
Video
Audio
Haptic TextureForce
3D CG model
Text
abcdefg
Dynamics
Christian Timmerer, Klagenfurt University, Austria 12
Kinesthetic / Tactile Effects (cont’d)
2009/07/03
Active interactions
Passive interactions
Christian Timmerer, Klagenfurt University, Austria 13
MPEG-V SI References• N10782, Text of ISO/IEC 23006-1 CD• N10783, Text of ISO/IEC 23006-2 CD• N10785, Text of ISO/IEC 23006-3 CD• N10786, Text of ISO/IEC 23006-4 CD
• Ad-hoc Group– metaverse@lists.uni-klu.ac.at– http://lists.uni-klu.ac.at/mailman/listinfo/metaverse
• Some selected publications– J. Cha, I. Oakley, Y.-S. Ho, Y. Kim, J. Ryu, “A Framework of Haptic Broadcasting”, IEEE Multimedia, in
print, Sept-Oct. 2009– J. Cha, S.-Y. Kim, Y.-S. Ho, J. Ryu, “3D Video Player System with Haptic Interaction based on Depth
Image-Based Representation”, IEEE Transactions on Consumer Electronics, Vol. 52, No.2, pp.477-484, May 2006
– M. Waltl, C. Timmerer, H. Hellwagner, A Test-Bed for Quality of Multimedia Experience Evaluation of Sensory Effects, Proceedings of the First International Workshop on Quality of Multimedia Experience (QoMEX 2009), San Diego, USA, July 29-31, 2009.
2009/07/03
Christian Timmerer, Klagenfurt University, Austria 14
Thank you for your attention
... questions, comments, etc. are welcome …
Ass.-Prof. Dipl.-Ing. Dr. Christian TimmererKlagenfurt University, Department of Information Technology (ITEC)
Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIAchristian.timmerer@itec.uni-klu.ac.at
http://research.timmerer.com/Tel: +43/463/2700 3621 Fax: +43/463/2700 3699
© Copyright: Christian Timmerer
2009/07/03
Recommended