Abstract Media SpacesRob Diaz-Marino

CPSC 781University of Calgary2005

OutlineWhat is Abstraction?Simple Media SpacesAbstract Media Spaces (AMS)Benefits and DrawbacksMethods for Abstracting MediaDesigning an AMSSummary

What is Abstraction?Guernica by Pablo PicassoReaction to the bombing of Guernica in World War II.

What is Abstraction? (2)Throwing away information [1]RealisticAbstractFemaleBrown HairYelling / AngryFemaleBlond HairYellingEmotion?FactAbstractionFemaleYelling / DistressedIn a barn/cellarPart not drawnNo contextLips, eyebrows, fingers, mouth, hairEyes on same side of head-Eyes in wrong placeDistorted body shapeHair on back of headBlue-gray colorImage croppedUnclear contextPhotoCartoonArt Piece

Simple Media Spacesand so on.

Simple Media Spaces (2)Literal TransmissionInput = OutputLow degree of abstractionSome loss from original (reality)Capture limitationsCompression bandwidth limitationsStill perceptually equivalent

Why use Abstraction in a MS?To control informationPreserve PrivacyShield sensitive detailsReduce DistractionEliminate unnecessary detailsRe-map awareness cuesReduce bandwidth needsBob is at his computerBob is not wearing clothesBob is has a fruit-basket hat on his headBob is yelling at his girlfriend on the phoneBob looks angrySomeone is at the computerThe person is mostly flesh-coloredThe person has something large on their headThe person is speaking loudlyNo details can be seen on their faceAbstraction[1]

Drawbacks of AbstractionLoss of informationUseful: Identity, Actions, Availability, etc.Incidental: Details, Emotional state, etc.Loss of understandingUnclear meaningUnclear contextBob is at his computerBob is not wearing clothesBob is has a fruit-basket hat on his headBob is yelling at his girlfriend on the phoneBob looks angrySomeone is at the computerThe person is mostly flesh-coloredThe person has something large on their headThe person is speaking loudlyNo details can be seen on their faceAbstraction

Methods of AbstractionSimple DegradationFeature Extraction (Silhouetting)

Simple DegradationVideoStill resembles originalEx. Mike Boyles Video Filters [4]Video-InVideo-OutBlur FilterPixelationVideo-Out

Simple Degradation (2)AudioStill resembles originalEchoMufflingAudio-InAudio OutAudio Out

Feature ExtractionVideoVision TechniquesMotion detectionPresence detectionEye trackingFace tracking

Feature Extraction (2)AudioEx. Smith et als Low Disturbance Audio [3]Speech Non-SpeechCan still recognize voiceCannot understand wordsSimilar to Blur Filter but for Audio!Audio-InAudio OutExtractSynthesize

Feature Extraction (3)TextEx. Syllable replacementJim: Hi! How are you doing?Bob: Doing okayJim: Are you busy?Bob: Im on the friggin phone!!Jim: Oh, sorry!Jim: Bla! Bla bla bla blabla?Bob: Blabla blablaJim: Bla bla blabla?Bob: Bla bla bla blabla bla!!Jim: Bla, blabla!Extract

Media TranslationConvert one media form to anotherNo direct translationFeature ExtractionSynthesizer Visualization, Sonification, etc.Video-InAudio-OutExtractSynthesize

Media Translation (2)Ex. AROMA [1]Peripheral awareness

Media Translation (3)Ex. Cambience (my thesis project)InputsWeb Cam (video)Feature extractionMotion detection, partitioning, thresholding, etc.OutputsAudio volume, pan, etc.

Translation PitfallsExtreme abstractionNo longer understandableUsable only as art pieceLearning CurveArbitrary mappingsUsers may need to see literal data [1]

Designing an AMSProcessingMust be done in REAL TIMECan lower sampling rate to compensatePeripheral vs. ForegroundDraw InspirationAmbient DisplaysVisualizations

Designing an AMS (2)3 ArchitecturesClient-side processingServer-side processingDistributed processing

AMS Architectures (1)Client-Side ProcessingTransmit raw data privacy risk!High Bandwidth usageLow CPU for Server, high for Clients

AMS Architectures (2)Server-Side ProcessingTransmit synthesized mediaHigh Bandwidth usageHigh CPU for Server, low for Clients

AMS Architectures (3)Distributed ProcessingTransmit extracted featuresLower Bandwidth usageLower CPU for Clients and Server

Designing an AMS (3)Ex. CambienceVideo InputFeature extraction on ServerTransmissionScalar valuesAudio OutputSound synthesis on Client

SummaryAbstract Media SpacesThrow away informationSimple DegradationFeature ExtractionCan provide a privacy shieldCan provide better peripheral awarenessAllow media re-mappingCan lower bandwidth usage

Questions?

ReferencesPedersen, E. R., Sokoler, T. (1997) AROMA: Abstract Representation of presence supporting Mutual Awareness. Proceedings of CHI97, 51-58.Wikipedia: The Free Encyclopedia. (n.d.) Retrieved October 2005 from http://en.wikipedia.org/wiki/Abstract_art, http://en.wikipedia.org/wiki/Pablo_Picasso, http://en.wikipedia.org/wiki/CubismSmith, I., Hudson, S. (1995) Low Disturbance Audio For Awareness and Privacy in Media Space Applications. In proceedings of ACM Multimedia 95, ACM Press, p. 91-97Boyle, M. (2005) Privacy in Media Spaces. PhD Thesis, Department of Computer Science, University of Calgary, Calgary, Alberta CANADA T2N 1N4. April.