VideoTagVideo Display and Recording

Aaron Shepler

Motivation

• Enable the user to view real-time video from the camera

• Allow the user to annotate on top of the video image

• Facilitate simple video recording

Previous / Related Work

• Point Grey Research has written a C++ wrapper for reading/writing to an AVI

• Microsoft is working on an InkCanvas control (to be released in Vista) that would potentially allow for something similar – inking on multimedia

Challenges

• How do I use the camera in C# when the interface is in C?– The .NET framework allows for easy

integration between different managed types of code: i.e. Managed C++ and Managed C#

– Integration between unmanaged code and managed code is not very well documented

Challenges

• How do I place the image on the screen once I have data using C#?– Lower level operations, such as direct access

to screen are not available

Challenges

• How do I manage the recording of the video– C# does not have built-in AVI recording

functionality– Function calls from C would also have to be

wrapped

Camera Approach / Solution

• Two possible approaches that allow for use of camera in C#– Link in flycapture camera methods directly

• Large blocks of unsafe { } code due to use of pointers

– Link in a library that simplifies the interface• Few lines of unsafe { } code

Camera Approach / Solution

• Mistake…– First tried to link in flycapture camera methods

directly• Unsafe { } blocks of code were everywhere due to

the use of pointers• Became a C-based program

– Learned my lesson the hard way

Camera Approach / Solution

• Encapsulated lower level camera functionality in a library…– Created a small library of methods

• Interface was well defined• Less time spent matching compatible types

Camera Approach / Solution

• CameraLib– Library with basic camera functionality

• void initialize();– Initializes the first camera on the FireWire bus, creates

context, etc

• void* grabImage(unsigned char*);– Retrieves an image based on the camera context

• void destroy();– Destroys the camera context

– Pure C library for use in managed code

Video Approach / Solution

• In order to place the image onto the screen, the InkPicture control was used– Allows the background image to be changed

while maintaining the ink on top.– Create thread in C#

• continuous capturing / refreshing the image

Recording Approach• Record to Avi using C# wrapper class

– Same refresh thread adds the current bitmap to the next frame in the AVI

– Pro:• Easy to access bitmaps out of the AVI later

– Con:• Large file is unhandy

• Avi wrapper written orginally by René Nyffenegger (modified by John Corinna), located at:– http://www.codeproject.com/csharp/steganodotnet4.asp

Demonstration

Discussion

• How can we reduce the slight flicker with the video/ink?– Double buffering?– User Drawn Control?

Discussion

• What methods of encoding the video data would be faster and more robust than AVI?– Various compression models?– Existing codecs?