An Interactive and Team Approachto Multimedia Design Curriculum

IEEE SIGNAL PROCESSING MAGAZINE [14] NOVEMBER 2005

[dsp EDUCATION]Min Wu and K.J. Ray Liu

1053-5888/05/$20.00©2005IEEE

Over the past decade,increasingly powerfultechnologies have made iteasier to compress, dis-tribute, and store multi-

media content. Many internationalmultimedia standards have been estab-lished to enable new technologies—HDTV, DVD, digital audio, and real-timeInternet games, just to name a few—to beenjoyed by millions of people around theglobe. Simultaneously, advances in wire-less and networking technologies, alongwith a significant decrease in cost for stor-age media, have led to the proliferation ofmultimedia data. This convergence of net-working, computing, and multimediatechnologies has collapsed the distancebetween the ability to create content andthe ability to consume content.

The merger of computing and com-munications has created a ubiquitousinfrastructure that brings digital multi-media closer to users, opening uptremendous educational and commer-cial opportunities in multimedia contentcreation, delivery, rendering, and archiv-ing for millions of users worldwide. Assuch, multimedia has become a basicskill demanded by an increasing numberof potential jobs for electrical engineer-ing/computer science (EE/CS) gradu-ates. Yet multimedia has been widelyregarded by educators as a very difficultsubject to teach at the undergraduatelevel because it involves a wide spectrumof issues, including coding, compres-sion, enhancement, communication,data mining, and rendering of speech,audio, image, video, and their combina-tions; it also involves interaction withdatabases, networking, and artificialintelligence. At the graduate level, oftenwe find each medium (e.g., image andspeech) is a subject taught separately,

focusing on specialized techniques andadvanced theories. Given the limitedtheoretical foundations of undergradu-ate students, one can imagine that it iseven harder to teach them multimediatechnologies in an interesting, yet non-superficial way.

Multimedia has many practicalaspects, with a variety of applicationsattractive to undergraduate students,especially members of the new digitalgeneration who are introduced to allsorts of digital media as children. A stu-dent must have some hands-on experi-ence in developing algorithms andperforming simulations before he/shecan have a level of true understandingand appreciation of the theoreticalaspects in advanced studies.

How much can an undergraduate stu-dent learn and practice during onesemester? And what is the best way forthem to learn multimedia? During thefall semester of 2001, the University ofMaryland, College Park, embarked on thedevelopment of a multimedia CapstoneDesign course targeting senior-level stu-dents. The course has been offered to sen-iors and first-year graduate studentsevery semester since. It serves as our firststep toward a new way of teaching under-graduate multimedia. The central idea isto teach students the principal conceptsthrough lectures, then to reinforce theirunderstanding through hands-on labexercises and team projects.

In this article, we intend to shareour experience and new ways of think-ing about curriculum development. Wehope it will be beneficial for colleaguesin multimedia signal processing areasfor use in developing or revising cur-riculum to fit the needs and resourcesof their own programs. We would like tocordially invite interested colleagues to

join us in building an internationalpartnership on multimedia research andeducation for a broader impact.

VISION AND CONCEPTSOF INNOVATIVEMULTIMEDIA EDUCATIONThe multimedia curriculum at the seniorlevel has high potential impact in termsof the number of prospective students.This makes it very important to thinkcarefully about the vision and conceptson which we want to base our uniqueapproach. As it is unrealistic to expect anundergraduate student to master theextensive theoretical background in amatter of months and finish with a per-fect project, our vision is to

■ offer strong intuitive componentsrevealing a solid scientific perspectivesuch that students with limited back-ground can participate and contribute ■ provide room for creative think-ing through open-ended problemformulations■ match students with various back-grounds to the corresponding hands-on, open-ended projects that willcapitalize on their existing skills andspecial talents■ utilize visualization, interactivetools, or computer-aided designfacilities to shorten lengthy learningand implementation time so thatthe focus can be on the appropriateengineering context.In addition to understanding the fun-

damental aspects of engineering disci-plines, students need skills beyond theirtechnical fields. The technology develop-ment paradigm in the IT era is being revo-lutionized; now someone with an idea andknow-how can pull together enoughresources—capital, talent, facilities, andmarket—to be able to realize his or her

[dsp EDUCATION] continued

IEEE SIGNAL PROCESSING MAGAZINE [16] NOVEMBER 2005

dream. This paradigm of students’ futuresis quite different from the traditional pathof finding engineering jobs in large corpo-rate and research laboratories and fittingin. The entrepreneurial spirit can beessential to the success of engineers of thedigital generation. We try to incorporateelements of entrepreneurship in the pro-posed curriculum through a term project.

A FUN WAY OF LEARNINGMULTIMEDIA Based on the vision and conceptsdescribed above, we developed and firstoffered the multimedia Capstone course(ENEE408G) in spring 2002. The experi-ment has been quite encouraging, as evi-denced from a report in an interview inthe summer 2003 issue of the UMD ECEConnections newsletter. Our course devel-opment has employed a number ofmethodologies that have made ENEE408Ga unique and attractive course.

WELL-DEVELOPED, HANDS-ON LAB ASSIGNMENTSAs shown in Table 1, one of the two par-allel tracks of the course covers the fun-damentals of four basic media types(speech, audio, image, and video)through eight-week lectures and labassignments. We use PowerPoint presen-tations and Flash demos to illustrate fun-damental concepts and multimediaexamples that cannot be vividly conveyedusing traditional blackboard teaching.

The lecture learning is complement-ed by four lab assignments on audio,speech, image, and video. Each lab con-sists of tasks of increasing difficultyand creativity levels. For example, inthe “Image Processing and DigitalPhotography” assignment, we first pro-vide students with conceptual under-standing through playing with apowerful image editing tool (e.g., PaintShop Pro or Adobe Photoshop) to accom-plish several image processing andenhancement tasks. We then give stu-dents a few MATLAB building blocks andask them to integrate and build a simpleJPEG-like image coder. Next, studentsare asked to study a sample program inthe Pocket PC multimedia programmingmanual that we have developed for them

and extend it to design a small PocketPCimage processing program usingMicrosoft’s embedded Visual C++/Basic.Finally, each student team is provided ahigh-end digital camera to learn digitalphotography first hand and create an artpiece using the image processing tech-niques that they have learned.

INNOVATIVE TERM PROJECT WITH ENTREPRENEURIAL SPIRITThe second track shown in Table 1 is aninnovative term project in which stu-dents work in teams to develop a creativeproduct on a desktop platform or amobile platform such as PocketPC. Sucha learning process is described in thecourse syllabus as follows:

Term Design Project: This is ateam-based final project on designand implementation of multimediasignal processing systems usinghandheld devices or desktop com-puters. Each student team will emu-late a high-tech company that will:

1) develop a product idea anddecide on system specifications of amultimedia product,

2) partition and coordinate thedesign tasks,

3) implement, test, and docu-ment the design, and

4) demonstrate and market theproduct.

BUILDING A TEAM CULTURE WITHSOUND ORGANIZATIONAL SKILLSWe recognize that it is unrealistic toexpect every student to master an exten-sive technical background and finishwith a perfect project in a matter ofmonths. Therefore, teamwork andsound team spirit are key to success. Weencourage students to use their imagina-tion and creativity to find an interestingidea that they are excited to develop;then we guide them to complete a proof-of-concept prototype within four to sixweeks through well-organized teamwork.

We experimented with various ways offorming student teams. For example, stu-dents are assigned at random to formteams for the first design project, and theyare assigned to work with different part-ners on the second project. From the thirdproject on, the students are asked to teamup voluntarily and stay with one team forthe remainder of the semester. This way“forces” all students to learn to work withpeople they are unfamiliar with, yet allowsfor some stability and enough time todevelop expertise and team culture for thefinal big project. Students have incentivesto behave well and contribute to the teamwork from the first project; otherwise,they may be embarrassed if no groupwants them as a member.

A peer evaluation form is collectedafter each project to give us an idea

IEEE SIGNAL PROCESSING MAGAZINE [16] NOVEMBER 2005

WEEK LECTURE LAB ASSIGNMENT TEAM PROJECT1 INTRODUCTION WARM UP ON

MATLAB ANDPOCKETPC

2 IMAGE PROCESSING IMAGE TEAM UP34 VIDEO CODING VIDEO5 IDEA DEVELOPMENT6 SPEECH PROCESSING SPEECH

AND RECOGNITION7 DEFINE SYSTEM SPEC8 AUDIO PROCESSING AUDIO SCHEDULING

AND RIGHTS MANAGEMENT

9 INSTRUCTOR’S APPROVAL OF DEVELOPMENT PLAN

10 PROJECT DEVELOPMENT: 1) JOB PARTITION, 2) PROGRESS REVIEW, 3) VERIFICATION, 4) INTEGRATION AND TESTING, AND 5) DOCUMENTATION

11121314 PROJECT PRESENTATION AND DEMO15 FINAL REPORT DUE

[TABLE 1] ENEE408G COURSE SCHEDULE CHART.

about team dynamics and help resolveany problems. The evaluation form alsoasks each student to briefly summarizehow the job is divided, processed, andintegrated by the team. One intent ofthis question is to ensure that a studentunderstands both his or her own roleand other teammates' roles in the proj-ect, regardless of whether the studentwas in a leader position or was being led.

INCORPORATE UP-TO-DATEDEVELOPMENT TOOLS AND DEVICESTo prepare students to meet the demandsof the IT workforce of the 21st century, itis important to equip them with behind-the-scenes knowledge as well as algo-rithm development and implementationskills. We incorporate the most up-to-date, state-of-the-art tools and devicesinto the course, such as PocketPC, digi-tal photo and video cameras, multimediaeditors, and speech processing software.Recognizing that many employers expectthat students have already mastered cut-ting-edge software tools and platforms,we have adopted in our course the latestplatforms on desktop PC and PocketPC,as well as major prototyping and devel-opment tools such as MATLAB, VisualStudio, and Embedded Visual Studio.Given today’s rapidly advancing pace inthe technology field, the ability to learnto use references and examples to quick-ly master new concepts and tools hasbecome an important skill for EE/CS stu-dents. This is why we have provided stu-dents with examples and reference

material to help them learn new toolsfor new platforms (such as program-ming on PocketPC), which is mostlyconducted on their own. Through labassignments and projects, students mas-ter the tools with first-hand experienceand extensive practice.

SYSTEMATIC ASSESSMENT OF STUDENTS’ LEARNING We have constructed a systematic assess-ment mechanism for student projectsand teamwork. For each lab and project,we have developed a grading sheet thatoutlines major tasks, requirements, andgrading criteria. At the beginning of theproject, we clearly convey to the studentsthe expectations and rationale behind thegrading in terms of developing key skillsuseful for their future careers. Alongwith our department undergraduateoffice, we have developed assessmentrubric on oral communications, writtenreport, and engineering designs, to quan-titatively assess students’ performanceand provide timely feedback that helpsstudents improve.

For team-oriented projects, we havedeveloped the teamwork evaluation ques-tionnaire to solicit students’ peer assess-ment. Handed out before the start of theproject, the questionnaire is intended toprovide constructive guidelines andincentives for students to develop effectivemechanisms for working together. Forexample, the importance of timely com-munication for effective teamwork isemphasized on the questionnaire througha multiple-choice question about how

[FIG1] Screen shots of the student project “Kanji Tutor.”

IEEE SIGNAL PROCESSING MAGAZINE [17] NOVEMBER 2005

[dsp EDUCATION] continued

IEEE SIGNAL PROCESSING MAGAZINE [18] NOVEMBER 2005

soon students communicate and respondto teammates, whether it is promptly(within a day), moderately (within two tothree days), or rarely. We complement thepeer evaluation with close observationand interaction with the student teamduring the lifespan of the project. Thisallows us to provide timely feedback andguidance for students to resolve issues inteam coordination and improve the effec-tiveness of teamwork.

EXAMPLES OF TERM PROJECTS“Kanji Tutor” is one example of capitaliz-ing on students’ special talents and cre-ativity. As a beginner learning Japanese,one student led her team to develop anovel PocketPC tool to facilitate learningto read and write sophisticatedJapanese/Chinese characters (“Kanji”).

As illustrated in Figure 1, the studentteam developed an algorithm to verifythe correctness of Kanji writing in differ-ent strictness levels. The team designed afriendly user interface and createdunique animations and quizzes to helpusers memorize Kanjis.

Inspired by a TV remote controllersoftware coming with PocketPC, anotherstudent team took just six weeks to com-plete the creation of “PocketPoint”shown in Figure 2. This prototypedemonstrates the concept of usingPocketPC to wirelessly control the flowof a desktop PowerPoint presentation; itprovides a mobile visual aid such asthumbnails or a preview of slides. Withthis unique course, the creativity andproof-of-concept capability achieved bythe undergraduate students are often far

above expectation. For example, an ideasimilar to PocketPoint was independent-ly investigated as a multiyear researchproject at another institution.

A student team led by an amateurguitar player developed “PocketInteractive Chords,” a PocketPC applica-tion that allows users to tune a guitarand learn to play basic guitar chords.Capturing guitar sound from the built-inmicrophone of PocketPC, the studentsused their signal processing knowledgeto develop estimators for tuning guitarstrings and identifying notes and chordsbeing played. They designed other funmodules and integrated them into acomplete program with a rich set of fea-tures, including tuning, tutoring, andinteractive quiz, as shown in Figure 3.

We have also exposed students to top-ics from the latest research and haveencouraged them to develop projectskeyed to emerging technologies. Securemultimedia multicast, content-basedaudio fingerprinting, and digital water-marking are just a few examples. Forthese research-oriented projects, weinvited graduate students to serve asmentors for the undergraduate teams.These efforts have provided an effectivevehicle for fostering interaction betweenundergraduate and graduate studentsand integrating research and undergrad-uate education.

SHARING AND COLLABORATION OF MULTIMEDIA RESEARCH AND EDUCATION Multimedia has become a vital compo-nent of the digital era, and it has funda-mentally changed the way we work andlive. Research related to multimedia hasgrown to encompass a number of areas,such as multimedia coding, communica-tions, synthesis, indexing, search,retrieval, recognition, understanding,security, and forensics. The global natureof information technology has nurtureda number of strong research groups withunique strengths worldwide. Multimediaresearch has evolved to a stage where asingle institution often specializes in asubset of active areas or particularmethodologies. Collaboration amongmultiple institutions across the globe

IEEE SIGNAL PROCESSING MAGAZINE [18] NOVEMBER 2005

[FIG2] Screen shots of the student project “PocketPoint.”

(a) (b)

[FIG3] Screen shots of the student project “Pocket Interactive Chords.”

(a) (b) (c)

IEEE SIGNAL PROCESSING MAGAZINE [19] NOVEMBER 2005

will provide a breeding ground for coop-erative research and education for multi-media technologies with much broaderimpact. This will, in turn, allow peopleall over the world to overcome physicaldistances and access a huge amount oftext, image, audio, and video informationthrough networks.

To respond to the global demand andthe technical skills required by the newgeneration of the workforce, we aresharing our development experienceand material. We intend to build aninternational partnership for a com-bined research and education programon multimedia. The program will offermultimedia education at the undergrad-uate level with a hands-on, team-orient-ed approach, and it will aid incooperatively developing and transfer-ring the latest results from multimedia-related research to education in a timelyfashion. Close collaboration in develop-ing and sharing research results andresources can expedite the transfer ofnew research into curriculum for theglobal, high-tech multimedia work-force. A set of course modules, includingpresentation material, lab assignments,and sample codes, will form a repositoryto enable multimedia educators world-wide to assemble suitable material forbuilding a multimedia course tailoredto their specific needs and facilities. Ifany reader is interested in teaming upand participating in this program,please feel free to contact the authors.

FINAL REMARKSA number of students have found themultimedia course added much to theirpersonal and professional interests.Student comments include:

“This course was my first full-fledged exposure to multimedia sig-nal processing. Now, I hope toinvestigate multimedia signal pro-

cessing in work or in graduateschool at least on some level. Thishas become one of my favoriteinterdisciplinary topics.”

“I had a lot of fun in yourcourse. I learned a lot, which Ihope to use very shortly. I have aproduct idea that I am going tostart designing . . .”

“The reason I applied for jobsinstead of graduate schools last fallis that I had not yet found a problemthat I was passionate about. . . .However, last semester, I finallyfound what I was looking for. I wantto devote myself to signal processingchallenges . . . Taking 408G openedmy eyes to all sorts of signal process-ing problems I would love to tackle.”Dr. Teddy Kumar, technical director of

Media Vision Laboratory of SarnoffCorporation, came to the University ofMaryland in early March to recruit students.A number of former and current ENEE408Gstudents were invited for interviews. In an e-mail to the instructors, Dr. Kumar wrote,“Almost all students said that the best coursethey had taken at the University of Marylandwas the multimedia signal processing course.They were all excited about their work init and the hands-on project experience.”

For more information, please visit theonline showcase at www.multimedia.umd.edu or contact the authors at kjr-liu@umd.edu and minwu@umd.edu.

AUTHORSMin Wu is an assistant professor of theDepartment of Electrical and ComputerEngineering and the Institute ofAdvanced Computer Studies at theUniversity of Maryland, College Park.

K.J. Ray Liu is a professor in theElectrical and Computer EngineeringDepartment and the Institute of SystemsResearch at the University of Maryland,College Park.

IEEE SIGNAL PROCESSING MAGAZINE [19] NOVEMBER 2005IEEE SIGNAL PROCESSING MAGAZINE [19] NOVEMBER 2005

[SP]