3.1 Audio, Video, Speech Synthesis and Recognition

2004 Prentice Hall, Inc. All rights reserved.

Multimedia: Audio, Video, Speech Synthesis and Recognition

Outline28.1 Introduction28.2 Audio and Video28.3 Adding Background Sounds with the bgsound Element28.4 Adding Video with the img Element’s dynsrc Property28.5 Adding Audio or Video with the embed Element28.6 Using the Windows Media Player ActiveX Control28.7 Microsoft Agent Control28.8 RealOne Player Plug-in28.9 Synchronized Multimedia Integration Language (SMIL)28.10 Scalable Vector Graphics (SVG)28.11 Web Resources


Objectives

• In this lesson, you will learn:– To enhance Web pages with sound and video.– To use the bgsound element to add background sounds.– To use the img element’s dynsrc property to incorporate

video into Web pages.– To use the embed element to add sound or video.– To use the Windows Media Player ActiveX control to play a

variety of media formats in Web pages.– To use the Microsoft Agent ActiveX control to create

animated characters that speak to users and respond to spoken commands from users.

– To embed RealOne Player™ to include streaming audio and video in a Web page.

– To embed video and create graphics in a Web page using SMIL and SVG.


28.1 Introduction

• Multimedia– Use of sound, images, graphics and video– Add sound, video, and animated characters to Web-based

applications

– Streaming technologies


28.2 Audio and Video

• Can be embedded in Web page• Can be downloaded “on-demand”

• Encoding algorithm or codec– Transforms raw audio or video into a format that Web

browsers can display


28.3 Adding Background Sounds with the bgsound Element

• bgsound element– src property

• Specifies the URL of audio clip to be played

– loop property• Specifies number of times audio clip will play

– balance property• Specifies balance between left and right speakers

– volume property• Determines volume of audio clip

2004 Prentice Hall, Inc.All rights reserved.

OutlineOutline

BackgroundAudio.html(1 of 4)

1 <?xml version = "1.0"?>

2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"

3 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

4

5 

6 

7

8 <html xmlns = "http://www.w3.org/1999/xhtml">

9 <head><title>The bgsound Element</title>

10 <bgsound id = "audio" src =

11 "http://msdn.microsoft.com/downloads/sounds/jazzgos.mid"

12 loop = "1"></bgsound>

13

14 <script type = "text/javascript">

15 

50 </script>


OutlineOutline


51 </head>

52

53 <body>

54 <h1>Background Music via the bgsound Element</h1>

55 <h2>Jazz Gospel</h2>

56

57 This sound is from the free sound downloads at the

58 <a href =

59 "http://msdn.microsoft.com/downloads/default.asp">

60 Microsoft Developer Network</a> downloads site.

61 <hr />

62 Use the fields below to change the number of iterations

63 and the volume for the audio clip<br />

64 Press <strong>Stop</strong> to stop playing the sound.

65 <br />Press <strong>Refresh</strong> to begin playing

66 the sound again.

67

68 <form name = "audioForm" action = "">

69 <p>Loop [-1 = loop forever]</p>

70 <input name = "loopit" type = "text" value = "1" />

71 <br />Volume [-10000 (low) to 0 (high)]

72 <input name = "vol" type = "text" value = "0" /><br />

73 <input type = "button" value = "Set Properties"

74 onclick = "changeProperties()" />

75


OutlineOutline


76 <input type = "button" value = "Stop Sound"

77 id = "stopButton" onClick = "stopSound()" />

78 </form>

79 </body>

80 </html>


28.4 Adding Video with the img Element’s dynsrc Property

• img element– Incorporates both images and videos– src property

• Indicates source is image

– dynsrc property• Indicates source is video clip

– start property• Specifies when video should start playing• Event fileopen

– Video should play as soon as it loads• Event mouseover

– Video should play when user position mouse over video


OutlineOutline

Dynamicimg.html(1 of 2)




4

5 

6 

7


9 <head>

10 <title>An Embedded Video Using the dynsrc Property</title>

11 <bgsound src =

12 "http://msdn.microsoft.com/downloads/sounds/carib.MID"

13 loop = "-1"></bgsound>

14 </head>

15

16 <body>

17 <h1>An Embedded Video Using the img element's

18 dynsrc Property</h1>

19 <h2>Car and Carribean Music</h2>

20 <table>

21 <tr><td><img dynsrc = "car_hi.wmv"

22 start = "mouseover" width = "180"

23 height = "135" loop = "-1"

24 alt = "Car driving in circles" /></td>

25 <td>This page will play the audio clip and video


OutlineOutline

Dynamicimg.html(2 of 2)

26 in a loop.<br />The video will not begin

27 playing until you move the mouse over the

28 video.<br />Press the browser’s<strong>Stop</strong>

29 button to stop playing the sound and the video.</td>

30 </tr>

31 </table>

32 </body>

33 </html>


28.5 Adding Audio or Video with the embed Element

• Element embed– Embeds media clip into Web page– Displays a graphical user interface (GUI)

– Supported by both Microsoft Internet Explorer and Netscape


OutlineOutline

EmbeddedAudio.html(1 of 3)




4

5 

6 

7


9 <head>

10 <title>Background Audio via the embed Element</title>

11 <style type = "text/css">

12 span { width: 600 }

13 .big { color: blue;

14 font-family: sans-serif;

15 font-size: 50pt;

16 font-weight: bold }

17 </style>

18


20 

44 </script>

45 </head>

46

47 <body onload = "start()">

48 <h1>Background Audio via the embed Element</h1>

49 <p>Click the text to stop the script.</p>

50


OutlineOutline


51 <p class = "big" align = "center">

52 <span onclick = "window.clearInterval( TimerID )"

53 id = "wft" style = "filter:wave(

54 add = 0, freq = 3, light = 0, phase = 0, strength = 5)">

55 WAVE FILTER EFFECT</p></span>

56

57 <p>These controls can be used to control the audio.</p>

58 <embed src = "humming.wav" loop = "true"></embed>

59 </body>

60 </html>


28.5 Adding Audio or Video with the embed Element

Play

Pause Stop Mute Volume


OutlineOutline

EmbeddedVideo.html(1 of 2)




4

5 

6 

7


9 <head>

10 <title>Video via the embed Element</title>

11 </head>

12

13 <body>

14 <h1>Displaying a Video using the embed Element</h1>

15 <h2>Car Driving in Circles</h2>

16

17 <table>

18 <tr><td><embed src = "car_hi.wmv" loop = "false"

19 width = "240" height = "176">

20 </embed></td>

21 </tr></table>

22 <hr />


OutlineOutline

EmbeddedVideo.html(2 of 2)

23 This page plays the video once.<br />

24 Use the controls on the embedded video player to play the

25 video again.

26 </body>

27 </html>


28.6 Using the Windows Media Player ActiveX Control

• object element– Embed Windows Media Player and ActiveX controls– Property id

• Specifies scripting name of element– width and height properties

• Specify width and height in pixels– property classid

• Specifies ActiveX control ID– Element param

• Specify parameter– Parameter FileName

• Specifies file containing media clip– AutoStart (boolean value)– ShowControls (boolean value)– Loop (boolean value)


OutlineOutline

MediaPlayer.html(1 of 3)




4

5 

6 

7


9 <head><title>Embedded Media Player Objects</title>


11 

23 </script>

24 </head>

25


OutlineOutline


26 <body>

27 <h1>

28 Audio and video through embedded Media Player objects

29 </h1>

30 <hr />

31 <table>

32 <tr><td valign = "top" align = "center">

33 <object id = "VideoPlayer" width = "200" height = "225"

34 classid =

35 "CLSID:22d6f312-b0f6-11d0-94ab-0080c74c7e95">

36 <param name = "FileName" value =

37 "car_hi.wmv" />

38 <param name = "AutoStart" value = "true" />

39 <param name = "ShowControls" value = "false" />

40 <param name = "Loop" value = "true" />

41 </object></td>

42 <td valign = "bottom" align = "center">

43 <p>Use the controls below to control the audio clip.</p>

44 <object id = "AudioPlayer"

45 classid =

46 "CLSID:22d6f312-b0f6-11d0-94ab-0080c74c7e95">

47 <param name = "FileName" value =

48 "http://msdn.microsoft.com/downloads/sounds/carib.mid" />

49 <param name = "AutoStart" value = "true" />

50 <param name = "Loop" value = "true" />


OutlineOutline


51 </object></td></tr>

52

53 <tr><td valign = "top" align = "center">

54 <input name = "video" type = "button" value =

55 "Pause Video" onclick = "toggleVideo( this )" />

56 </td></tr>

57 </table>

58 </body>

59 </html>


28.7 Microsoft Agent Control

• Interactive animated characters• Speaks

• Supports speech recognition


OutlineOutline

tutorial.html(1 of 12)




4

5 

6 

7


9 <head>

10 <title>Speech Recognition</title>

11

12 

13 <object id = "agent" width = "0" height = "0"

14 classid = "CLSID:D45FD31B-5C6E-11D1-9EC1-00C04FD7081F"

15 codebase = "#VERSION = 2, 0, 0, 0">

16 </object>

17

18 

19 <object width = "0" height = "0"

20 classid = "CLSID:B8F2846E-CE36-11D0-AC83-00C04FD97575"


22 </object>

23


OutlineOutline


24 

25 <object width = "0" height = "0"

26 classid = "CLSID:161FA781-A52C-11d0-8D7C-00A0C9034A7E"


28 </object>

29


31 

176 </script>

177

178 <script type = "text/javascript" for = "agent"

179 event = "Command( cmd )">

180 

183 </script>

184


186 event = "BalloonShow">

187 

193 </script>


OutlineOutline


194


196 event = "Click">

197 

202 </script>

203 </head>

204

205 <body style = "background-color: lemonchiffon"

206 onload = "loadAgent()">

207 <table border = "0">

208 <tr>

209 <th colspan = "4">

210 <h1 style = "color: blue">

211 Deitel Programming Tips

212 </h1>

213 </th>

214 </tr>

215 <tr>

216 <td align = "center" valign = "top" width = "120">

217 <span id = "gppSpan" style = "position : absolute;

218 left : 30; top : 80; width : 130;">


OutlineOutline


219 <img id = "gpp" src = "GPP_100h.gif"

220 alt = "Good Programming Practice" border =

221 "0" onclick = "imageSelectTip( this )" />

222 Good Programming Practices</span></td>


224 <span id = "seoSpan" style = "position : absolute;

225 left : 180; top : 80; width : 130;">

226 <img id = "seo" src = "SEO_100h.gif"

227 alt = "Software Engineering Observation"

228 border = "0"

229 onclick = "imageSelectTip( this )" />

230 Software Engineering Observations</span></td>


232 <span id = "perfSpan" style = "position : absolute;

233 left : 330; top : 80; width : 130;">

234 <img id = "perf" src = "PERF_100h.gif"

235 alt = "Performance Tip" border = "0"


237 Performance Tips</span></td>


239 <span id = "portSpan" style = "position : absolute;

240 left : 480; top : 80; width : 130;">

241 <img id = "port" src = "PORT_100h.gif"

242 alt = "Portability Tip" border = "0"



OutlineOutline


244 Portability Tips</span></td>

245 </tr>

246 <tr>


248 <span id = "guiSpan" style = "position : absolute;

249 left : 30; top : 260; width : 130;">

250 <img id = "gui" src = "GUI_100h.gif"

251 alt = "Look-and-Feel Observation" border =


253 Look-and-Feel Observations</span></td>


255 <span id = "eptSpan" style = "position : absolute;

256 left : 180; top : 260; width : 130;">

257 <img id = "ept" src = "EPT_100h.gif"

258 alt = "Error-Prevention Tip" border =


260 Error-Prevention Tips</span></td>


262 <span id = "cpeSpan" style = "position : absolute;

263 left : 330; top : 260; width : 130;">

264 <img id = "cpe" src = "CPE_100h.gif"

265 alt = "Common Programming Error" border =


267 Common Programming Errors</span></td>

268 </tr>


OutlineOutline


269 </table>

270 <img src = "agent_button.gif" style = "position: absolute;

271 bottom: 10px; right: 10px" />

272 </body>

273 </html>





Fig. 28.7 Peedy finishing introduction.



Fig. 28.8 Peedy ready to receive voice commands.



Fig. 28.9 Peedy receiving voice command.



Fig. 28.10 Peedy discussing Good Programming Practices.



Event Description BalloonHide Called when the text balloon for a character is

hidden. BalloonShow Called when the text balloon for a character is

shown. Hide Called when a character is hidden. Move Called when a character is moved on the screen. Show Called when a character is displayed on the

screen. Size Called when a character’s size is changed. Fig. 28.11 Other events for the Microsoft Agent control.



Property or method Description Properties Height The height of the character in pixels. Left The left edge of the character in pixels from the left of the

screen. Name The default name for the character. Speed The speed of the character’s speech. Top The top edge of the character in pixels from the top of the screen.

Width The width of the character in pixels. Methods Activate Sets the currently active character when multiple characters

appear on the screen. GestureAt Specifies that the character should gesture toward a location on

the screen that is specified in pixel coordinates from the upper-left corner of the screen.

Interrupt Interrupts the current animation. The next animation in the queue of animations for this character is then displayed.

StopAll Stops all animations of a specified type for the character. Fig. 28.12 Other properties and methods for the Character object.



Tag Description \Chr = string\ Specifies the tone of the voice. Possible values for string are

Normal (the default) for a normal tone of voice, Monotone for a monotone voice or Whisper for a whispered voice.

\Emp\ Emphasizes the next spoken word. \Lst\ Repeats the last statement spoken by the character. This tag must

be the only content of the string in the Speak method call. \Pau = number\ Pauses speech for number milliseconds. \Pit = number\ Changes the pitch of the character’s voice. This value must be

within the range 50 to 400 hertz for the Microsoft Agent speech engine.

\Spd = number\ Changes the speech speed to a value in the range 50 to 250. \Vol = number\ Changes the volume to a value in the range 0 (silent) to 65,535

(maximum volume). Fig. 28.13 Speech output tags.


28.8 RealOne Player Plug-in

• Element embed– Embed RealOne Player plug-ins for video and audio– Attribute type

• Specifies MIME type of embedded file

– Attributes width and height• Specify dimensions of space the control occupies

– Attribute autostart• Determines whether media start playing when page loads

– Attribute controls• Specifies which controls users can access

– Attribute src• Set to location of streaming media


OutlineOutline

StreamingMedia.html(1 of 3)




4

5 

6 

7


9 <head>

10 <title>Live Media!</title>

11


13 

24 </script>

25 </head>


OutlineOutline


26

27 <body>

28

29 <p>Pick from my favorite video streams:

30

31 <select id = "streamSelect" onchange =

32 "change( this.value )">

33 <option value = "">Select a station</option>

34 <option value = "0">NOAA</option>

35 <option value = "1">NASA</option>

36 </select></p>

37

38 <br />

39 <embed id = "videoControl" src = ""

40 type = "audio/x-pn-realaudio-plugin" width = "275"

41 height = "200" controls = "ImageWindow"

42 console = "streamingAudio"

43 autostart = "false" />

44 <br />

45 <embed id = "audioControl" src = ""

46 type = "audio/x-pn-realaudio-plugin" width = "275"

47 height = "40" controls = "ControlPanel"

48 console = "streamingAudio"

49 autostart = "false" />


OutlineOutline


50

51 </body>

52 </html>


28.9 Synchronized Multimedia Integration Language (SMIL)

• Coordinate wide range of multimedia elements• XML-based description language

• Organize text, audio, video to occur simultaneously or sequentially

• Provide time reference for all instances of text and media

• Specifies source and presentation of multimedia


OutlineOutline

exampleSMIL.smil(1 of 3)

1 <smil xmlns="http://www.w3.org/2000/SMIL20/CR/Language">

2

3 

4 

5

6 <head>

7 <layout>

8 <root-layout height = "300" width = "280"

9 background-color = "#bbbbee" title = "Example" />

10

11 <region id = "image1" width = "177" height = "230"

12 top = "35" left = "50" background-color = "#bbbbee" />

13 </layout>

14

15 <transition id = "wipeForward" dur = "2s" type = "barWipe" />

16 <transition id = "wipeBackward" dur = "2s" type = "barWipe"

17 subtype = "topToBottom" />

18

19 <transition id = "fadeIn" dur = "2s" type = "fade"

20 subtype = "fadeFromColor" fadeColor = "#bbbbee" />

21

22 <transition id = "fadeOut" dur = "2s" type = "fade"

23 subtype = "fadeToColor" fadeColor = "#bbbbee" />

24


OutlineOutline


25 <transition id = "crossFade" type = "fade" subtype = "crossfade"

26 dur = "2s" />

27

28 </head>

29 <body>

30 <seq>

31 <par>

32 <img src = "book1.jpg" region = "image1"

33 transIn = "wipeForward" transOut = "wipeForward"

34 alt = "book1" dur = "6s" fill = "transition"

35 fit = "fill" />

36 <audio src = "bounce.au" dur = ".5s" />

37 </par>

38 <par>

39 <img src = "book2.jpg" region = "image1" transIn = "fadeIn"

40 transOut = "fadeOut" alt = "book2" dur = "6s"

41 fit = "fill" fill = "transition" />


43 </par>

44 <par>


46 transIn = "wipeBackward" transOut = "fadeOut"

47 alt = "book3" dur = "6s" fit = "fill"

48 fill = "transition" />



OutlineOutline


50 </par>

51 <par>

52 <img src = "book4.jpg" region = "image1" transIn = "crossFade"

53 transOut = "fadeOut" alt = "book4" dur = "6s"



56 </par>

57 <par>


59 transIn = "wipeForward" transOut = "wipeBackward"

60 alt = "book5" dur = "6s" fit = "fill"

61 fill = "transition" />


63 </par>

64 <par>


66 transIn = "crossFade" alt = "book6" dur = "6s"



69 </par>

70 </seq>

71

72 </body>

73 </smil>


OutlineOutline

SMILexample.html(1 of 1)




4

5 

6 

7


9 <head>

10 <title>Embedding SMIL with RealOne Player</title>

11 </head>

12 <body>

13 <div style = "text-align: center">

14 <embed src = "exampleSMIL.smil"

15 controls = "ImageWindow"

16 type = "audio/x-pn-realaudio-plugin"

17 width = "280" height = "300" autostart = "true">

18 </embed>

19 </div>

20 </body>

21 </html>




28.10 Scalable Vector Graphics (SVG)

• Describes vector graphic data for JPEG, GIF, and PNG formats

• Vector graphics– Produced by mathematical equations

• XML vocabulary


OutlineOutline

shapes.svg(1 of 2)


2

3 

4 

5

6 <svg viewBox = "0 0 300 300" width = "300" height = "300">

7

8 

9 <g>

10 <path style = "fill: #eebb99" d = "M0,0 h300 v300 h-300 z"/>

11 </g>

12

13

14 

15 <g>

16

17 <circle style = "fill:green;" cx = "150" cy = "150" r = "50">

18 <animate attributeName = "opacity" attributeType = "CSS"

19 from = "0" to = "1" dur = "6s" />

20 </circle>

21

22 

23

24 <rect style = "fill: blue; stroke: white"

25 x = "50" y = "0" width = "100" height = "100">


OutlineOutline

shapes.svg(2 of 2)

26 <animate attributeName = "y" begin = "mouseover" dur = "2s"

27 values = "0; -50; 0; 20; 0; -10; 0; 5; 0; -3; 0; 1; 0" />

28 </rect>

29

30 

31

32 <text style = "fill: red; font-size: 24pt"

33 x = "25" y = "250"> Welcome to SVG!

34 <animateColor attributeName = "fill"

35 attributeType = "CSS" values = "red;blue;yellow;green;red"

36 dur = "10s" repeatCount = "indefinite"/>

37 </text>

38 </g>

39 </svg>



OutlineOutline

planet.svg(1 of 2)


2

3 

4 

5

6 <svg viewBox = "-500 -500 1000 1000">

7 <g id = "background">

8 <path style = "fill: black"

9 d = "M -2000,-2000 H 2000 V 2000 H -2000 Z" />

10 </g>

11

12 <circle id = "sun" style = "fill: yellow"

13 cx = "0" cy = "0" r = "100" />

14

15 <g>

16 <animateTransform attributeName = "transform"

17 type = "rotate" dur = "80s" from = "0" to = "360"

18 repeatCount = "indefinite" />

19

20 <circle id = "earth" style = "fill: blue"

21 cx = "400" cy = "0" r = "40" />

22

23 <g transform = "translate( 400 0 )">

24 <circle id = "moon" style = "fill: white"

25 cx = "70" cy = "0" r = "10">


OutlineOutline

planet.svg(2 of 2)

26 <animateTransform attributeName = "transform"

27 type = "rotate" dur = "20s" from = "360"

28 to = "0" repeatCount = "indefinite" />

29 </circle>

30 </g>

31 </g>

32 </svg>


28.11 Web Resources

• www.microsoft.com/windows/windowsmedia• www.streamingmedia.com• www.microsoft.com/msagent/downloads/default.asp• msdn.microsoft.com/downloads/default.asp• www.real.com• www.adobe.com/svg• www.service.real.com/help/library/guides/extend/embed.htm• www.nasa.gov/gallery/index.html• www.speech.cs.cmu.edu/comp.speech/SpeechLinks.html• www.mp3.com• www.mpeg.org• www.winamp.com• www.shoutcast.com• www.windowsmedia.com• www.research.att.com/~rws/Sable.v1_0.htm• www.w3.org/AudioVideo• smw.internet.com/smil/smilhome.html

Documents

3.1 Audio, Video, Speech Synthesis and Recognition