December 2010, 17(Suppl. 2): 26–31 www.sciencedirect.com/science/journal/10058885 http://www.jcupt.com

The Journal of China Universities of Posts and Telecommunications

Design and implementation of a streaming media system for symbian based smartphone

ZHANG Lin ( ), LUO Yin-zhuo, YUAN Meng-long, LUO Ming-shan

Key Lab of Universal Wireless Communications, Ministry of Education,

Beijing University of Posts and Telecommunications, Beijing 100876, China

Abstract

As the multimedia and network techniques having greatly developed, people could enjoy more and more media services from Internet every day, especially the streaming media service [1]. The main obstacle of utilization of this service is related to area limitation, the solution of the problem proposed in this paper is to port stream media-client application to mobile Symbian operating system (OS-based) platform. This approach declines the limitation related to network access availability and allows achieving all benefits of smart telephony and mobile Internet within the same portable wireless device. The system is designed under the client/server (C/S) structure based on real-time transport protocol (RTP) family protocols [2]. The system could providekinds of streaming media services under hybrid mobile Internet with good performance and support multi-user concurrency stably.

Keywords streaming media, mobile Internet, Symbian OS platform

1 Introduction1

With the development of mobile communication techniques, the mobile network data transmission rates become higher and higher. Simple textual service cannot satisfy the mobile users anymore. The multimedia application has been becoming more and more popular [3]. But traditional multimedia technology couldn’t fulfill users well. People couldn’t hear or watch the media file until it is completely downloaded. Especially for the audio files, the user experience is the most intolerable, because different song files may have the same file names, which means people may download the wrong file after waiting for a long time.

On the Internet, a multimedia technology, which is called ‘Streaming’, has been common used to deliver audio and video content for many years. This streaming media technology is based on RTP protocol family, which consists of real-time streaming protocol (RTSP), RTP and RTP control protocol (RTCP) [4–5].

Received date: 20-09-2010 Corresponding author: ZHANG Lin, E-mail: zhanglin@bupt.edu.cn DOI: 10.1016/S1005-8885(09)60587-2

Among these protocols, RTSP defines one to many applications to effectively transmit multimedia data through the IP network. It uses transmission control protocol (TCP) or RTP to implement data transmission. RTSP streaming media support functions such as pause, fast forward, rewind, jump and other operations, which make listening audio online just like listening CD. RTP protocol provides a time stamp, serial number and other data structures used to control the timely exile. RTP itself can neither provide a reliable delivery mechanism by transmitting data packets in order, nor provide flow control or congestion control. It relies on RTCP to provide these services. The main function of RTCP is to provide feedback of data transmission. Receiver on a regular basis to report information sent to the sender. At present, RTP-based multimedia applications for PC are very common on the Internet. However, it’s hard to find applications, which have the same functions like them, in the mobile Internet.

One possible solution to the abovementioned problem might be using mobile Symbian based platform S60 for porting stream media-client application to mobile Symbian OS-based platform. This approach would allow combining the most essential benefits of smart telephony and mobile Internet

Supplement 2 ZHANG Lin, et al. / Design and implementation of a streaming media system for… 27

within the same portable wireless device. At first, we need a non-commercial and editable RTP-based streaming server to provide specific streaming services. For this purpose, we choose the LIVE555 Streaming Server. Second, we need to implement a client application which could cooperate with the server and have scalability as broad as possible. The development of this application was done by means of S60 public application programming interfaces (APIs), which are available for 3rd party developers. Another positive issue of using S60 platform for development is that the share of S60 devices shipped to the market is noticeably big. This aspect should allow to cover a greater number of devices whose users might get benefits of using streaming media technology.

In this paper, we first introduce some related works in Sect. 2. Then, we present the audio stream application we developed in Sect. 3. In Sect. 4, we give a detail description of the implementation in Symbian OS platform and evaluate our application’s performance. We give our conclusion in Sect. 5.

2 Related works

Nowadays, there are so many companies and institutions have made a great contribution on the development of the RTP-based server in the world. The famous companies include Real Network, Apple and some open-source organizations like Live555 and so on. They have produced lots of RTP-based servers with good performance on the Internet.

For example, Helix Server and Real Player are produced by Real Network. They are using RTSP to implement streamed media data transferring between them. Helix Server can support all audio and video formats with stable performance. However, this server’s code is secret and there is no suitable client for mobile users. So, we need to find out a new way to develop our own application.

Fortunately, there are some open-source streaming media projects on the Internet, too. Among them, there is a project called Live555 Media Server, which also delivers streamed data via RTSP. We choose it to support our development work, not only because we can see its code freely and edit it easily, but also for the reason that it has stable performance on PC.

We also have done lots of research work on the mobile device’s software development kit (SDK). We find that there is RTP-based stack support in the Symbian V9.2 OS. But, the SDK just offers the API and it only supports some specific media file formats. Because of its poor scalability, we need to develop client application ourselves.

There are also some products in academia, too. In Ref. [6], the author implemented an embedded streaming media system. The system proved to play back MPEG 1/2/4 streaming data smoothly and also succeeds in streaming download. But their system was developed on specific chip. So it couldn’t give mobile users any benefit.

Nowadays, most of the multimedia players on Mobile terminals only support the traditional ‘Download first then listen’ mode. The others may support Streaming Play mode via extended hypertext transfer protocol (HTTP) protocol. But there is a barrier for HTTP protocol to provide streaming service for its lack of a session to guarantee long-lived service [7]. So, it’s still not the best way to implement streaming media applications. In the current mobile application market, however, there are not mobile streaming media player based on RTSP streaming protocol.

3 Audio streaming media application

In this section, we first present our system’s structure. Then, we give a brief description on system mechanisms.

3.1 System structure

Fig. 1 shows our system uses the C/S structure to develop. This is because our application has great demand on transferring streaming of high quality multimedia content from server to client, which means a greater bandwidth is needed for downlink, while the uplink is mainly used for control information, such as user’s selection and commands [5].

Fig. 1 Streaming media system structure

Under the consideration of multiple users with different bandwidth limitations, and in order to develop a robust system with good scalability and easy to use, our application consists of streaming server, content server, proxy server and encoding server.

The streaming server has responsibility for answering RTSP requests from clients. After receiving Setup and Play

28 The Journal of China Universities of Posts and Telecommunications 2010

messages, the streaming server will transfer the streaming media content to the clients. Streaming server is the core of this system structure. In order to meet more and more demands from different users, it often chooses means of making server clusters to achieve payload balancing and efficient implementation of the quality of service (QoS) strategy.

For the convenience of publishing the audio information, mobile users could send HTTP requests to the content server to get the audio file lists. In this way, it is easy for the system master to do effective management on streaming media resources.

In order to provide some personal services to mobile users, the proxy server and the structured query language (SQL) server are used to manage and store the users’ information. At last, the encoding server is mainly responsible for coding the media content for fulfilling the demand of transferring media data with real-time properties. Therefore, with this kind of system architecture, mobile users could enjoy the streaming audio services by choosing different internet accessing point (IAP) for their own specific mobile terminals.

3.2 System mechanisms

In this section, we give an example to show the application’s mechanisms.

When a mobile user opens our application on the mobile phone, the application needs to register to the proxy server first. The proxy server will check the received user’s information with the data stored in the SQL server. If the checking failed, the application will ask the user to create a new account. If the checking passed, the application will get a permission to visit the content server.

Then, the application will request the content server to get the streaming media file list, which includes the names, sizes, paths and some other brief information. When the list received, the application will extract some information to show on the screen so that user could choose one link to play. When the user clicks the link, the application will send a RTSP message, which includes the file path, to the streaming media server.

The first RTSP message is an OPTION request. The streaming media server will return the functions which the server could support. Then the client will send the second RTSP message which is a DESCRIBE request. In this way, the client could get a session description protocol (SDP) packet, from which the client could get the information to prepare for the next SETUP operation. When the server

received the SETUP request, it will give some instructions to guide the client to configure the parameters of the RTP and RTCP. After client finishing all the preparation, it will send a PLAY request to the server, and from then on, the streaming media will start transferring process. At last, the client will send a TEARDOWN request to the server to release the resources and finish the whole process.

4 Implementation and performance evaluation

4.1 System modules

Today meaning of software design is very crucial for successful application development. Proper design of an application can significantly improve quality of the application and decrease the amount of time required for its implementation. One of the most important steps needed to be passed through in the beginning of each functional design stage is definition of a high level architecture and emphasizing of basic logical units of the future application. This fragmentation is intended to reduce the complexity of application architecture from developer’s point of view and simplify further detailed design of emphasized components.

Fig. 2 depicts the system diagram of Streaming Media Player in Mobile Terminal. There are three key modules:

1) Net module, which is in charge of communicating with server and dealing with the streaming data. This module is further broken down into four parts: CRTSPEngine, CRTPEngine, CRTCPEngine and CHTTPEngine.

2) Cache manager module, it connects the other two modules with some cache management.

3) Player module, which can play the streaming data from Cache Manager Module. Fig. 2 describes the relationship between the modules.

Fig. 2 Application diagram

Supplement 2 ZHANG Lin, et al. / Design and implementation of a streaming media system for… 29

In this section, we discuss the interactions among the modules and their design issues, and present our solutions.

4.1.1 Net module

CRTSPEngine is used for establishing and controlling either a single or several time-synchronized streams of continuous media content. The process is implemented as section 3.2 and shown as in Fig. 3.

Fig. 3 CRTSPEngine flowchart

CRTPEngine is responsible for receiving streaming media data sent from server. Fig. 4 displays the process of the CRTPEngine.

Fig. 4 CRTPEngine flowchart

Class RTPInterface encapsulates the low-level socket function to listen, receive and send data. Class RTPReception- StatsDB saves information such as timestamp, serial number extracted from the RTP packets for sorting and statistical. Function StartNetworkReading() will listen the RTP packets. When the data comes, the socket will inform CRTPEngine through the callback function NetworkReadHandler(). CRTPEngine then analyzes the data according to the specification of RTP protocol. We can extract kinds of information, such as, RTP media types, synchronization source SSRC, timestamp, sequence number and so on. Finally it removes the head of the RTP packets to restore the original

streaming media data and sends the data to the Cache Manager Module by function doGetNextFrame().

CRTCPEngine is in charge of receiving the statistics data packets sent form server and sending RTCP packets to server. By the CRTSPEngine, the client periodically sends RTCP packets to server. Packets contain statistical information such as data packets have been sent and the number of lost packets. Server can use this information to dynamically change the transmission rate, and even to change the payload type to match current network conditions. Fig. 5 shows the process of receiving and sending RTCP packets.

Fig. 5 CRTCPEngine flowchart

CHTTPEngine is mainly responsible for obtaining channel information and song information. The server sends the information in the form of xml format. After receiving data, the mobile terminal will parse the data and extract the corresponding channel information and song information, then display the information in the form of a list.

4.1.2 Cache manager module

The application needs to do some cache management to share the memory between Net module and Player module. This module is responsible for connecting Net module and Player module. Processing and sharing of streaming media data is a big problem that has to be solved.

Symbian OS is a multitasking operating system. In order to achieve multi-task, while enabling the system to respond quickly and efficiently conduct of the event handling, and reduce the application programmer’s work burden, Symbian OS deliberately introduce the concept of active object (AO). Symbian OS also support multithreading. However, the switch cost in the same thread running between the active

30 The Journal of China Universities of Posts and Telecommunications 2010

object is lower than thread context, which makes system resources more intense. So, active objects are more suitable for event-driven multi-tasking. But Sometimes in order to guarantee the sustainability of the implementation of the tasks, such as to play a sound, we need a separate thread to deal with. Because multi-thread could provide better real-time performance, we choose the multi-thread mechanism to implement the data sharing.

In this module, we create two threads. One is for streaming data downloading, and the other is for playing. Fig. 6 depicts how the threads create and the interactions between them.

Fig. 6 Thread creation flowchart

4.1.3 Player module

In this module, we will adopt multimedia framework (MMF) provided by Symbian S60 platform to achieve the streaming playing. As a plug-in set of multimedia processing unit, MMF provides a common interface operator terminal hardware, supports different audio operations. It can be used for local playback and streaming playback.

Class CMdaAudioOutputStream provides API for streaming. Client application needs to implement class MMdaAudioOutputStreamCallback. It provides three callback functions to the client to prompt the results of streaming process. So that program can handle possible errors. These functions are MaoscOpenComplete (), MaoscBufferCopied () and MaoscPlayComplete (). In order to get the outcome of the streaming process, we need to inherit this observer class.

Fig. 7 provides an example of a typical use case of MMF API. First, we should initialize class CMdaAudioOutput- Stream. After the initialization completed, Multimedia Framework will call the MaoscOpenComplete () callback

function. It will point out whether the audio output is available or not. Then we can set the sampling rate and volume. After a successful initialization, we can call PlayL() function to play music. It calls FillBufferL () function to send data to the buffer and use the CMdaAudioOutputStream:: WriteL (const TDesC8 & aData) for playback. WriteL () is an asynchronous function. After we copy the data to audio hardware, MMF will call MmdaAudioOutputStreamCall- back::MaoscBufferCopied() method to inform the client application that data has been received and played. At this point, we can call FillBufferL() function to fill the buffer again in order to achieve continuous flow of play.

Fig. 7 An example of a typical use case of MMF API

4.2 Performance evaluation

According to the design of system architecture, we developed a streaming player on the Symbian platform, which has been tested in the actual cellular network. The experiment environment deployed here is China Mobile general packet radio service/enhanced data rate for GSM evolution (GPRS/EDGE) and wireless local area networks (WLAN). The cellular phone is Nokia N95, whose operating system is V9.2 with the S60 third edition user interface. It is equipped with ARM 11 whose frequency is 332 MHz. Its Memory is 160 MB with 30 MB running random access memory (RAM). Experiment result shows that the playback is relatively smooth, although it will be subject to the constraints of network conditions. Fig. 8(a) shows the song list we get from the content server . The music playing screen is shown in Fig. 8(b).

We also use the central processing unit monitor (CPUMonitor) application to test our application’s performance. Fig. 8(c) shows the resource possession status under the system idle condition. From Fig. 8(d), we can see that our application is so light-weighted that only needs 2 MB RAM to run on mobile terminal. At the same time, it merely doesn’t consume CPU because the percentage is so

Supplement 2 ZHANG Lin, et al. / Design and implementation of a streaming media system for… 31

small that adhere to zero. In comparison, we also test some similar applications.

Fig. 8(e) shows the performance of QQMusic, from which we can see it need 50% RAM more than our application and a little more CPU consumption. Fig. 8(f) shows the performance of TTPlayer. Although it has the same RAM possession as our application, it needs much more CPU consumption than ours. So, according to the experiment results, we can prove that our system is most suitable for the mobile network streaming media use.

(a) (b) (c)

(d) (e) (f)

Fig. 8 The snapshots of the application

5 Conclusions

In the paper, we present a RTP-based streaming system. It proves to fit the mobile network environment with stable

performance. The client could play audio streaming data smoothly with little hardware resources. Besides, it is so robust that the system could support multi-user synchronically. The future work will focus on the security during streaming transferring.

Acknowledgements

This work was supported by the Hi-Tech Research and Development Program of China (2009AA01Z246), the 111 Project (B08004), and the major project of Ministry of Industry and Informationization of China (2009ZX03006-007, 2010ZX03002-006).

Reference

1. Devlic A, Vojnic H, Enhancement of existing network infrastructure with media on demand functionality, Proceedings of the 7th International Conference on Telecommunications, IEEE, 11–13 June 2003, 2: 717–720

2. Dalal A C, Perry E. A new architecture for measuring and assessing streaming media quality, Passive and Active Measurement Workshop Proceedings, Apr, 2003, La Jolla, CA, USA. 2003: 223–231

3. Liu Y, Zhong G H, Liu Y, et al. The research of streaming media mutual digest authentication model based on RTSP protocol. Proceeding of the 2008 International Conference on Wavelet Analysis and Pattern Recognition, IEEE, 30–31 Aug. 2008, Hong Kong, 2008: 838–842

4. Schulzrinne H, Casner S, Frederick R, et al. RFC3550. RTP: A Transport Protocol for Real-Time Applications, Internet Engineering Task Force. 2003

5. Schulzrinne H, Rao A, Lanphier R. RFC2326. Real time streaming Protocol (RTSP), Internet Engineering Task Force. 1998

6. Zhang R L, Liu J, Gao Y X, et al. A realized embedded streaming media system. 2006 IET International Conference on Wireless, Mobile and Multimedia Networks, IEEE, Nov 6–9, 2006, Hangzhou, China. 2006: 1–4

7. Changwoo J, Shin, K G. A DAVIC video-on-demand system based on the RTSP. Proceedings of the 2001 Symposium on Applications and the Internet, IEEE, Jan. 2001, San Diego, CA, USA. 2001: 231–238