Embed Size (px)
Citation preview
UPTEC IT 11 016
Examensarbete 30 hpDecember 2011
An Evaluation of the PandaBoard as a Set-Top-Box in an Android Environment
Torbjörn Svangård
2
Teknisk- naturvetenskaplig fakultet UTH-enheten Besöksadress: Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0 Postadress: Box 536 751 21 Uppsala Telefon: 018 – 471 30 03 Telefax: 018 – 471 30 00 Hemsida: http://www.teknat.uu.se/student
Abstract
An Evaluation of the PandaBoard as a Set-Top-Box inan Android Enviroment
Torbjörn Svangård
The aim of this thesis is to evaluate the potential of the PandaBoard to take the roleas an Android based Set-Top-Box (STB). A STB is a device that converts an externalsignal into viewable video on a connected television. The PandaBoard is a low-costdevelopment platform intended for mobile software development and is based on achipset, OMAP4, developed and manufactured by Texas Instruments.
During the thesis implementation phase a PandaBoard was configured to run Android;a popular operating system mainly used in smartphones. A DVB-T2 device wasattached to one of the USB-ports on the PandaBoard in order to recieve terrestrialdigital television broadcasts.
The main obstacle was the fact that despite very promising video acceleratedhardware specifications on the PandaBoard, they were not available for evalutaion.During the thesis pilot study it became clear that hardware accelerated video playbackwas not yet supported for the PandaBoard Android-port. Programmable OpenGL ES2.0 shaders was used instead in order to offload the PandaBoard CPU as much aspossible.
Although this thesis may not be conclusive in terms of the impressive videoaccelration hardware present on the PandaBoard, the thesis will demonstrate whatcan be achieved with a dual core ARM Cortex-A9 processor together with OpenGLES 2.0 on a system running the Android operating system.
The thesis work was performed at the Syntronic Software Innovation office in Kista,Stockholm.
Tryckt av: Reprocentralen ITCISSN: 1401-5749, UPTEC IT11 016Examinator: Anders JanssonÄmnesgranskare: Arnold PearsHandledare: David Näslund
4
Sammanfattning
Syftet med denna rapport ar att utvardera potentialen av utvecklingskortet PandaBoard irollen som en Androidbaserad set-top-box (STB). En STB ar en enhet som anvands foratt omvandla en extern signal till en videostrom som kan visas pa en ansluten TV. EnPandaBoard ar ett utvecklingskort som ar avsedd for utveckling av programvara for sakallade smartphones. Pandaboardkortet baseras pa ett chipset, OMAP4, som utvecklatsoch tillverkas av Texas Instruments.
Under examensarbetets implementationsfas konfigurerades ett PandaBoardkort for attanvanda Android. Android ar ett populart operativsystem som aterfinns framst i smartamobiltelefoner. En DVB-T2 dongle kopplades in pa en av PandaBoardkortets USB-portarfor att fa tillgang till marksand digital-TV.
Det storsta hindret under arbetest gang var det faktum att trots mycket lovandespecifikationer for hardvaruaccelereread videouppspelning pa PandaBoardkortet, sa vardrivrutiner for Android dessvarre inte tillgangliga for utvardering. Under examensarbetetsforstudie blev det uppenbart att hardvaruaccelererad videouppspelning annu inte stoddesfor PandaBoardkortets androidport. Programmerbara OpenGL ES 2.0 shaders anvandes istallet for att avlasta PandaBoardkortets CPU sa mycket som mojligt.
Aven om denna uppsats inte ar uttommande vad galler den imponerande hardvarafor video acceleration pa PandaBoardkortet, sa kommer rapporten redogora vad som kanuppnas med en dual core ARM Cortex-A9-processor tillsammans med OpenGL ES 2.0 iett system med linuxdistributionen Android som operativsystem.
Examensarbetet utfordes pa plats hos Syntronic Software Innovation i Kista, Stockholm.
5
6
Contents
1 Background 111.1 The PandaBoard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.1.1 Ducati Sub System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.1.2 Pandroid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2 The DVB-project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.3 Digital Terrestrial Television in Sweden . . . . . . . . . . . . . . . . . . . . 13
1.3.1 Past and present . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.3.2 DTTV in the future . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.4 Content protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.5 Set-Top-Box requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2 Problem description 172.1 Linux kernel DVB-stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2 DVB-T2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3 Decoding Audio and Video . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3 Methods 19
4 Implementation 214.1 System peripherals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.1.1 Usb devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.1.2 HDMI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.1.3 The SD-Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.2 System software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.2.1 DVB Utils Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.2.2 FFMpeg Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254.2.3 Color space conversion . . . . . . . . . . . . . . . . . . . . . . . . . . 264.2.4 OpenGL ES 2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.3 Results and evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.3.1 System performance . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.3.2 Power consumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5 Conclusions 315.1 Unsolved problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.1.1 Audio playback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.1.2 Accelerated video playback . . . . . . . . . . . . . . . . . . . . . . . 31
6 Future work 33
Bibliography 35
7
CONTENTS CONTENTS
A PCTV nanoStick T2 290e PCB 37
8
Glossary
ADB Android Debug BridgeAVC Advanced Video CodingATSC Advanced Television Systems CommitteeCOFDM Coded Orthogonal Frequency Division ModulationDVB-T Digital Video Broadcasting - TerrestrialDVB-T2 Digital Video Broadcasting - Second Generation TerrestrialDTTV Digital Terrestrial TelevisionEPG Electronic Program GuideGLSL OpenGL Shading LanguageGPU Graphics Processing UnitHDTV High Definition TelevisionIVA-HD Image Video Accelerator - High DefinitionISS Imaging Sub SystemMPEG Moving Picture Experts GroupMUX MultiplexerOEM Original Equipment ManufacturersOFDM Orthogonal Frequency Division ModulationOMAP Open Multimedia Application PlatformOpenGL ES 2.0 Open Graphics Library for Embedded SystemsPCB Printed Circuit BoardSDTV Standard Definition TelevisionSIMD Single Instruction Multiple DataSoC System-on-a-ChipSTB Set-Top-BoxTI Texas Instruments
9
CONTENTS CONTENTS
10
Chapter 1
Background
1.1 The PandaBoard
The PandaBoard is a low-power and low-cost platform intended for mobile softwaredevelopment. It is equipped with a 4th generation Open Multimedia Application Platform(OMAP4) chip. The OMAP brand name is a family of System on Chips (SoC) for mobilemultimedia applications developed and manufactured by the American semiconductor andcomputer technology company Texas Instruments, widely known as TI.
The PandaBoard is equipped with the OMAP4430 SoC, which houses a dual-coreARM Cortex-A9 processor with each core running at 1GHz, a PowerVR SGX540 GraphicsProcessing Unit (GPU) and an Image Video Accelerator-High Definition (IVA-HD) SubSystem. The OMAP4 family is one of the first dual-core Cortex-A9-based architectures onthe market. A community of Linux experts and enthusiasts supports the PandaBord viathe website PandaBoard.org.
The PandaBoard supports playback of high definition video content at the HDTVstandard 1080p, perhaps more known by its marketing term ”Full HD”. The PandaBoard’shigh performance video media support is made possible thanks to its dedicated hardware;the Ducati Sub System.
1.1.1 Ducati Sub System
The Ducati sub system consists of two ARM Cortex-M3 processors, the Image VideoAccelerator - High Definition (IVA-HD) sub system and the Imaging Sub System (ISS).
The IVA-HD sub system is the hardware for encoding and decoding video up to 1080presolution at 30 frames per second. It supports multiple video codecs such as the Mpeg-4AVC, also known as H.264, codec. The ISS processes pixel data coming from an externalcamera sensor or from memory. The purpose of these sub systems is to offload video andimage processing from the main ARM Cortex-A9 CPU.
1.1.2 Pandroid
Pandroid is a Texas Instruments supported port of Android for the PandaBoard. Androidis an operating system for mobile devices, developed and maintained by Google. It consistsof a complete system based on the open source Linux kernel, middleware i.e. libraries thatcan be used by developers and applications. The applications are the actual programs thatthe user sees on a display and interacts with.
The difference between Pandroid and standard Android is a number of patches thatenables support for hardware found on the PandaBoard; such as TI specific hardware not
11
1.1. THE PANDABOARD CHAPTER 1. BACKGROUND
Figure 1.1: PandaBoard Layout from pandaboard.org (CC BY-SA 3.0)
yet included in the Linux kernel. Other more directly practical patches are also includedsuch as support for an USB mouse. Standard Android is designed for touch screen usageand lacks default USB mouse support.
For the implementation the Pandroid L27.10.2 released in March 2011 is used. It isbased on code from the Android version 2.2 named Froyo. This Pandroid release has thefollowing main features:
• External monitor via HDMI/DVI
• USB Keyboard
• USB Mouse with mouse cursor
• Wifi/802.11n
• Bluetooth
• Ethernet
• PowerVR SGX540 GPU Drivers
Note that neither the OMAP4 Ducati Sub System nor audio playback/recording is availablein this release. However OpenGL ES 2.0 is supported through the PowerVR GPU drivers.
12
CHAPTER 1. BACKGROUND 1.2. THE DVB-PROJECT
1.2 The DVB-project
The Digital Video Broadcast (DVB) project was established in September 1993 to developa common video broadcasting standard. It is a consortium consisting of over 250 differentcompanies and organizations from 35 countries with interest in open standards for deliveryof digital media services[16].
Although DVB usage is the most widespread DTTV standard, some alternatives exists.To name a few countries that do not use the DVB standards; USA uses the AdvancedTelevision Systems Committee (ATSC) standard, Japan use the Integrated Services DigitalBroadcasting (ISDB) standard. See 1.2 for an DTTV standards overview.
Figure 1.2: DTTV world coverage[1]
Currently there are over 500 million receivers in the world, compatible with a standardfrom the DVB-project.
The DVB-project has produced a number of widely used standards. The most importantare called DVB-S, DVB-C and DVB-T. The S stands for satellite, the C for cable and theT for terrestrial and, as the reader might assume, represents broadcasting standards forthese mediums. The focus for this thesis will be towards DVB-T and DVB-T2, the latteris the next generation terrestrial standard.
1.3 Digital Terrestrial Television in Sweden
1.3.1 Past and present
The political discussions regarding an introduction of Digital Terrestrial Television (DTTV)in Sweden started in the early 1990s. In June 1995 Lars Jeding was appointed by the Swedishgovernment, to investigate the country’s possibilities for a transition from analog to digitaltelevision. Jeding proposed in his report, From Massmedia to Multimedia – Digitalisationof Swedish Television[11], that Sweden should switch over to digital transmissions as soonas possible. He concluded that DTTV would provide an “alternative to digital broadcastingby satellite or cable” and that it would “attract commercial players”. The report generated
13
1.4. CONTENT PROTECTION CHAPTER 1. BACKGROUND
1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014
LarsJe
ding
sre
port
Dec
ision
toes
tabl
ishDTTV
broa
dcas
ting
DTTV
islaun
ched
Ana
log
switc
h-off
DVB-T
2m
peg4
tran
smiss
ions
laun
ched
Figure 1.3: Historical timeline of DTTV introduction in Sweden
a heated debate[4] whether DTTV should be introduced in Sweden or not. The decisionwas however taken 1 November 1997, by the Swedish Riksdag, to go over to DTTV.
According to the final report Digital TV commission[5]
1.3.2 DTTV in the future
A standard for mobile TV; DVB-H, emerged in November 2004 and trials have beenconducted in Sweden back in 2006. Despite the positive outcome[14] of the trials, DVB-His not currently provided in Sweden. Why was it not deployed? It seems, contradictively,that it was due to lack of public interest[18] and that Teracom needed all the frequencyspace they could get for HDTV.
The DVB-project is currently developing future standards for 3DTV, IPTV, and thenext generation hand-held TV (DVB-NGH).
1.4 Content protection
The DVB-CSA (Common Scrambling Algorithm) is a cryptographic method used to securethe Mpeg-2 transport stream payloads. Its main purpose is to support the use of scramblingand Conditional Access (CA) for digitally transmitted Pay-TV mainly in Europe.
DVB-CSA was specified by the European Telecommunications Standards Institute(ETSI) and adopted by the DVB consortium in May 1994. The algorithm was designedto last for at least ten years. Nearly twenty years later it still remains a robust solution.Threats to DVB-CSA has however increased along with technology and computing poweradvancements.
Original Equipment Manufacturers (OEM) were not allowed to implement the algorithmin software and it was only available under a Non-Disclosure Agreement (NDA) from anETSI custodian. This was due to security reasons. As a result, very little was publiclyknown about the CSA algorithm for almost a decade. This changed in the autumn of2002 when a Windows program called FreeDec emerged which implemented the CSA insoftware. FreeDec was reverse–engineered and the results were published on the web. Thepublication meant that the details about the algorithm became available to the public.Even now when the algorithm is known, no attack has been successful.
The details of CSA and are beyond the scope of this thesis.
They vary between broadcasters and are usually implemented on a smart card which isrequired to view encrypted pay-TV transmissions. The actual key for the CSA is calledcommon key and is usually changed every 10–120 seconds. The great relevance of CSA liesin the fact, that every encrypted digital Pay-TV transmission in Europe is secured using
14
CHAPTER 1. BACKGROUND 1.5. SET-TOP-BOX REQUIREMENTS
CSA. A practical break of CSA would therefore affect all broadcasters which would haveto exchange the hardware used to decrypt the transport streams. [3]
1.5 Set-Top-Box requirements
A Set-Top-Box (STB) is a device that converts an external signal into viewable video on aconnected television. The signal source is usually a satellite dish (DVB-S), an ethernetcable (IPTV), UHF/VHF antenna (DVB-T) or simply a coaxial cable (cable television,DVB-C). There are also hybrid STB’s which allow reception from multiple sources. Suchas a combination of DVB-S, DVB-T and DVB-C reception in the same device.
A traditional STB have in general only one additional feature other than providing theconnected television with audio and video; they are also capable of presenting ElectronicProgram Guide (EPG) in a more or less understandable way. The EPG information isnormally distributed via specific EPG-packets in the DVB Transport Stream.
STB’s can be equipped with a Common Interface (DVB-CI[7]) this interface is designedso that it is possible for service providers to use custom encryption algorithms for theirtransmissions.
Details of DVB decryption mechanisms are beyond the scope of this thesis as the focusonly lies on displaying unencrypted channels, so called Free-To-Air (FTA) channels.
15
1.5. SET-TOP-BOX REQUIREMENTS CHAPTER 1. BACKGROUND
16
Chapter 2
Problem description
The main question for this thesis is if the PandaBoard, and the OMAP4 in particular, ispowerful enough in terms of performance to be used as a Set-Top-Box, (STB). For thisthesis the external signal is the European digital terrestrial television standard DVB-Tand its successor DVB-T2.
The targeted operating system is Android, developed and maintained by Google.Android is an increasingly popular operating system found primarily in smartphones andtablets. The ARM processors main advantage is its low power consumption, this is thereason why it is found in mobile devices and embedded systems. The weakness of the ARMarchitecture has until recently been performance. Is it feasible to construct a low powerARM based STB for multimedia consumption?
In order to reach the thesis goals the following tasks were considered for the implemen-tation phase:
1. Setup the devlopment enviroment
2. Boot Pandroid on the PandaBoard using an SDCard
3. Add DVB support to the Pandroid Linux kernel
4. Create an application that can tune to a channel
5. Extract the transport stream from the DVB device
6. Split the transport stream into a Video and Audio stream
7. Decode the video and audio streams
8. Synchronized audio and video playback
2.1 Linux kernel DVB-stack
In order to be able to use a USB DVB-T dongle in Android the DVB-stack has to beactivated in the Android Linux kernel. Reconfiguration and recompilation of the kernelwill be needed. Compilation of Linux kernel drivers for the DVB-T and/or DVB-T2 USBdongle is also essential.
When the DVB-stack has been enabled in the kernel, software will be needed in orderto interface with the Linux kernel DVB-API. The DVB-API is used for operations suchas tuning and scanning channels. The Linux DVB-API is developed, maintained anddocumented by the LinuxTv.org[17] Project.
17
2.2. DVB-T2 CHAPTER 2. PROBLEM DESCRIPTION
2.2 DVB-T2
One of the implementation goals is to be able to receive and display the new DVB-T2transmissions that were introduced in Sweden on November 2010 by Teracom. The problemis that since the standard is new and few countries have adopted it, the available consumerhardware supporting DVB-T2 is limited. In fact during the time this thesis was written,only one DVB-T2 compatible USB device existed; the PCTV Systems nanoStick T2 290e.Device drivers is only officially supported for Windows operating system, this means nodrivers for the Linux kernel exist. The DVB-T2 goal is directly dependant on Linux kerneldrivers for the nanoStick. If no drivers would emerge during the time reserved for theimplementation phase, a DVB-T USB device with drivers already is in the Linux kernelwill be used instead.
It turned out that linux drivers for the nanoStick device emerged in April 2011, thismeant that it would be possible to compile these drivers into the Pandroid linux kernelbefore the thesis implementation phase ended.
Once we are able to get the datastream from a DVB-T2 or a DVB-T signal, the problemthat remains is to extract the different packages contained in the transport stream, thisprocess is called demultiplexing.
2.3 Decoding Audio and Video
The audio and video packets in the Transport Stream are encoded depending on the DVBstandard used. In Sweden the video stream is encoded using MPEG-2 and the audio streamis encoded with MPEG Audio Layer II (MP2) or Dolby Digital (AC-3), when sent usingDVB-T. The video stream sent using DVB-T2 is encoded with the newer MPEG-4 AVC,also known as H.264, standard and audio is encoded with the MPEG-4 HE-AAC standard.
Unfortunately Texas Instruments was not able to enable support for the Ducati SubSystem in Pandroid when the implementation was conducted. This meant that the dedicatedhardware on the PandaBoard that are supposed to be used when decoding video streamslike the two mentioned above were not available. How can video decoding be made possibleunder these circumstances?
Due to lack of support by Texas Instruments, audio playback is not enabled in Pandroidduring the implementation phase. If this issue is not solved by Texas Instruments it willnot be possible to playback the audio contained in the Transport Stream.
18
Chapter 3
Methods
The work on this thesis was performed at the Syntronic Software Innovation office in Kista,Stockholm. During the first 3-4 weeks work was focused on finding information about thePandaBoard and setting up the development enviroment. The development enviromentincluded a desktop computer with Ubuntu 10.10 (Maverick Meerkat) installed. Ubuntuwas a natural operating system choice, since the vast majority of the documentation forthe PandaBoard assumed Ubuntu as the host system. It was decided that a version controlsystem would be needed, not only for the implementation code, but also for keeping trackof changes in this written report. GIT was chosen as the primary system because it hasgained much popularity among the open source community.
The open mobile software development platform; PandaBoard was released in October2010. The primary source of information about it can be found at the PandaBoard.orgweb page. This web page is also intended to work as a community page and to connectsoftware developers with each other. Information was also found on their mailing list andIRC chat room.
One of the main goals for this thesis was to receive and decode the new DVB-T2transmissions, however Linux drivers for the PCTV nanoStick 290e usb dongle did notemerge until 8 April 2011. News about the development progress for the nanoStick Linuxdrivers could be found at a blog by a man called Steve Kerrison[12]. Once it was confirmedthat the nanoStick actually was working in Linux one of these devices was ordered bySyntronic and work was made on patching the Linux kernel in order to get it workingproperly on the existing system.
19
CHAPTER 3. METHODS
20
Chapter 4
Implementation
4.1 System peripherals
The most significant piece of external system hardware is of course the nanoStick DVB-T2usb dongle. However in order to interact with the system in real time, such as changingchannel, an usb mouse is used. An usb cable connecting the PandaBoard with the hostsystem is convenient for development and debugging purposes. A screen must of course beattached to the system and the system itself is running from and stored on a memory stickinserted into the PandaBoad. An overview of the system peripherals can be seen in figure4.1.
Figure 4.1: Peripherals connected to the PandaBoard
4.1.1 Usb devices
The preferred way to test programs for Android is by transferring and debugging theapplication via the Android Debug Bridge, or ADB in short. An ADB-connection is
21
4.1. SYSTEM PERIPHERALS CHAPTER 4. IMPLEMENTATION
established via an usb cable connected to an usb-port on the host system and a mini-usbconnector on the PandaBoard. There is also a RS-232 port on the PandaBoard that canbe used for debugging, a serial-to-usb converter was used for this project however only to alimited extent, mostly because ADB was more convenient.
The only HID-device, (Human Interface Device), used in this implementation is anusb-mouse. A standard Android system do does not include mouse support since theconventional way of interaction with an Android device is by using a touch screen interface.The Pandroid distribution is patched so that it supports a mouse. Instructions on how topatch Pandroid is documented on the Pandroid homepage.
In order for the system to receive DVB-T and DVB-T2 signals a PCTV nanoStick T2290e usb dongle is connected to an usb port. This device is the first consumer producton the market to support the DVB-T2 standard and as of April 2011, the first DVB-T2device with Linux kernel driver support. The nanoStick was shipped with a small antenna,however reception is poor with this kind of small antennas. For the implementation adifferent antenna was used instead.
The PCTV nanoStick T2 290e
The 1st November 2010 Swedish terrestrial broadcast service company, Teracom, startedsending with the new DVB-T2 standard in Sweden. A few weeks later, in November 25th2010, the PCTV nanoStick T2 290e was released. It was promoted with slogans suchas ”World’s first DVB-T2 receiver for PC”, even today, a half year later, few consumerDVB-T2 devices for PCs are out on the market.
Figure 4.2: The PCTV nanoStick T2 290e
The nanoStick’s main components is a NXP TDA18271HDC2 tuner, an Empia em28174controller chip and a Sony CXD2820R demodulator. The tuner and the controller chipdrivers was implemented previously in the kernel, since they are used in other DVB-Tdevices. The problem was the Sony demodulator. New code needed to be written for it towork in Linux. Pictures of the nanoStick PCB can be found in appendix A.
Below is an excerpt from an entry from Steve’s blog when an important milestone wasreached. It was early April when the last missing piece of the driver puzzle was laid by theFinnish Linux kernel hacker Antti Palosaari.
8th April 2011 - And so it begins
Another LinuxTV developer enters the fray - Antti Palosaari - bringing withhim a CXD2820R module! There are still a lot of details to work out when it
22
CHAPTER 4. IMPLEMENTATION 4.1. SYSTEM PERIPHERALS
comes to T2 support, but this should give us a working device. I cannot testuntil some time next week, but the keen hackers amongst you might want tohead over to Antti’s git repository to check it out.
Of course, those of you already on the linux-media mailing list will already knowthis. For those who don’t, here’s the message.
4.1.2 HDMI
A LCD screen is connected using a HDMI to DVI adaptor cable. The PandaBoard hastwo HDMI-connector slots, one of them hosting a HDMI-signal and the other a DVI-signal.The HDMI-connector with the DVI-signal was not enabled in Pandroid at the time theimplementation was performed. There where no problems using the HDMI-signal to a LCDdisplay with a DVI-port.
4.1.3 The SD-Card
The PandaBoard default boot device is the Secure Digital card slot. A Secure Digital flashmemory card, SD-card, is needed to be formatted in a proper configuration and have thenecessary boot-images in order to boot the Linux Kernel and Pandroid. The SD-card ispartitioned according to instructions on the PandaBoard homepage. Basically the SD-cardis partitioned with one boot partition and one rootfs partition. The boot partition containsthree important binary images. The X-Loader, U-Boot and uImage. The Android filesystem resides on the rootfs partition.
Figure 4.3: Content on the SD-card
X-loader, is a small first stage boot loader originating from the u-boot source code andis loaded into the internal static ram. The internal static ram is very small, this is why the
23
4.2. SYSTEM SOFTWARE CHAPTER 4. IMPLEMENTATION
x-loader is stripped down to a minimum and is used to initialize memory and the peripheraldevices needed to access and load the second stage loader, U-Boot, into main memory.
Universal Bootloader, or Das U-Boot, is a widely used open source boot loader forembedded devices. It’s name comes from an abbreviation of Das Unterseeboot, which isGerman for ”the submarine”. The U-Boots primary function is to load the Linux kerneluImage.
The Linux kernel 2.6.35 is configured and patched for use with Android accordingto instructions on the Pandroid web page. In order to activate the DVB-stack customconfigurations has to be made in the Linux compilation configuration. When using aDVB-T usb dongle with drivers included in Linux 2.6.35, it is only a matter of using thekernel menuconfig-tool, and enabling DVB-T support in the kernel. However, it is not thissimple when trying to use a brand new DVB-T2 usb dongle with Linux. First of all thekernel driver source code was released in April 2011, and this highly experimental driver isat the time of writing this report, not officially included in the Linux kernel yet. Thereis however a script provided by linuxtv.org that facilitates compilation of experimentalmultimedia drivers for the Linux kernel. The LinuxTV project, hosted at linuxtv.org,develops and maintains the DVB driver subsystem which is included in the Linux 2.6.xkernel.
4.2 System software
The main system covers a wide spectrum of technical areas. Some of these areas includeprogramming fragment shaders for OpenGL ES 2.0 hardware, utilizing 3rd party libraries(such as FFMpeg), patching, configuring and compiling of the Linux kernel to name a few.The main challenge has been to find a solution for each area and connect them with eachother into a usable system.
4.2.1 DVB Utils Library
In order to control a DVB device in Linux the DVB-stack must be included in the kernelwhen compiling. The DVB-stack is developed and maintained by the LinuxTV project.The code in the DVB Utils library is partly based on example code from the dvb-apps[17]tree, provided by LinuxTV.
When a DVB device has locked to a MUX a transport stream can be extracted fromthe device. The transport stream is part of the MPEG-2 specification.
4.2.2 FFMpeg Library
The FFMpeg project is an open source and cross-platform project that host several librariesfor multimedia processing. Two of its most widely used libraries are libavcodec andlibavformat. Many media formats don’t specify which codecs should be used to encodeaudio and video data. These media formats are containers that define how audio and videostreams should be combined into a single stream. In the case of DVB the container formatis called Transport Stream and is specified in the MPEG-2 standard. The libavformatlibrary deals with parsing the transport stream and separating, or demuxing, the streamscontained in it. The libavcodec library deals with decoding the audio and video streams.
In order to use FFMpeg in the thesis implementation, the following features has beenactivated when configuring and compiling FFMpeg:
• Video decoders: MPEG-2 and MPEG-4 AVC (aka h.264)
24
CHAPTER 4. IMPLEMENTATION 4.2. SYSTEM SOFTWARE
Figure 4.4: Simplified system structure
• Audio decoders: MPEG-2 Audio Layer II (mp2) and MPEG-1 Layer 3 (mp3)
• Demuxer: MPEG-2 Transport Stream
• NEON optimizations
Once the DVB device is locked to a mux, the Transport Stream can be read from thedevice. Each mux usually contains several standard definition television channels and it isneeded to specify what channel we are interested in reading data from. When demuxing isdone it is possible to extract the separate media elements, such as the video and the audiostream. For DVB-T the video stream is encoded with the MPEG-2 video format and forDVB-T2 the video format is MPEG-4 AVC (also known as h.264). The audio used in DVBbroadcasting is usually encoded with MPEG-2 Audio Layer II however audio suppport isnot included in the thesis implementation.
4.2.3 Color space conversion
When the video stream is decoded the resulting product is video frames in a format calledYUV420P.This format is used for video compression and human perception reasons. The Yin YUV is the frame luminance component. It measures 720x576 pixels and represents theframe brightness with values 0-255. The U and the V is two chrominance components thatonly is one fourth the size of the Y component. The reason for this is that the human eyeis much more sensitive to brightness compared to color information, it is therefore possibleto reduce the resolution of the color components with no loss in perceived quality. Thenumbers 420 (sometimes written 4:2:0) in YUV420P represents the chroma sub-sampling
25
4.2. SYSTEM SOFTWARE CHAPTER 4. IMPLEMENTATION
rates for Y, U and V component. This basically means that for each 4x4 pixel squarearea with Y components there are two 1x1 pixel chrominance componens, U and V. Thisrelationship is illustrated in Figure 4.5. Finally the letter P needs an explanation, it standsfor planar and means that the three components Y, U and V are grouped in three separateplanes as Figure 4.5 shows.
Figure 4.5: YUV420P SDTV frame layout
In order to render an YUV420P frame to a screen it has to be converted to RGBformat. This conversion process needs to be efficient and fast. At an early stage of theimplementation process the FFMpeg project was evaluated and some experiments wasconducted using a library called libswscale, part of FFMpeg. Libswscale has some functionsfor converting YUV frames to RGB, however results from the experiments showed thatit was not fast enough for real-time video rendering on the PandaBoard. At that point adistinct problem was identified; how should color space conversion be done?
The following conclusions was made:
• No support yet for the dedicated Ducati subsystem in Pandroid
• libswscale in FFMpeg was not fast enough
It was noted that some work had been done in adding NEON optimizations for thelibswscale color space conversion routines[2]. However the code seemed highly experimentaland was quickly dismissed as a viable solution. Instead focus went to the hardware thatwas supported in Pandroid; OpenGL ES 2.0. It was chosen to do the color space conversionsince it has a fragment shader that can be programmed, demonstrated in Listings 4.1, toperform arbitrary operations on individual pixels in a texture - very fast. Much effort wentinto researching how this could be implemented in the fragment shader using GLSL, aC-like OpenGL programming language.
The actual mathematical conversion is done according to recommendations specifiedby the International Telecommunications Union. There are differences in the conversiondetails depending on if the transmission is SDTV or HDTV. ITU-R BT.601[8] is used forSDTV and ITU-R BT.1847[9] is used for HDTV[13]. Figure 4.6 illustrate the conversion
26
CHAPTER 4. IMPLEMENTATION 4.2. SYSTEM SOFTWARE
matrix corresponding to ITU-R BT.601, according to [10] pages 19-20. In order tominimize implementation code complexity, a decision was made to treat HDTV as a SDTVtransmission as far as color space conversion is concerned.
R601
G601
B601
=
1.164 0 1.5961.164 −0.391 −0.8131.164 2.018 0
Y − 16U − 128V − 128
Figure 4.6: Color space conversion matrix according to ITU-R BT.601 used for SDTV
The YUV values in the above matrix are in the range of a byte i.e. 0-255. The pixelvalues in OpenGL ranges between 0.0 and 1.0 so the YUV values need to be divided with256 as seen in Figure 4.7.
Y − 16
256U − 128
256V − 128
256
=
Y − 0.0625U − 0.5V − 0.5
Figure 4.7: Conversion from samplesize 0-256 to 0.0-1.0
1 Y = texture2D (Ytex , v texCoord ) . r − 0 . 0 625 ;2 U = texture2D (Utex , v texCoord ) . r − 0 . 5 ;3 V = texture2D (Vtex , v texCoord ) . r − 0 . 5 ;45 mat3 m = mat3 (1 . 164 , 1 . 164 , 1 . 164 ,6 0 . 0 , −0.392 , 2 . 017 ,7 1 . 596 , −0.813 , 0 . 0 ) ;8 vec3 rgb = m ∗ vec3 (Y, U, V) ;9 g l FragCo lor = vec4 ( rgb , 1 . 0 ) ;
Listing 4.1: GLSL code from the implementation of YUV to RGB color space conversionin the OpenGL ES 2.0 fragment shader
4.2.4 OpenGL ES 2.0
The lack of support for the Ducati sub system, used for hardware accelerated video decoding,lead to the use of OpenGL ES 2.0 fragment shader. Inspiration for this kind of solutioncame from a number of open source projects. One project called Dead Penguin[6]. TheDead Penguin project has also, apart from having color space conversion in the fragmentshader, deinterlacing in the fragment shader. A very interesting feature that could bea possible future improvement for this project. Another example is the perhaps moreinfluential Chromium project[15]. They have also done an implementation of color spaceconversion in an OpenGL fragment shader.
Shortly after the decision was made to leverage on the OpenGL ES 2.0 accelerationhardware, the idea came to life that it would be interesting to present the decoded videostream as a cube that could be rotated. Each face of the cube rendering video except thetop and bottom face. The top was given a static texture with the Syntronic logo and the
27
4.3. RESULTS AND EVALUATION CHAPTER 4. IMPLEMENTATION
bottom face the Uppsala University logo. The cube can be rotated using an usb mouseconnected to the PandaBoard.
4.3 Results and evaluation
The research reported in this thesis has investigated if and how the PandaBoard mobilesoftware development platform can be used as a STB. A prototype system has beendeveloped demonstrating reception and displaying of digital terrestrial television. Theprototype system is based on Android and uses an external usb device for DVB-T andDVB-T2 reception. The system displays unencrypted channels sent over the Swedish digitalterrestrial television network.
Dedicated PandaBoard hardware for accelerated video decoding was not used due toTI lack of driver support for Pandroid. Another way of rendering the video stream wasdeveloped using the FFMPEG library. It was discovered that the OpenGL ES 2.0 fragmentshader could be used for accelerated color space conversion, this technique is implementedin the prototype system.
Figure 4.8: Prototype system
4.3.1 System performance
The system performance was evaluated by measuring the amount of frames per seconds(FPS) rendered on to the screen. For this evaluation the channels SVT1 (SD) and SVT1HD was compared to give an indication on the system video rendering performance. Nodeinterlacing or audio decoding was done when measuring the FPS.
28
CHAPTER 4. IMPLEMENTATION 4.3. RESULTS AND EVALUATION
Tests showed that when viewing SVT1 (SD) an average of 70 FPS was reached. 70 FPSis good since the video is only encoded with 25 FPS i.e. 280% more fps than required. Theextra processing power could be used for deinterlacing and audio decoding. When SVT1HD was viewed an average of 14,8 FPS was measured. This value is 59% less than the videostream at 25 FPS. All frames are not displayed resulting in a poor viewing experience.
4.3.2 Power consumption
The system power consumption was measured using an ammeter . In an idle state thesystem consumes 3,15 Watts without the nanoStick DVB-T2 USB device connected. Afterconnecting it, the power consumption increased to 4 Watts revealing that the nanoStickitself consume 0,5 Watts in an idle state. When a SDTV channel is tuned and renderedon a screen 5,75 Watts is consumed and the CPU load goes up to an average of 37%.When viewing a HDTV channel 7 Watts is used by the system and the CPU goes up to95%-100%.
29
4.3. RESULTS AND EVALUATION CHAPTER 4. IMPLEMENTATION
30
Chapter 5
Conclusions
5.1 Unsolved problems
5.1.1 Audio playback
One of the issues has been audio playback in Pandroid. The FFMPEG library, used in thisproject, includes codecs for the audio streams used in DVB broadcasts. This fuctionallitywas however never used since there was no ability to output the audio from the PandaBoardhardware.
Audio support was eventually added to Pandroid, unfortunately this was done afterthe implementation phase of this thesis. Below is an excerpt from the changelog on thePandroid webpage.
6/03/11 * Audio is working to some degree.
5.1.2 Accelerated video playback
The Pandaboard has great hardware for video playback, however it is not worth muchwhen it is not possible to use it. TI did not manage to support accelerated video playbackfor Pandroid during the implementation phase of this thesis. A different solution was foundin order to render video. The FFMPEG library supports the needed video codecs used inthe DVB-T and DVB-T2 standards. It was configured to take advantage of the advancedSIMD (Single Instruction Multiple Data), perhaps more known as NEON, support in theARM Cortex-A9 CPU. NEON provides standardized acceleration for media and signalprocessing applications.
The PandaBoard performed well using the FFMPEG library. SDTV signals recievedusing DVB-T was rendered to a screen in satisfying way. HDTV video received usingDVB-T2 was however too demanding for the system and video was not rendered fastenough, resulting in dropped and corrupt frames.
31
5.1. UNSOLVED PROBLEMS CHAPTER 5. CONCLUSIONS
32
Chapter 6
Future work
Unfortunately hardware accelerated video playback was not possible to implement in theprototype system for this thesis. Further evaluation of performance of this dedicatedhardware for video processing on the PandaBoard would be a very interesting developmentdirection to pursue in the future. When/if TI manages to enable such support in futureversions of Pandroid.
Audio support in a STB is of course essential and it should be included in a futuresystem, it seems that TI has managed to enable audio in Pandroid, however this was afterthe prototype system for this thesis was completed.
Other things that are missing in the prototype,that are needed in a STB, is EPGsupport (Electronic Program Guide). EPG data is sent in the MPEG2 Transport Streamand should be relatively easy to implement from a technical standpoint. It is howevernon-trivial to present this EPG information to the consumer in a satisfactory manner.The information should be easy to read and navigation through the data should be veryintuitive.
A remote control is the traditional and probably most common device for systeminteraction with a STB. There is an open source project called LIRC (Linux InfraredRemote Control) that could be useful in a future implementation. A remote control maynot, however, be the best method. It would be interesting to do further research into whatinteraction device is most suitable for a future STB-device.
33
CHAPTER 6. FUTURE WORK
34
Bibliography
[1] Wikipedia, November 2010. http://en.wikipedia.org/wiki/File:Digital_
broadcast_standards.svg.
[2] Allmann, J. FFMpeg fork with NEON optimizations for YUV to RGB color spaceconversion. https://github.com/j0sh/ffmpeg/tree/swscale-neon/libswscale/
arm.
[3] Benoit, H. Digital television: MPEG-1, MPEG-2 and principles of the DVB system,2. ed. ed. Focal press/Elsevier, Burlington, MA, 2004.
[4] Brown, A. Sweden: The digital threat to cultural sovereignty. In Digital TerrestrialTelevision in Europe, A. Brown and R. G. Picard, Eds. Lawrence Erlbaum AssociatesPublishers, Mahwah, New Jersy, 2005, pp. 203–221.
[5] Digital TV Commission. Digital TV switchover : 2005.09.19 - 2007.10.15 : TheDigital TV Commission’s final report, KU 2004:04., http://www.sweden.gov.se/sb/d/574/a/113008 visited February 2011 ed. Swedish Government Official Report,2008:35. Fritze, Stockholm, 2008.
[6] Essen, L. Dead Penguin - the set top box. http://www.deadpenguin.tv/.
[7] European Telecommunications Standards Institute (ETSI). EN 50221:”Common Interface Specification for Conditional Access and other Digital Video Broad-casting Decoder Applications”, February 1997. http://www.dvb.org/technology/
standards/En50221.V1.pdf.
[8] International Telecommunications Union. Recommendation ITU-R BT.601,Encoding Parameters of Digital Television for Studios. Geneva, 1992.
[9] International Telecommunications Union. Recommendation ITU-R BT.1847,1280 x 720, 16:9 progressively-captured image format for production and internationalprogramme exchange in the 50 Hz environment. Geneva, 2009.
[10] Jack, K. Video Demystified: A Handbook for the Digital Engineer, 4th Edition,4th ed. Newnes, Newton, MA, USA, 2005.
[11] Jeding, L. From Massmedia to Multimedia – Digitalisation of Swedish Television,http://www.sweden.gov.se/sb/d/8430/a/737 visited February 2011 ed. SwedishGovernment Official Report, 1996:25. Fritze, Stockholm, 1996.
[12] Kerrison, S. PCTV nanoStick T2 290e for Linux. http://stevekerrison.com/
290e/index.html.
35
BIBLIOGRAPHY BIBLIOGRAPHY
[13] Sveriges Television. HDTV - Expertinformation. http://svt.se/2.152197/1.
2408782/expert.
[14] Teracom. Konsumenterna efterfragar mobil tv. http://www.dvb-h.org/PDF/
Stockholm-Trial-Release.pdf, March 2007.
[15] The Chromium Authors. The Chromium projects. http://src.chromium.org/
svn/trunk/src/media/tools/player_x11/gles_video_renderer.cc.
[16] The DVB Project Office. Introduction to the dvb project - creating globalstandards for digital television. Fact Sheet http://www.dvb.org/technology/fact_sheets/DVB-Project_Factsheet.pdf. visited February 2011.
[17] The LinuxTV project. The LinuxTV project develops and maintains the DVB driversubsystem which is included in the Linux 2.6.x kernel. http://linuxtv.org/repo/.
[18] Zirn, T. Teracom kan tvingas offra mobil-tv-nat. http://computersweden.idg.se/2.2683/1.132101, November 2007.
36
Appendix A
PCTV nanoStick T2 290e PCB
Figure A.1: The top side contains a NXP TDA18271HDC2 tuner. Photograph courtesy ofSteve Kerrison
Figure A.2: The bottom side contains an Empia em28174 controller chip and a SonyCXD2820R demodulator. Photograph courtesy of Steve Kerrison
37
