Hindawi Publishing CorporationEURASIP Journal on Audio, Speech, and Music ProcessingVolume 2007, Article ID 51563, 14 pagesdoi:10.1155/2007/51563

Research ArticlePerceptual Coding of Audio Signals Using AdaptiveTime-Frequency Transform

Karthikeyan Umapathy and Sridhar Krishnan

Department of Electrical and Computer Engineering, Ryerson University, 350 Victoria Street, Toronto, ON, Canada M5B 2K3

Received 22 January 2006; Revised 10 November 2006; Accepted 5 July 2007

Recommended by Douglas S. Brungart

Wide band digital audio signals have a very high data-rate associated with them due to their complex nature and demand for high-quality reproduction. Although recent technological advancements have significantly reduced the cost of bandwidth and minia-turized storage facilities, the rapid increase in the volume of digital audio content constantly compels the need for better compres-sion algorithms. Over the years various perceptually lossless compression techniques have been introduced, and transform-basedcompression techniques have made a significant impact in recent years. In this paper, we propose one such transform-based com-pression technique, where the joint time-frequency (TF) properties of the nonstationary nature of the audio signals were exploitedin creating a compact energy representation of the signal in fewer coefficients. The decomposition coefficients were processed andperceptually filtered to retain only the relevant coefficients. Perceptual filtering (psychoacoustics) was applied in a novel way byanalyzing and performing TF specific psychoacoustics experiments. An added advantage of the proposed technique is that, dueto its signal adaptive nature, it does not need predetermined segmentation of audio signals for processing. Eight stereo audio sig-nal samples of different varieties were used in the study. Subjective (mean opinion score—MOS) listening tests were performedand the subjective difference grades (SDG) were used to compare the performance of the proposed coder with MP3, AAC, andHE-AAC encoders. Compression ratios in the range of 8 to 40 were achieved by the proposed technique with subjective differencegrades (SDG) ranging from –0.53 to –2.27.

Copyright © 2007 K. Umapathy and S. Krishnan. This is an open access article distributed under the Creative CommonsAttribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work isproperly cited.

1. INTRODUCTION

The proposed audio coding technique falls under the trans-form coder category. The usual methodology of a transform-based coding technique involves the following steps: (i)transforming the audio signal into frequency domain coef-ficients, (ii) processing the coefficients using psychoacous-tic models and computing the audio masking thresholds,(iii) controlling the quantizer resolution using the maskingthresholds, (iv) applying intelligent bit allocation schemes,and (v) enhancing the compression ratio with further loss-less compression schemes. A comprehensive review of manyexisting audio coding techniques can be found in the worksof Painter and Spanias [1]. The proposed technique nearlyfollows the above general transform coder methodologyhowever, unlike the existing techniques, the major part ofthe compression was achieved by exploiting the joint time-frequency (TF) properties of the audio signals. Hence, themain focus of this work would be in demonstrating thebenefits of using an adaptive time-frequency transformation

(ATFT) for coding the audio signals (i.e., improvement andnovelty in step (i)) and developing a psychoacoustic model(i.e., improvement and novelty in step (ii)) adapted to TFfunctions.

The block diagram of the proposed technique is shownin Figure 1. The ATFT used in this work was based on thematching pursuit algorithm [2]. The Matching pursuit algo-rithm is a general framework where any given signal can bemodeled/decomposed into a collection of iteratively selected,best matching signal functions from a redundant dictionary.The basis functions chosen to form the redundant dictio-nary determine the nature of the modeling/decomposition.When the redundant dictionary is formed using TF func-tions, the matching pursuit yields an ATFT [2]. The ATFTapproach provides higher TF resolution than the existing TFtechniques such as wavelets and wavelet packets [2]. Thishigh-resolution sparse decomposition enables us to achieve acompact representation of the audio signal in the transformdomain itself. Also, due to the adaptive nature of the ATFT,there was no need for signal segmentation.

2 EURASIP Journal on Audio, Speech, and Music Processing

Thresholdin

quiet(TIQ)

Wideband

audioTF

modeling

TFparameterprocessing

Masking QuantizerMediaor

channel

Perceptual filtering

Figure 1: Block diagram of the ATFT audio coder.

Psychoacoustics was applied in a novel way [3, 4] on theTF decomposition parameters to achieve further compres-sion. In most of the existing audio coding techniques, thefundamental decomposition components or building blocksare in the frequency domain with corresponding energy as-sociated with them. This makes it much easier for them toadapt the conventional, well-modeled psychoacoustics tech-niques into their encoding schemes. In few existing tech-niques [5, 6] based on sinusoidal modeling using matchingpursuits, psychoacoustics was applied either by scaling thedictionary elements or by defining a psychoacoustic adap-tive norm in the signal space. As the modeling was done us-ing a dictionary of sinusoids and segment-by-segment ba-sis approach [7, 8], these techniques do not qualify as a trueadaptive time-frequency transformation. Also, due to the factthat sinusoids were used in the modeling process, it was eas-ier to incorporate the existing psychoacoustics models intothese techniques. On the other hand, in ATFT, the signalwas modeled using TF functions which have a definite timeand frequency resolution (i.e., each individual TF functionis time limited and band limited), hence the existing psy-choacoustics models need to be adapted to apply on the TFfunctions.

The audio coding research is very dynamic and fastchanging. There are a variety of applications (offline, IPstreaming, embedding in video, etc.) and situations (networktraffic, multicast, conferencing, etc.) for which many spe-cific compression techniques were introduced. A universalcomparison of the proposed technique with all audio cod-ing techniques would be out of the scope of this paper. Theobjective of this paper is to demonstrate the application ofATFT for coding audio signals with some modifications tothe conventional blocks of transform-based coders. Hencewe restrict our comparison only with the two commonlyknown audio codecs MP3 and MPEG-4 AAC/HE-AAC [9–12]. These comparisons merely assess the performance of theproposed technique in terms of compression ratio achievedunder similar conditions against the mean opinion scores(MOS) [13].

Eight reference wideband audio signals (ACDC, DEFLE,ENYA, HARP, HARPSICHORD, PIANO, TUBULARBELL,VISIT) of different categories were used for our analysis.Each was a stereo signal of 20-second duration extractedfrom CD quality digital audio sampled at 44.1 kHz. TheACDC and DEFLE were rapidly varying rock-like audio sig-nals, ENYA and VISIT were signals with voice and hum-ming components, PIANO and HARP were slowly varyingclassical-like signals, HARPSICHORD and TUBULARBELLwere fast varying stringed instrumental audio signals. The

ACDC, DEFLE, ENYA, and VISIT are polyphonic soundswith many sound sources.

The paper is organized as follows: Section 2 covers theATFT algorithm, Section 3 describes the implementation ofpsychoacoustics, Sections 4 and 5 cover quantization, com-pression ratios and reconstruction process, Section 6 ex-plains the quality assessment of the proposed coder, Section 7covers results and discussion, and Section 8 summarizes theconclusions.

2. ATFT ALGORITHM

Audio signals are highly nonstationary in nature and thebest way to analyze them is to use a joint TF approach. TFtransformations can be performed either decomposing a sig-nal into a set of scaled, modulated, and translated versionsof a TF basis function or by computing the bilinear energydistributions (Cohen’s class) [14, 15]. TF distributions arenonparametric and mainly used for visualisation purposes.For the application in hand, the automatic choice wouldbe a parametric decomposition approach. There are vari-ety of TF decomposition techniques with different TF res-olution properties. Some examples in the increasing orderof TF resolution superiority are short-time Fourier trans-form (STFT), wavelets, wavelet packets, pursuit-based algo-rithms [14]. As explained in Section 1, the proposed ATFTtechnique was based on the matching pursuit algorithm withtime-frequency dictionaries. ATFT has excellent TF reso-lution properties (better than wavelets and wavelet pack-ets) and due to its adaptive nature (handling nonstation-arity), there is no need for signal segmentations. Flexiblesignal representations can be achieved as accurate as pos-sible depending upon the characteristics of the TF dictio-nary.

In the ATFT algorithm, any signal x(t) is decomposedinto a linear combination of TF functions gγn(t) selectedfrom a redundant dictionary of TF functions [2]. In this con-text, redundant dictionary means that the dictionary is over-complete and contains much more than the minimum re-quired basis functions, that is, a collection of nonorthogonalbasis functions, that is, much larger than the minimum re-quired basis functions to span the given signal space. UsingATFT, we can model any given signal x(t) as

x(t) =∞∑

n=0angγn(t), (1)

K. Umapathy and S. Krishnan 3

where

gγn(t) =1√sng(t − pnsn

)exp

{j(2π fnt + φn

)}(2)

and an are the expansion coefficients.The scale factor sn, also called as octave parameter, is used

to control the width of the window function, and the param-eter pn controls the temporal placement. The parameters fnand φn are the frequency and phase of the exponential func-tion, respectively. The index γn represents a particular com-bination of the TF decomposition parameters (sn, pn, fn, andφn). The signal x(t) is projected over a redundant dictionaryof TF functions with all possible combinations of scaling,translations, and modulations. The dictionary of TF func-tions can either suitably be modified or selected based on theapplication in hand. When x(t) is real and discrete, like theaudio signals in the proposed technique, we use a dictionaryof real and discrete TF functions. Due to the redundant orovercomplete nature of the dictionary it gives extreme flex-ibility to choose the best fit for the local signal structures(local optimisation) [2]. This extreme flexibility enables tomodel a signal as accurate as possible with the minimumnumber of TF functions providing a compact approximationof the signal.

In our technique, we used the Gabor dictionary (Gaus-sian functions) which has the best TF localization proper-ties [15]. At each iteration, the best correlated TF functionwas selected from the Gabor dictionary. The remaining signalcalled the residue was further decomposed in the same wayat each iteration subdividing them into TF functions. AfterM iterations, signal x(t) could be expressed as

x(t) =M−1∑

n=0

⟨Rnx, gγn

⟩gγn(t) + RMx(t), (3)

where the first part of (3) is the decomposed TF functionsuntil M iterations, and the second part is the residue whichwill be decomposed in the subsequent iterations. This pro-cess was repeated till all the energy of the signal was decom-posed. At each iteration, some portion of the signal energywas modeled with an optimal TF resolution in the TF plane.Over iterations, it can be observed that the captured energyincreases and the residue energy falls. Based on the signalcontent, the value of M could be very high for a completedecomposition (i.e., residue energy = 0). Examples of Gaus-sian TF functions with different scale and modulation pa-rameters are shown in Figure 2. The order of computationalcomplexity for one iteration of the ATFT algorithm is givenby O(N logN) where N is the length of the signal samples.The time complexity of the ATFT algorithm increases withthe increase in the number of iterations required to modela signal, which in turn depends on the nature of the signal.Compared to this, the computational complexity of MDCT(in MP3 and AAC) is only O(N logN) (same as FFT).

Any signal could be expressed as a combination of coher-ent and noncoherent signal structures. Here the term “co-herent signal structures” means those signal structures thathave a definite TF localisation (or) exhibit high correlation

Time positionpn

Center frequency

fn

Highercenter frequency

TF functions withsmaller scale

Scale or octavesn

Figure 2: Gaussian TF function with different scale andmodulationparameters.

with the TF dictionary elements. In general, the ATFT al-gorithm models the coherent signal structures well withinthe first few 100 iterations, which in most cases contributeto > 90% of the signal energy. On the other hand, the non-coherent noise like structures cannot be easily modeled sincethey do not have a definite TF localisation or correlation withdictionary elements. Hence, these noncoherent structures arebroken down by the ATFT into smaller components to searchfor coherent structures. This process repeats until the wholeresidue information is diluted across the whole TF dictionary[2]. From a compression point of view, it would be desirableto keep the number of iterations (M ≪ N) as low as possibleand at the same time sufficient enough to model the signalwithout introducing perceptual distortions. Considering thisrequirement, an adaptive limit has to be set for controllingthe number of iterations. The energy capture rate (signal en-ergy capture rate per iteration) could be used to achieve this.By monitoring the cumulative energy capture over iterationswe could set a limit to stop the decomposition when a par-ticular amount of signal energy was captured. The minimumnumber of iterations required to model a signal without in-troducing perceptual distortions depends on the signal com-position and the length of the signal.

In theory, due to the adaptive nature of the ATFT decom-position, it is not necessary to segment the signals. However,due to the computational resource limitations (Pentium III,933MHZ with 1GB RAM), we decomposed the signals in5-seconds durations. The larger the duration decomposed,the more efficient is the ATFT modeling. This is because ifthe signal is not sufficiently long, we cannot efficiently uti-lize longer TF functions (highest possible scale) to approxi-mate the signal. As the longer TF functions cover larger sig-nal segments and also capture more signal energy in the ini-tial iterations, they help to reduce the total number of TFfunctions required to model a signal. Each TF function has adefinite time and frequency localization, which means all the

4 EURASIP Journal on Audio, Speech, and Music Processing

0.2

0.1

0

−0.1−0.2

Amplitude

(a.u.)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2×105Time samples

Sample signal

(a)

1

0.8

0.6

0.4

0.2

0

Energy

(a.u.)

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

×10−3

Number of TF functions

Energy curve

99.5% of the signal energy

(b)

Figure 3: Energy cutoff of a sample signal (au:arbitrary units).

information about the occurrences of each of the TF func-tions in time and frequency of the signal is available. Thisflexibility helps us later in our processing to group the TFfunctions corresponding to any short time segments of thesignal for computing the psychoacoustic thresholds. In otherwords, the complete length of the audio signal can be firstdecomposed into TF functions and later the TF functionscorresponding to any short time segment of the signal canbe grouped together. In comparison, most of the DCT- andMDCT-based existing techniques have to segment the sig-nals into time frames and process them sequentially. This isneeded to account for the nonstationarity associated with theaudio signals and also to maintain a low-signal delay in en-coding and decoding.

In the proposed technique for a signal duration of 5-second, the limit was set to be the number of iterationsneeded to capture 99.5% of the signal energy or to a maxi-mum of 10 000 iterations. For a signal with less noncoherentstructures, 99.5% of signal energy could be modeled with alower number of TF functions than a signal with more non-coherent structures. In most cases, a 99.5% of energy cap-ture nearly characterizes the audio signal completely. Theupper limit of the iterations is fixed to 10 000 iterations toreduce the computational load. Figure 3 demonstrates thenumber of TF functions needed for a sample signal. In thefigure, the right panel (b) shows the energy capture curvefor the sample signal in the left panel (a) with number of TFfunctions in the X-axis and the normalized energy in the Y-axis. On average, it was observed that 6000 TF functions areneeded to represent a signal of 5-second-duration sampled at44.1 kHz. Using the above procedure, all eight (ACDC, DE-FLE, ENYA, HARP, HARPSICHORD, PIANO, TUBULAR-BELL, VISIT) reference wideband audio signals were decom-posed into their respective number of TF functions.

3. IMPLEMENTATION OF PSYCHOACOUSTICS

In this work, psychoacoustics was applied in a novel way onthe TF functions obtained by decomposition. In the conven-tional method, the signal is segmented into short time seg-ments and transformed into frequency domain coefficients.These individual frequency components are used to computethe psychoacoustic masking thresholds and accordingly theirquantization resolutions are controlled. In contrast, in our

approach we computed the psychoacoustic masking prop-erties of individual TF functions and used them to decidewhether a TF function with certain energy was perceptuallyrelevant or not based on its time occurrence with other TFfunctions. TF functions are the basic components of the pro-posed technique and each TF function has a certain time andfrequency support in the TF plane. So their psychoacousticalproperties have to be studied by taking them as a whole toarrive at a suitable psychoacoustical model.

3.1. Threshold-in-quiet (TiQ)

TiQ is the minimum audible threshold below which we donot perceive a signal component. TF functions form fun-damental building blocks of the proposed coder and theycan take all possible combinations of time duration and fre-quency. However in the ATFT algorithm implementation,they could take any time width between 22 samples (90 mi-croseconds) to 214 samples (0.4 second) in steps with any fre-quency between 0 and 22 050Hz (max frequency). The timesupport of a frequency component also plays an importantrole in the hearing process. From our experiments we ob-served that longer duration TF functions were heard muchbetter even with lower energy levels than the shorter dura-tion TF functions. Hence, out of all the possible durations ofthe TF functions, the highest possible time duration of 16 384samples corresponding to the octave 14 (the term octave isfrom the implementation nomenclature, i.e., the scale factordoubles in each step) was the most sensitive TF function fordifferent combinations of frequencies. This forms the worstcase TF function in ourmodeling for which our ears aremoresensitive. So it is obvious that this TF function has to be usedto obtain the worst case threshold in quiet (TiQ) curve forour model. The curve obtained in this way will hold good forall other TF functions with all possible combinations of time-widths and center frequencies. Figure 4 demonstrates the dif-ferent modulated versions of the TF function with maximumtime-width (octave 14).

3.2. Experimental setup

Experiments were performed with 5 listeners to arrive atthe TiQ curve for the above-mentioned TF function withmaximum time width. The experimental setup consisted

K. Umapathy and S. Krishnan 5

(a) (b) (c) (d)

Figure 4: TF function with time width of 16 384 samples modulated at different center frequencies.

of a Windows 2000 PC (Intel Pentium III 933MHz), cre-ative sound blaster PCI card, high-quality head phones(Sennheiser HD490), and Matlab software package.

The TF functions (duration 0.4 seconds) with differentcenter frequencies were played to each of the listeners. Itshould be noted that the “frequency” here means the centerfrequency of the TF function and not the absolute frequencyas used in regular psychoacoustics experiments. In general,each of the TF functions will have a center frequency anda frequency spread based on the time width they can take.For this experiment as we are using only the TF functionwith the longest width (duration 0.4 second), the frequencyspread is fixed. For each frequency setting the amplitude ofthe TF function was reduced in steps until the listener couldno longer hear the TF function anymore. Once this point isreached, the amplitude of the TF function is increased andplayed back to the listener to confirm the correct point ofminimum audibility. This is repeated for the following valuesof center frequencies: 10Hz, 100Hz, 500Hz, 1 kHz, 2 kHz,4 kHz, 6 kHz, 8 kHz, 10 kHz, 12 kHz, 16 kHz, and 20 kHz.The minimum audible amplitude level for each frequencysetting was recorded. The values obtained from 5 listenerswere averaged to obtain the absolute threshold of audibilityfor TF functions.

To reduce the computational complexity, the frequencyrange is divided into three bands of low frequency (500Hzand below), sensitive frequencies (500Hz to 15 kHz), andhigh frequencies (15 kHz and above). The experimentalvalues were averaged to get uniform thresholding for thelow- and high-frequency bands. In the middle or sensitiveband, the lowest averaged experimental value was selected asthreshold of audibility throughout the band. Figure 5 illus-trates the averaged TiQ curve superimposed on the actualTiQ curve. The TF functions are grouped into the above-mentioned three frequency groups. Amplitude values of theTF functions are calculated from their energy and octave val-ues. These amplitude values are checked with the TiQ averagevalues. The TF functions whose amplitude values fall belowthe averaged TiQ values were discarded.

3.3. Audiomasking applied to TF functions

Similar to TiQ, the existing masking techniques cannot beused directly on the proposed coder for the same reasons ex-plained earlier. So masking experiments were conducted toarrive at masking thresholds for TF functions with different

100

10−1

10−2

10−3

10−4

Amplitude

(a.u.)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2×104Frequency (Hz)

TIQ curve

Figure 5: Average thresholding applied to TiQ curve. Solid linedenotes the actual TiQ curve and dashed line denotes the appliedthreshold. (au:arbitrary units).

time-widths with a similar experimental setup as describedin Section 3.2. The possible time duration of TF functionsvaries between 22 to 214 in steps of powers of 2, each of thetime width TF function was examined for its masking prop-erties. Each of this different duration TF functions, can oc-cur at any point in time with frequencies between 20Hz to20 kHz. Out of the possible durations of the TF functionsthe shorter durations (22 to 27) are transient-like structureswhich have larger bandwidths but little time support. Re-moving these TF functions in the process of masking will in-troduce more tonal artifacts in the reconstructed signal. Thishappens because the complex frequency pattern of the sig-nal is disturbed to some extent. Hence, these functions werepreserved and not used for masking purposes.

The remaining TF functions with time widths (28 to 214)were used for the masking experiments. TF functions witheach of these time widths (durations from 256 to 16 384 sam-ples) were tested for their masking capabilities with othertime-width TF functions at various energies and frequencies.The TF functions were first grouped into equivalents of 400time samples (10 milliseconds). This is possible as each of theTF functions has the precise information about its time oc-currence. Once they were grouped into time slots equivalent

6 EURASIP Journal on Audio, Speech, and Music Processing

10ms10ms

10ms 10ms

(a)

Maskee Masker

Masker

Maskee

Masker

Maskee

Masker Maskee

(b)

Figure 6: (a) Illustration of few possible time occurrences of two TF functions as masker and maskee, (b) possible masking conditions thatcan occur within the 10 milliseconds time slot.

to 10 milliseconds, the TF functions falling in each time slotwere divided into 25 critical bands based on their center fre-quencies. In each critical band, the TF function with high-est energy was located. Relative energy difference of this TFfunction with the remaining TF functions in the same crit-ical band was computed. Using a lookup table, each of theremaining TF functions was verified if it would be maskedby the relative energy difference with the TF function havingthe highest energy. The experimental procedure for comput-ing the lookup table of masking thresholds will be explainedin subsequent paragraphs. The TF functions which fall be-low the masking threshold defined by the lookup tables willbe discarded.

As shown in Figure 6(a) within the 10 milliseconds du-ration the location of masker and maskee TF functions canoccur anywhere. The worst case situation would be when themasker TF function occurs at the beginning of the time slot,and the maskee TF function occurs at the end of the time slotor vice versa. So all of our testing was done for this worst casescenario by placing the masker TF function and the maskeeTF function at the maximum distance of 10 milliseconds.

Based on the duration of masker and maskee TF func-tions, one of the following could occur as depicted inFigure 6(b).

(1) Masker andmaskee are apart in time within the 10mil-liseconds, in which case they do not occur simultane-ously. In this situation masking is achieved due to tem-poral masking effects where a strong occurring maskermasks preceding and following weak signals in timedomain.

(2) Masker duration is large enough that the maskee du-ration falls within the masker (two scenarios shown inFigure 6(b)) even after a 400 samples shift. In this case,simultaneous masking occurs.

(3) Masker duration is shorter than the maskee duration.In this case, both simultaneous and temporal mask-ings are achieved. The simultaneous masking occurs

during the duration of the masker when the maskee isalso present. Temporal masking occurs before and af-ter the duration of the masker.

Four sets of experiments were conducted with masker TFfunction (normalized in amplitude) taking center frequencyof 150Hz, 1 kHz, 4.8 kHz, and 10.5 kHz (critical band centerfrequencies) and the maskee TF function taking center fre-quency of 200Hz, 1.1 kHz, 5.3 kHz, and 12 kHz (correspond-ing critical band upper limits), respectively. As the mask-ing thresholds depend also on the frequency separation ofmasker and maskee, maximum separation from the criticalband center frequency was taken for our experiments formaskee TF functions. TF functions of each time width wereused as maskers to measure their masking capabilities on theremaining of each time width TF functions for all the above 4different frequency sets. Both (masker and maskee TF func-tions) were placed apart with 10 millisecond duration andplayed to the listeners. Each time the amplitude of the mas-kee TF function was reduced till the listener perceived onlythe masker TF function, or in other words, until there was nodifference observed between the masker TF function playedindividually or played together with the maskee TF function.At this point, the masker TF function’s energy was sufficientto mask the maskee TF function. The difference in their ener-gies is calculated in dB and used as the masking threshold forthe particular time-width maskee TF function when occur-ring simultaneously with that particular time-width maskerTF function. Once all the measurements were finished, eachtime-width TF function was analyzed as a maskee against allthe remaining time-width TF functions as masker. An av-erage energy difference was computed for each time-widthTF function below which they will be masked by any othertime-width TF functions. Five different listeners participatedin the test and their average masking curves for each time-width of TF functions were computed. Figure 7 shows thedifferent masking curves obtained for different durations ofTF functions. The X-axis represents the different time-width

K. Umapathy and S. Krishnan 7

55

50

45

40

35

30

25

20

15

RelativedB

difference

withmasker

8 9 10 11 12 13 14

Time width of maskee TF functions 2x

Masking curves

Masker freq. Maskee freq.

10500–11250Hz

10500–12000Hz

4800–5300Hz

150–200Hz

1000–1080Hz

Figure 7: Masking curves for different time width of TF functions.

TF functions and the Y-axis represents the relative energydifference with the masker in dB.

The masking curve obtained for critical band center fre-quency 10.5 kHz deviates from the remaining curves consid-erably. This is due to the fact that the frequency separationbetween themasker and themaskee becomes very high at thisband. This is because we use for all our experiments the up-per limit of the critical band as the maskee frequency to sim-ulate the worst case scenario. To demonstrate this frequencyseparation dependence on masking performance, a secondmasking curve was obtained for the critical band with a cen-ter frequency of 10.5 kHz for masker but this time the fre-quency separation between masker and maskee was reducedby half. The curve dropped down explaining the increase inmasking performance, that is, when the frequency separationbetween themasker andmaskee was reduced, the average rel-ative dB difference required for masking also reduces.

From these curves it could be observed that the mask-ing curves of critical bands with center frequencies 150Hz,1 kHz, and 4.8 kHz remain almost the same. Hence, themasking curve obtained for 1 kHz was used as the lookuptable for the first 20 critical bands. The remaining 5 crit-ical bands use the masking curve obtained for the criticalband with a centre frequency of 10.5 kHz (with 12 kHz up-per limit) as the lookup table. These lookup tables were usedto verify if a TF function will be masked by the relative dBdifference of it with the TF function having highest energywithin the same critical band.

The flow chart in Figure 8 gives an overview of the mask-ing implementation used in the proposed coder.

4. QUANTIZATION

Most of the existing transform-based coders rely on con-trolling the quantizer resolution based on psychoacousticthresholds to achieve compression. Unlike this, the proposedtechnique achieves a major part of the compression in thetransformation itself followed by perceptual filtering. That is,

TF functions

Sort the TFfunctions intotime slots of

10ms

TF functions in each time slot are divided into25 critical bands based on their center frequency

Verification of each TF function with themasking threshold based on lookup tables

Lookuptables

Store indexof TF functionsto be removed

Check ifall time slotsprocessed

No

YesDiscard the TFfunctions &proceed toquantization

· · · 25 critical bands · · ·

Figure 8: Flow chart of the masking procedure.

when the number of iterations M needed to model a signalis very low compared to the length of the signal, we just needM×L bits. Where L is the number of bits needed to quantizethe 5 TF parameters that represent a TF function. Hence, welimit our research work to scalar quantizers as the focus ofthe research mainly lies on the TF transformation block andthe psychoacoustics block rather than the usual subblocks ofthe data-compression application.

As explained earlier, each of the five parameters energy(an), centre frequency ( fn), time position (pn), octave (sn),and phase (φn) are needed to represent a TF function andthereby the signal itself. These five parameters were to bequantized in such a way that the quantization error intro-duced was imperceptible while, at the same time, obtaininggood compression. Each of the five parameters has differentcharacteristics and dynamic range. After careful analysis ofthem, the following bit allocations were made. In arriving atthe final bit allocations informal MOS tests were conductedto compare the quality of the 8 audio samples before and af-ter quantization stage.

In total, 54 bits are needed to represent each TF func-tion without introducing significant perceptual quantizationnoise in the reconstructed signal. The final form of data forM TF functions will contain the following:

(1) energy parameter (log companded) =M ∗ 12 bits;(2) time position parameter =M ∗ 15 bits;(3) center frequency parameter =M ∗ 13 bits;

8 EURASIP Journal on Audio, Speech, and Music Processing

1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

Energy

(a.u.)

0 50 100 150 200 250 300 350

Number of TF functions

Curve-fitted energy curve

Original energy curve

Compressed curve

Figure 9: Log companded original and curve-fitted energy curvefor a sample signal (au:arbitrary units).

(4) phase parameter =M ∗ 10 bits;

(5) octave parameter =M ∗ 4 bits.

The sum of all the above (= 54 ∗ M bits) will be the totalnumber of bits transmitted or stored representing an audiosegment of duration 5 seconds. The energy parameter afterlog companding was observed to be a very smooth curveas shown in Figure 9. Fitting a curve to the energy param-eter further reduces the bitrate. Nearly 90% of the energyis present in the first few 100 TF functions and hence theyare not used for curve fitting. The remaining number of TFfunctions is divided into equal lengths of 50 points on thecurve. Only the values corresponding to these 50 points needto be sent with the first few original 100 values. The distancebetween these 50 points can be treated as linear comparingthe spread of total number of TF functions. In the recon-struction stage, these 50 points can be interpolated linearlyto the original number of points. The error introduced inthis procedure was very small due to the smooth slope ofthe curve. Moreover, this error was introduced only in the10% energy of the signal which was not perceived. To bet-ter explain the benefit of the proposed curve fitting approachin reducing the bitrate, let us take an example of transmit-ting 5000 TF functions. To transmit the energy parameterfor 5000 TF functions (without applying curve fitting) willrequire 5000 ∗ 12 bits = 60 000 bits. With curve fitting, saywe preserve the energy parameter for the first 150 TF func-tions and thereafter select the energy parameter from every50th TF function in the remaining 4850 TF functions. Thiswill result in [150 + (4850/50 = 97)] = 247 values of the en-ergy parameter requiring only 247∗12 = 2964 bits for trans-mission. We see a massive reduction in bits due to curve fit-ting. Figure 9 demonstrates the original curve superimposedwith the fitted curve. Every kth point in the compressed curvecorresponds to actually the (3 + k)∗ 50th point in the origi-nal curve. A correlation value of 1 was achieved between theoriginal curve and the interpolated reconstructed curve.

With just a simple scalar quantizer and curve fitting ofthe energy parameter, the proposed coder achieves high com-pression ratios. Although a scalar quantizer was used to re-duce the computational complexity of the proposed coder,sophisticated vector quantization techniques can be easily in-corporated to further increase the coding efficiency. The 5parameters of the TF function can be treated as one vec-tor and accordingly quantized using predefined codebooks.Once the vector is quantized, only the index of the codebookneeds to be transmitted for each set of TF parameters result-ing in a large reduction of the total number of bits. How-ever, designing the codebooks would be challenging as thedynamic ranges of the 5 TF parameters are drastically differ-ent. Apart from reducing the number of total bits, the quan-tization stage can also be utilized to control the bitrates suit-able for constant bitrate (CBR) applications.

5. COMPRESSION RATIOS

Compression ratios achieved by the proposed coder werecomputed for the eight sample signals as described below.

(1) As explained earlier, the total number of bits needed torepresent each TF function is 54.

(2) The energy parameter is curve fitted and only the first150 points in addition to the curve-fitted point need tobe coded.

(3) So the total number of bits needed forM iterations fora 5 second duration of the signal is TB1 = (M ∗ 42) +((150+C)∗12), where C is the number of curve-fittedpoints, andM is the number of perceptually importantfunctions.

(4) The total number of bits needed for a CD quality 16 bitPCM technique for a 5 second duration of the signalsampled at 44 100Hz is TB2 = 44 100 ∗ 5 ∗ 16 =3 528 000.

(5) The compression ratio can be expressed as the ratio ofthe number of bits needed by the proposed coder to thenumber of bits needed by the CD quality 16 bit PCMtechnique for the same length of the signal, that is,

Compression ratio = TB2

TB1. (4)

(6) The overall compression ratio for a signal was then cal-culated by averaging all the 5 seconds duration seg-ments of the signal for both the channels.

The proposed coder is based on an adaptive signal transfor-mation technique, that is, the content of the signal and thedictionary of basis functions used to model the signal play animportant role in determining how compact a signal can berepresented (compressed). Hence, variable bitrate (VBR) isthe best way to present the performance benefit of using anadaptive decomposition approach. The inherent variabilityintroduced in the number of TF functions required to modela signal and thereby the compression is one of the highlightsof using ATFT. Although VBR would be more appropriate topresent the performance benefit of the proposed coder, CBRmode has its own advantages when used with applications

K. Umapathy and S. Krishnan 9

that demand network transmissions over constant bitratechannels with limited delays. The proposed coder can also beused in CBRmode by fixing the number of TF functions usedfor representing signal segments, however due to the signaladaptive nature of the proposed coder, this would compro-mise the quality at instances where signal segments demanda higher number of TF functions for perceptually lossless re-production. Hence, we choose to present the results of theproposed coder using only the VBR mode.

We compare the proposed coder with two existing pop-ular and state-of-the-art audio coders viz MP3 (MPEG 1layer 3) and MPEG-4 AAC/HE-AAC. Advanced audio cod-ing (AAC) is the current industrial standard which was ini-tially developed for multichannel surround signals (MPEG-2AAC [16]). The transformation technique used is the mod-ified discrete cosine transform (MDCT). Compared to mp3which uses a polyphase filter bank and an MDCT, new cod-ing tools were introduced to enhance the performance. Thecore of MPEG-4 AAC is basically the MPEG-2 AAC butwith added tools to incorporate additional coding enhance-ments and MPEG-4 features so that a broad range of appli-cations are covered. There are many application specific pro-files that can be chosen to adaptively configure the MPEG-4audio for the user needs. It is claimed that at 128 kbps theMPEG-4 AAC is indistinguishable from the original audiosignal [17]. As there are ample studies in the literature [9, 11,12, 16, 18, 19] available for both MP3 and MPEG-2/4 AAC,more details about these techniques are not provided in thispaper.

As the proposed coder is of VBR type, in our first com-parison we compare the proposed coder with both the MP3and MPEG-4 AAC coders in VBR mode. All eight sam-ple signals were MP3 coded using the Lame MP3 encoder(version 1.2, Engine 3.88 Alpha 8) in VBR mode [20, 21].For the MPEG-4 AAC, we used the AAC encoder devel-oped by PysTel research (currently ahead software). As thereare many profiles possible in AAC, we choose the followingsuitable profile for our comparison-VBR high quality withmain long-term prediction (LTP) [10]. All eight signals wereMPEG-4 AAC encoded. The average bitrates for each sig-nal for both MP3 and MPEG-4 AAC was found using theWinamp decoder [22]. These average bitrates were used tocalculate the compression ratio as described below.

(1) Bitrate for a CD quality 16 bit PCM technique for 1-second stereo signal is given by TB3 = 2∗ 44 100∗ 16.

(2) The average bitrate/s achieved by (MP3 or MPEG-4AAC) in VBR mode = TB4.

(3) Compression ratio achieved by (MP3 or MPEG-4AAC) = TB3/TB4.

The 2nd, 4th, and 6th columns of Table 1 show the com-pression ratio (CR) achieved by the MP3, MPEG-4 AAC,and the proposed ATFT coders for the set of 8 sample au-dio files. It is evident from the table that the proposed coderhas better compression ratios than MP3. When comparingwith MPEG-4 AAC, 5 out of 8 signals are either comparableor have better compression ratios than the MPEG-4 AAC. Itis noteworthy to mention that for slowmusic (classical type),

the ATFT coder provides 3 to 4 times better comparison thanMPEG-4 AAC or MP3. The compression ratio alone cannotbe used to evaluate an audio coder. The compressed audiosignals has to undergo a subjective evaluation to comparethe quality achieved with respect to the original signal. Thecombination of the subjective rating and the compression ra-tio will provide a true evaluation of the coder performance.A second comparison was also performed by comparing theHE-AAC profile of the MPEG-4 audio at the same bitrates tothat was achieved by the ATFT coder in the VBRmode. Moredetails on the HE-AAC profile of the MPEG-4 audio will bediscussed in the subsequent sections. A subjective evaluationwas performed as will be explained in Section 6.

Before performing the subjective evaluation, the signalhas to be reconstructed. The reconstruction process is astraight forward process of linearly adding all the TF func-tions with their corresponding five TF parameters. In orderto do that, first the TF parameters modified for reducing thebitrates have to be expanded back to their original forms.The log-compressed energy curve was log expanded after re-covering back all the curve points using interpolation on theequally placed 50 length points. The energy curve was multi-plied with the normalization factor to bring the energy pa-rameter as it was during the decomposition of the signal.The restored parameters (energy, time-position, centre fre-quency, phase, and octave) were fed to the ATFT algorithmto reconstruct the signal. The reconstructed signal was thensmoothed using a third order Savitzky-Golay [23] filter andsaved in a playable format.

Figure 10 demonstrates a sample signal (/“HARP”/) andits reconstructed version and the corresponding spectro-grams. It can be clearly observed from the reconstructed sig-nal spectrogram compared with the original signal spectro-gram, how accurately the ATFT technique has filtered outthe irrelevant components from the signal (evident fromTable 1-(/“HARP”/)-high compression ratio vs. acceptablequality). The accuracy in adaptive filtering of the irrelevantcomponents is made possible by the TF resolution providedby the ATFT algorithm.

6. QUALITY ASSESSMENT OF THE PROPOSED CODER

6.1. Subjective evaluation of ATFT coder

Subjective evaluation of audio quality is needed to assessthe audio codec performance. We use the subjective evalu-ation method recommended by ITU-R standards (BS. 1116).It is called a “double blind triple stimulus with hidden ref-erence” [1, 13]. In this method, listeners are provided withthree stimuli A, B, and C for each sample under test. A is thereference/original signal, B and C are assigned to either ofthe reference/original signal or the compressed signal undertest. Basically the reference signal is hidden in either B or Cand the other choice is assigned to the compressed (or im-paired) signal. The choice of reference or compressed signalfor B and C is completely randomized. For each sample au-dio signal, listeners listen to all three (A, B, C) stimuli, andcompare A with B and A with C. After each comparison of Awith B, and A with C, they grade the quality of the B and C

10 EURASIP Journal on Audio, Speech, and Music Processing

Table 1: Compression ratio (CR) and subjective difference grades (SDG). MP3-moving picture experts group I layer 3, AAC-MPEG-4 AAC,moving picture experts group 4 advanced audio coding-VBR main LTP profile, ATFT:adaptive time-frequency transform.

Samples MP3 AAC ATFT

— CR SDG CR SDG CR SDG

ACDC 7.5 0.067 9.3 −0.067 8.4 −0.93DEFLE 7.7 −0.2 9.5 −0.067 8.3 −1.73ENYA 9 0 9.6 −0.133 20.6 −0.8HARP 11 −0.067 9.4 −0.067 36.3 −1HARPSICHORD 8.5 −0.067 10.2 0.33 9.3 −0.73PIANO 13.6 0.067 9.6 −0.2 40 −0.8TUBULARBELL 8.3 0 10.1 0.067 10.5 −0.53VISIT 8.4 −0.067 11.5 0 11.6 −2.27Average 9.3 −0.03 9.9 −0.02 18.3 −1.1

0.2

0.1

0

−0.1

−0.2

Amplitude

(a.u.)

1 2 3 4×105Time samples

Original

(a)

×104

2

1.5

1

0.5

0

Frequency

(Hz)

0 2 4 6 8

Time (s)

Original

(b)

0.2

0.1

0

−0.1

−0.2

Amplitude

(a.u.)

1 2 3 4×105Time samples

Reconstructed

(c)

×104

2

1.5

1

0.5

0

Frequency

(Hz)

0 2 4 6 8

Time (s)

Reconstructed

(d)

Figure 10: Example of a sample original (/“HARP”/) and the reconstructed signal with their respective spectrograms.X-axes for the originaland reconstructed signal are in time samples, and X-axes for the spectrogram of the original and the reconstructed signal are in equivalenttime in seconds. Note that the sampling frequency = 44.1 kHz (au:arbitrary units).

signals with respect to A in 5 levels from 1 to 5. The levels 1to 5 corresponds to (1) unsatisfactory (or) very annoying, (2)poor (or) annoying, (3) fair (or) slightly annoying, (4) good(or) perceptible but not annoying, and (5) excellent (or) im-perceptible [1, 13]. A subjective difference grade (SDG) [1]

is computed by subtracting the absolute score assigned tothe hidden reference from the absolute score assigned to thecompressed signal. It is given by

SDG = Grade{compressed} −Grade{reference}. (5)

K. Umapathy and S. Krishnan 11

Thresholdin

quiet(TIQ)

Wideband

audioTF

modeling

TFparameterprocessing

Masking QuantizerMediaor

channel

Perceptual filtering

Reconstructed Reconstructed Reconstructed

A B C

Figure 11: Block diagram explaining MOS choices A, B, and C for the subjective evaluation of perceptual filtering and quantization stages.

Table 2: Average SDG for PFO and QO (PFO:perceptually filteredoutput, QO:quantization output).

Samples PFO QO

ACDC −0.8 −0.4DEFLEP −0.6 −0.6ENYA −0.8 −1HARP −0.8 −0.6HARPSICHORD 0 −0.6PIANO −0.2 −0.8TUBULARBELL −0.6 −1.2VISIT −0.2 −0.4Average −0.5 −0.7

Accordingly, the scale of SDG will range from (−4 to 0)with the following interpretation, (−4) unsatisfactory (or)very annoying, (−3) poor (or) annoying, (−2) fair (or)slightly annoying, (−1) good (or) perceptible but not an-noying, and (0) excellent (or) imperceptible. Fifteen listen-ers (randomly selected) participated in the MOS studies andevaluated all the 3 audio coders (MP3, AAC, and ATFT inVBR mode). The average SDG was computed for each of theaudio sample. The 3rd, 5th, and 7th columns of Table 1 showthe SDGs obtained for MP3, AAC, and ATFT coders, respec-tively. MP3 and AAC SDGs fall very close to the Impercepti-ble (0) region, whereas the proposed ATFT SDGs are spreadout between −0.53 to −2.27.

A second listening test was performed using the HE-AAC v1/v2 (also known as MPEG-4 AAC v2, AAC PLUS v2)encoder [12]. The HE-AAC encoder is an enhanced high-efficiency version of the AAC with improved audio quality.The HE-AAC v1 encoder comprises of the basic AAC andspectral band replication (SBR) technologies whereas the v2encoder comprises of AAC, SBR, and parametric stereo (PS)coding technologies. The HE-AAC v2 encoder is rated as thebest audio codec at low bitrates. In the second test, SDG werecomputed for the 8 audio samples by encoding them usingHE-AAC v1/v2 coder at the same bitrates/compression ratiosas that of the ATFT. Six listeners participated in the secondMOS study and the obtained SDG are shown in Table 3. De-tailed discussion of the compression ratio versus subjectiveevaluation scores is given in Section 7.

6.2. Subjective evaluation of perceptual filtering andquantization stages

In order to evaluate the performance of the developed per-ceptual model and the quantization stage, another listening

test was conducted with 5 listeners. The procedure as de-scribed in Section 6.1 was repeated but the choices A, B, andC were assigned as shown in Figure 11. The output of theTF decomposition (TF modeling stage) forms the input tothe perceptual filtering module, hence the reference A wasassigned to the reconstructed signal at the output of the TFmodeling stage. Similarly, choice B was assigned to the re-constructed signal at the output of the perceptual filteringmodule and C to the reconstructed signal at the output of thequantization stage. Listeners were asked to rate the choices B(perceptual filtering output) and C (quantization stage out-put) with the reference A (TF modeling output) on a scale of1 to 5 as explained in Section 6.1.

The results were averaged for the five listeners and givenin Table 2. From Table 2, it can be observed on an average,SDGs of −0.5 and −0.7 were achieved for the perceptualfiltering stage and the quantization stage, respectively. TheSDG scores indicate that the novel perceptual filtering tech-nique proposed is performing exceedingly well with the eightsample signals and the noise introduced in the process ofquantization affects the output quality minimally. Interest-ingly, it can be noted from Table 2 that the signals ACDC,DEFLEP, and VISIT have better SDG scores when the ref-erence is the reconstructed output signal of the TF modelingblock thanwhen the reference is the original signal. This givesa valid clue that the low SDG scores achieved by these signals(as seen in Table 1) are not due to the perceptual filtering orthe quantization stages but may be due to TF modeling withsymmetrical type Gaussian functions.

7. RESULTS ANDDISCUSSION

7.1. Performance comparison in VBRmode

The compression ratios (CR) and the SDG for all threecoders (MP3, AAC, and ATFT) are shown in Table 1. All thecoders were tested in the VBR mode. For the proposed tech-nique, VBRwas the best way to present the performance ben-efit of using an adaptive decomposition approach. In ATFT,the type of the signal and the characteristics of the TF func-tions (type of dictionary) control the number of transfor-mation parameters required to approximate the signal andthereby the compression ratio. The inherent variability intro-duced in the number of TF functions required tomodel a sig-nal is one of the highlights of using ATFT. Hence, we chooseto present comparison of the coders in the VBR mode.

The results show that the MP3 and AAC coders per-form well with excellent SDG scores (Imperceptible) at a

12 EURASIP Journal on Audio, Speech, and Music Processing

Table 3: SDG comparison of MPEG-4 AAC PLUS coder for the same compression ratio/bitrate achieved by the proposed ATFT coder.Modes: 1. HE-high efficiency AAC v1 (AAC and spectral band replication (SBR)) and 2. HEv2-high efficiency AAC v2 (AAC, SBR andparametric stereo (PS) coding).

Samples HE-AAC v1/2 ATFT

— MODE Kbps CR SDG SDG

ACDC HE 168 8.4 0.17 −0.93DEFLE HE 170 8.3 0.17 −1.73ENYA HEv2 68 20.6 −0.17 −0.8HARP HEv2 38 36.3 0 −1HARPSICHORD HE 151 9.3 −0.33 −0.73PIANO HEv2 35 40 0.17 −0.8TUBULARBELL HE 134 10.5 0 −0.53VISIT HE 121 11.6 0.33 −2.27Average — 111 18.3 0.04 −1.1

45

40

35

30

25

20

15

10

5

Com

pression

ratio(C

R)

−4 −3 −2 −1 0 1

MP3AACATFT

Very annoying ImperceptibleSubjective difference grade (SDG)

Figure 12: Subjective difference grade (SDG) versus compressionratios (CR).

compression ratio around 10. The proposed coder does notperform well with all of the eight samples. Out of the 8samples, 6 samples have an SDG between −0.53 to −1(imperceptible-perceptible but not annoying) and 2 sampleshave SDG below −1. Out of the 6 samples with SDGs be-tween (−0.53 and −1), 3 samples (ENYA, HARP, and PI-ANO) have compression ratios 2 to 4 times higher thanMP3 and AAC and 3 samples (ACDC, HARPSICHORD, andTUBULARBELL) have comparable compression ratios withmoderate SDGs.

Figure 12 shows the comparison of all three coders byplotting the samples with their SDGs in X-axis and compres-sion ratios in the Y-axis. If we can virtually divide this plot

in segments of SDGs (horizontally) and the compression ra-tios (vertically), then the ideal desirable coder performanceshould be in the right top corner of the plot (high compres-sion ratios and excellent SDG scores). This is followed nextby the right bottom corner (low compression ratios and ex-cellent SDG scores) and so on as we move from right to leftin the plot. Here, the terms “low” and “high” compressionratios are used in a relative sense based on the compressionratios achieved by all the 3 coders in this study. From the plotit can be seen the MP3 and AAC coders occupy the right bot-tom corner, whereas the samples fromATFT coder are spreadover. As mentioned earlier, 3 out the 8 samples of the ATFTcoder occupy the right top corner however only with mod-erate SDGs that are much less than the MP3 and the AAC.Three out of the remaining 5 samples of the ATFT coder oc-cupy the right bottom corner, however again with only mod-erate SDGs that are less than MP3 and AAC. The remain-ing 2 samples perform the worst occupying the left bottomcorner.

We analyzed the poorly performing ATFT-coded signalsDEFLE and VISIT. DEFLE is a rapidly varying rock-like sig-nal with minimal voice components and VISIT is a sig-nal with dominant voice components. We observed that thesymmetrical and smooth Gaussian dictionary used in thisstudy does not model the transients well, which are the mainfeatures of all rapidly varying signals like DEFLE. This ineffi-cient modeling of transients by the symmetrical Gaussian TFfunctions resulted in the poor SDG for the DEFLE. A moreappropriate dictionary would be a damped sinusoids dictio-nary [24] which can better model the transient like decay-ing structures in audio signals. However, a single dictionaryalone may not be sufficient to model all types of signal struc-tures. The second signal VISIT has significant amount(s)of voice components. Even though, the main voice compo-nents are modeled well by the ATFT, the noise like hissingand shrilling sounds (noncoherent structures) could not bemodeled within the decomposition limit of 10 000 iterations.These hissing and shrilling sounds actually add to the pleas-antness of the music. Any distortion in them is easily per-ceived which could have reduced the SDG of the signal to the

K. Umapathy and S. Krishnan 13

lowest of the group −2.27. The poor performances with thetwo audio sample cases could be addressed by using a hybriddictionary of TF functions and residue coding the noncoher-ent structures separately. However, this would increase thecomputational complexity of the coder and reduce the com-pression ratios.

7.2. Performance comparisonwith same bitrates(high-efficiency AAC)

In the second performance comparison of the ATFT coder,we choose to test the high-efficiency profile of the MPEG-4AAC v2 at the same bitrates as that of the ATFT coder. Asper [12], the HE-AAC v2 improves the coding gain of theAAC by 4 times and outperforms most of the existing codersin audio quality especially at low bitrates. All the 8 sampleswere encoded using the HE-AAC v1/v2 encoder at the samebitrates as that of the VBR bitrates of the ATFT. The choicefor the v1 or the v2 encoder was determined by the target bi-trates (below 70 kbps v2 was used). From the SDGs obtainedas shown in Table 3, it is very evident that the HE-AAC pro-files of the MPEG-4 audio codec outperforms the proposedATFT coder for the same bitrates. However, it would not befair to compare the HE-AAC directly with the ATFT, sinceATFT is a basic coder without the additional enhancementsachieved by SBR or any form of parametric coding.

Although we did not include standalone sinusoidalcoders in our comparison, the MPEG-4 HE-AAC v2 in-cludes the parametric stereo coding based on the transient-sinusoid-noise (TSN) model and is derived from the MPEG-4 audio sinusoidal coding (also abbreviated as SSC) [25]. TheTSNmodel though in existence for quite some time for audioand speech coding, received much attention recently with itsinclusion in the MPEG-4 audio standard for low-bitrate ap-plications. The formal verification tests of the MPEG-4 SSCindicate that the SSC coder performs either comparable to orbetter thanMPEG-4 AAC even at lower bitrates thanMPEG-4 AAC [25]. Another recent well-known family of sinusoidalcodec (the SiCAS codec and its variants) is from the researchgroup of Heusdens et al. and Philips Research Laboratories[26]. It is shown that the psychoacoustics model used in thisfamily of codecs is better than the MPEG I Layer I-II psy-choacoustics model [27]. The subjective listening tests indi-cate that this family of codec performs equal to or better thanMPEG-4 at low bitrates (16 Kbps (HILN), 24 Kbps (SSC),and 32Kbps (AAC)) [26]. Various improvements have beenproposed for this codec family over the years with an in-cremental performance gain [28, 29]. Interestingly, the im-provements proposed in using amplitude modulated sinu-soids over constant amplitude sinusoids indicate the migra-tion of these approaches towards completely adaptive signaldecompositions such as the one proposed in the ATFT coder[29].

8. CONCLUSIONS

This paper presented a novel ATFT coding technique forwideband audio signals. The proposed approach demon-

strated the application of adaptive time-frequency transformfor audio coding and the development of a novel psychoa-coustics model adapted to TF functions. The compressionstrategy was changed from the conventional way of control-ling quantizer resolution to achieving majority of the com-pression in the transformation itself. Eight stereo sample sig-nals were used in the study. Listening tests were conductedand the performance comparison of the proposed coder withMP3 and AAC coders was presented. From the preliminaryresults, although the proposed coder achieves high com-pression ratios, its SDG scores are well below the MP3 andAAC family of coders. The proposed coder however performsmoderately well for slowly varying classical-like signals withacceptable SDGs. The proposed coder is not as refined as thestate-of-the-art commercial coders, which to some extent ex-plains its poor performance. Future work involves testing theproposed coder with a hybrid dictionary of TF functions andincluding professional refinements.

REFERENCES

[1] T. Painter and A. Spanias, “Perceptual coding of digital audio,”Proceedings of the IEEE, vol. 88, no. 4, pp. 451–515, 2000.

[2] S. G. Mallat and Z. Zhang, “Matching pursuits with time-frequency dictionaries,” IEEE Transactions on Signal Process-ing, vol. 41, no. 12, pp. 3397–3415, 1993.

[3] K. Umapathy and S. Krishnan, “Joint time-frequency codingof audio signals,” in Proceedings of the 5th WSES/IEEE Multi-conference on Circuits, Systems, Communications, and Comput-ers (CSCC ’01), pp. 32–36, Crete, Greece, July 2001.

[4] K. Umapathy and S. Krishnan, “Low bit-rate coding of wide-band audio signals,” in Proceedings of IASTED InternationalConference on Signal Processing, Pattern Recognition and Appli-cations (SPPRA ’01), pp. 101–105, Rhodes, Greece, July 2001.

[5] R. Heusdens, R. Vafin, and W. B. Kleijn, “Sinusoidal modelingusing psychoacoustic-adaptive matching pursuits,” IEEE Sig-nal Processing Letters, vol. 9, no. 8, pp. 262–265, 2002.

[6] T. S. Verma and T. H. Y. Meng, “Sinusoidal modeling us-ing frame-based perceptually weighted matching pursuits,”in Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP ’99), vol. 2, pp. 981–984,Phoenix, Ariz, USA, March 1999.

[7] R. Heusdens, J. Jensen, P. Korten, and R. Vafin, “Rate-distortion optimal high-resolution differential quantisationfor sinusoidal coding of audio and speech,” in IEEE Workshopon Applications of Signal Processing to Audio and Acoustics (AS-PAA ’05), pp. 243–246, New Paltz, NY, USA, October 2005.

[8] R. Heusdens and J. Jensen, “Jointly optimal time segmen-tation, component selection and quantization for sinusoidalcoding of audio and speech,” in Proceedings of IEEE Inter-national Conference on Acoustics, Speech and Signal Process-ing (ICASSP ’05), vol. 3, pp. 193–196, Philadelphia, Pa, USA,March 2005.

[9] I. JTC1/SC29/WG11, “Overview of the MPEG-4 standard,” inInternational Organization for Standardisation, March 2002.

[10] J. Herre and B. Grill, “Overview of MPEG-4 audio and its ap-plications in mobile communications,” in Proceedings of the5th International Conference on Signal Processing (ICSP ’00),vol. 1, pp. 11–20, Beijing, China, August 2000.

[11] J. Herre, K. Brandenburg, et al., “Second generationISO/MPEG audio layer-3 coding,” in The 98th Audio Engineer-ing Society Convention (AES ’95), Paris, France, February 1995.

14 EURASIP Journal on Audio, Speech, and Music Processing

[12] S. Meltzer and G. Moser, “MPEG-4 HE-AAC v2—audio cod-ing for today’s digital media world,” Tech. Rep., EBU TechnicalReview, Geneva, Switzerland, January 2006.

[13] T. Ryden, “Using listening tests to assess audio codecs,” in Col-lected Papers on Digital Audio Bit Rate Reduction, N. Gilchristand C. Grewin, Eds., pp. 115–125, Audio Engineering Society,New York, NY, USA, 1996.

[14] S. Mallat, A wavelet Tour of Signal Processing, Academic Press,San Diego, Calif, USA, 1998.

[15] L. Cohen, “Time-frequency distributions: a review,” Proceed-ings of the IEEE, vol. 77, no. 7, pp. 941–981, 1989.

[16] K. Brandenburg and M. Bosi, “MPEG-2 advanced audio cod-ing: overview and applications,” in The 103rd Audio Engineer-ing Society Convention, p. 4641, Ney York, NY, USA, August1997.

[17] http://www.apple.com/MPEG4/aac/.[18] E. Eberlein and H. Popp, “Layer-3, a flexible coding standard,”

in The 94th Audio Engineering Society Convention, p. 3493,Berlin, Germany, March 1993.

[19] http://www.iis.fraunhofer.de/bf/amm/.[20] http://lame.sourceforge.net/index.php.[21] http://www.mp3dev.org/.[22] http://www.winamp.com/.[23] S. J. Orfanidis, Introduction to Signal Processing, Prentice-Hall,

Englewood Cliffs, NJ, USA, 1996.[24] M. M. Goodwin, Adaptive Signal Models: Theory, Algorithms

and Audio Applications, Kluwer Academic Publishers, Boston,Mass, USA, 1998.

[25] ISO/IEC JTC 1/SC 29/WG 11N6675, “Report on the verifica-tion tests of MPEG-4 parametric coding for high quality au-dio,” in International Organization for Standardisation, July2004.

[26] R. Heusdens, J. Jensen, W. B. Kleijn, et al., “Bit-rate scalableintraframe sinusoidal audio coding based on rate-distortionoptimization,” Journal of the Audio Engineering Society, vol. 54,no. 3, pp. 167–188, 2006.

[27] S. van de Par, A. Kohlrausch, R. Heusdens, J. Jensen, and S. H.Jensen, “A perceptual model for sinusoidal audio coding basedon spectral integration,” EURASIP Journal on Applied SignalProcessing, vol. 2005, no. 9, pp. 1292–1304, 2005.

[28] P. Korten, J. Jensen, and R. Heusdens, “High-resolution spher-ical quantization of sinusoidal parameters,” IEEE Transactionson Audio, Speech, and Language Processing, vol. 15, no. 3, pp.966–981, 2007.

[29] M. G. Christensen and S. van de Par, “Efficient parametriccoding of transients,” IEEE Transactions on Audio, Speech andLanguage Processing, vol. 14, no. 4, pp. 1340–1351, 2006.