0018-9162/07/$25.00 © 2007 IEEE September 2007 23P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y

P E R S P E C T I V E S

Long-standing techniques for

performance evaluation of

computer designs are

beginning to fail. Computers

increasingly interact with other

computers, humans, and the

outside world, leading to

scenario-oriented computing,

an emerging category of

design that will enable future

consumer devices and usher

in a new era of performance

evaluation.

A New Era ofPerformance Evaluation

Sean M. Pieper University of Wisconsin-Madison

JoAnn M. PaulVirginia Tech

Michael J. SchulteUniversity of Wisconsin-Madison

S ince 1943, researchers have used latency and throughputas the primary metrics to describe computer performance.These metrics served us well because we used computers in fairly simple ways.

The unspoken assumption is that data is available ondemand and only its quantity and content can affect executiontime. This implies batch-style execution, in which measuring eachprogram’s speed, including the operating system, in isolation candetermine overall performance. For performance-evaluation pur-poses, programs are merely an extension of instructions—reducedlatency and higher throughput are always better. This perspectiveinforms the design of benchmark suites such as those from theStandard Performance Evaluation Corporation (SPEC)1 and theEmbedded Microprocessor Benchmark Consortium (EEMBC),2

which are composed of batch-style jobs executed in isolation.For many new computer systems, such evaluation is misleading.

Computers increasingly interact with humans, the physical world,and each other—often simultaneously. Overall performance in thiscontext is a function not only of individual applications, but alsoof their interactions as they contend for resources both internal andexternal to the device. Cell phones, for example, often performsome baseband processing in software. Wireless communicationsarrive over time rather than on demand, and strict requirementsdictate when outputs must occur. Other tasks such as video-conferencing depend on this software, but they also can competewith it for memory and processing resources. I/O subsystems overwhich the processor has little or no control and interdependenciesbetween unrelated programs break the batch processing model,but they are essential aspects of this new computing style.

Researchers must describe modern computer usage in terms ofscenarios consisting of numerous I/O streams, timing information,and parallel tasks that enter and leave the system, rather than interms of programs executing in isolation from the physical worldand each other.3 Such use represents a new style of computing,which we call scenario-oriented to contrast it with other well-established computing styles such as general-purpose and appli-cation-specific. Table 1 compares these three styles.

Evaluation methods designed for general-purpose and appli-cation-specific computing are insufficient for scenario-orientedcomputing. Existing benchmarks do not reflect modern usage,and their metrics fail to describe performance as perceived byend users.

COMPUTER USAGE EVOLUTIONAs Figure 1a illustrates, with traditional computer usage, a sin-

gle task, T1, enters the system, executes for some period of time,

24 Computer

and then completes. Some time later, another task, T2,enters the system and takes its turn. This model abstractsschedulers and other operating system features to sim-plify performance evaluation. There is no contentionbetween T1 and T2. Only program control flowsequences and interleaves data access; there is a singleinput stream and a single output stream.

In contrast, as Figure 1b depicts, modern usage is morecomplicated. Many tasks operate simultaneously, con-tend for resources, and communicate with each other.Unlike traditional usage, both asynchronous and stream-ing I/Os such as alerts and user inputs, webcams, andmusic are important to the functionality, and there is notnecessarily a one-to-one mapping from inputs to outputs.

An arbitration layer uses preemptive scheduling toallow for interleaved execution and to enable multiplelogical I/O streams to share a single physical link. Thislayer supports real-time requirements and user demandfor concurrency. Advanced hardware also enables simul-taneous execution. Complex interactions between tasksand hardware resources prevent describing an entire sys-tem’s performance in terms of a single task evaluated inisolation or even of multiple tasks run one after theother. This break from traditional assumptions on com-puting causes Amdahl’s law to fail for bounded systemswith heterogeneous resources in that slowing downsome tasks can actually improve overall performance.4

While Figure 1a accurately describes scientific andengineering usage, the computing industry has expandedbeyond the small market of engineers and scientists whouse computers to develop and run batch-style programsto an ever-growing group of nontechnical users. Softwarethat uses processing power to deliver both new func-tionality and increasing ease of use has made this growthpossible. These nontechnical users currently buy billionsof processors every year in the form of cell phones, set-top boxes, and music players; and they expect thesedevices to make their lives easier and more enjoyable.

Examples such as e-mail, Web browsing, and gamingillustrate how researchers have historically harnessedincreased processing power to create a virtual infra-structure unique to computing. These applications domore than simply facilitate problem solving; they actu-

ally create entirely new technological foundations thatincrease demand for computing. In turn, the applica-tions themselves become ever more sophisticated. E-mail, for example, dates back to at least 1972, butincreased memory and processing capability, as well asmultithreaded operating systems, have expanded itscapability far beyond the transmission of text messages.

Modern e-mail programs have integrated this basefunctionality with features such spell checking, junk-mail filtering, scheduling, sort and search functions,HTML support, image decompression, encryption, andvirus checking. As a result, e-mail is significantly moreuseful, but it is also more complicated and computa-tionally intensive.

This trend is not confined to general-purpose devicesand their applications. Cell phones have reached thepoint where they no longer can be considered traditionalapplication-specific devices. They now use multithreadedoperating systems, such as Symbian and Windows CE,and they can run applications such as video-editing soft-ware and 3D games that would traditionally run on aPC. Users are thinking less in terms of general-purposecomputers or single-purpose systems, such as phones,and more in terms of programmable convergence devicesthat integrate into various aspects of their lives. Theyexpect such devices to facilitate common tasks and toenable novel ways of interacting with the world.

TURNING POINT IN HARDWARE DESIGNModern users’ demands are rapidly outpacing the capa-

bilities of existing design methodologies. The computercommunity has been in this position before. Vacuumtubes gave way to discrete transistors, then came simpleintegrated circuits, followed by very large-scale integra-tion systems. In each case, the response was to createentirely new foundational principles. As uniprocessordesign hits its limits, researchers must find new designmethodologies to deliver next-generation functionality.

Sematech’s most recent International TechnologyRoadmap for Semiconductors suggests that single-coredesigns cannot scale beyond 20 to 25 million transistors.Multiprocessor designs with “SOC-like integration ofless efficient, general-purpose processor cores with more

Table 1. Comparison of general-purpose, application-specific, and scenario-oriented computing.

Computing style User programmability Design Performance evaluation Inputs

General-purpose Complete programmability Balanced performance Each application evaluated Sequenced by applicationindividually

Application-specific Limited or no Excellent performance Compared against known Timed to external referenceprogrammability for a single application requirements

Scenario-oriented Can install software for Variety of uses, but Holistic evaluation of Both sequenced by applicationsnew functionality emphasizes performance scenario components and and timed to external reference

of some their interactions

September 2007 25

efficient special-purpose ‘helper engines’” are projectedto be the next step in computer evolution.5 Developersexpect as many as 63 processing engines—cores and custom logic blocks—on a single chip by 2009. Themigration to single-chip heterogeneous multiprocessors(SCHMs) will pick up over the next few years and ulti-mately allow exponential increases in performance to con-tinue while reducing reliance on clock scaling.5

Two early SCHM architectures for commercialdevices are the Sony, Toshiba, and IBM (STI) Cell andthe Sandbridge Sandblaster.6,7 Cell is geared toward set-top boxes and game consoles, while Sandblaster targetswireless handsets. In both of these areas, the most diffi-cult problems, such as physics processing for games andbaseband processing in cell phones, contain significantdata- and thread-level parallelism. To exploit this par-allelism, both Cell and Sandblaster combine a cluster ofsingle-instruction, multiple-data processors with a sin-gle scalar processor. Cell uses its scalar processor to coor-dinate the SIMD units, and Sandblaster uses its to handleuser-interface tasks. Neither architecture reserves proces-sors for specific functions. As a result of their nontradi-tional design and programming model, processors suchas Cell and Sandblaster have been described as system-on-chip designs, but this is not strictly accurate. Both

Cell and Sandblaster are more accurately described as“processors of processors.”

SoC descends from application-specific integrated cir-cuit design and provides a methodology to rapidlydevelop integrated circuits for complex, but well-defined, task sets that are fixed at design time. The SoCdesign style divides the chip into several units; some ofthese units can be programmable, but each has a fixedpurpose. Cell and Sandblaster diverge from this modelby considering the entire chip as a programmable devicethat must be able to dynamically reallocate resources.They also are intended for devices that are marketablebased on their compelling features, rather than sheerprocessing power.

SCENARIO-ORIENTED COMPUTINGThe changes in usage combined with developments in

technology point to a new organizing principle fordesign—rather than being general-purpose or applica-tion-specific, computing is becoming scenario-oriented.Consider an onboard navigation system that determinesits current location using GPS, and receives verbalinstructions, such as “Go to 1600 Pennsylvania Avenue.”

In response to the user’s command, the system con-nects to a map server and checks for traffic advisories,

1 TT 2 T1 T3

T4 T5 T3 T5

Computer

T3

ComputerInput stream Output stream

Program

T 1 T2

Active task timeline Active task timeline

Traditional

(a)

Modern

(b)

Arbitration layer

Storage

Internet

World

User

Storage

Internet

Speakers

Display<85 Hz

<22 KHz

<10 Mbps

MouseKeyboardMicrophoneTablet

WebcamGPS

HTMLE-mailPeer-to-peerGames

HDDFlashRAM

MoviesPicturesGUIs

MusicAlerts Speech

Peer-to-peerGamesWeb feeds

Save-filesLogsDownloads

Figure 1.Traditional and modern computer usage. (a) Common traditional tasks include compilation, data compression, physics

simulation, placement and routing, and discrete event simulation. (b) Common modern tasks include image manipulation, video

and audio playback, e-mail, virus scanning, Web browsing with dynamic content, voice over IP, and 3D gaming.

26 Computer

calculates and displays an optimized route, and trans-mits the directions through speech synthesis software asthe user nears the destination. If a traffic advisory arrives,the computer drops the speech synthesis and seeks analternate route. Unlike application-specific computing,the processor performs different tasks over time, butunlike general-purpose computing, these tasks share acommon goal.

In contrast with the assumptions of both general-pur-pose and application-specific design, actual usage ofdevices such as cell phones, PDAs,and set-top boxes is modal. Usersview these devices differently accord-ing to their immediate purpose—they might use a smart phone as ascheduler, music player, game-play-ing device, digital camera, or simplyas a phone. These devices also canimplement a single mode in severalways—for example, a user mightplay songs in several different for-mats while using the device as a music player. The user’sexpectations distinguish the modes, rather than theactual hardware or software that enables them.

Because customers expect a variety of uses for a finiteamount of silicon, heterogeneous programmable coresbecome the central elements in scenario-oriented hard-ware. In contrast to the fixed-purpose resources in SoCand other application-specific design styles, the pro-cessing power of these cores is intended for a wide rangeof tasks. Unlike general-purpose computers, scenario-oriented devices must accommodate varying demandsfor different types of processing within a finite amountof silicon and certain time constraints. Heterogeneity isa response to this challenge. Software designers canleverage modality to inform scheduling decisions anduse heterogeneous cores more effectively.

FAILURE OF EXISTING METRICS AND BENCHMARKS

A benchmark suite is a set of applications that pro-vide a representative sample of usage. SPEC CPU,1 theprimary benchmark suite computer architects use, con-tains a variety of real engineering and scientific appli-cations that are selected and modified to run withminimal operating system support and interaction. TheEEMBC benchmarks, which contain representative ker-nels and applications from the embedded domain, sup-port embedded systems design.2 The Stanford SPLASHbenchmark suite, which measures the runtime of paral-lelizable algorithms, provides similar services for tradi-tional multiprocessor architectures.8 All applications inthese benchmark suites are batch jobs and are executedin isolation. Figure 1a illustrates this type of usage.

Performance in SPEC is measured as speedup, s = �reference/�measured, over a reference system. Because the

design goal is to provide excellent performance underarbitrary usage, each application’s speedup is treatedequally, using the geometric mean to generate a com-posite score. The geometric mean places greater weighton entries with low performance than those with highperformance—if a single result is 0, the entire output is0, giving this entry infinite weight. This rewards bal-anced performance, which is appropriate for general-purpose usage, but does not accurately describescenario-oriented performance.

SPEC also includes SPEC_rate, athroughput measurement intendedfor multiprocessor systems. To gen-erate SPEC_rate scores, the com-puter executes n copies of each tasksimultaneously and then measuresthe time to complete all n copies.This measurement anticipates homo-geneous usage appropriate to indus-trial applications such as simulation,Web hosting, database processing,

and supercomputing. Scenario-oriented design, in con-trast, anticipates diverse usage, as is common in mostconsumer applications.

Some newer benchmarks such as Business ApplicationsPerformance’s SYSmark and Futuremark’s 3Dmark9 aremore representative of commercial use. SYSmark evalu-ates computer performance in a business setting. It usescommon commercial applications such as AdobeAcrobat Reader, Macromedia Dreamweaver, McAfeeVirusScan, and Microsoft Office in combination withinput events and data generated by observing real users.Multiple applications execute together under differentscenarios, such as communication (e-mail and Webbrowsing) and data analysis (database queries andspreadsheet operations). The benchmark reports sepa-rate scores for each scenario. SYSmark focuses onresponse times rather than runtimes, reflecting the factthat many applications are event-driven and can go idlewhile the user is not interacting with them.10

SYSmark comes closer than SPEC to describing mod-ern usage, but it still does not include any real-timeapplications. Real-time tasks such as streaming media,baseband processing, and voice recognition are essen-tial to multimedia, mobile usage, and human-computerinteraction. Their absence limits SYSmark’s ability todescribe scenario-oriented usage. SYSmark’s focus oncurrent usage also limits its applicability to scenario-ori-ented hardware design. Several years can pass betweenthe time developers make fundamental design decisionsand when a new device hits the market. Designers needthe ability to evaluate performance under anticipatedfuture workloads.

3Dmark evaluates gaming performance under next-generation loads. It measures the real-time frame rate ofa set of games with extremely demanding graphics.

The processing power

of heterogeneous

programmable cores

is intended

for a wide range

of tasks.

3Dmark originally focused ongraphics processing units, butrecently it has added a CPU por-tion to model the impact of AI and physics calculations on framerate. While 3Dmark can describereal-time performance of futuregaming workloads, its depen-dence on frame rate as a figure ofmerit limits its applicability toother areas.

While researchers have investedmuch effort and creativity in thedesign of these benchmarks andtheir associated metrics, they areinsufficient for guiding scenario-oriented design for the followingreasons:

• Their composite metricsweight all applicationsequally. This is a relic ofsharing general-purpose processors for batch jobs.With interactive usage, fast responses to some eventsare more important than the response time to others.

• They judge hardware on its ability to accelerate, ratherthan enable. Customers expect increasing integrationof new features such as speech recognition, ratherthan faster execution of existing features such as spellcheckers.

• They can’t describe cooperation across tasks. If sev-eral tasks operate toward a common purpose, theacceleration of some tasks is not necessarily benefi-cial and can even degrade overall performance.

Fundamentally, none of these approaches identifiesthe modifications that would improve a scenario-ori-ented design. Computer architecture has historicallybeen the art of identifying performance bottlenecks andthen identifying performance facilitators such as cachesand branch predictors to alleviate these bottlenecks.Different performance facilitators will exist for futurearchitectures, but these will facilitate critical cases—theinstances when application software overloads the sys-tem and performance rapidly deteriorates. Scenario-ori-ented benchmarks must enable designers to identifycritical cases and, in doing so, aid the discovery of newperformance facilitators.

EVALUATING SCENARIO-ORIENTEDPERFORMANCE

Figures 2 and 3 describe a hypothetical system’s per-formance in an onboard navigation scenario in terms ofusefulness and timeliness. Usefulness indicates the degreeto which the device helps the user navigate, and timeli-ness indicates the device’s ability to perform calculations

in a timely manner. Although subjective, human or evenmarketing studies can measure usefulness. Timeliness isa complex metric—some deadlines can be more impor-tant than others (this relative importance, of course, isalso subjective) and creating a composite can be diffi-cult. The importance of such metrics lies in bringing per-formance evaluation in line with user satisfaction.

The navigation system is assumed to involve speechrecognition, route optimization, Web search, and graph-ical display, executing on a SCHM with both reduced-instruction-set computer and SIMD cores. The speechand display software run better on the SIMD cores,while the route optimization and Web search run betteron the RISC cores.

Figure 2 compares the performance of several possi-ble implementations of a navigation scenario as the com-ponent algorithms’ complexities vary. Each group ofbars represents an implementation, and each bar—whose height indicates relative computational com-plexity—represents a task. Reasons for complexitychanges can include algorithm selection, the amount ofdata the system is processing, or control dependencies oninput values. We assume that, given infinite processingpower, higher complexity results in increased usefulness.

Figure 2 illustrates three important points:

• Software and hardware are not evaluated indepen-dently.

• Adding a new feature can significantly increase adevice’s usefulness even if the individual quality ofother features is sacrificed.

• The relative amount of computing power dedicatedto each feature has a significant effect on usefulnessand timeliness.

September 2007 27

Implementationdescription 1 No software 2 Basic function 3 Best without speech 4 RISC load too high 5 Inadequate speech 6 Inadequate display 7 Speech and display equal 8 Optimal balance 9 SIMD load too high 10 All load too high

Implementation1 2 3 4 5 6 7 8 9 10

Quali

ty

Speech recognitionRoute optimizationWeb searchDisplay

UsefulnessTimeliness

Figure 2.Timeliness and usefulness of various implementations of a navigation scenario.

28 Computer

In implementation 1 in Figure 2, no software is run-ning. As a result, no deadlines are missed, and the time-liness rating is perfect. However, the lack of functionalitybrings the usefulness score to zero.

Implementation 8 does not have perfect timeliness, butit incorporates enough functionality in a sufficiently timelymanner that the device is very useful. In implementation10, the load is too high, and all deadlines are missed.Timeliness bottoms out, and this degradation destroys use-fulness. On a more powerful processor, however, timeli-ness would improve, and the usefulness of these morecomplex implementations would increase with it. Softwaremust match hardware to optimize performance.

Comparing implementations 1-4, which do notinclude speech recognition, with implementations 5-8shows the impact of adding an additional feature.Usefulness hits a local maximum in implementation 3,and then begins to decrease because the requirementsfor route optimization and Web search are too high, andthey must be performed on SIMD processors where theyexecute less efficiently. Using these processors to imple-ment speech recognition, rather than improve existingfeatures, will make the device far more useful overall.

Implementations 5-8 also demonstrate the importanceof striking the correct balance. When the speech recog-nition is prioritized too much in implementation 6, thedisplay must be sacrificed to maintain timeliness. Inimplementations 7 and 8, the balance adjusts toimprove usefulness without reducing timeliness. Thisleads to an unequal division of computing resources.

Figure 3 illustrates a navigation system’s dynamicbehavior and the impact of adding an extra feature to

this system. The figure shows dif-ferent situations in roughlychronological order from left toright. The navigation system isidle until it receives a verbalrequest, which triggers a chain ofcomputation that continues untilthe system finds a route. Fromthen on, the navigation systemperiodically polls the GPS andtraffic advisory Web sites andannounces each turn. Two barsillustrate each situation. Thegreen bar shows the timeliness ofthe navigation system executingalone, while the yellow bar showsthe timeliness when the system isdownloading and displaying astreaming video at the same time.

This type of graph can revealunexpected consequences ofadding new functionality thatmight not be apparent from themacroscopic view of Figure 2. For

example, downloading map data is not a problem forthe base navigation system, so we might not thinkabout how adding a new feature affects performance.Figure 3, however, reveals that this is the worst casewhen simultaneously streaming video. Further analy-sis might reveal that limited bandwidth is to blame,which could lead to solutions such as compressing mapdata or designing around a more sophisticated wire-less protocol.

Figures 2 and 3 demonstrate the potential for new per-formance representations to isolate critical cases anddescribe tradeoffs. When performance evaluation breaksfree of traditional assumptions, many representationsbecome possible.

TAKING PERFORMANCE EVALUATION INTO A NEW ERA

Because it differs so vastly from both general-purposeand application-specific computing, scenario-orientedcomputing requires an overhaul of performance evalu-ation. We can divide this challenge into benchmark selec-tion and metric design.

Scenario-oriented benchmark criteriaBenchmark selection is the problem of defining a com-

putational load in terms of inputs (programs and data)and timing information. New scenario-oriented bench-marks should satisfy the following criteria:

• Include software and hardware interactions. Taskscan wait for each other, communicate, spawn chil-dren, and leave the system. As a result, the processor’s

Base navigation system (BNS)BNS while streaming video

BNS idle Processingrequest

Downloadingmaps

Parsingdata

Optimizingroute

CheckingGPS

Checkingadvisories

Instructingdriver

Situation

Tim

elin

ess

Figure 3. Navigation system timeliness.The green bar shows the timeliness of the navi-

gation system executing alone, while the yellow bar describes timeliness when the sys-

tem is downloading and displaying a streaming video at the same time.

load can change dramatically during usage. Figure 3illustrates this type of dynamic behavior as well as anexample in which multiple tasks compete for ashared hardware resource.

• Provide timing information for inputs and outputs.Because a mouse click can spark a chain of calculations,its occurrence relative to other events is important. Forexample, Figure 3 shows how processing directionswhile streaming a video can hurt performance, but run-ning the streaming video is fine when the base naviga-tion system is idle. Outputs such as video and musichave associated timing requirements that play an impor-tant role in determining perceived quality.

• Exercise critical cases. Scenarios should isolate theknees of performance curves. Because this will varyaccording to the underlying hardware, mechanismsmust exist to adjust requirements in a variety of ways.Implementation 9 in Figure 2 illustrates this type ofoperating point.

• Describe sets of usage modes. Although they aregeared toward certain uses, scenario-oriented proces-sors can provide additional value in other modes.Quantitative evaluation of tradeoffs between pri-mary and secondary modes is necessary. For exam-ple, is it better to have an excellent handheld TV orto sacrifice some TV functionality to allow using thesame device as a phone?

We propose a fundamental change in the structure of benchmarks for scenario-oriented computing.Accordingly, metrics for evaluation must also change.

Scenario-oriented metric criteriaMetric design is the problem of quantitatively assess-

ing the execution of a benchmark. Scenario-oriented per-formance metrics should satisfy the following criteria:

• Differentiate application elements by relative impacton usefulness. Users do not have equal performancerequirements for all tasks or even all portions of asingle task. In Figure 2, for example, speech recog-nition is slightly more important than displaybecause drivers try to avoid looking away from theroad, and speech recognition assists with thisrequirement.

• Account for nonlinearity. A hard real-time task willnot perform correctly if it can’t meet its deadlines.Once it meets all deadlines, however, there might beno benefit to further accelerating that task. Humaninteraction is similar—beneath some threshold,humans can’t perceive faster response times. Forexample, a graphics task doesn’t benefit from framerates higher than the monitor can support.

• Describe critical cases. Understanding the tradeoffsin a scenario-oriented computer requires knowingwhen the interaction of time, data, functionality, and

hardware causes overall performance to degrade. Forexample, there should be a way to describe exactlywhat happens in implementation 9 in Figure 2 thatcauses performance to rapidly drop or to explain theinteractions that occur in Figure 3 when download-ing a map while displaying a streaming video.

• Have a visual representation. Pictures and graphsare powerful tools for rapidly conveying complexinformation and tradeoffs. Developers often use barcharts in conjunction with SPEC to demonstrateimprovements in throughput and latency. Scenario-oriented designers will frequently need to select oneapproach from numerous alternatives when a single“best” choice is not apparent. Intuitive ways ofexpressing the results of future benchmarks are nec-essary to guide such decisions.

These properties for future benchmarks and metricsdepart sharply in structure and focus from those thathistorically have guided computer design. This overhaulis necessary for computer designers to develop and eval-uate the new principles required to deliver compellingdevices to end users.

D evelopers must reconsider performance evaluationin light of emerging hardware and software trends.A new era of scenario-oriented computing is

dawning. The means to evaluate new performance facil-itators and decisions shape, both directly and indirectly,approaches to design and their ultimate success—orfailure. The advancement of scenario-oriented design,therefore, hinges on the development of appropriateevaluation methods. We invite the community to joinus in considering this challenge. Contact us at soar@ece.wisc.edu or visit www.ece.wisc.edu/~soar for moreinformation. ■

Acknowledgments

This work was supported in part by the NationalScience Foundation under grants 0607934 and0606675. Any opinions, findings, and conclusions orrecommendations expressed in this material are thoseof the authors and do not necessarily reflect the views ofthe NSF.

References

1. J.L. Henning, “SPEC CPU2000: Measuring CPU Performancein the New Millennium,” Computer, July 2000, pp. 28-35;www.spec.org.

2. M. Levy, “Evaluating Digital Entertainment System Perfor-mance,” Computer, July 2005, pp. 68-72; www.eembc.org.

September 2007 29

30 Computer

3. J.M. Paul, D.E. Thomas, and A. Bobrek, “Scenario-OrientedDesign for Single-Chip Heterogeneous Multiprocessors,”IEEE Trans. VLSI, Aug. 2006, pp. 868-880.

4. J.M. Paul and B.H. Meyer, “Amdahl’s Law Revisited for Sin-gle Chip Systems,” Int’l J. Parallel Programming, Apr. 2007,pp. 101-123.

5. Sematech, International Technology Roadmap for Semicon-ductors (ITRS), 2005; www.itrs.net/Links/2005ITRS/Home2005.htm.

6. H.P. Hofstee, “Power-Efficient Processor Architecture and theCell Processor,” Proc. 11th Conf. High-Performance Com-puting Architectures, IEEE CS Press, 2005, pp. 258-262.

7. M.J. Schulte et al., “A Low-Power Multithreaded Processorfor Software Defined Radio,” J. VLSI Signal Processing Sys-tems, June 2006, pp. 143-159.

8. S.C. Woo et al., “The SPLASH-2 Programs: Characterizationand Methodological Considerations,” Proc. 22nd Ann. Int’lSymp. Computer Architecture, ACM Press, 1995, pp. 24-36.

9. N. Renqvist and M. Kallinen, “3DMark06,” white paperv1.0.2, Jan. 2006; www.futuremark.com/products/3dmark06.

10. J.M. Sammons and C.H. Sauer, “Measuring the Performanceof Personal Computers,” Proc. 37th IEEE Computer SocietyInt’l Computer Conf. (Compcon 92), IEEE CS Press, 1992,pp. 311-313.

Sean M. Pieper is a PhD student in the Department of Elec-trical and Computer Engineering at the University of Wis-consin-Madison. His research interests include scenario-oriented and power-efficient computer architectures. Pieperreceived an MS in electrical and computer engineering fromCarnegie Mellon University. Contact him at spieper@wisc.edu.

JoAnn M. Paul is an associate professor in the Departmentof Electrical and Computer Engineering at Virginia Tech.Her research interests include the design, modeling, simu-lation, and evaluation of single-chip heterogeneous multi-processors. Paul received a PhD in electrical engineeringfrom the University of Pittsburgh. She is a member of theIEEE and the IEEE Computer Society. Contact her atjmpaul@vt.edu.

Michael J. Schulte is an associate professor in the Depart-ment of Electrical and Computer Engineering at the Uni-versity of Wisconsin-Madison. His research interests includehigh-performance embedded processors, computer archi-tecture, domain-specific systems, and computer arithmetic.Schulte received a PhD in electrical engineering from theUniversity of Texas at Austin. He is a senior member of theIEEE and the IEEE Computer Society. Contact him atschulte@ece.wisc.edu.

■ Monthly updates highlight the latest additions to the digital library from all 23 peer-reviewed Computer Society periodicals.

■ New links access recent Computer Society conference publications.

■ Sponsors offer readers special deals on products and events.

Available for FREE to members, students, and computing professionals.

Visit http://www.computer.org/services/csdl_subscribe

For the IEEE Computer Society Digital Library E-Mail Newsletter

Sign

Up

Toda

y