The Texas Advanced Computing Center: It’s Not Just For ...media2.planning.org/APA2012/Presentations/S553_Using Supercomputers to Implement...The Texas Advanced Computing Center:

The Texas Advanced Computing Center:

It’s Not Just For Science Any More

Jay Boisseau, Director

April 16, 2012

Motivation for Advanced Computing:

It Goes Back to the Beginning*

As the Universe expanded and cooled, atomic particles were created, the forces of nature ‘split,’ and galaxies and stars formed.

The resulting Universe, and everything in it, is governed by mathematical equations!!!

Understanding the world, and the universe, means being able to describe it and predict its behavior. – Our mathematical ‘language’ for describing and predicting the

behavior of physical systems is

• calculus and differential equations

• statistics and probability

*My PhD is in astronomy, so I really go back to the beginning!

The Universality of Math is Why

Computing is So Important in Science!

“Computers are

incredibly fast,

accurate, and stupid;

humans are incredibly

slow, inaccurate and

brilliant; together they

are powerful beyond

imagination.” (attributed to Albert Einstein)

What is Advanced Computing?

• Advanced computing means ‘advanced’ in

– nature of hardware

– nature of software

– scale of systems

– scale applications/jobs

– purpose of systems

• TACC develops, provides, supports, and

helps people use the advanced computing

technologies to transform science & society

What is TACC?

• TACC is an advanced computing, or

‘supercomputing,’ center at UT Austin

• Provides big and fast computing, storage,

and visualization

• Has deep expertise in advanced computing

• Enables open science research across the

US (and internationally through collaborations

among users)

TACC Mission

To enable discoveries

that advance science and society

through the application of

advanced computing technologies.

TACC Vision & Strategy

• Provide the most powerful, capable computing

technologies and techniques that enable people—

researchers, educators, developers, engineers,

businessmen, etc.—to advance science and society.

• Provide leadership in the advanced computing

community in technology R&D, support, education,

and expertise to ensure maximum impact of current

and future technologies in diverse applications.

• Enable transformational science and societal impacts

that change, influence, and improve our

understanding of the world, and the world itself.

Lonestar: Enabling World-Class Science

• HPC capabilities

– 1888 Dell 1955 blades, dual socket

– 6-core Intel Westmere (3.3GHz)

– 302 Tflops, 43 TB memory

• Shared memory capabilities

– 16 1TB shared memory nodes

• Remote visualization capabilities

– 16 Nvidia Fermis -> 48 (2012)

• 750 US research projects! HPC

Vis/

Data Analysis

Shared

memory

High

throughout

Global high speed storage

Stampede – Coming January 2013

• 10 petaflops (PF) peak performance

– 2 PF Linux cluster: Dell nodes w/Intel Xeon E5 procs

– 8 PF Intel MIC co-processors

– 250+ TB memory

– 14+ PB disk

– 56Gb FDR InfiniBand

• 16 1TB shared memory nodes

• 128 Nvidia Kepler 2 GPUs

TACC Is Leader in Visualization, Too!

Quantum Chemistry GeoSciences Natural Convection Gravity Map

Bioinformatics Orbital Debris Turbulent Flow CT Models

Longhorn: Most Powerful Interactive

Remote Visualization System in the World

Longhorn specs

– 256 Dell Quad-core Intel Nehalem Nodes

• 8 cores/nodes, 2048 total cores

– 128 NVIDIA Quadroplexes

• 4GPUs/node, 512 total GPUs)

– QDR InfiniBand Interconnect

– NSF award for $7M to enable

remote visualization across

the US

Stallion: Highest Resolution Display

Environment in the World

Massive Computing Requires

Massive Data Storage: Corral…

• 5 petabytes (replicated) of

DataDirect Networks online

disk storage

• Mulitple access mechanisms – MySQL & Postgres SQL databases

– Lustre parallel filesystem

– iRODS

– Web-based access

• Can easily expand to petabytes

• Designed for hosting data

collections!

... and the Ranch Data Archival System

• 40 petabytes tape capacity

in Sun StorageTek Silos

– 10,000 1TB tapes, 6000 5TB

– Used for long-term

storage

– Access provided to users

of other TACC resources

– Potential for up to 100PB

CyberShake: Creating Better Hazard Analysis Maps PI: Thomas Jordan, University of Southern California

Project uses supercomputing

to determine impact of

earthquakes on buildings

based on how they propagate

through crust material

Impact: create safer buildings,

protect life and minimize

damage

Getting Ahead of the Spill PI: Gordon Wells and Clint Dawson, UT Austin

Project used satellite data

and supercomputing to

predict evolving oil spill—

where the oil was going

Impact: helped target

remediation efforts for

cleanup

Science at the Center of the Storm PI: Frank Marks, NOAA

A simulation of Hurricane Ike on TACC's

Ranger supercomputer shortly before the storm

made landfall in Galveston, Texas, on Sept. 13,

2008.

• Project used

supercomputing to model

Hurricanes Ike and

Gustav in 2008,

forecasting track and

intensity.

• Impact: improved

accuracy of NOAA

hurricane predictions

H1N1 Flu Outbreak Simulation Ned Dimitrov, Lauren Meyers, UT Comp. Bio

• Project modeled potential

spread of epidemic based

on locations and

transportation

• Impact: produced insights

toward helping understand

how to minimize chance of

pandemic by placement of

antiviral drugs

Computational Science is Not Just

Modeling and Simulation

• Modeling & simulation

– Simulation of mathematical models

– Must store, visualize/analyze simulation output

• Data-enabled science

– Facilitated by digital data collection

– Often, no formal mathematical laws -> statistics

• Mine/analyze data (then work toward models)

– “Physics vs. stamp collecting” (Rutherford): not true

– “End of science (simulation)” (Wired): even less true

A Window on the Archive of the Future PI: Maria Esteva, Weijia Xiu, Texas Advanced Computing Center

• Project with the

National Archives and

Records Administration

(NARA) is developing

experimental workflows

and visualization tools

to represent, analyze,

and interact with

massive datasets. A tree map , information visualization representing the

entire Federal Records Collection.

Impact: these tools will help future archivists organize government

documents, while furthering public access to open records.

“Big Data”

• Computing power and storage growing

rapidly, but…

• Digital data measurement & collection is

growing even more rapidly

• Understanding the world requires analyzing

data, not just solving questions

• “Big data” is the new big buzzword in

computing, in business and government as

well as science

Big Data is the Big Deal

• Obama administration revealed $200M “Big

Data” initiative on March 29, 2012

– NSF

– NIH

– DOE

– DOE

– DARPA

– USGS

http://www.whitehouse.gov/blog/2012/03/29/big-data-big-deal

TACC Is Going Big Into Big Data

Just upgraded Corral, Ranch

Already supporting many data-driven projects

Hiring new data mining, data mgmt experts

Stampede project will seek out new data –

driven projects

Just received $10M commitment for

designing, deploying new data services

TACC & Sustainable Places Project

• TACC is new member of Capitol Area Texas

Sustainability (CATS) “Sustainable Places Analytics

Tool” project

• Modeling cities is a big data problem

– Mining vast amounts of data (big fast storage)

• People, buildings, autos, companies, power, water…

– Executing huge statistical models (big fast compute)

• Traffic, development, environmental quality….

– And interpreting results at all scales (big viz)

• Seeing citywide data with resolution of homes, autos

• Future : massive data mining, ensemble models,

scenario/what-if decision simulations…

Summary

Advanced computing technologies go beyond

mainstream IT—allow us to ask questions at

the edge of human knowledge, understanding

We are witnessing a data explosion that

offers potential for examining world in ways

we can’t yet model from first principles

TACC aims to lead in enabling data-driven

research and decision-making, including

urban planning, power grid modeling, etc.

Capital

Area of

Texas

Sustainability

CONSORTIUM

CATS Sustainable Places Project

$3.7 Million HUD Sustainable Communities Planning Grant

Implementation of CAMPO 2035 Transportation Plan focusing on 37 ‘Activity Centers’

SPP goals map to HUD Livability Principles

Three Strategies:

• Planning Demonstration Sites

• Analytic Tool

• Public Engagement

Austin (Economic) Factors

Focus on development around proposed urban rail plan (Development Oriented Transit)

1. Taxes – The city property tax and sales tax revenues to be

achieved.

2. Jobs – The number of jobs created by investment near urban rail

stops. 3. Savings – Accruing to development or the community.

4. Business Productivity – The amount of productivity gains businesses

can expect to achieve by reducing commute times for employees.

Austin Demonstration Site

Seaholm District

Mueller

Seaholm District

• +/- $1B in pipeline

• $85 M taxes NPV @ 30 years

• 2500 jobs

• 3544 residential units

Mueller

• 700 acre reuse of municipal airport

• $250 M in pipeline /$1 B Project

• $43 - $48 M taxes NPV @ 20 years

• 4,213 - 7,009 jobs

• 7,868 - 9,410 residential units

• Activity Centers:

Planning Demonstration Sites

Activity

Centers:

Central Austin

& Mueller

Hutto

Elgin

Lockhart

Dripping

Springs

Sustainable Places Analytic Tool

• UT-Austin Center for Sustainable Development

• Envision Tomorrow + (Core Software)

• Collaboration with Fregonese and University Utah for the

Scenario Tool

• Joint with Local Consultants at Demo Sites

• HDR Associates (Non-market Valuation)

• CTR – Micro-simulation with CAMPO model

• Criterion Planners – INDEX Data Wizard

• TACC – Hosting, visualization, web interface

Sustainable Places Analytical

Resources Collection (SPARC)

ESRI Application Server Open Source Application Server

PosgreSQL / PostGIS Database Server

Transactional PL/PGSQL, Python

Desktop ArcGIS Non ET + Tools • Oden Fiscal • HDR SROI • UT NMC DTA • Dr Ming Zhang

INDEX Data Quality Wizard • External shapefile input • Clean planning data stored

ArcMap Scenario Designer • Spatial editing • Topology, network edits ET + Tool • Indicators • Reporting/visualization

ArcSDE

ArcGIS Server Mapnik, Geoserver, MapServer

Web Browser (using AJAX)

Web Portal • Publish scenario results • Scenario polling (Metroquest-like dashboard) • User communication, email, txt msg, social media

Scenario Designer (non-spatial) • Project & scenario mgmt • Palette management painting polygons • Edit attribute data Scenario Analysis (long-term functionality) • Policy alerts/tutorials • 3D visualization (read-only?) • Calculate indicators, rank scenarios by goals • Scenario promotion/approval system

3D scenario

design

• Procedural

• CityEngine?

Design Analytics in Practice (setting the stage for advanced computing)

Scott Polikov 4.16.12

Central Texas Activity Centers Initiative

• Building on Envision Cent. Texas Plan

• HUD-USDOT-EPA Livability Principles

• Refined into “Austin Factors”

• Focus on Return on Investment

Linkages to Livability

HUD Sustainable Communities

Planning

Housing Credit Programs

TIGER Transit Investments

Leverage HUD Funding

Programs

Transit-Based Neighborhoods

Regional Sustainability

Coordinated Investment

Select Austin Activity Centers Analyzed

• Fiskville TOD

• Rainey Street, Downtown

• South Shore, Downtown

Fiskville TOD, Airport Boulevard

Airport Boulevard Corridor



This TOD will be an additional station on the existing Metrorail Red Line


Use of rail on site:

• Take advantage of existing operation in corridor by adding a stop

• Provide stop across from new Travis County employment center

Redevelopment Potential:

• Leverage new county site across the street and form-based redevelopment of Airport Boulevard within demographic center of Austin

Fiskville Annual

New Activity Square Footage Taxable Value Taxable Sales/Lodging Jobs City Sales/Lodging Tax Rev City Property Tax Rev

Residential 554,842 $73,270,589 $352,505

Commercial 88,183 $11,249,003 $27,192,625 225 $271,926 $54,119

Parking 648,572 $31,965,879 $153,788

Totals 1,291,597 $116,485,471 $27,192,625 225 $271,926 $560,412

Replaced Activity 87,162 $2,779,749 134 $13,373

Net New Activity Square Footage Taxable Value Taxable Sales/Lodging Jobs City Sales/Lodging Tax Rev City Property Tax Rev

Total 1,204,435 $113,705,722 $27,192,625 91 $271,926 $547,038

3 New Buildings (conservative)

$113 Million in new taxable value

91 New jobs created

$750,000 in new tax revenue (annual)

North Shore/Rainey Street, Downtown Austin

North Shore/Rainey Street

• Rainey Street existed as a vibrant single family neighborhood

• 2004 Rainey Street rezoned CBD, after a period of decline

• Proximity to Convention Center

(Source: Statesman) (Source: Downtown Austin Blog)


• The site contains exican American Cultural Center, flanked by Lady Bird Lake

• Since the Rainey street area has become a designation, parking has become challenging

• The rail and future structured parking could be coordinated to leverage redevelopment

(Source: Austin Business Journal)

(Source: Austin Chronicle)

Future urban rail station

Future urban rail line to South shore

Lady Bird Lake

Mexican American Cultural Center

Convention Center


North Shore/Rainey Street Future Urban Rail


Lady Bird Lake

Office and Residential Towers

Mexican American Cultural Center

Convention Center

Future Manchester Hotel

Future Hotel/ Residential

High-rise office and Residential Towers


North Shore/Rainey Street Potential Redevelopment (illustrative)

Bars in adapted SF buildings



• The rail would serve as a connector to south of the river and north in to downtown, the Capitol Complex and the University of Texas.

• Access to rail stop from Rainey Street improves connections to the existing regional hike and bike trail

Redevelopment Potential:

• CBD zoning, the proximity to downtown and IH-35 frontage justify high rises


2,913 New jobs created

$5.2 Million in new tax revenue (annual)

North Shore Annual


Residential 2,439,039 322,099,490 0 0 0 $1,549,621

Commercial 1,554,739 227,110,494 57,492,186 3,072 $2,157,681 $1,092,629

Parking 2,047,718 117,687,915 0 0 0 $566,197

Totals 6,041,496 $666,897,899 $57,492,186 3,072 $2,157,681 $3,208,446

Replaced Activity 77,024 $4,430,612 $3,264,625 104 $32,646 $21,316

Replaced SF with no tax base 150,702 N/A $0 54 $0 $0


Total 5,813,770 $662,467,287 $54,227,561 2,913 $2,125,035 $3,187,130

South Shore, Downtown Austin


• Several viable existing office uses

• Much of this site either exists under continuous single ownership or under a PUD that has been in place since 1989.

(Source: Statesman)

• The site is access point Lady Bird Lake Hike and Bike Trail and viewing area for the Congress Ave Bridge Bat Colony

• Street frontage is vacant retail or parking lot with land at $200/SF

(Source: Statesman)


Greater South River City Combined Neighborhood Plan, Adopted in 2005

• “New and existing businesses can support enhanced public transportation along commercial roadways.”

(Source: forum.skyscraperpage.com)

(Source: homeaway.com)

Update to the Waterfront Overlay, Adopted in 2009

• “provide a more harmonious interaction and transition between urban development and the parkland and shoreline of Lady Bird Lake”’

• “Encouraging appropriate mixed-use and residential development along the waterfront and urban edge”



• The proposed transit bridge would provide crossing for bikes, pedestrian and rail, breaking through congestion currently constraining downtown

• Rail station would connect current and future office and retail uses

Redevelopment potential:

• Long term redevelopment potential of the site could provide neighborhood amenities as well as high end central city office and living in context of $200/SF land costs

Opportunities and constraints:

• Site contains super blocks that limit access to lake

• Height restrictions in relation to the price of land make new development unfeasible

• The build out of the existing PUD on COX site will not achieve objectives stated in the Waterfront Overlay and Neighborhood Plan (approved in 2005), failing to give priority to waterfront or pedestrian environment

• AIA SDAT will be undertaken summer

Existing Development:

• Approximately 250,000 SF of Office • Mostly single tenant – TxDOT and

Austin American-Statesman


Future urban rail line to downtown

Lady Bird Lake

Crockett Property

Austin Statesman – Cox Property


Future urban rail line to downtown

Lady Bird Lake

Office and Residential Towers

Office tower with garage parking

Multifamily with ground floor restaurant

Residential tower with ground floor retail

Open space with access to lake and trail

Street-fronting mixed-use

Existing 5 story office

Potential Development (illustrative):

• Multifamily - 984 Units • Office – 1,567,924 SF • Retail – 158,517 SF • Restaurant – 80,849 SF


3,525 New jobs created

$3 Million in new tax revenue (annual)

South Shore Annual


Residential 1,180,578 156,084,217 0 0 $0 $750,921

Commercial 1,807,289 251,775,409 $69,868,100 4,963 $698,681 $1,211,291

Parking 1,522,340 95,800,430 0 0 $0 $460,896

Totals 4,510,207 $503,660,057 $69,868,100 4,963 $698,681 $2,423,109

Replaced Activity 330,640 $6,734,080 $3,695,450 659 36,955 $32,398

Replaced SF with no tax base 272,901 N/A 0 780 $0 $0


Total 3,906,666 $496,925,977 $66,172,650 3,525 $661,727 $2,390,711

• Value is created when sites are leveraged through multimodal access in context of good planning and zoning

• Sites can realize increased intensity and still coexist with existing neighborhoods

• City tax base benefits when potential is unleashed through public-private investments as demonstrated by analytics

Conclusions (spreadsheet)

Regional Infrastructure Investments

Private Investment

Local Govt. Decision Making

Leverage Local Funds

Transit-Based Neighborhoods

Regional Sustainability

Coordinated Investment

Conclusions (advanced computing)

www.gatewayplanning.com

Documents

The Texas Advanced Computing Center: It’s Not Just For ...media2.planning.org/APA2012/Presentations/S553_Using Supercomputers to Implement...The Texas Advanced Computing Center: