Chapter 1 Geoscience Application Challenges to Computing Infrastructure

C. Yang and C. Xu, 2013. Chapter 1 Geoscience Application Challenges to Computing Infrastructure, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 1-15.

Chapter 1 Geoscience Application Challenges to Computing Infrastructure

Chaowei Yang and Chen Xu

1


Learning Objectives

Develop conceptual background understanding of

1. the 21st century geoscience challenges 2. the computing requirements posed by the

challenges3. the emergence of cloud computing4. the advantage and disadvantages of cloud

computing

2


Learning Modules

1. The 21st century geoscience challenges Energy, Emergency Response, Climate Change. Sustainable Development

2. The computing requirements posed by the challengesEnough computing power, Responding in real time, Saving energy, Saving budget, Improving accessibility

3. The emergence of cloud computingDistributed computing, on-demand services, computing sharing and cost savings, reliability, the emergency of cloud computing

4. The advantage and disadvantages of cloud computing

3


The 21st century geoscience challenges: Energy

• 80% energy supply is fulfilled by fossil fuels, which is not sustainable because it’s limited and its consumption will impact climate/environment

• Energy consumption is increasing at a rate of 2% per year and double the consumption as of today by 2040.

• The objective is to conserve energy and control global warming under 2°above pre-industry

• Method: Better monitoring and management of energy consumption by dynamically matching needs and support dynamic collaboration across regional and global energy needs with advanced computing and communication technologies.

4


The 21st century geoscience challenges: Emergency Response

• Disasters and emergencies, such as Tsunami and avian influenza, are increasing rapidly in both frequency and severity because of climate change, population explosion, and infrastructure increase.

• Responding to the emergencies needs fast integration of massive amount of data.

• Knowledge needs to be utilized to make sense of the big data in a rapid fashion.

• A virtually unlimited computing and communication infrastructure is demanded to support such response.

5


Emergency Response: Hurricane Katrina Example

6

For responding to Hurricane Katrina, FEMA produced a flooding water depth map, which is critical in helping both potential victims to decide either to or how to evacuate and first responders to deploy to the right location. But it took FEMA several weeks to produce the map and being able to produce such a map in minutes or house can help save thousands more lives. But the capability to integrate and fuse the Big Data involved to produce the information requires virtually unlimited computing resources on demand.

(Courtesy of Department of Homeland Security)


The 21st century geoscience challenges: Climate Change

• Climate is one of the biggest challenges and require us to advance our understanding of it, limit its magnitude and adapt to the change.

• Understanding climate change will require climate models to simulate hundreds of parameters.

• Limit the magnitude will require well conceived simulation of potential change.

• Adapting to climate change will require massive amount of multi-disciplinary and multi-scale simulations .

• All the requirements will demand large amount (virtually unlimited when needed) of computing resources.

7


The 21st century geoscience challenges: Sustainable Development

• Sustainability of critical and includes the complex interrelationships among natural environment, economic vitality, and healthy communities

• Freshwater is such an example and require good planning for supplying based on comprehensive modeling of water consumptions at various scales from local to global.

• Another example is urban planning that needs computing support to provide different scenarios.

• Sound scientific knowledge based decision is needed to mitigate increasing population stress to urban growth

• Any of the sustainability challenges will require Big Data processing to balance local to global and dynamically changing demands for resource and preservation.

8


Learning Modules





9


The computing requirements posed by the challenges: Enough computing power

• Computing Technology is advanced rapidly in the past decades in producing GPU, fast CPU and network , bigger storage and RAM, all of them are available for a single computer to improve the capacity. But not enough to address the geoscience challenges.

• HPC was used for simulation but can’t meet on demand requirements.

• Crowd sourcing and citizen computing were utilized for distributed Data

• Timely access to computing resources is not possible with these existing computing resources to guarantee a reasonable time frame for computing.

10


The computing requirements posed by the challenges: Responding in real time

• Emergency response systems will require real time integration big data and delivering of information.

• Other decision support scenarios, such as driving routing and computing infrastructure scheduling.

• Spiking access to computing resources is demanded through the Internet.

• A computing infrastructure provides real time and spiking responding capability is desired.

11


The computing requirements posed by the challenges: Saving energy

• Energy Consumption is reduced for running computers from desktop to HPC systems.

• Running a dedicated computing device will waste energy when not in use.

• It is desirable to pool and share the computing resources. • This share should be cross geographic regions because of the

demands will normally be spatiotemporally distributed across the globe.

12


The computing requirements posed by the challenges: Saving budget

• Our lab HPC is normally 10-20% utilized. But we have to pay the full life cycle operation including system admin and resources.

• Our spike computing demand may go beyond our own HPC capacity. Purchasing more will demand significant budget increase.

• Would be ideal to share the computing while having access to computing capacity as needed but only pay for the portion we used such facility.

13


The computing requirements posed by the challenges: Improving accessibility

• Smart devices and Internet grows fast and brings computing power to us almost anywhere anytime to sending texts and surfing the Internet.

• The network access also gave us the opportunity to access physically distributed computing resources located at data centers.

• It would be ideal to access the more advanced computing power using the smart devices from anywhere anytime.

14


Learning Modules





15


The emergence of cloud computing

• Distributed computing• On-demand services• Computing sharing and cost savings• Reliability• The emergency of cloud computing

16


The emergence of cloud computing: Distributed computing

• We have been sharing computing power back in the early stage of computers when suing mainframe computing.

• This sharing provides the idea for utility computing and cloud computing in a way the computing consumption is charged or consumed based on usage.

• Telecommunication and Internet expanded the geographic scope within which a computing capability can be shared.

• The use of cloud to represent the computer network linking computing resources and users sparkled the name cloud computing.

• Through the cloud represented network, computing service can be delivered as a type of utility through the Internet.

17


The emergence of cloud computing: On-demand services

• Service based architecture enables computing to be delivered on demand as a service

• Computing can also be delivered as utility • Consumer of computing services can request pay per use

service from computing providers• On demand service for computing capability became possible.

18


The emergence of cloud computing: Computing sharing and cost savings

• Utility computing helps computing consumers save budget from maintaining computing systems.

• No up front cost is needed and cost is charged to operation. • Dynamic computing demands can be matched with flexible

computing resources. • Computing providers utilized virtualization to share computing

resources to reduce system idle time and maximize return on investment

19


The emergence of cloud computing: Reliability

• Computing redundancy improves reliability from cloud computing providers.

• New hardware can be seamlessly added to the computing resource pool.

• The availability of multiple computing providers also reduce exclusive reliance on a single provider.

• Study shows web-based application can achieve better online availability with cloud computing.

20


The emergence of cloud computing

• Advancements in computing and communication technologies enables computing capabilities (computing, storage, network, application) to be delivered as services.

• Consumers are released from purchasing a underutilized computing facility.

• Amazon became the first provider of cloud computing in 2006 and matured in the past 7 years to provide the best and biggest cloud computing service to world wide cloud computing consumers.

• Microsoft started Azure in 2008 and matured in the past years to provide cloud computing for (mainly) windows users.

• Others (google, apple, facebook, etc.) joined the cloud computing race.

21


Learning Modules





22


The advantage of cloud computing

•Access to computing power beyond that of traditional supercomputers•Saving cost by using more effective and powerful management•Having better reliability •Accessing to multiple cloud computing resource through standardized API or GUI•Having more control over computing resources than that of traditional clusters• A new computing paradigm with virtually unlimited computing

power to satisfy dynamic geoscience computing demands by sharing and pooling computing resources.

23


The disadvantage of cloud computing

24

• Centralization of computing power to limited locations makes the network infrastructure a bottleneck. For example, the 2010 Amazon Reston data center network outage caused global impact to EC2 users.

• Sharing of computing resources caused the ultimate control loss of the computing recourses by a consumer, therefore, security becomes a concern.

• Transmitting big data and fully utilized computing devices will not benefit from commercial cloud, and will increase cost according to several studies to be introduced in this book.


Discussion Questions

1. Discuss what other geoscience challenges that pose computing challenges.

2. Enumerate several challenges that can not be meet by current computing infrastructure without cloud computing.

3. What types of costs are involved in computing procurement? Which part can be saved if using cloud computing?

4. Why Amazon became the first the most successful cloud computing provider?

5. What research are you doing and will it require computing support, and would cloud computing help?

25


References

A good resource for grid/citizen computing:Anderson, D. P., J. Cobb, E. Korpela, M. Lebofsky, and D. Werthimer. 2002. SETI@home: An experiment in public-resource computing. Communications of the ACM 45, no. 11: 56–61.

A good resource for cloud computing: Armbrust, M., A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee et al. 2010. A view of cloud computing. Communications of the ACM 53, no. 4:50–58.

A good resource for spatial cloud computing: Yang, C., M. Goodchild, Q. Huang, D. Nebert, R. Raskin, Y. Xu, M. Bambacus, and D. Fay. 2011. Spatial cloud computing: How can the geospatial sciences use and help shape cloud computing? International Journal of Digital Earth 4, no. 4: 305–329.

26

Education

Chapter 1 Geoscience Application Challenges to Computing Infrastructure