File RepositoryCloud Computing Topic

Lina Qu439-Distributed System by Prof. Thiruvathukal

2011.05.01

Outline

1. Introduction2. Cloud Computing3. Google App Engine™ platform4. File Repository5. Summary6. References

1. Introduction

Using the cloud computing platform, Google App Engine™ platform in this case, to develop and deploy a simple application ---- File repository.

Through this project, we can learn briefly:• Cloud Computing and its service models;• Google App Engine(GAE) as develop platform;• Big Table as a distributed storage system;• An example project actually runs on GAE.

2. Cloud Computing2.1 What is Cloud Computing?

A large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted, virtualized, dynamically-scalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the Internet[1].

Cloud computing refers to the provision of computational resources on demand via a computer network, such as applications, databases, file services, email, etc. (----Wikipedia)

2. Cloud Computing2.2 Key characters of Cloud Computing

• On-demand self-service• Broad network access• Resource pooling• Rapid elasticity• Measured Service

2. Cloud Computing2.3 Service models of Cloud Computing

1. Software as a Service (SaaS)

2. Platform as a Service (PaaS)

3. Infrastructure as a Service (IaaS)

Google Pack, Salesforce.com, NetSuite.

Google App Engine, Windows® Azure, Force.com.

Amazon EC2 (Elastic Cloud Computing) Service, S3 (Simple Storage Service), IBM® Cloud.

2. Cloud Computing2.3 Service models of Cloud Computing

3. Google App Engine™ platform3.1 What is Google App Engine?

It lets you run your web applications on Google’s infrastructure.

3. Google App Engine™ platform3.2 Why Google App Engine?

• Easy to build- Easy to get Started[17]

- Simple app configuration• Easy to maintain- Cost efficient hosting- Risk free trial period- The reliability, performance and security of Google's

infrastructure• Easy to scale- Automatic scalability- APIs

3. Google App Engine™ platform3.3 Google App Engine Architecture

urlfech

mail

imagesapp

stdlibPythonVM

process

memcache datastore

req/respstateless APIs R/O FS

statefulAPIs

Source: [15]

3. Google App Engine™ platform3.3 Google App Engine Architecture

The App Engine request handling architecture[13].

3. Google App Engine™ platform3.4 Key Concepts in Google App Engine

• Sandbox: -Security; It isolates your application in its own secure, reliable environment ; the hardware, operating system and physical location of the web server.

3. Google App Engine™ platform3.4 Key Concepts in Google App Engine

• Bigtable: A Distributed Storage System for Structured Data[8].

-It uses the distributed Google File System(GFS[25]) to store log and data files.-It uses Google SSTable file format to store data. (persistent, ordered immutable map from keys to values)-It relies on a highly-available and persistent distributed lock service called Chubby.-It is designed to scale to a very large size.-It does not support a full relational data model.

3. Google App Engine™ platform3.4 Key Concepts in Google App Engine

• Datastore: A distributed data storage service grows with your data.

-Abstraction on top of Bigtable; It provides a scalable, transactional storage system for creating, storing, and querying data objects.-Two data storage options[18, 19]: The Master/Slave Datastore and the High Replication Datastore.- APIs For Java: Java Data Objects(JDO) 2.3 and Java Persistence API (JPA) 1.0

3. Google App Engine™ platform3.4 Key Concepts in Google App Engine

• Hierarchical Datastore

OBJECT ORIENTED RELATIONAL DATABASE DATASTORE

Class Table Kind

Object Record Entity

Attribute Column Property

3. Google App Engine™ platform3.4 Key Concepts in Google App Engine

• Indexes- index.yaml; the development server automatically

determines the index configuration for you[22].- Properties are automatically indexed by type+value[15,

21]. • Queries

-JDOQL: is similar to SQL, but is more appropriate for object-oriented databases like the App Engine datastore.-GQL[24]: is a SQL-like language for retrieving entities or keys from the App Engine scalable datastore.

3. Google App Engine™ platform3.5 Develop Web app with Google App Engine

GAE Application Development Cycle[10]

3. Google App Engine™ platform3.6 Managing Your App

•

3. Google App Engine™ platform3.6 Managing Your App

• Quotas-5M pageviews/month-Channel API Calls: 657K calls-Stored Data: 1GB-Datastore API Calls: 141M calss-Number of Indexes: 200-Requests(includes HTTPS): 43.2M requests-CPU Time: 6.5 CPU-hours/day-Task Queue API Calls: 100K-UrlFetch API Calls: 657K calls-XMPP API Calls: 46M calls

3. Google App Engine™ platform3.7 Support

• Java[7]

-Java 6-Common Java web technology standards-Plugin for the Eclipse IDE-Other languages: JRuby, JavaScript(Thino), Scala.-Spring, Struts 1&2, Apache Commons FileUpload, etc.

• Python[27]

-Complete Python standard library-Django-CGI or WSGI

4. File Repository4.1 Function

• Sign In with Google Accounts

• File Upload

• File Download

• File List

• Log Out

4. File Repository4.2 Code

• Configure<servlet><servlet-name>upload</servlet-name><servlet-class>loyola.lqu.server.servlet.FileUploadServlet</servlet-class></servlet><servlet-mapping><servlet-name>upload</servlet-name><url-pattern>/upload</url-pattern></servlet-mapping>

4. File Repository4.2 Code

• Sign In<td><%UserService userService = UserServiceFactory.getUserService();if (!userService.isUserLoggedIn()) {%>Please <ahref="<%=userService.createLoginURL("/jsp/home.jsp")%>">signin</a> to upload your file and manage the repository.<% } else { response.sendRedirect("/jsp/home.jsp"); } %></td>

4. File Repository4.2 Code

• Deploy<application>lqu-filerepo</application><version>1</version>

4. File Repository4.3 Technology

• Apache Commons FileUpload[28]

• Apache HttpClient

• JSP(CSS, JavaScript)

• The Blobstore Java API[29]

• The Users Java API[30]

5. Summary

• Google App Engine• Cloud Computing

6. Reference1. Cloud Computing and Grid Computing 360-Degree Compared, Ian Foster, Yong

Zhao, Ioan Raicu, Shiyong Lu.

2. Cloud computing service models, Part 2: Platform as a Service : Learn the benefits of PaaS.

3. Cloud and industry, Part 1: PaaS best practices and patterns : Learn about the requirements and functions of three models to deliver industry solutions, IaaS, PaaS, SaaS, and how you can use best practices and patterns with the PaaS framework in particular to deploy and manage cloud computing solutions.

4. Connecting to the cloud, Part 1: Leverage the cloud in applications: Explore cloud computing and the offerings from the major cloud platform vendors: Amazon, Google, Microsoft®, and SalesForce.com.

5. Google App Engine Home Page : Learn more about Google App Engine™ platform.

6. Install Google Plugin for Eclipse: Get started with the plug-ins.

6. Reference7. Will it play in App Engine for Java?: Lists the level of compatibility of various

Java technologies and App Engine (Java).

8. What's BigTable?: Read the Google research publication to find out.

9. Cloud computing fundamentals: Aid you in assimilating the reality of the revolution, so you can use it for your own profit and well being.

10. Stanford University Wiki – Google App Engine Project.

11. Slatkin, Brett. Building Scalable Web Applications with Google App Engine. Google IO 2008.

12. Ashcraft, Ken. Best Practices - Building a Production Quality Application on Google App Engine. Google IO 2008.

13. Dan Sanderson, Programming Google App Engine. O’Reilly, GoogleTM Press.

14. Getting Started: Java. Helpful sample from Google App Engine site.

6. Reference15. Presentation by Guido van Rossum. Google App Engine: Run your web

applications on Google's infrastructure. Stanford EE Computer Systems Colloquium. 5 Nov 2008.

16. Why App Engine. Brief explain why developers choose Google App Engine.

17. Google App Engine Samples. A list of samples and its source code.

18. Choosing a Datastore(Java). Comparing the Data Storage Options, choose the right one.

19. Datastore Overview(Python).

20. Barrett, Ryan. Under the Covers of the Google App Engine Datastore. Google IO 2008.

21. Introduction to Indexes.

22. Java Index Configuration Using YAML.

6. Reference23. Queries in JDO.

24. GQL Reference.

25. The Google File System. Get a in-depth understanding of GFS.

26. Google App Engine Articles. Articles related to Google App Engine in many aspects by category.

27. Google App Engine Python SDK Release Notes.

28. Apache Commons FileUpload.

29. The Blobstore Java API.

30. The Users Java API.

31. Google File Service on Google App Engine for Java. Main Reference for code.

EndCloud Computing Topic

Lina Qu439-Distributed System by Prof. Thiruvathukal

2011.05.01