Java and Groovy onGoogle App Engine

--->

Ken Kousenken.kousen@kousenit.com

Google App Engine

Google's "cloud computing" solution

    Run applications on Google's scalable infrastructure

    Pay based on resources used        storage, bandwidth            measured by gigabyte

    For free,         up to 500 MB of storage and        up to 5 million page views/month

Running Java on GAE

    Original GAE version:        Python interpreter and standard libraries

    New, "Early Look" version:        JVM available            So Java works, as well as other languages                that compile to the JVM                i.e., Groovy, Scala, JRuby, ...

    

Java "Early Look"

Need to register at http://appengine.google.com     Originally expected 10K developers    Demand was so high, increased to 25K    Now open to ALL

Download SDK

Download Eclipse plugin (optional)

Can run locally or to myapp.appspot.com    If Google Apps user, can map local http address        (example later)

Sandbox Environment

Limited access to underlying OS

    Manage app through application console

    No writing to file system        Can read files uploaded by app

    Datastore available (discussed below)

    Services available (described next)

Welcome to 1999Let's go back to those thrilling days of yesteryear...       Ricky Martin,        Livin' La Vida Loca

    Jar-Jar Binks                      Bill and Monica

  

Also in 1999

On Dec. 17th, Sun released Servlet 2.2 specification    Established structure of a WAR file

    http://www.youtube.com/watch?v=EVESMxs4rbA

    Ever since then, you could deploy a WAR file        to an application server

    Unfortunately, GAE doesn't know from WAR files

WAR, huh, ... what is it good for?

Actually, that's not quite true    GAE does know about war structure,        just not war files

    gae_app\        src\            java code        war\            normal war stuff, including classes and lib dirs            WEB-INF\                web.xml, etc

GAE LimitationsGAE running on Java 6, but with some limitations

Once a request is sent to the client, no further processing can be doneRequest will be terminated if longer than 30 sec to complete, throwing an exception

Also,No socketsNo threads or timersNo JNISystem.gc(), System.exit(...), etc, do nothingSecurity issues, esp. with class loaders

GAE War Quirks

Sample web.xml file    Has DTD based on Web App 2.3

    But also includes xmlns and version attributes        (version = 2.5, no less)

It's confused

Scalability

About 30 active dynamic requests simultaneously

Average server-side req processing time is 75 ms

       --> about 400 requests/sec without additional latency    (note static files not affected by this limit)

GAE ServicesURL Fetch    URL wrapper that uses Google infrastructure to        retrieve resourcesMail    Send emailMemcache    "high performance in-memory key-value cache"Image Manipulation    Resize, copy, rotate, flip JPEG and PNG imagesUser    Detect if user is logged in and is administrator

Welcome to the 2300s

    Maybe by then we'll have moved beyond relational DBs        (but I doubt it...)

Datastore

Distributed data storage    transactional

Filtering and sorting by property value type

NOT a traditional relational database    Google uses the term "schemaless"

Uses optimistic concurrency control with retries

GAE and Persistence

GAE doesn't know from relational

    All persistence defined by @annotations on classes

        JDO           O'Reilly book (c) 2003 

        JPA            But JDO is the default                 (wait, what? JDO? what's up with that?)

GAE Persistence

GAE object datastore based on BigTable    BigTable is a massively scalable,        distributed storage system        used by Google for lots of things

    For Java, API uses DataNucleus bytecode enhancer

    (No, I'd never heard of it either...)

Software Development Kit

App Engine SDK    Includes web server (jetty)    Emulates all the GAE services

SDK includes an upload tool to deploy app to GAE

Command line tools included

Project Development

Supports Java 5 and Java 6 (Standard Edition)

    GAE deployment is Java 6

Google Plugin for Eclipse    version for 3.3 (Europa)        http://dl.google.com/eclipse/plugin/3.3

    version for 3.4 (Ganymede)        http://dl.google.com/eclipse/plugin/3.4

Project Development

SDK comes as a zip file    Includes sample applications

    Some use GWT:  Google Web Toolkit        How to do Ajax without doing Ajax            by doing it in Java

    GWT also comes with the Eclipse plugin

Demo: Recommended Books

Warning: User Interface Poisoning Hazard Ahead

Entity Class

@PersistenceCapable(identityType = IdentityType.APPLICATION)public class Book {@PrimaryKey        @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)private Long id;@Persistentprivate String asin;@Persistentprivate String recommendation;

Entity Class

// Other attributes populated by XML response (not persistent!)private String title;private String author;  // multiple are separated by commasprivate String formattedPrice;private String mediumImageURL;private String detailPageURL;

    // contructors, getter and setter methods    // equals, hashCode, toString overrides as desired}

Entity Class

Book represents book at Amazon.com

Amazon provides Product Advertising API    formerly Amazon Associates Service    http://docs.amazonwebservices.com/AWSECommerceService

RESTful (sort of) web service    Append parameters to request        URL = base?Service=AWSECommerceService            &Operation=ItemLookup            &ASIN=...isbn...   // ... etc ...

Amazon Web Service

Input URL --> Output XML

Amazon Web Service

As of May, 2009    Renamed Product Advertising API (ugh)

As of August, 2009    All REST requests must be digitally signed    Sample code given in Java        Uses javax.crypto classes and Apache Commons Codec

    

DAO Interface

public interface BookDAO {Book findById(Long id);Book findByAsin(String asin);Set<Book> findAllBooks();Long addBook(Book b);boolean removeBook(Long id);}

DAO Implementation

public class JdoBookDAO implements BookDAO {  private PersistenceManagerFactory pmf = PMF.get();

  @Override  public Long addBook(Book b) {Book book = findByAsin(b.getAsin());if (book != null) {    return book.getId();}PersistenceManager pm = pmf.getPersistenceManager();

DAO Implementation

try {    pm.currentTransaction().begin();    pm.makePersistent(b);    pm.currentTransaction().commit();} finally {    if (pm.currentTransaction().isActive()) {pm.currentTransaction().commit();    }            pm.close();}return b.getId();    }

DAO Implementationpublic Set<Book> findAllBooks() {Set<Book> results = new HashSet<Book>();PersistenceManager pm = pmf.getPersistenceManager();Query q = pm.newQuery(Book.class);q.setOrdering("asin desc");try {    List<Book> books = (List<Book>) q.execute();    for (Book b : books) {        results.add(b);    }} finally {    pm.close();}return results;    }

Use Cases

Servlet implementations of use cases    List all books    Add a new book    Remove a book

Each is mapped in web.xml file

Each goes through a service class to fill in book details

Each forwards to JSP

List All Books

    public void doGet(    HttpServletRequest req, HttpServletResponse resp) throws IOException, ServletException {AmazonBookService service = new AmazonBookService();Set<Book> books = (Set<Book>) service.getBooks();req.setAttribute("books", books);req.getRequestDispatcher("books.jsp").forward(req, resp);    }

What is that AmazonBookService?    It converts XML to Book instances

    Enter Groovy...

Amazon Book Service

Class implemented in Groovy:

class AmazonBookService {def baseUrl = 'http://ecs.amazonaws.com/onca/xml'def params = ['Service':'AWSECommerceService',             'Operation':'ItemLookup',             'AWSAccessKeyId':'... long ugly key ...',             'AssociateTag':'kouitinc-20',             'ResponseGroup':'Medium']BookDAO dao = DAOFactory.instance.bookDAO

Amazon Book Service

def getBooks() {def books = dao.findAllBooks()books.each { book ->book = fillInBookDetails(book)}return books }

Amazon Book Service

def fillInBookDetails(Book book) {def queryString = params.collect { k,v -> "$k=$v" }.join('&')def url = "${baseUrl}?${queryString}&ItemId=${book.asin}"def response = new XmlSlurper().parse(url)def item = response.Items.Itembook.title = item.ItemAttributes.Titlebook.author =             item.ItemAttributes.Author.collect { it }.join(', ')book.formattedPrice =             item.ItemAttributes.ListPrice.FormattedPricebook.mediumImageURL = item.MediumImage.URLbook.detailPageURL = item.DetailPageURLreturn book    }

One annoying thing

For Groovy classes,    GAE needs all compiled classes in        war/WEB-INF/classes/ ...        (normal for WAR files)

    But couldn't get that and Groovy Eclipse plugin        to keep them in classpath at same time

    Had to manually copy compiled classes to proper dir

    Grrr... but at least it worked  (probably Eclipse issue)

JSP

books.jsp displays all the books in the request

<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %><%@ page isELIgnored="false" %>...      <c:forEach items="${books}" var="book">       <tr>            <td><a href="${book.detailPageURL}">                 <img alt="${book.title} picture"                        src="${book.mediumImageURL}">                 </a></td>        

JSP

    <td>        <ul>            <li>ISBN: ${book.asin}</li>            <li style="font-weight: bold;">            <a href="${book.detailPageURL}">${book.title}</a></li>            <li style="font-style: italic;">${book.author}</li>            <li>${book.formattedPrice}</li>        </ul>    </td>    <td>${book.recommendation}</td>

Resulting Display

Admin CapabilitiesUse GAE UserService class

UserService us = UserServiceFactory.getUserService();     if (us.isUserLoggedIn()) {      <p><a href="<%=         us.createLogoutURL("/listbooks") %>">Logout</a></p>    } else {    <p><a href="<%=         us.createLoginURL("/listbooks") %>">Login</a></p>    }

Yeah, ugly scriptlet code, but so be it

Admin Capabilities

If logged in user and user is registered admin,    Can add new books    Can delete existing books

Otherwise links aren't available

Probably best to make a custom JSP tag out of this    Or use a real security solution        JSecurity, Spring Security, etc.

Easy enough for a demo, though

Admin Capabilities

Can set billing limit, too    Mine is $2/day,        so don't bother writing those scripts ... :)

Other Services

URL fetch service        Just opening a URL means we're using it already        Has more capabilities for alternate request types

Mail service    Can only send, not receive    Limits on size, frequency, but seem reasonable

Memcache    API implements JCache interface

Memcache

Caching capability based on javax.cache package    JCache: JSR 107 (under development)

    Cache cache = CacheManager.instance.        cacheFactory.createCache(props)    ...    if (cache.get(book.asin)) {        book = cache.get(book.asin)    } else {book = fillInBookDetails(book)cache.put(book.asin,book)    }

Scheduling

CRON jobs supported in cron.xml

<?xml version="1.0" encoding="UTF-8"?><cronentries>    <cron>        <url>/listbooks</url>        <description>Repopulate the cache every day at            5am</description>        <schedule>every day 05:00</schedule>    </cron></cronentries>

Admin Console

Administrative console located at    http://appengine.google.com   for non-Google Apps or    http://appengine.google.com/a/domain.comif the domain is using Google Apps

Current limit of 10 apps per user

Can check activity, quotas, etc.

GAE vs. Amazon EC2

Amazon EC2 installs images    You need to configure server, JVM, etc.

GAE is a sandbox with provided services    Detailed admin console available    Java persistence framework built in

The Quest for the Holy Grails

Grails framework    Spring + Hibernate + Groovy

Modular -- built on plugins

    Now there's a GAE plugin        Uninstall hibernate, can then use JDO        Some quirks, but it's early yet

    Will eventually be the easiest way to build a GAE app

Grails GAE Plugin1. Create Grails app with matching name    (Workaround available if not possible)

2. Uninstall Hibernate plugin

3. Install app-engine plugin (set APPENGINE_HOME)

4. Set version to 1 (or some other int)    GAE doesn't like decimal version numbers

5. Deploy using package command

Opinionated Conclusions: Bad Stuff

Must be able to deploy a WAR file    Hard to take GAE seriously otherwise    Expect this to be easy in release version

Must support the full Java EE standard    Sun (Oracle?) is very upset about this...    Also hard to port existing apps otherwise

Should support other persistence options    Especially Hibernate    Spring would be nice, too (might be asking a lot)    But non-relational issues will persist (sorry)

Opinionated Conclusions: Good Stuff

Easy to use cloud framework    Great entry-level tool

Scales like Google

Free!     You get a fully-functioning system for free    Did I mention that it's free?

    (okay, within limits, but it's free for a while)

LinksGoogle App Engine home    http://code.google.com/appengine

Source code    git://github.com/kousen/recommended-books.git 

Deployed App URL    http://recommended-books.kousenit.com 

GAE Plugin for Grails    http://grails.org/plugin/app-engine