Transcript No Slide Title

Google App Engine For Java
www.ociweb.com
© 2009 Object Computing, Inc. (OCI)
Revised 05/05/09
All rights reserved. No part of these notes may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording or otherwise, without the prior, written permission of Object Computing, Inc. (OCI)
Google App Engine For Scala
www.ociweb.com
© 2009 Object Computing, Inc. (OCI)
Revised 05/05/09
All rights reserved. No part of these notes may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording or otherwise, without the prior, written permission of Object Computing, Inc. (OCI)
Google App Engine For Groovy
www.ociweb.com
© 2009 Object Computing, Inc. (OCI)
Revised 05/05/09
All rights reserved. No part of these notes may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording or otherwise, without the prior, written permission of Object Computing, Inc. (OCI)
Google App Engine For JRuby
www.ociweb.com
© 2009 Object Computing, Inc. (OCI)
Revised 05/05/09
All rights reserved. No part of these notes may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording or otherwise, without the prior, written permission of Object Computing, Inc. (OCI)
Google App Engine For Clojure
www.ociweb.com
© 2009 Object Computing, Inc. (OCI)
Revised 05/05/09
All rights reserved. No part of these notes may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording or otherwise, without the prior, written permission of Object Computing, Inc. (OCI)
Overview
• Google App Engine was first introduced in April 2008
– Initial release was based on Python
• Design Goals:
– Easy to use
– Easy to scale
– Free to get started
• Differences from traditional hosting include:
– No upfront costs
– No access given to specific server, virtual or otherwise
– Don’t have to worry about fault-tolerance, scalability, setting up data stores,
access control .
• Still considered a “Preview Release”
– A least it’s not “Beta” ;)
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
6
Google App Engine Stack
• Components:
– Web serving infrastructure
•
•
•
•
Provides URL (<application name>.appspot.com)
Execute your code
Serve static content
Handles logging
– Language runtime
• First Python, now Java too.
– SDK
• Provides development environment
– Web base admin interface
• Tools to manage and monitor your application
– Scalable persistence layer
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
7
Google App Engine for Java
In April 2009, Google App Engine for Java is released
• Characteristics:
– Runs Java application using a Java 6 JVM
– Uses the Java Servlet standard
• More or less compatible with 2.4 spec
– Run in a secured sandbox
– Provides standard interfaces to scalable persistence layer using
• JDO (2.3)
• JPA (1.0)
– Other Services include:
•
•
•
•
•
•
Access to Google Users service
URL Fetch service
Mail service
Images service
Memcache service
XMPP service
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
8
Limitations
Because GAE/J applications run in a secured sandbox there
are limitations:
• To ensure applications do not interfere with other apps,
Applications can not:
–
–
–
–
–
Spawn threads
Write to a local file system including using temp files
Make arbitrary network connections
Invoke native code
Stream data to clients in response to single request
• SDK uses a class “white list” for core JRE classes.
Notable omissions:
– RMI and CORBA
– AWT and Swing classes including Graphics and Image
– ImageIO
• What about generating images ? (We’ll cover this later)
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
9
Limitations
• Application are optimized for short requests.
– Google imposes a 30 second limit after which an Exception is
thrown
– Requests are limited to 1MB
– Responses are limited to 10MB
• Increased from 1 MB in version 1.2.1 (May13th, 2009)
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
10
Limitations
• Your application and data is on Google’s infrastructure
• Once you create an application, you can not delete it !
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
11
Quotas
• There are two types of quotas for GAE applications:
– Billable quota
•
•
•
•
Resource maximums set by application administrator
Prevent usage from exceeding budget
Every application gets some billable quota for free
If billing is enabled, billable quotes can be raised above the free levels
– Only billed for amount of resource above free thresholds
– Fixed quota
• Resource maximum set be the App Engine
• Ensure integrity of the system
• Fixed quotas are raised when billing is enabled, but are not billable
– There are both Daily and Per-Minute Quotas
• Per-Minute Quotas prevent Daily Quotas from being consumed too
quickly
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
12
Resources Subject to Quotas
• Billable Quota Resources:
–
–
–
–
Outgoing and Incoming Bandwidth
CPU Time
Stored Data
Recipients E-Mailed
• Fixed Quota Resources
– Requests
• Both HTTP and HTTPS requests
– E-mail
• Data Size
• Number and size of attachments
– Various API Calls to DataStore, Mail, URLFetch, Users, Image,
Memcache and XMPP services.
• When Resources are depleted:
– Bandwidth, CPU, and Requests result in HTTP 403 code
– Others will result in various Exceptions
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
13
Some Current Quotas and Costs
Resource
Free Default Quota
Billing Enabled Quota
Cost
Daily Limit
Max. Rate
Daily Limit
Max. Rate
Requests
1,300,000
7,400 / min.
43,000,000
30,000 / min.
N/A
Outgoing
Bandwidth
1 GB 2, 4
56 MB / min.
1,046 GB 1
740 MB / min.
$0.12 /
GB
Incoming
Bandwidth
1 GB 2, 4
56 MB / min.
1,046 GB 1
740 MB / min.
$0.10 /
GB
CPU Time
6.5 CPU
hours 3
15 CPU-min. /
min.
1,729 CPUhours 1
72 CPU-min. /
min.
$0.10 /
CPU hr.
Stored Data
1 GB
N/A
No Max 1
N/A
$0.15 /
GB /
month
Recipients Emailed
2,000
8 / min.
7,400,000 1
5,100 / min.
$0.0001
per
1 Anything
above Free Default Quota is billed
2 Changed from 10 GB on May 25th, 2009
3 Changed from 46 CPU hours on May 25th, 2009
4 Changed to total 1GB incoming and outgoing
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
14
SDK for Java
• Emulates Google’s server environment
– Uses a Jetty-based solution for the development server
• Allows code to developed locally
– Includes shell scripts and/or Ant tasks for
• For starting and stopping the development server
– Path to war directory passed as parameter
•
– There are ways of leveraging Maven
• http://gae-j-maven.appspot.com documents using archetype
– Include blank application template for new projects
• GAE/J expects a specific project directory structure:
src/
META-INF/
war/
WEB-INF/
lib/
classes/
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
...Java source code...
...other configuration...
...JSPs, images, data files...
...app configuration...
...JARs for libraries...
...compiled classes...
15
Eclipse Plugin
• Includes SDK
• Includes GWT
– Maybe you should use it. Hint, hint, nudge, nudge
• Facilitates App Engine development
– Provides a Google App Engine project type
• Uses expected project directory structure
– Can execute using development server within Eclipse
– Can debug using Eclipse
– Can deploy to Google from Eclipse
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
16
Configuring
• GAE/J follows the J2EE standards for web applications
– web.xml minus features:
• App Engine supports the <load-on-startup> element for servlet
declarations. However, the load actually occurs during the first request
handled by the web server instance, not prior to it.
• App Engine supports <mime-mapping> elements for specifying the
MIME type to use for resources whose filenames end with certain
extensions. MIME mappings only apply to servlets, not to static files.
Static files use a fixed list of mappings of filename extensions to MIME
types.
• Some deployment descriptor elements can take a human readable
display name, description and icon for use in IDEs. App Engine doesn't
use these, and ignores them.
• App Engine doesn't support JNDI environment variables (<env-entry>).
• App Engine doesn't support EJB resources (<resource-ref>).
• The <distributable> element is ignored.
• Servlet scheduling with <run-at> is not supported
– Google App Engine does have service for CRON tasks
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
17
Configuring
• Has extra configuration file appengine-web.xml which
specifies:
– Name of application:
• <application name>.appspot.com
–
–
–
–
–
–
Version of application
Logging
Enabling HTTP Sessions
Enabling SSL
System properties and environment variables
Differentiating between static and resource files
• By default all files under /war directory are both resource and static files
except .JSP and those under /WEB-INF
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
18
Running in Development
• Eclipse provides a Run Configuration type “Web
Application”
• SDK provides scripts and ant tasks to execute
development server
• The development web server includes a console web
application
– New in SDK version 1.2.5
– Use to browse and manage the local datastore
– Append “/_ah/admin” to the URL
• http://localhost:8080/_ah/admin
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
19
Admin Console
Google provides online monitoring on application deployed to
its infrastructure
• Access via appengine.google.com
– Sign in using your Google account
• You can use the Administration Console to:
– Create a new application, and set up a free appspot.com subdomain, or a top-level domain name of your choosing
– Invite other people to be developers for your application, so they
can access the Console and upload new versions of the code
– View access data and error logs, and analyze traffic
– Browse your application's datastore and manage indexes
– View the status of your application's scheduled tasks
– Test new versions of your application, and switch the version that
your users see
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
20
Persistence
The persistence layer for Google App Engine application is
known as DataStore
• Based on BigTable
– Proprietary database system build on Google File System (GFS)
• Not SQL
• Schema-less object database
– Supports arbitrary new properties or columns
•
•
•
•
Supports transactions for writes
Provides SQL-like queries
Doesn’t support joins
Can use either Java Data Objects (JDO) or Java
Persistence API (JPA)
– Interfaces provided by the DataNucleus Access Platform
• http://www.datanucleus.org/
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
21
Using JDO
• Configured using /src/META-INF/jdoconfig.xml file
– Copied to /war/WEB-INF/classes/META-INF in war file
– Has App Engine specific contents
• Requires class enhancement
– A post-compilation step that processes annotations and does some
bytecode manipulation to facilitate persistence
• http://www.datanucleus.org/products/accessplatform/enhancer.html
– Ant macro provided in SDK
– Does automatically by Eclipse plugin
• Provides JDOQL query mechanism
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
22
Unsupported JDO Features
• Unowned relationships.
– You can implement unowned relationships using explicit Key values. JDO's
syntax for unowned relationships may be supported in a future release.
• Owned many-to-many relationships.
• "Join" queries.
– You cannot use a field of a child entity in a filter when performing a query on
the parent kind.
• JDOQL grouping and other aggregate queries.
• Polymorphic queries.
– You cannot perform a query of a class to get instances of a subclass. Each
class is represented by a separate entity kind in the datastore.
• Only @PersistenceCapable, IdentityType.APPLICATION
is supported.
• Composite primary keys !!
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
23
Sample JDO Persistent Class
@PersistenceCapable(identityType =
IdentityType.APPLICATION)
public class Employee {
@PrimaryKey
@Persistent(valueStrategy =
IdGeneratorStrategy.IDENTITY)
private Long id;
@Persistent
private String firstName;
@Persistent
private String lastName;
@Persistent
private Date hireDate;
// Constructors, Getters and Setters
// JDO does not require Getters and Setters
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
24
Persisting with JDO
• JDO Uses a PersistanceManager to perform various operations:
PersistenceManager pm = PMF.get().getPersistenceManager();
try {
pm.makePersistent(new Employee("Alfred", "Smith",
new Date()));
} finally {
pm.close();
}
– Any child records have not being persisted will be persisted as well
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
25
Keys used by JDO
• Keys – There are four types of primary keys:
– Long
• Can be automatically generated by the DataStore and populate when
instances is saved:
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long id;
– Unencoded String
• The application sets the value before saving
– Key
• com.google.appengine.api.datastore.Key
• Can be set by application or automatically generated if left null
– Key as Encoded String
• Makes the application more portable by not depending on Key class
• Can be set by application or automatically generated if left null
– Objects can easily be retrieved using its key:
• Employee e = pm.getObjectById(Employee.class, k);
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
26
JDO Queries
• JDO supports queries
Query query = pm.newQuery(Employee.class);
query.setFilter("lastName == lastNameParam");
query.setOrdering("hireDate desc");
query.declareParameters("String lastNameParam");
try {
List<Employee> results = (List<Employee>)query.execute("Smith");
} finally {
query.closeAll();
}
• Deletes can be done via query
– Query.deletePersistentAll(<params>)
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
27
Using JPA
• Configured using persistence.xml file
– Must be in the app's war/WEB-INF/classes/META-INF/ directory
– Future release of Eclipse plugin will deploy this automatically
– Specifies DataNucleus as persistence provider
• Also requires class enhancement
– Bytecode manipulation done here
– Ant macro provided
– Done automatically by Eclipse plugin
• Not as well documented by Google
– They seem to favor JDO at this point in the documentation
• JPA more suited for ORM persistence ?
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
28
Sample JPA Persistent Class
@Entity
public class Employee {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String firstName;
private String lastName;
private Date hireDate;
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
29
Unsupported JPA Features
• Owned many-to-many relationships, and unowned
relationships. You can implement unowned relationships
using explicit Key values, though type checking is not
enforced in the API.
• "Join" queries. You cannot use a field of a child entity in a
filter when performing a query on the parent kind. Note
that you can test the parent's relationship field directly in
query using a key.
• Aggregation queries (group by, having, sum, avg, max,
min)
• Polymorphic queries. You cannot perform a query of a
class to get instances of a subclass. Each class is
represented by a separate entity kind in the datastore
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
30
Upcoming Features
• Service for storing and serving large files
• Incoming email support
• Cursors for continuing results of Datastore queries past
the 1000 entity limit
• Alerting system for exceptions in your application
• DataStore dump and restore facility
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
31
Users Service
• A UserService object can be retrieved using a provided Factory:
UserService userService =
UserServiceFactory.getUserService();
• The UserService class provides methods:
–
–
–
–
isUserLoggedIn – Is current user logged in using Google account
isUserAdmin – Is current user an admin for the application
getCurrentUser – Retrieve User object of current logged in user (null otherwise)
createLoginURL – Returns a URL where a user can log in
createLogoutURL – Return a URL where the user can be logged out
• The User class provides:
– getEmail – E-mail address associated with Google account
– getAuthDomain - Domain name to which user has authenticated
• "gmail.com" for normal Google authentication.
– getNickname – Human readable identifier with application or e-mail
• Access to GAE/J applications can controlled using securityconstraint element in web.xml
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
32
Mail Service
• Supports the JavaMail (javax.mail) interface:
Session session = Session.getDefaultInstance(props, null);
try {
Message msg = new MimeMessage(session);
msg.setFrom(new InternetAddress("[email protected]"));
msg.addRecipient(Message.RecipientType.TO,
new InternetAddress("[email protected]", "Mr. User"));
msg.setSubject("Your Example.com account");
msg.setText(msgBody);
Transport.send(msg);
} catch (AddressException e) { // ...
} catch (MessagingException e) { // ...
• You do not need to provide any SMTP server configuration.
– App Engine will always use the Mail service for sending messages.
• Development server does not send E-mail, logs it instead
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
33
URL Fetch Service
• Uses java.net.URLConnection and related classes
• Must use the standard ports for HTTP (80) and HTTPS
(443)
• Uses an HTTP/1.1 compliant proxy
• Waits up to 5 seconds for the remote server to respond
– Throws exception on timeout
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
34
Images Service
• App Engine provides the ability to manipulate image data
using a dedicated Images service:
– Resize, rotate, flip, and crop images
• Includes “I'm Feeling Lucky” transform to adjust both color and contrast
to optimal levels
–
–
–
–
Composite multiple images into a single image
Convert image data between several formats
Enhance photographs using an predefined algorithm
Provide information about an image, such as its format, width,
height, and a histogram of color values.
– Accepts image data in the JPEG, PNG, GIF (including animated
GIF), BMP, TIFF and ICO formats.
– Returns transformed images in the JPEG and PNG formats
– 1 MB limit on image data sent or received from service
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
35
Memcache Service
• The Memcache Java API implements the JCache
interface (javax.cache),
–
–
–
–
A draft standard described by JSR 107
Provides a Map-like interface to cached data
Keys and values can be of any Serializable type or class
Values stored in Memcache are retained as long as possible
• If new values are added, least recently used values are evicted if
memory is low
• Values can be given an expiration date
– Cache not saved to disk
• Service failure may cause values to become unavailable
– 1 MB limit on values in Memcache
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
36
XMPP Service
• An App Engine application can send and receive instant
messages to and from any XMPP-compatible instant
messaging service
– Google Talk and Google Wave
– An app can send messages of any type defined in RFC 3921
• Send and receive chat messages,
• Send chat invites
• Request status information.
– Incoming XMPP messages are handled by request handlers,
similar to web requests.
• Limitations
– Currently, an app cannot participate in group chats
– Only receive messages of the "chat" and "normal" types
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
37
Cron Service
• The App Engine Cron Service allows you to configure
regularly scheduled tasks that operate at defined times or
regular intervals
– A cron job will invoke a URL at a given time of day
• Subject to the same limits and quotas as a normal HTTP request,
including the request time limit.
– A cron.xml file resides in the /WEB-INF directory of the application
– Typically URLs accessed by cron jobs have access restricted to
application administrators
– Admin Console allows monitoring of state of jobs
– No cron support in development.
• Of course one can use their local server’s cron and task scheduler to
accomplish the same
– Supports English-like time specifications
• every 5 minutes
• every monday 09:00
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
38
Sample cron.xml
<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
<cron>
<url>/recache</url>
<description>Repopulate cache every 2 minutes</description>
<schedule>every 2 minutes</schedule>
</cron>
<cron>
<url>/weeklyreport</url>
<description>Mail out a weekly report</description>
<schedule>every monday 08:30</schedule>
<timezone>America/New_York</timezone>
</cron>
</cronentries>
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
39
Task Queue Service
• Currently this is an experimental service
– Can change and become incompatible even in minor releases
• Allows applications to perform work outside of a user
request
– But initiated by a user request
– Background work is organized into discrete units called Tasks
• There are quotas on these as well
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
40
Status of Frameworks (Java)
• Struts – Unknown, but rumored to be not working
– Who cares ?
• Struts 2 – Works with minor tweaks
– OgnlRuntime.setSecurityManager(null);
• Spring MVC – Works with minor issues:
– May use unsupported class, javax.naming.InitialContextFactory
• Wicket – Works with tweaks:
– http://www.danwalmsley.com/2009/04/08/apache-wicket-on-google-appengine-for-java/
– http://stronglytypedblog.blogspot.com/2009/04/wicket-on-google-appengine.html
• Tapestry – 5.0.18 works, but 5.1+ has issues:
– Uses unsupported class, javax.xml.stream.XMLInputFactory
• JSF - JSF 1.1 seems to be working but 1.2 is not
– Seems to have issues due to the bundling of JSP 2.1.
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
41
Status of Frameworks (Scala)
• Lift
– Mostly working minus some functionality:
• Comet support built on top of Scala actors which may spawn threads
• Lesser known web frameworks are all known to work
– Step
– Pinky
– Sweet
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
42
Status of Frameworks (Groovy)
• Grails
– Officially supported in Grails 1.1.1
– Plugin provided
• http://grails.org/plugin/app-engine
• Gaelyk
– New lightweight toolkit especially for the GAE/J
• Uses “Groovelets” as controllers
• Support templates with embedded Groovy as views
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
43
Statius of Frameworks (JRuby)
• Rails
– Working
– http://olabini.com/blog/2009/04/jruby-on-rails-on-google-appengine/
• Sinatra
– Working
– http://blog.bigcurl.de/2009/04/running-sinatra-apps-on-google.html
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
44
Status of Frameworks (Clojure)
• Compojure – Works with minor problem areas:
– http://elhumidor.blogspot.com/2009/04/clojure-on-googleappengine.html
– Issues
• Agents, clojure.parallel library, and future will not work (Threads)
• Shared references (Vars, Refs, and Atoms) may have problems due to
distributed nature of the Google infrastructure
Copyright © 2009 by Object Computing, Inc. (OCI).
All rights reserved.
45