Upload
sandeepmathoor8218
View
233
Download
0
Embed Size (px)
Citation preview
8/2/2019 App Engine Testing Techniques
1/44
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
2/44
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
3/44
Testing Techniques ForGoogle App EngineMax RossMay 20, 2010
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
4/44
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
5/44
http://bit.ly/GAETesting
View live notes and ask questions about thissession on Google Wave
Tuesday, May 25, 2010
http://bit.ly/9XLCAshttp://bit.ly/9XLCAs8/2/2019 App Engine Testing Techniques
6/44
Who Am I?
Max!
Software Engineer at Google
Member of the App Engine team
Datastore
JPA/JDO Implementation
Java Runtime
http://gae-java-persistence.blogspot.com/
I feel naked and sad without good test coverage
4
Tuesday, May 25, 2010
http://gae-java-persistence.blogspot.com/http://gae-java-persistence.blogspot.com/http://gae-java-persistence.blogspot.com/8/2/2019 App Engine Testing Techniques
7/44
Agenda
Developer Testing
Testing App Engine Apps Locally
Testing App Engine Apps In The Cloud
Wrap Up
Questions
5
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
8/44Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
9/44
8/2/2019 App Engine Testing Techniques
10/44
Terminology
Unit test
tests a single method/class
behavior of dependencies controlled by the test
Integration test
tests interaction between multiple methods, classes, orcomponents
behavior of dependencies mightbe controlled by the test
End to end test
tests behavior by driving requests through external accesspoints
for App Engine that means HTTP
7
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
11/44
Soapbox: Why Developer Testing Is Important
Demonstrates correctness of new code
important
Demonstrates continued correctness of old code
more important
Provides a safety net when making large or risky changes
code always evolves organically
refactoring can help restore order
Treat your tests as first class citizens in your code base
please Deploying to App Engine doesnt change any of this!
8
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
12/44
Testing @ Google
part of our engineering culture
guiceberry
googletest
googlemock
thread-weaver
googletesting.blogspot.com
9
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
13/44
App Engine Execution Environments
Cloud
serving HTTP traffic
does not have to be prod
Local
serving HTTP traffic
equivalent functionality
nonequivalent scale
Tests!
serving HTTP traffic and test harness requests local and cloud
do we need App Engine specific testing strategies?
10
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
14/44Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
15/44
Testing App Engine Apps
Locally
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
16/44
Local Testing - Whats The Same?
Frameworks
JUnit, pyUnit, Selenium
Best practices
design
avoiding flakiness
isolation
End to end testing
This is great news!
12
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
17/44
Local Testing - Whats Different?
RPC layer assumes dev appserver sets up the environment
What if there is no dev appserver?
set up the environment by hand or... use the App Engine Testing APIs!
currently Java only
13
// from com.google.appengine.api.datastore.DatastoreApiHelper.java
static String getCurrentAppId() {
ApiProxy.Environment environment = ApiProxy.getCurrentEnvironment();
if (environment == null) {
throw new NullPointerException(
"No API environment is registered for this thread.");}
return environment.getAppId();
}
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
18/44
Local Testing - Demo!
Guestbook Persistence
Integration test with the datastore and task queue
high(er)-availability writes
14
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
19/44Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
20/44
Testing App Engine Apps In
The Cloud
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
21/44
I Want To Run My Tests In The Cloud!
Why?
Fidelity
true behavior and performance
Efficiency
run large test suites faster
let someone else manage your test grid
16
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
22/44
Cloud Testing - Whats The Same?
Same similarities as local testing
frameworks
best practices
end to end testing
This is also great news!
17
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
23/44
Cloud Testing - Whats Different?
Each test must complete in 30 seconds
App code and test code must obey sandbox restrictions
threads, direct network access, etc
Must invoke tests via HTTP
no need to worry about setting up the environment
GAE Cloud Cover can help
18
Tests AppIn Process
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
24/44
Cloud Testing - Whats Different?
Each test must complete in 30 seconds
App code and test code must obey sandbox restrictions
threads, direct network access, etc
Must invoke tests via HTTP
no need to worry about setting up the environment
GAE Cloud Cover can help
18
Tests
XAppIn Process
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
25/44
Cloud Testing - Whats Different?
Each test must complete in 30 seconds
App code and test code must obey sandbox restrictions
threads, direct network access, etc
Must invoke tests via HTTP
no need to worry about setting up the environment
GAE Cloud Cover can help
18
T
ests AppCloudCover
CloudCover
UI
HTTP
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
26/44
Cloud Testing - Simple Demo!
GAE Cloud Cover is designed to run existing test suites
Guestbook Persistence tests running in the cloud:
install JAR
configure Servlet
replace LocalServiceTestHelper
extend JUnit3Config
deploy
run
curl up under a cozy warm blanket of cloud test coverage
19
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
27/44
How It Works
A Suite is a collection of Tests
GAE Cloud Cover creates one task queue task per Test
Tests execute in parallel
number of workers determined by queue config
GAE is good at fan-out!
20
Task Queue
Worker 1 Worker 2
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
28/44
How It Works
A Suite is a collection of Tests
GAE Cloud Cover creates one task queue task per Test
Tests execute in parallel
number of workers determined by queue config
GAE is good at fan-out!
20
SUITE
Test 1
Test 2
Test 3 Task Queue
Worker 1 Worker 2
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
29/44
How It Works
A Suite is a collection of Tests
GAE Cloud Cover creates one task queue task per Test
Tests execute in parallel
number of workers determined by queue config
GAE is good at fan-out!
20
SUITE
Test 1Test 2Test 3
Task Queue
Worker 1 Worker 2
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
30/44
How It Works
A Suite is a collection of Tests
GAE Cloud Cover creates one task queue task per Test
Tests execute in parallel
number of workers determined by queue config
GAE is good at fan-out!
20
SUITE
Task Queue
Worker 1 Worker 2
Test 2
Test 3
Test 1
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
31/44
Isolation Problems
Tests must be thread-safe
Test data must be thread-safe
Non thread-safety yields flakiness Flaky tests are worse than consistently failing tests
21
Thread 1: testReadThruCache Thread 2: testQuery
1) populate cache 1) perform query
2) install ErrorDelegate 2) verify results
3) fetch val from cache
Thread 1: testCreateUser Thread 2: testDeleteUser
1) create User m 1) create User m
2) assert m exists 2) delete m
3) assert m does not exist
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
32/44
Isolation Problems
Tests must be thread-safe
Test data must be thread-safe
Non thread-safety yields flakiness Flaky tests are worse than consistently failing tests
21
Thread 1: testReadThruCache Thread 2: testQuery
1) populate cache 1) perform query
2) install ErrorDelegate 2) verify results
3) fetch val from cache
Thread 1: testCreateUser Thread 2: testDeleteUser
1) create User m 1) create User m
2) assert m exists 2) delete m
3) assert m does not exist
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
33/44
8/2/2019 App Engine Testing Techniques
34/44
Isolation Solutions
Use ThreadsafeDelegate instead of ApiProxy.Delegate
Use Namespaces to enforce test data isolation
coming soon to an App Engine SDK near you
implemented as a way to stripe persistent data with arbitraryidentifiers
this is exactly what we need!
22
Kind Namespace Key
Greeting testCreateUser max
Greeting testDeleteUser max
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
35/44
Cloud Testing - Fancy Demo!
App Engine JDO/JPA implementation test suite
userland library
1300+ unit and integration tests
makes extensive use of the datastore
uses custom ApiProxy.Delegate implementations
23
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
36/44
Running Arbitrary Test Suites
Why limit ourselves to test suites for App Engine apps?
GAE Cloud Cover is framework agnostic
requires a small plugin to hook up a framework
com.google.appengine.testing.cloudcover.spi
implement 3 simple interfaces
If you have a large test suite, App Engine can be your grid!
24
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
37/44
Cloud Testing - Arbitrary Demo!
Google Collections Framework
http://code.google.com/p/google-collections/
44,776 tests
Has nothing to do with App Engine
25
Tuesday, May 25, 2010
http://code.google.com/p/google-collections/http://code.google.com/p/google-collections/8/2/2019 App Engine Testing Techniques
38/44
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
39/44
Wrap Up
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
40/44
Looking Ahead
Testing APIs
Python
running tests inside the local sandbox
launching dev appserver from tests
GAE Cloud Cover:
IDE-integration
support for dynamically generated tests
project is completely open source
http://code.google.com/p/cloudcover/
Help!
27
Tuesday, May 25, 2010
http://code.google.com/p/cloudcover/http://code.google.com/p/cloudcover/8/2/2019 App Engine Testing Techniques
41/44
Conclusions
Developer testing is Important!
Your testing skillz transfer
App Engine Testing API makes local testing easy
GAE Cloud Cover makes cloud testing easy
You dont have to build an App Engine app to use AppEngine
28
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
42/44
Tuesday, May 25, 2010
8/2/2019 App Engine Testing Techniques
43/44
Questions
http://bit.ly/GAETesting
Tuesday, May 25, 2010
http://bit.ly/9XLCAshttp://bit.ly/9XLCAs8/2/2019 App Engine Testing Techniques
44/44