Upload
corey-rogers
View
220
Download
1
Embed Size (px)
Citation preview
Java application delivery
Deploying without going offline
Me
Frederic Jambukeswaran, Founder, rapidcloud.io
Previously:
VP, Technology EXPO Communications
CTO Pronto/IAC
VP, Engineering Sportnet
Focus on Deployments
I’m going to focus almost exclusively on deploying your application
My assumption is that you have a separate build process which packages your artifact (jar, war, etc) and prepares your static assets Maven, SBT, Ant, Grunt, etc are all tools to prepare your application
Deploying works by picking up prebuilt artifacts and handling the delivery to your servers Yes you can build your artifacts for each (server) instance if you need to, however it quickly
starts to become a bottleneck if you have many instances or a complex environment.
Deployment Expectations
Zero downtime (at least never offline)
Fast (not slow)
100% automated
Reliable
Simplest Basic Deployment (stop/start)
The simplest deployment strategy in any environment is to shutdown the application, copy the assets and startup again. Unfortunately this leads to our app being offline and inaccessible.
Acceptable for some use cases, maybe on dev or a lightly changed/used application, but it creates a poor user experience and creates a hesitancy to deploy (BAD)
Parallel Deployments to the Rescue!
Parallel deployments allow you to maintain availability by launching your new application while your old one is still handling requests
You are able to verify the new instance is healthy and ready to receive traffic before activating it
You can give your old application time to finish processing existing requests
Downside
They are more difficult to setup and may require application changes in how static assets, databases or sessions are handled
Some gotchas
Http sessions Since we are launching new instances, http sessions will be lost unless you
Setup session replication
Use a centralized session store (memcache, etc)
Don’t use sessions (how cool is it that Play Framework doesn’t use http sessions?)
Database Schemas Since you’re old app and new app will be running at the same time, your schema changes must be backwards
compatible
Static assets You may be serving requests for two different versions at the same time. So your static assets will get
confused.
Use asset fingerprinting (asset auto versioning) to prevent cache poisoning. (OK/Kinda)
Separate your static assets and deliver them for any version (GREAT!)
Server Configurations
We are going to cover how to execute this in the following environments
AWS / Autoscaling (dynamic instances)
Single server (static instance)
Multiple servers (static instances)
AWS : Overview
AWS allows you to “instruct” them on how to launch an server Think of your application in terms of dynamic instances while hold everything
JVM, WWW server, etc
Don’t think about just deploying a war file, think about deploying an entire server instance
How do we keep track of them all? Amazon’s load balancers work together with the auto scaling group to automatically add new
instances to into rotation
Overall…. Very elastic, very cloudy, very awesome.
AWS : Deploying (Mindset)
Think about it as deploying new instances not deploying new code These new instances are configured to run your new code
Then we “undeploy” older code, by terminating the older instances
AWS : Auto Scaling
A very powerful service to manage instances by configuration. Setup rules to tell amazon how many instances you want, what type they are, and under what
conditions it should create /destroy instances.
Example: You can scale up to 10 instances during high load and down to 2 when load is low.
If an instance dies or gets unhealthy, it can automatically be replaced.
Amazon’s load balancers work together with the auto scaling group to automatically add new instances to into rotation
Overall…. Very elastic, very cloudy, very awesome.
What does it look like
Load Balancer
Launch new server group
Deregister the old server group
After the old instances have cooled down terminate them
App 1
Group 1
App 2
Group 2
Load Balancer
App 2
Group 2
Load Balancer
App 1
Group 1
App 2
Group 2
Load Balancer
App 1
Group 1
AWS : Alternatives (which I don’t like)
Launch a new ELB for each deployment and change the DNS settings Tinkering with DNS is not something that I would like to do it
New ELB may need time to scale up in the case of high traffic
Have your autoscaling groups pull the “latest” production built and kill old instance one by one (allowing autoscaling to rebuild with new instances) If your new production build is unstable for some reason, you’ve “broken” your autoscaling
group
What's the catch?
Have to learn Amazon’s environment Powerful, flexible but complicated
Have to think about your instances differently No more www1, www2, frey, Aryn, Baratheon, etc
You have “groups” and you won’t always know how many instances each group holds
To deploy a new application you’ll need to handle orchestrating and managing AWS services
Example: Through the AWS API
To take advantage of autoscaling, AWS needs to be able to launch, configure and deploy your application to an instance completely by itself
http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/as-scale-based-on-demand.html
Imagine an ELB here
AWS Network Components
Key parts of AWS’s networking infrastructure Regions
Availability Zones
VPCs
Subnets
Security Groups
Subnet
Availabilty Zone
VPC
Region
You configure these network
services
Regions and Availability Zones
Regions are separate geographic areas and are independent of each other us-east-1 – Virginia
us-west-1 – Northern California
us-west-2 - Oregon
Availability zones exist within a region and are isolated locations Communication across zones and within a region are low latency/fast
Ex: us-east-1a, us-east-1b
Virtual Private Cloud (VPC)
Create your own logically isolated section of the amazon cloud. You can setup subnets, ip ranges, etc.
AWS automatically creates a VPC for you(for all accounts created > Dec-04 2013)
VPC exists within a region and across availability zones
Subnet
A subnet exists within a VPC and an availability zone (do not span zones)
Public subnet – Traffic is routed to the internet
Private subject – Traffic doesn’t have a route to the internet
Security Groups
Act like firewalls controlling inbound and output network traffic
Associate one or more security groups to each of your instances
Defined at the region level
Security Groups
You can limit access to another security group
You can limit access to an ip range
AWS Network Components (Repeated)
Key parts of AWS’s networking infrastructure Regions
Availability Zones
VPCs
Subnets
Security Groups
Subnet
Availabilty Zone
VPC
Region
You configure these network
services
Configuring AWS to Launch Instances
AMIs
Launch Configurations
Auto Scaling Groups
Load Balancers
Amazon Machine Image (AMI)
These images are the foundation of any server instance you create Specify the OS, installed software, etc
Amazon provides some good ones to use as a starting point RedHat, Ubuntu, etc
AMIs
Customizing an AMI
To speed up launching instances you can take one of Amazon’s AMIs and launch an instance with that.
Customize that instance with all your key software (Java 8, Log Entries, etc)
Create your own AMI from that image
Use your customized AMI for future instances
Its really that easy….
Launch Configurations
What is it? Set of rules to define how to launch an instance
Key elements Security groups
Type of instance (small, medium, t2, m3)
User data (meta data field)
Startup Script Launch configurations can be setup to run a bash script on startup
Configured through the user data attribute
Its an idea way to bootstrap your instance, since the the script carry your instance settngs (i.e. application, environment, role and build_id)
I like to use a boot.sh file built into my AMI and have my launch configuration script just delegate to it (keeps my launch config meta data small)
Code: Create Launch Configuration
AmazonAutoScalingClient autoScalingClient = new AmazonAutoScalingClient();
CreateLaunchConfigurationRequest request = new CreateLaunchConfigurationRequest();
request.withImageId("abc-123-ami");
request.withIamInstanceProfile("application-server");
request.withKeyName("default-ssh-key");
request.withLaunchConfigurationName( "myapplication-staging-cron-lc-" + System.currentTimeMillis());
request.withSecurityGroups(new String[] { "sg-123-abc", "sg-123-def" });
request.withInstanceType("t2.small");
request.withAssociatePublicIpAddress(true);
request.setUserData(toBase64("#! /bin/bash –e\n/opt/boot/boot.sh myapplication staging cron build_1232"));
autoScalingClient.createLaunchConfiguration(request);
Auto Scaling Groups
Auto scaling groups define how you want manage a group of instances
Key elements Capacity of your group
Subnet
Load balancer
Healthcheck to use to monitor the instances
Meta data (tags)
Auto scaling policies
Code: Create Autoscaling Group
CreateAutoScalingGroupRequest createGroup = new CreateAutoScalingGroupRequest();
createGroup.setAutoScalingGroupName("myapplication-staging-cron-asg" + System.currentTimeMillis());
createGroup.setDesiredCapacity(2);
createGroup.setHealthCheckGracePeriod(300); /* how much time before healthchecks have to pass */
createGroup.setHealthCheckType("ELB");
List<String> elbNames = new ArrayList<String>();
elbNames.add("myapplication-staging-elb");
createGroup.setLoadBalancerNames(elbNames);
createGroup.setLaunchConfigurationName(launchConfigurationName);
createGroup.setMaxSize(10);
createGroup.setMinSize(2);
createGroup.setVPCZoneIdentifier("subnet"); /* subnet id for the instances */
createGroup.setDefaultCooldown(300);/* seconds between scaling availabilities */
autoScalingClient.createAutoScalingGroup(createGroup);
Elastic Load Balancer
Elastic Load Balancer (ELB) High availability load balancer which automatically scales up with traffic
Capable of determining the state of an instance with http healthchecks
Deregister and check the status of instances in the ELB using the AWS SDK
Code: Check Instances in ELB
private Integer getReadyCount(List<String> newInstances, String elbName) {
AmazonElasticLoadBalancingClient elbClient = new AmazonElasticLoadBalancingClient();
DescribeInstanceHealthRequest request = new DescribeInstanceHealthRequest(elbName);
DescribeInstanceHealthResult describeInstanceHealth = elbClient.describeInstanceHealth(request);
Integer readyCount = 0;
for(InstanceState state : describeInstanceHealth.getInstanceStates()) {
if(state.getState().equals("InService")) {
if(newInstances.contains(state.getInstanceId())) {
logger.info("{} instance is ready", state.getInstanceId());
readyCount = readyCount +1;
}
}
}
return readyCount;
}
Tip: Name your entities carefully
Use very consistent naming, it can get confusing quickly
Key all entities by: Environment – prod, stage, dev
Application – sso, datafactory, etc
Role – www, cron, worker, etc
Creation timestamp (for uniqueness)
Example:
prod-sso-www-asg-1411586408
prod-sso-cron-asg-1411586408
prod-sso-cron-lc-1411562401
S3 : Great place to store assets
Scalable data store
Ideal place to hold application artifacts
static assets
S3 : Storing Artifacts (jar/war)
Use S3 to house your jar/war files for deployment. Setup a bucket for the artifacts.
repository.mycompany/myapplication/$build_id/myapplication.war
The $build_id will allow you to have multiple versions active (staging, dev, prod, etc)
If Jenkin’s hosted on AWS you can leverage AWS’s command line tools to copy your assets up to a bucket after the build step (lots of other ways to move assets to s3 as well)
aws s3 cp target/universal/myapplication-1.0.zip "s3://<mu bucket>/versions/$BUILD_ID/myapplication-1.0.zip" ;
S3: Managing static assets
You can store your static assets (css/js/images) a public s3 bucket: static.mycompany/myapplication/$build_id/(js|css|img)/…
Serve static assets from your public bucket http://static.mycompany.s3-us-east1.amazonaws.com/
OR front them with a CDN like cloudfront or akamai
Self Initializing (I know I’m repeating)
It’s important to understand with AWS Auto scaling, your instances need to be able to fully and completely initialize themselves in a repeatable and reliable manner.
AWS is going to launch your instance relying on your AMI and launch configuration. Use your launch configuration script to fetch any assets and complete any instance
initialization.
Example Deployment Steps
Identify the build_id to deploy (argument to Jenkins, detect latest build on s3, etc)
Using the AWS SDK/API you then
Create new launch configuration (with meta data defining which app/build/role/env it is for)
Create autoscaling group (direct it to use your new launch config)
Wait for the group to be ready (@ least 1 member is “InService”)
Poll the ELB till the new instances are ready
Get the list of old instances from the ELB
Clean up
Deregister old instances from the ELB
Give your instance time to cooldown
Find all the old auto scaling groups set their counts to 0 and wait for them to terminate their instances
Remove old autoscaling groups and launch configuration
Orchestrating with Jenkins
Jenkins is a great way to pull it all together
Separate build jobs from deployment jobs Build job (one per application)
Checkout application code, package, copy artifact and static assets to S3
Deploy job (one per application/role/environment)
Instruct AWS to launch a new auto scaling group with new application version and tear down old ones
Naming
Suggestion: [application]-[environment]-[role]-[task]
Example:
rapidcloud-build-ci
rapidcloud-prod-worker-deploy
rapidcloud-prod-cron-deploy
rapidcloud-stage-worker-deploy
rapidcloud-stage-cron-deploy
Watch Out
You’ll have new and old instances working side by side for some time Static assets which have changed and are not delivered via S3 will be delivered inconsistently
You’ll use a lot of instances Since each deployment spins up an entirely new application layer, you’ll create and destroy
more instances
Make sure you have enough allocation to handle the occasional spikes
If you use S3 to store assets/artifacts, you’ll need to clean them out periodically
This takes some time to setup, give yourself a few days to write the scripts/configs to manage the deployment to AWS
NON- AWS environments
We’ll cover single server and multiple server deployments now
Single Server
A single server anywhere, cloud or physical hardware.
Very low cost server setup (ex: digital ocean 2gb ram $20/month)
Still need to deploy without going offline
No more rebuilding whole instances now….
What does it look like
www App:8080 www App:8080
App:8081
www App:8080
App:8081
www
App:8081
Start new application instance on 8081
Once new application is readySwitch nginx to 8081
After old application has cooled downStop the instance on 8080
Environment Configuration
Use a reverse proxy to accept www requests and route them to the application Nginx, Apache, Lighttpd
Configure the application to run on any one of two ports (ex: 8080 and 8081)
With each deployment alternate the port we use for the new application and point the reverse proxy to this new port
Reverse Proxy Configuration (nginx)
Nginx..proxy configuration:
location / {
proxy_buffering off;
proxy_pass http://127.0.0.1:8080;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
}
..
When switching ports, deploy an updated configuration (or change the port) and reload
Ex: sudo /usr/local/bin/nginx -s reload;
Since there is a proxy between the client and
app, we need to forward along the clients IP address
This is where nginx will send the request, we’ll
change this port with each
deployment
How To: Tomcat on 2 ports
For Tomcat install it across two instances with a central catalina home:
CATALINA_HOME=/opt/tomcat
Instances
/opt/tomcat/instances/8080
/opt/tomcat/instances/8081
Modify the server.xml in /opt/tomcat/instance/8081/conf to change or disable the default ports (8080, 844, 8009, 8005)
To startup the 8080 instance:
Execute: /opt/tomcat/bin/catalina.sh run
With environment variables:
CATALINA_HOME=/opt/tomcatCATALINA_BASE=/opt/tomcat/instances/8080
These environment variables tell tomcat where to look for the
webapp and its binaries
How To: Play on 2 ports
For Play install the application in one of two locations
/op/play/9000/
And
/opt/play/9001
To startup the 9000 instance:
/opt/play/9000/bin/start -Dhttp.port=9000 -Dpidfile.path=/opt/play/9000/play.pid
Example Deployment Steps
Find running instance (8080 or 8081) Do this by finding which port reverse proxy is using (truth)
Copy application package (war, etc) into the “other” instance 8081 (if 8080 is running)
Startup the new instance (supervisor is great for this)
Verify the new instance (curl, etc)
Activate the new application by switching nginx to the new port
Wait for old requests to cool down (sleep, etc)
Shutdown old application
Bash Snippet – Check Port
NGINX_PORT=8080;
TARGET_PORT=8081;
echo '[echo]: Checking /etc/nginx/nginx.conf to find the active nginx application port.';
if grep "8081" /etc/nginx/nginx.conf
then
NGINX_PORT=8081;
TARGET_PORT=8080;
fi
Bash Snippet - Verify new application
RETURN_CODE=`curl -m30 -s -o /dev/null -w "%{http_code}" http://localhost:8080/am/i/healthy`;
if [ "$RETURN_CODE" == '200' ]
then
echo '[echo]: Url check returned 200 status, success.';
else
echo "[echo]: Url check returned $RETURN_CODE status, failure.";
exit 1;
fi
Watch Out
You’ll need enough memory, since two copies of the application will be running
Logs files will go to two different locations, and keep swapping with each deployment Using a log manager to help, or just check both files
Multi-server Setup
It gets a little more interesting with multiple servers
You’ll need either a load balancer or a reverse proxy to distribute traffic to your application instances
In this walk though, I’ll assume Two or more servers as reverse proxies
Use round robin DNS to route requests to the proxies and DNS failover to handle unexpected “bumps”
Diagram – www / app servers separate
www
www app
app
DNS
Proxy Configuration (nginx)
http {
upstream application {
server srv1.example.com:8080;
server srv2.example.com:8080;
}
server {
listen 80;
location / {
proxy_pass http://application;
}
}
}
This is how nginx knows where to route traffic, by default it’ll round robin
through the list (behavior is configurable)
Example Deployment Steps
For each application instance (cycle through them) Take the application offline (wait, offline, yes, signal the app to stop accepting requests)
Wait for the application to cooldown
Shutdown the application
Copy the new assets
Start the application up again.
Verify the application started up successfully
Taking your application offline
When deploying we want to “take an application offline” so nginx stops sending requests, but allow it to continue processing existing ones
Add “offline” state to your application Once offline allnew calls return a http status 500
Can be built into a ServletFilter or Play action annotation
Can be triggered via url, file, etc
Watch out
Nginx latency Identifying proxy health is done as it executes requests meaning increased latency during
deployments
Definitely expect multiple versions live at the same time
Rapidcloud.io
Rapidcloud.io is a cloud management and application deployment service for Java developers.
Join our Beta and get our starter package free for 12 months
Appendix
Supervisord
Tomcat Parallel Deplotment
Amazon SDK
Supervisord
Supervisord is a great service for managing instances.
Uses a declarative configuration to define services
Supervisor Example
[program:tomcat_8080]
command = /opt/tomcat/bin/catalina.sh run
user = tomcat
directory = /opt/tomcat/
process_name = tomcat_8080
autorestart = true
startsecs = 30
stopwaitsecs = 10
redirect_stderr = false
priority = 600
startretries = 1
stdout_logfile=/var/log/tomcat/tomcat_8080.out.log
stderr_logfile=/var/log/tomcat/tomcat_8080.err.log
stdout_logfile_maxbytes = 50MB
stdout_logfile_backups = 10
stderr_logfile_maxbytes = 50MB
stderr_logfile_backups = 10
environment=CATALINA_HOME=/opt/tomcat,CATALINA_BASE=/opt/tomcat/instances/8080,@ENVIRONMENT_VARS@
Tomcat Parallel Deployment
Tomcat has a great built in parallel deployment engine. Once you turn it on, use it by numbering each war ROOT.war#00001
ROOt.war#00002
Tomcat will deploy the highest number (string sorted)
Pro Super easy
No log file confusion
Doesn’t activate if the new war fails
Cons Not sure when the new war is completely active
Some applications don’t play nice (memory leaks, or port conflicts)
Amazon SDK
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.9.4</version>
</dependency>
http://aws.amazon.com/sdk-for-java/