Upload
carlos-ferreira
View
789
Download
3
Embed Size (px)
DESCRIPTION
Bluemix Meetup Cambridge July 22, 2014 Carlos Ferreira will share his experiences moving a Python Django application from Heroku to Bluemix. As part of this workshop you will learn how to: • Determine the right delivery solution for your business problem. • Choose a programming stack and PaaS provider. • Move your app on to Bluemix • Collaborate and continuously deliver with DevOps Services
Citation preview
© 2014 IBM Corporation
Deliver apps faster using Python, Django, Bluemix and DevOps Services
Carlos Ferreira – IBM
22 July 2014
© 2014 IBM Corporation2
Agenda
What is the right delivery solution for your business problem?
Choose a programming stack and PaaS provider
Move your app on to Bluemix
Collaborate and continuously deliver on DevOps Services
Source: If applicable, describe source origin
IBM Presentation Template Full Version
© Copyright IBM Corporation 2014. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
© 2014 IBM Corporation3
What Is the right delivery solution for your business problem?
First understand the business problem and business role user requirements
Method Role Needs
Platform as a service Developers Customized - Upload code and reuse services
Software as a service Business Users Generalized - Access commercial function in utility model
Infrastructure as a service IT Managers Transparent - Provision and manage infrastructure
© 2014 IBM Corporation4
What Is the right delivery solution for your business problem?
Example requirements to consider
Application Category– Mobile -> Android Google Play or Apple IOS plus PaaS for persistence.– Web App– “Systems of Engagement” -> PaaS– “Systems of Record” Depends PaaS, SaaS or IaaS
Security– Data privacy concerns?
• In country• In company, extranet and internet
– Collaborating with partners and clients?• SSO Authentication• Authorization
Ecosystem - Are the right services available? Is there vendor lock-in?– Public/private, managed/unmanaged or both– PaaS Business advantage – Licensing– Dev time, runtime and operational
Agility / Flexibility / Scalability
Model cost at initial and full build out state and include on going maintenance costs– Including services– Development, test and production environments
Consider the complete lifecycyle for your requirements.– Develop/Build, Run and Manage – Try before you buy
© 2014 IBM Corporation5
What Is the right delivery solution for your business problem?
For chosen delivery method chose: – public/private– managed/unmanaged– both
PaaS allows developers to develop applications and upload source code to PaaS where it is complied and run including account and application specific resource requirements
PaaS provides dynamic containers to compile, build, and run applications– Bluemix http://bluemix.net/– CloudFoundry - Open source http://cloudfoundry.org/– Heroku
Selecting a PaaS http://www.slideshare.net/altoros/slides-how-to-select-a-paas
© 2014 IBM Corporation6
Agenda
Is PaaS the right solution for your business problem?
Choose a programming stack, identify services and PaaS provider
Move your app on to Bluemix
Collaborate and continuously deliver on DevOps Services
Source: If applicable, describe source origin
IBM Presentation Template Full Version
© 2014 IBM Corporation7
Choosing and planning a stack – Use Case Web App
Identify:
Providing Application Services
Consuming Application Services
Application Framework and or Pattern
Manage Services
Runtime Services
DevOps Services
PaaS Provider ( Bluemix )PaaS Provider
Runtime Services
Application Framework
Other Providers
ApplicationServices
DevOps Services
ApplicationServices
© 2014 IBM Corporation8
Identify - Application Frameworks, components and services
Choose an actively supported Ecosystem
http://www.ohloh.net/ Check out stacks
Language: Python
Application Framework: Django- MVC Web Application framework
© 2014 IBM Corporation9
Identify - Runtime Boilerplates are known patterns and stacks
Python is not listed as a runtime but does work with Bluemix
Use Buildpacks to use other runtimes not listed under boilerplates
Python works on Bluemix with these Buildpacks: – https://github.com/ephoning/heroku-buildpack-python – https://github.com/cloudfoundry/python-buildpack
© 2014 IBM Corporation10
Identify Services – Runtime to Application Components
Runtime Process / Service Application Component Provides
Web Container gunicorn==18.0
Django==1.6.2
dj-static==0.0.5
Serves Django pages and static content
PostgreSQL Database django-toolbelt==0.0.1
psycopg2==2.5.2
Data persistence and queries
Rabbit MQ + transactions via separate command-line process
amqp==1.4.5
celery==3.1.11
django-celery==3.1.10
Transactions
Solr* via separate command-line Tomcat process
pysolr==3.2.0 Indexing and search
Via Add On SendGrid Mail services for registration/activation and password reset etc.
*Applications that aren’t available as services Bluemix can be ported as command line process with corresponding buildpacks
© 2014 IBM Corporation11
Choosing and planning a stack – Use Case Web App
Identify:
Providing Application Services
Consuming Application Services
Application Framework and or Pattern
Manage Services
Runtime Services
DevOps Services
PaaS Provider ( Bluemix )PaaS Provider
Runtime Services
Application Framework
Other Providers
ApplicationServices
DevOps Services
ApplicationServices
© 2014 IBM Corporation12
Move your app on to Bluemix
Allows for app portability to Bluemix from other PaaS providers– Vendors that support Cloudfoundry– Heroku and Bluemix both use Buildpacks as supporting technology
Take advantage of this commonality for development and deployment and avoiding vendor lock in
Getting Started with Bluemix: https://www.ng.bluemix.net/docs/#starters/index.html
Django pre-requisites still apply:– https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/gunicorn/
© 2014 IBM Corporation13
Move your app on to Bluemix - Getting Started
Get an account
Access bluemix Web site here:
https://ace.ng.bluemix.net/
Click on the “Sign Up” for Bluemix button.
Read quick start
http://www.ng.bluemix.net/docs/QuickStart.jsp
© 2014 IBM Corporation14
Move your app on to Bluemix - Getting Started
Installing the cf tool to do “cf” commands instead of “heroku” commands
Before you begin, install the command line tool that is used to upload and manage your application. BlueMix uses a tool called cf command line interface.
Download cf command line interface from https://github.com/cloudfoundry/cli
The cf command is a program that you need to install. Select the appropriate installer for your operating system. After the cf command tool is installed, you can use cf -v command to verify its version as follows:
$ cf -v
cf version 6.0.0-90db382
© 2014 IBM Corporation15
Move your app on to Bluemix - Getting Started
Login
Login. Be sure to use the right endpoint.
$cf login [-a API_URL] [-u USERNAME] [-p PASSWORD] [-o ORG] [-s SPACE]
To see the end point type:
$ cf –t
API endpoint: https://api.ng.bluemix.net (API version: 2.2.0)
Not logged in. Use 'cf login' to log in.
I had to reset mine to the from the beta site.
$ cf api https://api.ng.bluemix.net
© 2014 IBM Corporation16
Move your app on to Bluemix - Getting Started
Get help getting answers to your questions
For Bluemix and DevOps services go to IBM.com forums https://developer.ibm.com/answers/questions/10523/python-run-time-environment-availability/?community=bluemix
For Python questions use https://Stackoverflow.com
Cloudfoundry command line help: $cf help
© 2014 IBM Corporation17
Move your app on to Bluemix - Getting Started
Generate your requirements.txt file
Identifies the services and components required by the application
Python BuildPack expects this file
$pip freeze > requirements.txt
httplib2==0.8kombu==3.0.15oauth2==1.5.211oauthlib==0.6.1pika==0.9.13psycopg2==2.5.2pysolr==3.2.0pystache==0.5.4python-openid==2.2.5python-social-auth==0.1.24python-twitter==1.3.1pytz==2014.3requests==2.3.0requests-oauthlib==0.4.0simplejson==3.5.2six==1.6.1static==1.0.2
Django==1.6.2amqp==1.4.5anyjson==0.3.3autoenv==1.0.0billiard==3.3.0.17celery==3.1.11dj-database-url==0.3.0dj-static==0.0.5django-celery==3.1.10django-pagination==1.0.7django-registration==1.0django-taggit==0.11.2django-toolbelt==0.0.1gunicorn==18.0
© 2014 IBM Corporation18
Move your app on to Bluemix- manifest.yml
Contains important settings for deploying the application, the host machine, the services that the application uses, number of instances, memory, routes and more.
Learn more about deploying using manifest.yml http://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#services-block.
Things to look out for– Manifest.yml syntax.
• Use this guide: http://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#services-block.
• Get syntax right by validating manifest YAML file format:http://yaml-online-parser.appspot.com/
• Use this Bluemix app by Ryan Baxter to generate a manifest file: http://cfmanigen.mybluemix.net
– Precedence Between Manifests, Command line Options, and Most Recent Values • Covered later
© 2014 IBM Corporation19
Move your app on to Bluemix - Procfile, deployment manifest.yml or -c
Two ways you can run python django
A. Using development server using python manage.py runserver
or script that includes it. Example run.sh command:#!/bin/bash#echo STARTINGD RUN.SH FOR STARTING DJANGO SERVER PORT IS $PORT##echo ENVIRONMENT VAIRABLES IN RUN.SH#printenv if [ -z "$VCAP_APP_PORT" ]; then SERVER_PORT=5000; else SERVER_PORT="$VCAP_APP_PORT";fiecho [$0] port is------------------- $SERVER_PORTpython manage.py syncdb --noinputecho "from django.contrib.auth.models
import User; User.objects.create_superuser('admin', '[email protected]', 'admin')" | python manage.py shell
echo [$0] Starting Django Server...python manage.py runserver 0.0.0.0:$SERVER_PORT --noreload
Not recommended for production environments
© 2014 IBM Corporation20
Move your app on to Bluemix - Procfile, deployment manifest.yml or -c
A. Using development server using python manage.py runserver
Example manifest:applications:- name: firstdjango memory: 256M command: bash ./run.sh buildpack: https://github.com/cloudfoundry/python-buildpack path: . services: - postgresql-django
© 2014 IBM Corporation21
Move your app on to Bluemix - Procfile, deployment manifest.yml or -c
A. Using development server using python manage.py runserver
Example requirements.txt:Django==1.6.2dj-database-url==0.3.0psycopg2==2.5.2
© 2014 IBM Corporation22
Move your app on to Bluemix - Procfile, deployment manifest.yml or -c
B. Using Gunicorn and wsgi. Using push command python ./wsgi.py
Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX http://gunicorn.org/
Example manifest fileapplications:- name: simpledjango memory: 256M command: python ./wsgi.py buildpack: https://github.com/ephoning/heroku-buildpack-python path: "." services: - postgresql-9.1
© 2014 IBM Corporation23
Move your app on to Bluemix - Procfile, deployment manifest.yml or -c
B. Using Gunicorn and wsgi.
Gunicorn requires you specify a procfile. Like this:
Using push command python ./wsgi
web: gunicorn app_name.wsgi --workers 2
© 2014 IBM Corporation24
Move your app on to Bluemix - Procfile, deployment manifest.yml or -c
B. Using Gunicorn and wsgi.
Example requirements.txt:Django==1.6.2dj-database-url==0.3.0psycopg2==2.5.2dj-static==0.0.5django-toolbelt==0.0.1gunicorn==18.0static==1.0.2wsgiref==0.1.2
© 2014 IBM Corporation25
Move your app on to Bluemix – Django example multiple push sequence
1. First create database service and push web app without the db setup command. It deploys successfully.
$cf push app_name -m 256M -b https://github.com/ephoning/heroku-buildpack-python.git
2. Then bind my app to it.
© 2014 IBM Corporation26
Move your app on to Bluemix – Django example push sequence
3. Now restart the app with command to populate DB. Now that you have a connection to it. Instead stop the app and then redeploy using the cf push command and this time append the -c "setup_db.sh" option.
$cf push app_name -m 256M -b https://github.com/ephoning/heroku-buildpack-python.git -c "bash ./setup_db.sh"
setup_db.sh#!/bin/bashecho "Trying----------------------------- to synch"python manage.py syncdb --noinputecho "Finish manage.py synchdb ----------------------------"echo "from django.contrib.auth.models import User;
User.objects.create_superuser('carlos', '[email protected]', 'carlos')" | python manage.py shell
echo "Finish creating super user ----------------------------"
4. Now restart the app with command to run.sh without create users cf push command and this time append the -c “bash ./run.sh" option.
© 2014 IBM Corporation27
Move your app on to Bluemix – access console on Bluemix
These haven’t been verified and your mileage may vary depending on runtime / buildpack you are using
– https://developer.ibm.com/answers/questions/11152/how-can-i-run-rake-mirgatedb-with-ruby-on-rail-bluemix/
– https://www.ng.bluemix.net/docs/#cli/index.html#cfcommands Look at the section on Retrieving environment variables and using a WebSocket console driver.
– https://developer.ibm.com/bluemix/2014/07/16/easy-ssh-sessions-bluemix/
Cloudfoundry doesn’t provide a way of accessing the consoleOther options available to synch DB and create super user
© 2014 IBM Corporation28
Move your app on to Bluemix – Troubleshooting
Files and logs – Only accessible if you successfully deploy
$cf –logs app_name from command line window to see what is happening before then
© 2014 IBM Corporation29
Move your app on to Bluemix – Things to keep in mind
Using environment variables to access databases and other service
File system is transient
Deployment tasks starting apps order is important
Multi-scaling don’t expect second request to be persisted in memory
© 2014 IBM Corporation30
Move your app on to Bluemix – Set environment variables
Use environment variables to reflect the differences in running you app from dev, test and production environments.
– For things like database, accounts etc.
$cf set-env app_name name value
© 2014 IBM Corporation31
Move your app on to Bluemix – Using environment variables to access databases
Django .settings file
Set local client environment variable to do local client testing:
postgres://userid:password@localhost:5432/test
On bluemix to see what your environment variables are set to
$cf logs your_app /logs/env.log
© 2014 IBM Corporation32
Move your app on to Bluemix – File system is transient
Django .settings file
Local file system storage is short-lived. When an application instance crashes or stops, the resources assigned to that instance are reclaimed by the platform
Instances of the same application do not share a local file system. Each application instance runs in its own isolated container. Thus a file written by one instance is not visible to other instances of the same application.
Current Path where App is installed
© 2014 IBM Corporation33
Move your app on to Bluemix – Starting apps order is important
1. The -c command-line option, for example: $ cf push my-app -c "node my-app.js“
2. The command attribute in the application manifest, for example: command: node my-app.js
3. The buildpack, which provides a start command appropriate for a particular type of application.
How cf push Determines its Default Start Command
The first time you deploy an application, cf push uses the buildpack start command by default. After that, cf push defaults to whatever start command was used for the previous push.
To override these defaults, provide the -c option, or the command attribute in the manifest. When you provide start commands both at the command line and in the manifest, cf push ignores the command in the manifest.
Forcing cf push to use the Buildpack Start Command
To force cf to use the buildpack start command, specify a start command of null.
You can specify a null start command in one of two ways.
Using the -c command-line option:
$ cf push my-app -c "null" Using the command attribute in the application manifest:
command: null
Source: Pivotal Web Services
© 2014 IBM Corporation34
Move your app on to Bluemix - Deployment task order is important
Invoke CF push multiple times to deploy your application:
1. Create your DB service before you doing CF push commands.Bind DB service to your application
2. Subsequent db migration
$cf push app_name --no-start -b "https://github.com/ephoning/heroku-buildpack-python.git" -c "python manage.py syncdb --noinput"
Note: After this step the database has been created but the application itself has not started, because the normal start command is not used.
3. To clear out earlier cached commands and start app
$cf push crawl -m 256M -b https://github.com/ephoning/heroku-buildpack-python.git -c "null"
Note: This example assumes that the normal start command for your application is the one provided by the buildpack, which the -c 'null' option forces cf to use.
© 2014 IBM Corporation35
Move your app on to Bluemix - Mutliscaling
HTTP Sessions Not Persisted or Replicated– Cloud Foundry supports session affinity or sticky sessions for incoming HTTP requests
to applications if a jsessionid cookie is used.
See Bluechatter tutorial on architecturally scaling apps on cloud: https://www.youtube.com/watch?v=i7_dQQy40ZQ
Add transactional support to improve performance of long running requests Celery for Django using RabbitMQ
© 2014 IBM Corporation36
Demonstration
Using development server approach using python manage.py runserver– http://goo.gl/wTHV7L
© 2014 IBM Corporation37
Move your app on to Bluemix – Tutorials, Webcast and sample projects
Tutorials on Django– http://nbviewer.ipython.org/github/agconti/Django-IPython-Tutorial/blob/master/
django_tut-official.ipynb
Tutorials and demonstrations used in this presentation– Deploy to Bluemix http://goo.gl/wTHV7L– Setup Eclipse and Using DevOps Services with Bluemix goo.gl/Dn4jdj
Sample projects from this presentation– https://hub.jazz.net/project/carlosf/crawl%202/overview– https://hub.jazz.net/project/carlosf/django/overview
© 2014 IBM Corporation38
Agenda
Is PaaS the right solution for your business problem?
Choose a programming stack and PaaS services provider
Move your app on to Bluemix
Collaborate and continuously deliver on DevOps Services
IBM Presentation Template Full Version
© 2014 IBM Corporation39
IBM DevOps Services
Develop, build, track, plan and deploy your apps to Bluemix
Software configuration management (SCM)̶YStore and manage the code for your applications in the hosted Git repository or
Jazz source control
Integrated development environment ̶YUse built-in Web IDE (Orion), Eclipse, Visual Studio, Rational Team Concert –
or leverage your tool of choice
Agile planning & tracking, team collaboration̶YSeamlessly adopt Agile practices̶YEasily share work and collaborate with team members with task boards and
Dashboards
Continuous Delivery Pipeline̶YBuild automation̶YAutomated deployment to Bluemix
© 2014 IBM Corporation40
40
IBM BluemixDevOps Services
(powered byJazzHub)
Configure build process and auto-deploy
Connect Eclipse client to DevOps services and Team Share or load code
IBM DevOps Services and Bluemix is now ready for continuous delivery. Delivered changes will automatically deploy application to Bluemix.
Setting up continuous delivery environment is easy in IBM Bluemix…
Edit code of app source code
Automatically links to Bluemix SQL
DB
WAS Liberty
Create new IBM DevOps Services project 5
2
1
3
4
CPO Lunch and Learn - IBM Confidential
Alternatively simply CF push to deploy from on local client
A
© 2014 IBM Corporation41
Many ways to get started - Projects already exist in DevOps Services
Projects already exist in DevOps Services
Repository tools use either Git or Jazz SCM:– Working with a Git project http://goo.gl/A6XJo4– Working with a Jazz SCM project http://goo.gl/ckLrRC
© 2014 IBM Corporation42
DevOps Services – Use Jazz SCM
Follow this YouTube tutorial http://goo.gl/Dn4jdj to:– Set up your Eclipse client to with Rational Team Concert – Create a DevOps Project Area– Push your app to the DevOps Project Area– Deploy your app to Bluemix– Collaborate with others by synchronizing your workspace with the changes from others
© 2014 IBM Corporation43
DevOps Services – Collaborate with others
Find other projects that are doing similar things
Collaborate and do development with others on your team – Notifications
© 2014 IBM Corporation44
DevOps Services – Things to keep in mind
Workflow from going from command line push, to deployed app on bluemix, to a project area on JazzHub.
TIP: It doesn’t move your code there you have to push / check in deliver from your client.
Using Eclipse client with RTC pluginTIP: use your jazz.net id with your ibm.com password to connect to DevOps Services on
JazzHub
You have multiple optionsTIP: You can use Git or SCM with DevOps or just use Bluemix by itself
Your client workspace changes you delivered don’t show up in project areaTIP: Make sure you accept changes in your repository workspace in your project area.
© 2014 IBM Corporation45
Innovate and build something cool !!!
Solve your business problem
Choose a programming stack and Bluemix
Collaborate and continuously deliver on DevOps Services
Send me your Bluemix and DevOps feedback at [email protected]
Follow me on Twitter: @carlosTweets
Learn more on my blog: bluemixstirred.wordpress.com
© 2014 IBM Corporation46
https://bluemix.net