OpenStack HorizonControlling the Cloud using Django

LA Django Meetup, February 18, 2014

David LapsleyOpenStack Engineer, @metacloudinc

@devlaps, dlapsley@metacloud.com

About the Presenter

• David is a Lead OpenStack Software Engineer at Metacloud• David has been using Python for over 10 years and Django for 5 years

• Education:– Ph. D in Electrical and Electronics Engineering from the University of Melbourne– B. Sc. and B. E. from Monash University.

• Personal Stuff:– Originally from Melbourne, Australia. Now based in LA (via Boston). In his spare time, he

enjoys spending time with his family, cycling, and not shoveling snow

Our Agenda

• Introduction to Cloud Computing and OpenStack

• OpenStack Horizon– Controlling the Cloud with Django– Interesting Patterns– Example– Contributing to Horizon– Challenges and Future Directions

OpenStackLinux for the Cloud

Cloud Computing in Action

Creating a virtual server on a Public OpenStack Cloud

Cloud Computing in Action

Cloud Computing in Action

Cloud Computing in Action

Cloud Computing in Action

Cloud Computing in Action

Cloud Computing Model

Virtualization

Cloud Computing

“Cloud computing is a model for enabling convenient, on-demand network access to a

shared pool of configurable computing resources (e.g., networks, servers, storage,

applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider

interaction.”http://www.nist.gov/itl/cloud/

OpenStack

• Open Source Cloud Platform– Build public/private clouds– Multi-tenant– Virtual machines on demand– Storage volumes

• Founded in 2010 by Rackspace and NASA• Since then, enormous growth…

Some interesting OpenStack facts …

OpenStack Source Code

OpenStack Source Code

OpenStack Contributors

14, 175 People132 Countries

http://www.openstack.org

“Linux for the Cloud”

OpenStack Projects

• Nova (Compute)– Virtual servers on demand

• Nova (Network)– Manages virtual network resources

• VM Registration (Glance)– Catalog and manage server images

• Identity (Keystone)– Unified authentication and authorization

OpenStack Projects

• Object Storage (Swift)– Secure, reliable object storage

• Block Storage (Cinder)– Persistent block storage to VMs

• Dashboard (Horizon)– Web-based UI for all OpenStack Services

– Many, many, more: Heat, Neutron, Ceilometer, Reddwarf, ..

OpenStack HorizonControlling the Cloud with Django

Horizon Overview

• Django-based application that provides access to OpenStack services

• Typically deployed as an Apache WSGI application

• Leverages well known existing technologies– Bootstrap, jQuery, Underscore.js, AngularJS, D3.js,

Rickshaw, LESS CSS• Extends Django stack to enhance application

extensibility

Django Stack

Horizon Stack

Horizon UI Structure (logical)

Sidebar

Panel Group

Panel

User Info

Panel Content

Branding

Dashboard

Horizon UI Structure (logical)

Project Dropdown

Project Dashboard

Horizon UI Structure (CSS)

Horizon Base Demo

Admin Overview

Project Overview

Launching an Instance

Horizon Base Demo

Horizon Base Demo

Horizon Base Demo

Instance List

Filtering

Sorting

Horizon Base Demo

Row Actions

Table Actions

Table Actions

Table Actions

Instance Details

Instance Details

Instance Console

OpenStack HorizonInteresting Patterns

Dashboards and Panels

• Horizon provides a flexible framework for creating Dashboards and Panels

• Panels are grouped into PanelGroups• PanelGroups into Dashboards

Dashboard App

• Dashboards are created as Django Applications

• Dashboard modules partitioned into:– static/

• Static media (css, js, img)

– templates/• Django templates

– python modules:• dashboard.py module which includes the class used by

Horizon

‣openstack_dashboard‣dashboards

‣admin‣ladjango

‣static‣ladjango

‣css‣img‣js

‣templates‣ladjango__init__.pydashboard.py

Dashboard Directory Structure

INSTALLED_APPS = ( ... 'horizon', 'openstack_dashboard.dashboards.project', 'openstack_dashboard.dashboards.admin', 'openstack_dashboard.dashboards.metacloud', 'openstack_dashboard.dashboards.settings', 'openstack_dashboard.dashboards.ladjango', ...)

settings.py

from django.utils.translation import ugettext_lazy as _ import horizon class BasePanelGroup(horizon.PanelGroup): slug = "overview" name = _("Overview") panels = (”hypervisors",) class LADjango(horizon.Dashboard): name = _("LA Django") slug = "ladjango" panels = (BasePanelGroup,) default_panel = "hypervisors" roles = ("admin",) horizon.register(LADjango)

dashboard.py

Starting Point

LA Django Dashboard

Dashboard Tab

Panel Group

Panel

• Panels are created as Python Modules• Panel modules partitioned into:

– static/• Static media (css, js, img)

– templates/• Django templates

– python modules:• urls.py, views.py, panel.py• tables.py, forms.py, tabs.py, tests.py

‣ladjango‣ hypervisors__init__.pypanel.pyurls.pyviews.pytests.pytables.py...

‣ static‣ ladjango

‣ hypervisors‣ css‣ img‣ js

‣ templates‣ ladjango

‣ hypervisorsindex.html...

__init__.pydashboard.py

Panel Directory Structure

from django.utils.translation import ugettext_lazy as _ import horizon from openstack_dashboard.dashboards.ladjango import dashboard class Hypervisors(horizon.Panel): name = _(”Hypervisors") slug = 'hypervisors' dashboard.LADjango.register(Hypervisors)

panel.py

LA Django Dashboard

Panel Nav Entry

View Module

• View module ties together everything– Tables– Templates– API Calls

• Horizon base views:– APIView, LoginView, MultiTableView,

DataTableView, MixedDataTableView, TabView, TabbedTableView, WorkflowView

views.py

from openstack_dashboard import api

from openstack_dashboard.dashboards.ladjango.hypervisors \

import tables as hypervisor_tables

class HypervisorsIndexView(tables.DataTableView):

table_class = hypervisor_tables.AdminHypervisorsTable

template_name = 'ladjango/hypervisors/index.html’

def get_data(self):

hypervisors = []

states = {}

hypervisors = api.nova.hypervisor_list(self.request)

for state in api.nova.service_list(self.request):

if state.binary == 'nova-compute':

states[state.host] = {'state': state.state,

'status': state.status}

for h in hypervisors:

h.service.update(states[getattr(h, h.NAME_ATTR)])

return hypervisors

Table Module

• Table classes provide framework for creating tables with:– consistent look and feel– configurable table_actions row– configurable row_actions– select/multi-select column– sorting– pagination

• Functionality is split server- and client-side, however implementation is all server-side

tables.py

class HypervisorsFilterAction(tables.FilterAction):

def filter(self, table, hypervisors, filter_string):

"""Naive case-insensitive search."""

q = filter_string.lower()

return [hypervisor for hypervisor in hypervisors

if q in hypervisor.name.lower()]

class EnableAction(tables.BatchAction):

...

class DisableAction(tables.BatchAction):

name = 'disable'

classes = ('btn-danger',)

def allowed(self, request, hypervisor):

return hypervisor.service.get('status') == 'enabled'

def action(self, request, obj_id):

hypervisor = api.nova.hypervisor_get(request, obj_id)

host = getattr(hypervisor, hypervisor.NAME_ATTR)

return api.nova.service_disable(request, host, 'nova-compute')

tables.py

def search_link(x):

return "/admin/instances?q={0}".format(x.hypervisor_hostname)

class AdminHypervisorsTable(tables.DataTable):

hypervisor_hostname = tables.Column(

"hypervisor_hostname", verbose_name=_("Hostname"))

state = tables.Column(

lambda hyp: hyp.service.get('state', _("UNKNOWN")).title(),

verbose_name=_("State"))

running_vms = tables.Column(

"running_vms",

link=search_link,

verbose_name=_("Instances"))

...

class Meta:

name = "hypervisors"

verbose_name = _("Hypervisors")

table_actions = (HypervisorsFilterAction, )

row_actions = (EnableAction, DisableAction)

Template

• Standard Django template format• Typically leverage base horizon templates (e.g.

base.html)

index.html

{% extends 'base.html' %}

{% load i18n horizon humanize sizeformat %}

{% block title %}{% trans "Hypervisors" %}{% endblock %}

{% block page_header %}

{% include "horizon/common/_page_header.html" with title=_("All Hypervisors") %}

{% endblock page_header %}

{% block main %}

{{ table.render }}

{% endblock %}

URLs Modules

• Provides URL to View mappings

from django.conf.urls import patterns from django.conf.urls import url from openstack_dashboard.dashboards.ladjango.hypervisors\

import views urlpatterns = patterns( 'openstack_dashboard.dashboards.ladjango.hypervisors.views' url(r'^$', views.IndexView.as_view(), name='index'),)

urls.py

Completed Dashboard!

Nav Entries

Column Sorting

Table Action

State Aware Row ActionsPanel Rendering

Data Retrieval RPC

Linking

Click through to Instances

Authentication

• Keystone manages all Authentication for OpenStack

• To access an OpenStack service:– authenticate with Keystone– Obtain a TOKEN– Use TOKEN for transactions with OpenStack

service• Horizon passes all Auth requests to Keystone

via CUSTOM_BACKENDS

class MetacloudKeystoneBackend(KeystoneBackend):

def authenticate(self, request=None, username=None, password=None, user_domain_name=None, auth_url=None):

keystone_client = get_keystone_client() client = keystone_client.Client( user_domain_name=user_domain_name, username=username, password=password, auth_url=auth_url, insecure=insecure, cacert=ca_cert, debug=settings.DEBUG) # auth_ref gets assigned here…

# If we made it here we succeeded. Create our User! user = create_user_from_token( request, Token(auth_ref)) request.user = user

return user

backend.py

Customization Hooks

• Change Site Title, Logo, Brand Links• Modify Dashboards and Panels• Change Button Styles• Use Custom Stylesheets• Use Custom Javascript

Custom Overrides Module

• For site-wide customization, Horizon enables you to define a python module that will be loaded after Horizon Site has been configured

• Customizations can include:– Registering or unregistering panels from an

existing dashboard– Modifying dashboard or panel attributes– Moving panels between dashboards– Modifying attributes of existing UI elements

HORIZON_CONFIG = {

...

'customization_module':

'openstack_dashboard.dashboards.ladjango.overrides',

'test_enabled': True,

}

local_settings.py

overrides.py

from openstack_dashboard.dashboards.ladjango.test import panel as \

test_panel

from openstack_dashboard.dashboards.ladjango import dashboard \

as ladjango_dashboard

from django.conf import settings

import horizon

LADJANGO_DASHBOARD_SETTINGS = horizon.get_dashboard('ladjango')

if settings.HORIZON_CONFIG.get('test_enabled'):

LADJANGO_DASHBOARD_SETTINGS.register(test_panel.Tests)

ladjango_dashboard.BasePanels.panels += ('ui', )

Full LA Django Dashboard

Test Panel

Custom CSS and Javascript

• Horizon templates provides blocks for custom CSS and Javascript

• To add custom CSS/JS, can either extend existing templates, or replace with your own custom templates

<!DOCTYPE html>

<html>

<head>

<title>{% block title %}{% endblock %} - {% site_branding %}</title>

{% block css %}

{% include "_stylesheets.html" %}

{% endblock %}

. . .

</head>

<body id="{% block body_id %}{% endblock %}">

{% block content %}

. . .

{% endblock %}

<div id="footer”>{% block footer %}{% endblock %}</div>

{% block js %}

{% include "horizon/_scripts.html" %}

{% endblock %}

</body>

</html>

base.html

index.html

{% extends 'base.html' %}

{% load i18n %}

{% block title %}{% trans "Volumes" %}{% endblock %}

{% block css %}

{% include "ladjango/_stylesheets.html" %}

{% endblock %}

{% block page_header %}

{% include "horizon/common/_page_header.html" with title=_("Volumes") %}

{% endblock page_header %}

{% block main %}

<div id="volumes">{{ volumes_table.render }}</div>

<div id="volume-types">{{ volume_types_table.render }}</div>

{% endblock %}

{% block js%}

{% include "ladjango/_scripts.html" %}

{% endblock %}

Horizon Base View

Custom View

About Metacloud

Metacloud

Takes the best of OpenStack and enhances it to deliver a fully scalable, highly

available, and customizable cloud platform.

Metacloud OpenStack

• High Availability

• Scalability• Patches/Fixes• Enhanced UI

Cloud Operations

• Install• Monitor• Upgrade

Services

• On-Prem Private Cloud

• Hosted Private Cloud

Metacloud

Metacloud

• Community Support• Contribute to OpenStack as much as possible

– Bug fixes, features– Help out with code reviews, documentation– Participate in design summits

OpenStack HorizonContributing

Devstack and Contributing

• Devstack:– “A documented shell script to build complete

OpenStack development environments.”– http://devstack.org

• Contributing to Horizon:– http://docs.openstack.org/developer/horizon/

contributing.html

Challenges and Future Directions

Challenges

• Bugs!• Performance• Evolving Architecture in a Smooth Manner

Future Directions

• Evolving the client: AngularJS• Search/Data Service Model

Lastly…

References

• IRC channels (freenode.net)– #openstack-dev– #openstack-horizon– #openstack-meeting

• Web:– http://docs.openstack.org/developer/horizon/– https://launchpad.net/horizon– http://devstack.org– https://github.com/openstack– https://github.com/openstack/horizon– http://docs.openstack.org/developer/horizon/– http://docs.openstack.org/developer/horizon/topics/settings.html– https://wiki.openstack.org/wiki/Gerrit_Workflow

References

• Web:– http://www.stackalytics.com– http://activity.openstack.org/dash/browser/– http://gabrielhurley.github.io/slides/openstack/

building_on_horizon/– http://www.solinea.com/blog/openstack-grizzly-

architecture-revisited

For more information visit:

metacloud.com

Thank You

dlapsley@metacloud.com

@devlaps