Embed Size (px)
Citation preview
Building Service Platforms using OpenStack and CEPH Malte Dreyer
Founded in 1810 Faculties: 9 Students: 35 000 (5 000 International) Degree Programs: 185 Professorships: 419(483)
Academic Staff: 2 000 Non-Academic: 1 500 3 Campuses, ~180 Buildings
Humboldt-Universität zu Berlin
Computer and Media Service
Broad Range of Services
Hardware Service
Identity Management
Image Editing
Mail Service
Mailing Lists
Map Service
Media Repository
Moodle
Printing Service
Public Computers
Publication Repository
Resource Management SW
Scanning Service
Shibboleth
Software Service
Statistics SW Service
Storage Area Network
Survey Service
Terminalserver Service
Repositories
Training
Trouble Ticket Systems
User Certificates
User Support
VPN
Video Conferences
Video Service
WLAN
Web Content Management
Web Files
Web Hosting
Website Management
Wiki and Blog Farms
Windows Domains
ePublishing
ADS
AFS
Accounts
Anti Virus Software
Audio / Video Equipment
Backup-Service
Calendar Service
Campus Management SW
Chat Service
Compute Clusters
Computer Networks
Computer Training Rooms
Database Service
Managed Desktops
File Exchange
File Service
Graphics Service
Customizations and
feature developments
Media Repository Farm
Joint projects and
software developments
Laudatio Corpus Repository
Projects in Close Cooperation with Scholars
"Products of research are not just publications.”
NSF senior policy specialist Beth Strausser.
Biographical Sketch(es), has been revised to rename the “Publications”
section to “Products” and amend terminology and instructions accordingly.
13 January 2013: "National Science Foundation’s Merit Review Criteria: Review and Revisions” Chapter II.C.2.f(i)(c),
1. Recommended open access to scholarly papers of
publicly funded research
2. Recommended open access to all digital outputs of
publicly funded research
3. Mandated open access to scholarly papers of publicly
funded research
4. Mandated open access to all digital outputs of
publicly funded research
5. Enforced, mandated open access to scholarly papers
of publicly funded research
6. Enforced, mandated open access to all digital
outputs of publicly funded research
80 individual projects
80 setups
80 databases
80 configs
Custom software
Fedora Repository
CakePHP
Elastic Search
PID-Server (Handle)
jQuery / Bootstrap
Apache Xerces
+RedMine for Tickets
From a Data Center Point of View
Cloud as e-Infrastructure
Supporting Research
Infrastructure Servers–Storage-
Network
Infrastructure Servers–Storage-
Network
Infrastructure Servers–Storage-
Network
Platform OS & Basic Services
Platform OS & Basic Services
Applications Applications and Data
IaaS
PaaS
SaaS
XaaS/EaaS: Software-led Infrastructure
(Software Defined Data Center)
Backup
Database
Identity
Monitoring
Network
Security
Storage
…………………. as a Service
Media Repository
Seafile
Repository xyz
…………………. as a Service
Flexible e-Infrastructure…
reduce technology stack for project staff
concentrate on what you are really interested in
keep services alive
beyond project lifetime
easy testing
environments
We Took a Look At…
(Being aware that these are different in type)
• amazon web services
• Apache Cloudstack
• Docker / LXC
• Eucalyptus
• Microsoft Azure
• OpenNebula
• OpenStack
• VMware
And still use Proxmox for VM management
for many services
… and Chose OpenStack
still under heavy development
Macro Scripting
„Manage VMs like a Linux Process“
Pet or Cattle
Images: Sheep: Public Domain; Toy Dog: By Toyloverz (Own work) [CC BY-SA
3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
OpenStack Component Architecture
• Preise [TEUR]
2016 2017 2018
WDM (Licht) 200 200
Linecards 10 Gbps x20
200
MLX (Router) Chassis 8x
175 175
Linecards (26) 10Gbps x8
286 286
Summe 661 661 200
CEPH Server SAN
Figure to the left: By Inktank, CC-BY-SA
http://ceph.com/docs/master/architecture/
0
200
400
600
800
1000
1200
0
20
40
60
80
100
120
140
160
1 2 3 4 5 6 7 8
Aggregated MB/s
MB/s
Number of instances
Read from disk
Read from cache
Aggregated readfrom disk
Aggregated readfrom cache
Seafile: Deployment in OpenStack
Using Ansible and Heat
HEAT
my_port01:
type: OS::Neutron::Port
properties:
admin_state_up: true
name: Port01
network_id: { get_resource: my_first_network }
my_server07:
type: OS::Nova::Server
properties:
name: seaf07
key_name: { get_param: keypair_name }
flavor: { get_param: machine_flavor }
image: { get_param: image_id }
networks: [{ "port": { get_resource: my_port07 }, …}]
user_data: |
#!/bin/bash -v
….
cinder_volume07:
type: OS::Cinder::Volume
properties:
size: { get_param: volume_size }
volume_attachment07:
type: OS::Cinder::VolumeAttachment
properties:
volume_id: { get_resource: cinder_volume07 }
instance_uuid: { get_resource: my_server07 }
mountpoint: /dev/vdb
ANSIBLE Playbook
# update packet cache
- hosts: all
tasks:
- apt:
- name: apply common configuration to all nodes
hosts: all
roles:
- common
- name: deploy and configure memcached server
hosts: memcached
roles:
- memcached
- name: deploy MySQL and configure the databases
hosts: db
roles:
- db
- name: deploy seafile and apply configuration
hosts: nodes
roles:
- nodes
- name: deploy loadbalancer
hosts: loadbalancer
roles:
- loadbalancer
tags:
- loadbalancer
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Organizational Responsibilities
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Organizational Responsibilities /1
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Organizational Responsibilities /2
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Organizational Responsibilities /3
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Organizational Responsibilities /4
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Organizational Responsibilities /5
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Organizational Responsibilities /6
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Organizational Responsibilities: New Distributed Competency
Build new competencies
• OpenStack is complex and additional technologies are needed
• An advanced training program
Changing system administration styles
• Develop vs. configure manually step by step
Re-Use what others built already (and share again)
• Read scripts from other people
Be patient when OpenStack is still immature, or Neutron fails again
• So far it develops/improves very fast
• Dogfooding builds trust
Organizational Challenges and Conclusions
