THE EVOLUTION OF GLANCE API On the Way f rom v1 to v3

This graphic was selected is show placement and can be replaced with another image.

2

•  @br14nr •  Freenode: rosmaita

•  OpenStack ATC since Folsom

•  Driver for Glance and Searchlight

ABOUT ME

Brian Rosmaita Senior Software Developer, Rackspace

3

GLANCE AT RACKSPACE

(Official Glance Logo)

4

OUTLINE

•  When Nikhil asked me to fill in for him, I took the title too literally

•  The Images v1 API – Not an exhaustive look, just some highlights

–  Including interesting observations

•  The Images v2 API – Also including interesting observations, and also not exhaustive

•  The Future – Just a little foreshadowing for Alex’s presentation

5

IN THE BEGINNING …

There was Nova …

… well, Nova and Swift

6

THE EVOLUTION OF OPENSTACK

•  Stuart McLaren created a really nice diagram of the evolution of OpenStack projects for a talk at the Liberty design summit in Vancouver, and he kindly agreed to let me use it today

•  Powerpoint, however, is unkindly displaying the PDF he gave me as a solid black box, so I will bail out to Acrobat here to show you the diagram

AUSTIN

8

AUSTIN RELEASE

•  October 21, 2010

•  Release of – OpenStack Object Store

– OpenStack Compute

– OpenStack Clients

9

C o r y W r i g h t a n d R i c k H a r r i s h a d o r i g i n a l l y c r e a t e d a s e t o f e n d p o i n t s

c a l l e d T e l l e r a n d P a r a l l a x t h a t e v e n t u a l l y b e c a m e t h e G l a n c e r e g i s t r y

a n d i m a g e s t r e a m i n g s e r v i c e s … T h e y w e r e m o d e l e d a f t e r s i m i l a r i m a g e

s e r v i c e s i n s i d e t h e S l i c e h o s t i n f r a s t r u c t u r e , i f I r e m e m b e r c o r r e c t l y .

Jay Pipes

10

AUSTIN RELEASE

•  Partial implementation of an image registration and caching proxy named Glance

•  Not enabled by default

•  Only used by Nova with the OpenStack API – When Nova uses the EC2 API, the existing S3-like objectstore is used for VM image

storage

•  Known issues –  Images without ramdisks are not supported at this time

– OpenStack Object Store does not support objects larger than 5 GB

11

I s t a r t e d w o r k i n g w i t h C o r y a n d R i c k o n w h a t w o u l d b e c o m e G l a n c e ( f r o m t h e T e l l e r a n d P a r a l l a x c o d e ) i n t h e

A u s t i n t i m e f r a m e . F o r t h e l i f e o f m e , I c a n ’ t r e m e m b e r i f w e h a d a n y t h i n g

c a l l e d a P T L b a c k t h e n , o r e v e n w h e n I b e c a m e G l a n c e ’ s P T L , o r e v e n

w h y ! : ) I t h i n k I j u s t k i n d o f f e l l i n t o t h e s p o t o f P T L f o r G l a n c e a r o u n d t h e

A u s t i n r e l e a s e . Jay Pipes

BEXAR

13

BEXAR RELEASE

Features

•  Glance APIs (for registry and delivery) were unified, and a specific client class created –  Images API version 1.0

•  Support for uploading disk images directly through the glance REST-ful API •  Documentation for Glance is now available

•  The dependency on Twisted was removed. Glance now uses only Eventlet for its server-side internals

14

BEXAR RELEASE

Known issues

•  The S3 and Swift backends do not currently support the POST /images/ API command in the Glance API – These backends only currently support fetching disk images via GET calls.

•  Support for storing disk images in S3 and Swift directly through the Glance API is planned for the Cactus release

CACTUS

16

CACTUS RELEASE

Features

•  Images API continues at version 1.0 •  Introduction of the “glance” CLI tool

•  Introduction of disk_format and container_format metadata

•  Checksum verification for uploaded images

•  “Lots of other changes should go unnoticed” –  like: support for database schema migration, non-static versioning or standard use of

paste.deploy, ConfigParser and optparse

DIABLO

18

DIABLO RELEASE

API Enhancements

•  Versioning in the API – Needed because Images API version 1.1 is released!

– 1.1 becomes CURRENT, 1.0 becomes SUPPORTED

•  Custom ordering of results

•  Results filtering

•  Results limiting and pagination

•  New ISO disk format

19

DIABLO RELEASE

Usability and performance improvements

•  Local filesystem LRU cache •  Delayed deletion of images

•  Integration with Keystone authentication

•  Sharing images between tenants

•  Event notifications

•  Internationalization of error and output messages

ESSEX

21

ESSEX RELEASE

Stabilization and Usability

•  Images API v 1.1 is CURRENT, 1.0 is SUPPORTED (unchanged from Diablo) •  Contributors fixed 185 bugs and implemented 11 blueprints – Role-based access control

–  Introduction of the ‘protected’ image property

– Configurable number of Glance API processes

– Copy image data from external locations

–  Image upload progress bar in the Glance CLI

FOLSOM

23

FOLSOM RELEASE

Features

•  Images API version 2.0 is introduced – 2.0 and 1.1 are CURRENT

– 1.0 is SUPPORTED

•  The python-glanceclient is introduced

•  Client SSL certificate validation on glance-api

•  Tenant-specific storage in Swift

•  Image replication using glance-replicator

24

WHY VERSION 2?

•  Version 1 of the Images API was not designed for human consumption – GET v1/images/UUID returns image data in payload, image properties in headers • Number of of allowed headers can be configured at multiple places in the request chain before

Glance ever sees them

• Various components in the request chain think nothing of monkeying with headers

• GET v2/{tenant_id}/servers/UUID returns a representation of an instance, not the instance itself –  On analogy, GET v1/images/UUID should return a representation of the image

– Architectural considerations: v1 designed to use the Glance Registry (extra layer between the Images API and the DB API)

25

WHY VERSION 2?

•  Version 2 is will include features suitable for making the Images API “public” – Anti-spam image sharing

– Property protections

–  Image “import” vs. image “upload”

•  Architectural considerations – Domain model

– Glance registry becomes optional

GRIZZLY

27

GRIZZLY RELEASE

Features

•  Images API version 2.1 is introduced – 2.1, 1.1 are CURRENT

– 2.0, 1.0 are SUPPORTED

•  API v2 Image Sharing

•  JSON PATCH draft 10 support

•  glance-control status: programmatically expose the status of the glance services

•  glance-manage downgrade: explicitly migrates a database down to the requested version

HAVANA

29

HAVANA RELEASE

Features

•  Images API version 2.2 is introduced – 2.2, 1.1 are CURRENT

– 2.1, 2.0, 1.0 are SUPPORTED

•  Multiple image locations

•  Property protections

•  Registry API

•  Policies on image membership •  Backend storage support: Sheepdog, Cinder, GridFS

ICEHOUSE

31

ICEHOUSE RELEASE

Features

•  Images API versions: no changes •  Add VMware Datastore as Storage Backend

•  Adding image location selection strategy

•  A new field ‘virtual_size’ is added for image

•  API message localization

•  The calculation of storage quotas has been improved •  0-based indices for location entries (per JSON-pointer RFC6901)

JUNO

33

JUNO RELEASE

Features

•  Images API versions: no changes •  Asynchronous Processing – v2/tasks endpoint introduced

•  Pull of glance.store into its own library •  Metadata Definitions Catalog – v2/metadefs endpoint introduced

•  Restricted policy for downloading images

34

JUNO RELEASE

Mission statement change

•  “original” mission statement – Merged June 11, 2014

– https://review.openstack.org/#/c/98001/

•  To provide services to store, browse, share, distribute and manage bootable disk images

35

JUNO RELEASE

Mission statement change

•  Merged July 10, 2014 – https://review.openstack.org/#/c/98002/

•  To provide a service where users can upload and discover data assets that are meant to be used with other services, like images for Nova and templates for Heat.

KILO

37

KILO RELEASE

Features

•  Enhanced Image import process – Basic support for Image conversion, Image Introspection

•  Reload configuration files on SIGHUP signal

•  Metadefs: – Tag support, Support for multivalue operators, Software Metadata Definitions

•  Allow None values to be returned from the API

•  Digest algorithm is now configurable

38

KILO RELEASE

API changes

•  Image “deactivation” (2 new API calls) •  Images API versions – 2.3 introduced as CURRENT

– 1.1 becomes SUPPORTED

– 1.0, 2.0, 2.1, 2.2 continue as SUPPORTED

•  Catalog Index Service experimental API

39

KILO RELEASE

API changes

•  Downgrading v1 to SUPPORTED is a step toward the deprecation and ultimate elimination of the v1 API

LIBERTY

41

LIBERT Y RELEASE

Features

•  Added support for uploading signed images •  Scrubbing of images in parallel is now possible

•  The health of a Glance node can be monitored using the healthcheck middleware

•  python-glanceclient now defaults to using Images API v2

•  Catalog Index Service becomes the “Searchlight” project

42

T h e c u r r e n t G l a n c e ( v 1 a n d v 2 ) a r c h i t e c t u r e t r i e s t o h a n d l e t h r e e

d o m a i n s n a m e l y m e t a d a t a r e g i s t r y , i m a g e f o r m a t , a n d i n t e r f a c e t o s t o r e

i m a g e d a t a - b l o b s . I t b e c o m e s i n e f f e c t i v e a t h a n d l i n g e a c h o f t h e m

a t l a r g e s c a l e m a i n l y a s t h e s e p a r a t i o n o f c o n c e r n i s w e a k .

Nikhil Komawar

43

I f i n d a n i n d e x i n g m e c h a n i s m b e t t e r a t h a n d l i n g l a r g e s c a l e i m a g e &

r e l a t e d m e t a d a t a l i k e t h e p r o p e r t i e s , t a g s , a n d m e m b e r s i n f o ; a d e d i c a t e d t r a n s f e r l o g i c t o b e t t e r h a n d l e l a r g e s c a l e d a t a b l o b m o v e m e n t ; a n d s o m e a s y n c h r o n o u s w a y t o c l e a n l y h a n d l e

t h e i m a g e i n t e r - o p e r a b l e f o r m a t t i n g .

Nikhil Komawar

44

I n K i l o a n d L i b e r t y , I h a v e a t t e m p t e d t o p r o v i d e a p l a t f o r m t o d e v e l o p e r s ,

o p e r a t o r s a n d p r o d u c t m a n a g e r s a l i k e t o b r i n g G l a n c e t o a s t a t e t h a t w o u l d

h a n d l e e a c h o f t h e s e d o m a i n s i n a b e t t e r m a n n e r .

Nikhil Komawar

45

LIBERT Y RELEASE

API changes

•  The Artifacts API is available for use •  Artifacts API is exposed as v3 on the Glance endpoint – 3.0 is EXPERIMENTAL

– 2.3 is CURRENT

– 2.2, 2.1, 2.0, 1.1, 1.0 are SUPPORTED

46

WHY VERSION 3?

•  Generalization of Glance from cataloging and delivery of VM images to cataloging and delivery of “digital artifacts”

•  Some of the shortcomings of Glance Images are addressed by artifacts –  Images are unversioned, artifacts are versioned

–  Images are independent, artifacts can have dependency relations

–  Images have a single payload

•  An artifact cataloging and delivery project would duplicate a lot of Glance functionality

MITAKA

48

WHAT ABOUT MITAKA?

•  That’s what we’re here in Tokyo to decide! – Check your conference schedule for the Glance design sessions, pretty much running all

day tomorrow

•  Post summit: – You can see what’s been approved for development at specs.openstack.org

– For items still under discussion, go to review.openstack.org • Search for status:open project:openstack/glance-specs

– There will be a “priorities” document among the specs shortly after the summit • https://review.openstack.org/#/c/230392

Leadership team was selected to show placement and each can be replaced with another image/name.

49

THANKS TO THOSE WHO HELPED MAKE THIS TALK NECESSARY

JAY PIPES

Bexar, Cactus, Diablo, Essex

BRIAN WALDON

Folsom, Grizzly

MARK WASHENBERGER

Havana, Icehouse, Juno

NIKHIL KOMAWAR

Kilo, Liberty

FLAVIO PERCOCO

Mitaka

50

H i n d s i g h t i s , o f c o u r s e , 2 0 / 2 0 . I f I h a d i t a l l t o d o o v e r a g a i n , I p r o b a b l y

w o u l d p u s h f o r n o t h a v i n g a n A P I /i m a g e s t r e a m i n g e n d p o i n t a t a l l i n G l a n c e , a n d j u s t h a v e G l a n c e b e a

g e n e r a l p u r p o s e m e t a d a t a r e g i s t r y .

Jay Pipes

O N E FA N AT I C A L P L A C E | S A N A N T O N I O , T X 7 8 2 1 8

U S S A L E S : 1 - 8 0 0 - 9 6 1 - 2 8 8 8 | U S S U P P O R T: 1 - 8 0 0 - 9 6 1 - 4 4 5 4 | W W W . R A C K S PA C E . C O M

© RAC K S PAC E LT D . | RAC K S PAC E® AN D FAN AT I C AL S U P P ORT ® ARE S ERV I C E M ARK S OF RAC K S PAC E U S , I N C . REG I ST ERED I N T HE U N I T ED STAT ES AN D OT HER C OU N T RI ES . | W W W.RAC KS PAC E .C OM

US

Thank you

DON’T GO ANYWHERE!

And now … ALEX TIVELKOV to speak about the v3 API