© 2016 VMware Inc. All rights reserved.

“Together, or Not At All?”DefCore: The Interoperability Standard for OpenStack

Mark T. Voelker, OpenStack ArchitectMay 20, 2016

2

Perhaps inherent in that ambition is the promise of interoperability.

Code against one set of capabilities and API’s, take your pick of many public clouds, distributions, appliances, and services.

As it turns out, even clouds that are on some level the same can look and act very differently.

OpenStack aims to be a “ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds, regardless of size…”

3

There are ~4,650 config options in the tc-approved-release OpenStack projects.And ~1,073 policy.json configurations (as of late 2015).

OpenStack has a lot of nerd knobs that can affect how a cloud behaves. In fact…

4

You can also put things in front of OpenStack that change the behavior users see…like firewalls.

…and load balancers, API gateways, SSL configurations…

5

The mechanics of what’s under the hood may change behavior too…

(take image formats supported by various virt drivers & storage platforms for example)

6Think you can code against that?

Mark T. Voelker (@marktvoelker)• OpenStack Architect @ VMware, DefCore Committee co-chair, Triangle OpenStack Meetup co-founder,

OpenStack Foundation Member #54, OpenStack ATC, Ex-Puppet OpenStack core dev• Fact: can be bribed with doughnuts• In copious (hah!) spare time: OpenStack solutions, Big Data, Massively Scalable Data Centers, DevOps,

making sawdust with extreme prejudice, raising two great kids with my awesome wife in North Carolina

“A computer nerd….is somebody who uses a computer in order to use a computer.” –Douglas Adams

8

[note: this talk will be slightly more entertaining if you’re a science fiction fan…

…otherwise it will merely be somewhat informative.]

9

Why Should Vendors Care About Interoperability Standards?• It’s good for your users

• It helps you promote your product

• It helps applications be developed for your platform

• It’s now required if you want to call your product OpenStack.

10

def list_images():  “”” A function to list images. Because all OpenStack Powered Platforms can do that…somehow.””"  if $cloud == ‘vendorA’:     # TODO: this also works for vendorX     list_images_via_nova_image_api()  elif $cloud == ‘vendorB’:     # TODO: this also worked for vendorY last week but now, um?     list_images_via_glance_v1()  elif $cloud == ‘vendorC’:     list_images_via_glance_v2()  else:     # I dunno what cloud this is, but it’s OpenStack Powered! So something must work. # Resort to trial and error since we don’t know.     try:        list_images_via_nova_image_api()     except NopeError:         # D’oh, guess that wasn’t it…         try:            list_images_via_glance_v1()         except StillNopeError:            # Aww…well third time’s the charm?             try:               list_images_via_glance_v2()            except NopeNopeNopeError:               rage_quit()

This function could also be called:

not_winning()

11

Someone should make a standard.

Situation:Interoperability is hard.

12

Soon

Soon:

Want to prevent that?

When the community works together with the DefCore Committee, we can.

To understand how, you need to know how OpenStack is governed.

13

OpenStack is primarily governed by two bodies:

Technical Committee

& the Board of Directors

(you decide which is which)

14

The Technical Committee provides:“technical leadership for OpenStack as a whole…..enforces OpenStack ideals (Openness, Transparency, Commonality, Integration, Quality…), decides on issues affecting multiple projects…..”

15

“The Board of Directors provides strategic & financial oversight of Foundation resources and staff.”

Which includes

these things

openstack.org/marketplace

16

DefCore is a Board activity.

• One Director serves as co-chair, other co-chair elected by participants.

• It’s work and procedures must ultimately be approved by a vote of the Board, not the +2’s of it’s most trusted reviewers.

• It produces “Guidelines”, not infrastructure code.

• It can use the OpenStack trademark and logo as both a carrot and stick.

• It can make requirements for products that call themselves OpenStack.

17

So what’s a Guideline, then?

• A list of Capabilities that products must support.

• A list of Tests products must pass to prove it.

• A list of Designated Sections of OpenStack code they must use to provide those Capabilities.

(also: a list of exceptions & things that might be required in the future)

18

What’s the cadence of Guidelines?• New Guideline every 6 months• Each Guideline covers 3 OpenStack releases• Only the newest 2 can be used for new logo

program certification

19

Things you won’t find in DefCore Guidelines:

• Stuff that end users don’t see or can’t use:

• Admin-only API’s• RPC API’s• DB Schema• HA Requirements

• Stuff that’s intentionally pluggable:

• Virt/net/storage drivers• Middlewares• Specific databases

• Stuff that doesn’t have tests

• Stuff that’s being deprecated (usually…more on that in a minute)

20

How do we decide what gets in?

10 Guiding Principles12 Core CriteriaA giant list of tests

New Capabilities must be “advisory” (non-required) for one Guideline before becoming required.

21

DefCore Criteria(Currently all weighted roughly equally, but that will likely change a bit in the next Guideline.)

https://github.com/openstack/defcore/blob/master/doc/source/process/CoreCriteria.rst

22

Say more to me about these tests….

• Must be under TC governance• All Tempest today, but in-tree suites that can be run by

Tempest are likely to be introduced soon• Must work on all releases covered by a Guideline• Typically run via the RefStack wrapper which reports results to

refstack.openstack.org and shows which Guidelines you meet

Details: http://openstack.org/interop

http://refstack.openstack.org

23

Sometimes things go amiss…Tests can be “flagged” (not required for the duration of the Guideline) in some cases:• Capability fails to meet Criteria

(e.g. was scored incorrectly)

• Test fails/is skipped due to an accepted bug in the project

• Test fails/is skipped due to an accepted bug in the test

• Test fails because it requires non-DefCore Capabilities

• Test reflects an implementation choice that isn’t widely deployed even though the Capability itself is.

24

When does all this happen?

• Summit-3 months: preliminary draft

• Summit-2 months: ID new Capabilities

• Summit-1 month: Score Capabilities

• Summit: “Solid” draft

• Summit+1 month: Self-testing

• Summit+2 months: Test Flagging

• Summit+3 months: Board Approval

(Note: 2015 was weird in that we had a very accelerated schedule to get DefCore bootstrapped…above is what it looks like from now on.)

25

Why isn’t $my_favorite_thing in the current DefCore Guideline?

26

27

• It didn’t meet Criteria (scored too low)

• It wasn’t scored in time (scoring is surprisingly hard to get right)

• It was admin-only or driver-specific

• That project isn’t yet widely deployed

• There wasn’t a test for it

• It didn’t score highly across all releases covered in that Guideline

• Nobody brought it up yet (DefCore only became required last year)

28

I’m an OpenStack Developer

I have a really cool new feature...

How do I get it into the DefCore Guidelines?

29

I’m the DefCore Committee co-chair.

I have a blog post you should read!

http://markvoelker.github.io/blog/defcore-for-devs/

• Document it well

• Ensure it has usable tests

• Foster adoption among users, SDK’s, & other projects

• Be patient: needs to be in 3 OpenStack releases

30

Do we get to offer feedback?

Absolutely!

• Feedback built into DefCore scoring cycles

• Feedback encouraged for Advisory capabilities

• Feedback encouraged via flag requests

• Feedback via User Survey• Feedback via RefStack

community-visible results (you may also buy me a beer & bend my ear about DefCore anytime)

31

How do I make RefStack work for me?It’s actually not that hard.

Instructions here which boil down to:1. Download refstack-client.2. Run the “setup_env” script.3. Configure tempest for your cloud.4. Run refstack-client to execute tests.5. Upload results to refstack.openstack.org and review.

32

Why run all the tests & upload results?

Because more data is better.

This gives us additional data about what works in your cloud. With data from lots of clouds, we can make better scoring decisions in the future when considering adding Capabilities to Guidelines.

33

What if I don’t pass all the required tests?

Don’t panic.

• Figure out why some tests failed. • Was it environmental (e.g. a timeout due to

storage being slow)? Tweaking tempest config may help.

• Was it due to a bug in the test? • Was it due to a bug in OpenStack?

• Do you have grounds for requesting a flag?• Valid reasons for flagging a test and how to

do it can be found here.• Talk to us!

• We have an interest in helping you succeed.

• interop@openstack.org is here to help!• Catch us on IRC at #openstack-defcore or

#openstack-refstack.

34

What’s Next?

• 2016.07 scoring going on now • New: networking, images, more

• More testing• Vendors currently testing against 2016.07 draft

• DefCore considering feedback from year 1• TC resolution on proxy API’s• TC resolution on moving DefCore tests Tempest• Possibility of vertical Powered programs

• Report on “top five” interoperability issues• Interoperability test definition spec

35

Sometimes the enemy is us.

• Sometimes projects are slow to adopt support for each others’ new API’s and features.

• Sometimes projects provide multiple ways to do the same things (and sometimes they’re mutually exclusive).

• Sometimes we don’t have good data about what’s really supported.

• Sometimes tests use admin credentials unnecessarily or lump many Capabilities into one test.

36

For this to work, we have to communicate as one community.

• Board of Directors/DefCore Committee

• Technical Committee/technical contributors

• End Users

• Vendors

Want to learn more?

• 2016.01 Guideline• 2015.07 Guideline• Next Guideline draft• Open Reviews (including some

for 2016.07 Guideline)• Core Criteria• DefCore Committee procedural

overview• How to submit patches• Lexicon of DefCore terms• DefCore wiki & meeting Info

38“Do what I do. Hold on tight and pretend it’s a plan!”

39

Thank you.

[with apologies to fine the folks at the BBC’s “Doctor Who”]

(please don’t have me arrested)