Guide to the MidoNet Pluginversion 3.0-3.0.1-1 for Fuel 7.0

Contents1 Revision history 3

2 Document purpose 4

3 Introduction 5

3.1 Requirements 5

3.2 Known Limitations 5

3.3 Changes in MidoNet plugin 3.0.1 5

4 Installation Guide 6

4.1 Install the Plugin 6

4.2 Enable Experimental Features 6

4.3 Create the MidoNet roles 6

4.3.1 NSDB role 7

4.3.2 Gateway role 7

4.4 Edit the Fuel deployment graph dependency cycle 8

4.5 Verifying installation 9

5 MidoNet Fuel Plugin User Guide 10

5.1 MidoNet Networks 10

5.1.1 Neutron without MidoNet plugin 10

5.1.2 Neutron with MidoNet plugin 11

5.2 Select Environment 12

5.3 Install Midokura Enterprise MidoNet (Optional) 13

5.4 Configure MidoNet Plugin 13

5.5 Assign Roles to Nodes 15

5.6 Finish environment configuration 15

5.7 Operations and Troubleshooting 16

5.7.1 Operating MidoNet 16

5.7.2 Troubleshooting MidoNet Fuel deployment 16

5.7.3 Note on Fuel upgrades 16

6 Appendix B - licenses 17

6.1 Third Party Components Used in MidoNet OSS 17

6.2 Puppet Modules 18

7 Appendix B - references 19

1 Revision history

Version Revision date Editor Comment

0.1 15.01.2016 Carmela Rubinos(carmela@midokura.com)

Initial release

0.2 29.02.2016 Samir Ibradzic(samir@midokura.com)

Version 3.0.1 changes

0.3 07.03.2016 Samir Ibradzic(samir@midokura.com)

Added release notes andtroubleshooting

0.3a 29.06.2016 Samir Ibradzic(samir@midokura.com)

Formating fixes, added "Verifyinginstalltion" and "Note on FuelUpgrades" sections

2 Document purposeThis document will guide you through the steps of install, configure and use the MidoNet plugin version3.0.1 for Fuel 7.0.

3 IntroductionMidoNet is an Apache licensed production grade network virtualization software forInfrastructure-as-a-Service (IaaS) clouds. Plugin for Fuel 7.0 provides the puppet manifests to install allthe components to deploy easily MidoNet with Fuel in both lab or production environments.

Fuel MidoNet plugin is capable of deploying MidoNet v2015.06 on top of Mirantis OpenStack Fuel version7.0 (including Maintenance Update 2). There are no prerequisites to use the MidoNet plugin: MidoNet isOpen Source, and the plugin sets the repositories from where download and install MidoNet packages.

This plugin also supports Midokura Enterprise MidoNet (MEM) installation by allowing the user to choosethe option from the Fuel Web UI. The packages are available to download from a passwordprotected-repository. The needed credentials will be provided by Midokura.

3.1 Requirements

Requirement Version/Comment

Fuel 7.0

MidoNet plugin for Fuel 3.0.1

3.2 Known Limitations

• The plugin has some limitations regarding node count scalability for NSDB (Network State Database)and OpenStack Controller role nodes. Once number of nodes with such roles have been determinedon initial deployment, it can not be changed. Compute role nodes are not affected by this limitation,current plugin version supports Compute scalability.

• Current version of plugin can only deploy single MidoNet Gareway role node. MidoNet itself supportsany number of gateway nodes, it is only a plugin limitation, additional nodes needs to be set upmanually.

3.3 Changes in MidoNet plugin 3.0.1

• New features:

• Support for Fuel 7.0 Maintenance Release 2

• Added post_install.sh script that sets up Fuel experimental features, customized MidoNet rolesand custom deployment graph

• Bug fixes:

• Provided updated python-neutron-plugin-midonet debian package

• MidoNet BGP gateway role parameters

• Neutron db creation timeouts

• Fix some OSTF test scenarios

• Numerous documentation fixes

4 Installation Guide

4.1 Install the PluginTo install the MidoNet Fuel plugin:

1. Download the plugin from the Fuel Plugin Catalog

2. Log into Fuel Master node and install the plugin using the Fuel CLI:

# fuel plugins --install midonet-fuel-plugin-3.0-3.0.1-1.noarch.rpm

3. Verify that the plugin is installed correctly:

# fuel pluginsid | name | version | package_version---|---------|---------|----------------9 | midonet | 3.0.1 | 3.0.0

4.2 Enable Experimental FeaturesTo be able to use MidoNet Fuel plugin correctly, Fuel Experimental Features must be enabled. This isdone automatically by plugin post-install script, assuming the plugin is installed on top of clean Fuelmaster installation. In case the process has to be done manually, follow these steps:

1. Modify the /etc/fuel/version.yaml file in Fuel Master host to add experimental to thefeature_groups list in the VERSION section, just below mirantis item:

VERSION:...feature_groups: - mirantis - experimental

2. Restart the Nailgun container with dependencies by running:

# dockerctl restart nailgun# dockerctl restart nginx# dockerctl shell cobbler# cobbler sync# exit

3. Make sure the nginx and the nailgun docker services finished the restart process before go on withthe new section:

# dockerctl check

4.3 Create the MidoNet rolesMidoNet core functionality depends on two roles that are not provided by Fuel on default:

• the NSDB role, which deploys the Network State DataBase services, namely ZooKeeper andCassandra.

• the Gateway role, which provides the MidoNet gateway service needed for handling externalOpenStack traffic. (See MidoNet Fuel Plugin User Guide for more info about networking in MidoNet)

The above roles are added automatically by plugin post-install script, assuming the plugin is installed ontop of clean Fuel master installation. In case the process has to be done manually, follow these steps:

4.3.1 NSDB role

1. Create a YAML file with the NSDB role definition, like this:

name: nsdbmeta: name: Network State Database for MidoNet description: MidoNet Synchronization Servicesvolumes_roles_mapping: - allocate_size: min id: os

2. Name it, for instance, nsdb.yaml. Push the role for the environment (Ubuntu 2015.1.0-7.0)using the Fuel CLI:

$ fuel release---|----------------------|-------------|------------------|-------------2 | Kilo on Ubuntu 14.04 | available | Ubuntu | 2015.1.0-7.01 | Kilo on CentOS 6.5 | unavailable | CentOS | 2015.1.0-7.0

$ fuel role --create --release 2 --file nsdb.yaml

4.3.2 Gateway role

1. Create the role for MidoNet Gateway by creating a file called gateway.yaml with the followingcontents:

name: midonet-gwmeta: name: MidoNet HA Gateway description: MidoNet Gatewayvolumes_roles_mapping:- allocate_size: min id: os

2. Create the role for the environment (Ubuntu 2015.1.0-7.0) using the Fuel CLI:

$ fuel release---|----------------------|-------------|------------------|-------------2 | Kilo on Ubuntu 14.04 | available | Ubuntu | 2015.1.0-7.01 | Kilo on CentOS 6.5 | unavailable | CentOS | 2015.1.0-7.0

$ fuel role --create --release 2 --file gateway.yaml

4.4 Edit the Fuel deployment graph dependency cycleThe roles that were just added needs to be accompanied with appropriate deployment tasks, so thatFuel's Deployment Graph is fully populated. Again, needed deployment tasks are added automatically byplugin post-install script, assuming the plugin is installed on top of clean Fuel master installation. In casethe process still has to be done manually, follow these steps:

1. Create a group type for Fuel 7.0 in a YAML file called /tmp/midonet_groups.yaml with thefollowing content:

- id: nsdb parameters: strategy: type: parallel requires: - deploy_start required_for: - deploy_end role: - nsdb type: group tasks: - logging - hiera - globals - netconfig- id: midonet-gw parameters: strategy: type: parallel required_for: - deploy_end requires: - deploy_start role: - midonet-gw tasks: - logging - hiera - globals - netconfig type: group

2. Download the deployment tasks for the release 2 (Ubuntu 2015.1.0-7.0):

$ fuel release---|----------------------|-------------|------------------|-------------2 | Kilo on Ubuntu 14.04 | available | Ubuntu | 2015.1.0-7.01 | Kilo on CentOS 6.5 | unavailable | CentOS | 2015.1.0-7.0

fuel rel --rel 2 --deployment-tasks --download

3. A file ./release_2/deployment_tasks.yaml will be downloaded.

4. Without moving from your current directory, append the /tmp/midonet_groups.yaml file into thedeployment_tasks.yaml:

cat /tmp/midonet_groups.yaml >> ./release_2/deployment_tasks.yaml

5. Upload the edited deployment-tasks file to the release 2:

fuel rel --rel 2 --deployment-tasks --upload

6. Current Fuel Plugins framework only allows to apply tasks on pre_deployment and post_deploymentstages, adding these groups and tasks into the main graph will allow NSDB and Gateway associatedtasks to:

• Configure logging to see Puppet and MCollective logs related to the tasks from the Fuel WebUI.

• Access to hiera variables.

• Access to global variables.

• Configure the IP addresses for each Fuel network type.

4.5 Verifying installationMidoNet MEM Fuel Plugin replaces Neutron default OpenvSwitch networking plugin in a typicalOpenStack deployment based on Neutron. Therefore, the goal of the verifying the system that has justbeen deployed is to to ensure that plugin installation was successful as well to ensure that OpenStackNeutron networking supported by MidoNet is fully functional. Basic fuel OSTF health check provides asolid base for verifying basic OpenStack as well as MidoNet functionality, it is highly advised to make surethat this particular set of OSTF tests are passing correctly:

1. Sanity tests:

• Request list of networks

2. Functional tests:

• Check network connectivity from instance via floating IP (requires functional BGP setup)

• Check network parameters

• Launch instance with file injection (requires functional BGP setup)

Although the list is somewhat short, executing these functional tests successfully will ensure that a largerset of MidoNet functionality is operating correctly. Just to mention few: MidoNet NSDB, MidoNet RESTAPI, MidoNet Neutron plugin integration, MidoNet networking agents, MidoNet BGP gateway includingFloating IPs.

5 MidoNet Fuel Plugin User GuideOnce the Fuel MidoNet plugin has been installed (following Installation Guide), you can create OpenStackenvironments that use MidoNet SDN controller as a Neutron back-end.

5.1 MidoNet NetworksMidoNet changes the behavior of default Neutron deployments, understanding what MidoNet plugin does,especially in regard to external networks, is essential to configure and use MidoNet Fuel plugin properly.

MidoNet plugin is compatible with both Neutron + GRE and Neutron + VxLAN network tunnelingoverlays, so let's focus on showing the differences beteewn the Neutron default ML2 deployments first.

5.1.1 Neutron without MidoNet pluginFuel 7.0 reference architecture contains some useful informaition in Neutron Network Topologies section.First, let's have an overview of Neutron-default ML2 topolgy:

In this topology, red, or "North" network represents the Public Internet, including Floating IP subnetassigned to OpenStack cloud. That means API access to services and Virtual Machines' Floating IPsshare the same L2/L3 network. This topology overloads the Controllers' traffic, since Neutron L3 agentservice is running on the controller, answers all ARP requests coming from "North" traffic that belong toVirtual Machines' Floating IPs, does NAT on all of the traffic destined to Floating IP assigned to VirtualMachines and places the resulting packets in the overlay of the green, "South" network (br-tun).

Even in an HA deployment, the L3 agent only runs on one of the Controllers, and only gets spawned inanother host if the previous one loses connectivity (active-standby Corosync / Pacemaker HA setup).

Node hosting Neutron Controller has to:

• Serve the API requests coming from users

• Run the data and RPC messaging services (Rabbitmq and MySQL is running on the controllers aswell)

• Handle all the North-South traffic that comes to and from the Virtual Machines.

5.1.2 Neutron with MidoNet pluginWith MidoNet, Neutron separates the control traffic from the data traffic. Even the Floating IPs live in thenetwork overlay. Floating IP subnet is separated from the services API network range (called PublicNetwork on Fuel and represented by the red network below) and MidoNet gateway advertises the routesthat belong to Floating Ranges to BGP peers. So MidoNet plugin forces you to define a new Network onits settings, and allocation-range from environment settings get overridden.

MidoNet deployment topology:

On this topology diagram:

• External Public & API networks is the red one on the diagram. Only Controllers (access toOpenStack APIs and Horizon) and Gateway need access to this network. On the external side of thisunderlay we expect an ISP BGP router(s), ready to learn our OpenStack Floating IP subnet route soit can pass traffic to our virtual machines.

• Private network underlay is the green one on the diagram. All the traffic between virtual machines istunneled by MidoNet on top of this network. Including traffic to and form floating IP addresses.

• Management network is the blue one. All nodes need to be connected to it, this network is used foraccess to NSDB nodes in order to access virtual networks topology and flow information.

• PXE/Admin network is the gray one. Needed by Fuel master to orchestrate the deployment.

• Storage network is not shown on the diagram, as it is out of scope of this guide (and NEutron &MidoNet itself).

MidoNet gateway is native distributed system, one can place as many gateways necessary, soNorth-South traffic can be distributed and balanced. Once BGP sessions are established and routes areexchanged between BGP "peers", each North-to-South network packet gets routed from the ExternalPublic API network to one of the MidoNet gateways. It does not matter which of them gets the packet,they work as if they are a single entity. MidoNet gateway sends the inbound packet directly to theCompute that hosts the target virtual machine.

In this way controller nodes gets significantly less overloaded, since they only need to answer userrequests and they don't handle VM traffic at all (the only exception is the metadata traffic at VMprovisioning time).

Following the learned concepts, we are ready to create a Fuel environment that uses MidoNet.

5.2 Select Environment

1. When creating the environment in the Fuel UI wizard, choose Neutron with tunnelingsegmentation (second option) on the Network tab.

After that, you will be able to choose between GRE or VXLAN segmentation. MidoNet works withboth.

2. MidoNet plugin does not interact with the rest of the options, so choose whatever your deploymentdemands on them. Follow instructions from the official Mirantis OpenStack documentation to finishthe configuration.

Alternatively, this can be done in fuel cli:

$ fuel env --create --name test-deployment --rel 2 --net neutron --nst tun

Once the environment is created, open the Settings tab of the Fuel Web UI.

5.3 Install Midokura Enterprise MidoNet (Optional)

1. Installing Midokura Enterprise MidoNet, you will be able to use some specific features from MidoNetonly available on the Enterprise version.

2. Activate the option Install Midokura Enterprise MidoNet.

3. Select the Midokura Enterprise MidoNet (MEM) version and fill the Username and Password fieldsfor downloading the packages from the repository.

5.4 Configure MidoNet Plugin

1. Configuring the MidoNet plugin for Fuel, you will override most of the options of the Public Networksection of the Settings tab of the environment:

Fuel will still reserve IP addresses of the IP range (first row) to assign API-accessible IPs to theOpenStack services, but the rest will be overridden by the plugin options that you are about toconfigure, making the Floating Network full-overlay and pure floating.

2. Activate the option Assign public networks to all nodes. By default, Fuel only gives public accessto Controllers. We need to enable this option in order to have external connectivity to Gatewaynodes.

3. Select the plugin checkbox and fill the options:

Let's explain them:

• Tunnel Type: Here you can choose between GRE or VxLAN as tunneling technology. Both aresupported by MidoNet, but VxLAN is recommended for its performance.

• Floating Network subnet Public Network CIDR**: This option represents the CIDR ofNeutron's external network (overriding Public Network CIDR for the default Neutron ML2plugin). This subnet MUST NOT be the same as the Public Network CIDR section of theSettings tab of the environment. Since there is no option to fine-tune this kind of networkseparation control within Fuel core, one must use MidoNet Fuel plugin settings to do it.

• Floating Network Gateway IP: The Gateway IP address to the MidoNet Virtual IP subnet. ThisIP address is usually set to the first available IP in the subnet. Make sure that the address reallybelongs to the Floating Network subnet CIDR.

• Floating Network Range Start and Floating Network Range End: First and last IP address ofthe Floating range of IPs available for use on virtual machines.

• BGP routing subnet: IP subnet in which BGP peers resides. Both local and remote BGP peerIP addresses must belong to this subnet.

• BGP local IP address and BGP local AS: This pair of parameters identifies BGP peer local toMidoNet gateway. They are usually given by ISP to be set into your networking equipment (inthis case your MidoNet gateway) by the network administrators. "AS number" stands forAutonomous System Number.

• BGP peer IP address and BGP peer AS: This pair of parameters usually identifies BGP peeron the side of your ISP. They are usually given by ISP to be set into your BGP peer so thatthose peers know where to find each other.

5.5 Assign Roles to Nodes

1. Go to the Nodes tab and you will see the Network State DataBase and MidoNet HA Gateway rolesavailable to be assigned to roles.

2. Some general advice to be followed:

• Gateway role should be given to a dedicated node.

• NSDB role can be combined with any other roles, but note that it needs at least 4GB RAM foritself (dedicated storage hihgly recommended).

5.6 Finish environment configuration

1. Run network verification check

2. Press Deploy button to once you are done with environment configuration.

5.7 Operations and TroubleshootingA successful deployment done with MidoNet Fuel plugin will produce fully working OpenStackenvironment, with MidoNet Neutron network back-end. MidoNet is fully compatible with Neutron and NovaAPIs and most of its aspects can be directly managed by OpenStack Horizon WEB interface, as well asNeutron API.

5.7.1 Operating MidoNetFor advanced networking features supported by MidoNet please see MidoNet Operations Guide. Forgeneral MidoNet troubleshooting, assuming the deployment went fine, please see MidoNetTroubleshooting Guide.

5.7.2 Troubleshooting MidoNet Fuel deploymentIn a case MidoNet Fuel deployment failed for some reason, first thing to do is to make sure that theinitiated deployment satisfies the plugin Known Limitations.

In a case MidoNet Fuel deployment failed for some other reason, useful thing to be checked are variouslog outputs available in Fuel WEB UI. Click on the Logs tab and observe logging information. Default logdisplayed in the WEB interface shows "Web backend" logs, which are too general to provide anytroubleshooting information, we want to check "Astute" logs, which can be selected by clicking Sourcedrop down menu, followed by clicking Show button. In case of deployment errors, important messageswill be shown in red, identifying which stage of deployment may have failed, and on which node(s).

Next step is to look into how deployment tasks were executed at target nodes. After identifying nodes inprevious step, select "Other servers" in the Logs drop-down menu, following by selecting an appropriatenode in Node and "puppet" in Source drop-down menus. Again, important failures should be marked inred. Depending on user's level of understanding of these messages, they should either be included inMidoNet support claims to help to help the troubleshooting or an action can be taken by user to preventissue from happening on re-deployment.

5.7.3 Note on Fuel upgradesFuel provides special mechanism for upgrading locally hosted (usually on a Fuel master node itself)Operating System as well as OpenStack repositories. This is achieved with executing followingcommands on Fuel master node (assuming environment number 1 deployed on nodes 2, 3, 4, 5 and 6):

# fuel-createmirror -M# fuel --env 1 node --node-id 2 3 4 5 6 --tasks upload_core_repos

For deployments based on MidoNet this will cause Neutron services to be restarted. This is an expectedbehavior, and it will not affect network functionality on VMs, virtual routers or gateways.

6 Appendix B - licenses

6.1 Third Party Components Used in MidoNet OSS

Name Project Web Site License

akka https://typesafe.com/community/core-projects/akka

Apache 2.0

ApacheCassandra

http://cassandra.apache.org Apache 2.0

ApacheCommons

http://commons.apache.org/ Apache 2.0

Apache Server http://httpd.apache.org Apache 2.0

Apache Tomcat http://tomcat.apache.org Apache 2.0

ApacheZookeeper

http://zookeeper.apache.org Apache 2.0

AspectJ http://projects.eclipse.org/projects/tools.aspectj EPL 1.0

Curator http://curator.apache.org Apache 2.0

Disruptor https://github.com/LMAX-Exchange/disruptor Apache 2.0

EqualsVerifier https://github.com/jqno/equalsverifier Apache 2.0

guava https://github.com/google/guava Apache 2.0

Guice https://github.com/google/guice Apache 2.0

Hamcrest http://hamcrest.org/ BSD Three Clause

HibernateValidator

http://hibernate.org/validator Apache 2.0

HttpComponents

http://hc.apache.org Apache 2.0

infinispan http://infinispan.org/ Apache 2.0

Jackson http://jackson.codehaus.org Apache 2.0

Java https://www.java.com Oracle Binary Code LicenseAgreement

Jcabi Aspects http://aspects.jcabi.com/index.html BSD Three Clause

Jetty http://eclipse.org/jetty/ Apache 2.0. and Eclipse 1.0

jminix https://code.google.com/p/jminix/ Apache 2.0

JMockit http://jmockit.org MIT

jna https://github.com/twall/jna LGPL 2.1. v4.0 and later: Apache2.0.

JsonPath https://github.com/jayway/JsonPath Apache 2.0

JSch http://www.jcraft.com BSD-style

LOGBack http://logback.qos.ch EPL 1.0. and LGPL 2.1

Metrics https://dropwizard.github.io/metrics Apache 2.0

mockito https://github.com/mockito/mockito MIT

netty http://netty.io Apache 2.0

NGINX http://nginx.org BSD Two Clause

Open vSwitch http://openvswitch.org Apache 2.0

powermock https://code.google.com/p/powermock Apache 2.0

protobuf https://developers.google.com/protocol-buffers BSD Three Clause

RxJava http://reactivex.io Apache 2.0

scala http://scala-lang.org BSD Three Clause

scala-logging https://github.com/typesafehub/scala-logging Apache 2.0

typesafeconf https://github.com/typesafehub/config Apache 2.0

ScalaCheck http://scalacheck.org BSD Three Clause

ScalaTest http://scalatest.org Apache 2.0

Scallop https://github.com/scallop/scallop MIT

slf4j http://www.slf4j.org MIT

6.2 Puppet Modules

Name License

midonet-midonet Apache 2.0

ripienaar-module_data Apache 2.0

puppetlabs-inifile Apache 2.0

deric-zookeeper Apache 2.0

midonet-cassandra Apache 2.0

puppetlabs-apt Apache 2.0

puppetlabs-java Apache 2.0

puppetlabs-tomcat Apache 2.0

7 Appendix B - references

• MidoNet Web Site

• MidoNet v2015.06 Documentation

• MidoNet v2015.06 Code

• Midokura Enterprise MidoNet (MEM) v1.9 Documentation

• Midokura Enterprise MidoNet (MEM) 30 Day Trial

• Fuel Enable Experimental Features

• Fuel Plugins Catalog