Peter JohnsonBrad MillerFredric Silberberg

BuildingandContributingtoWPILib

Agenda

• What is the WPILib Suite?• Why build WPILib?• How to build the library• Building NetworkTables• How to contribute your changes• What’s next?

WPILibMission• Enable FIRST teams to focus on writing game-specific software rather than on

hardware details – “raise the floor, don’t lower the ceiling”– Enable teams with limited programming knowledge or mentor experience to do as much

as possible– Enable teams with intermediate programming knowledge to use powerful tools such as

PID to improve their robot performance– Enable teams with advanced programming knowledge to use the full power of the system

• Support the Kit of Parts control system hardware (e.g. controllers, sensors)• Provide parity across all officially supported languages (C++, Java, LabView)

– Enable teams to pick the language of their choice without worrying about supported features

• To this end, the library and associated tools need to be robust, reliable, maintainable, and understandable!

WPILib SuiteofProjects

Project PrimaryLanguage Purpose

WPILibC C++ UserlibraryforC++robot programs

WPILibJ Java UserlibraryforJavarobot programs

HAL C++ Low-levelhardwarelibraryforC++andJava

SmartDashboard Java Graphical dashboard

SFX JRuby Graphical dashboard

RobotBuilder Java Code generator

Eclipse Plugins Java Integrateddevelopmentenvironment, example programs

NTcore Library C++ NetworkTablescorelibraryfor C++andJava

FRCSim C++ andJava Robotprogramsimulationenvironment

GRIP Java Graphicalbuilder for imageprocessing

JavaInstaller Java Installsembedded Java runtimeonroboRIO

WhyBuildWPILib?

• Make local modifications for your robot projects– Fix issues that you find– Add extensions

• Contribute changes back to the project for all teams– Changes accepted to the library will continue to be maintained– Your improvements will benefit all FRC teams

BuildingWPILib

Robot Library for C++ and Java

NTcore

GRIPSmartDashboard

HAL

FRCARMToolchain

WPILibC

NIroboRioLibraries

WPILibJ

EclipsePlugins

RobotBuilder

Partof“allwpilib”repository

Separaterepositories

Legend

WPILib SuiteDependencyTree

JavaInstaller

Background:Gradle

• An open source build automation system• Uses a Groovy-based format rather than a declarative

project description (ant, maven, make)• Plugin based architecture• Great support for Java and native languages (C++)• Good multi-project support and scales to large projects• Integrations for all the popular IDEs

• Gradle Build System is used for (almost) all components– Exceptions are eclipse plugins and SFX– Build artifacts are published as Maven-style dependencies

apply plugin: 'java'apply plugin: 'maven-publish'

sourceSets {shared

}

dependencies {sharedCompile 'edu.wpi.first.wpilib.networktables.java:NetworkTables:3.0.0-SNAPSHOT:arm'sharedRuntime 'edu.wpi.first.wpilib.networktables.java:NetworkTables:3.0.0-SNAPSHOT:arm'

}

Background:Gradle

Background:Maven

• Build automation tool that focuses mostly on Java• Used in 3 places for WPILib Suite:

– Eclipse Plugins• Gradle support for building eclipse plugins isn't good

– SFX • Hasn't been converted

– Maven Dependency Repository• Used extensively through all projects• Gradle has robust support for reading and publishing

BuildingWPILib

• Obtain source• Open terminal in source directory• ./gradlew build

WhatDidthatDo?

1. Checked for dependency updates2. Built the HAL3. Built WPILibJ and WPILibC4. Built libraries are in the build directories of each

subproject• hal/build• wpilibj/build• wpilibc/build

BuildingaRobotProgramwithYourWPILib Build

• ~/wpilib/java/current/ant/build.properties• Edit wpilib.jar to point to your built version of WPILib

Gradle Tasks

Taskname Descriptiontasks Completelistofallavailabletasksbuild Buildslibrariesand/orprojectapplicationpublish Publishesmvn dependencies tolocalrepositoryclean Removesallbuildartifacts

• ./gradlew task-name• Many tasks for building each individual target, see tasks --all for a full list

Mavendependencies• Maven-style dependencies are used for interproject

dependencies• Hierarchy:

– Group ID: edu.wpi.first.wpilib– Artifact ID: wpilibj, wpilibc, networktables.cpp, networktables.java,

etc.– Version: Currently transitioning to semantic versioning. For more

info, see semver.org• Full format

– As a string : ‘group_id:artifact_id:version_id’– As a folder hierarchy: group/id/artifact_id/version_id

MavenDependencyResolution

• Hierarchical Resolution• Tries the first level, then falls back to the next level, and so on• We have 2 levels

– Local repository - ~/releases/maven/development• Local builds publish to here

– Global Repository –http://first.wpi.edu/FRC/roborio/maven/development

• Will be transitioning to an development repository (for building WPILib for developers) and a release repository (for teams who want to use Gradle or Maven to build robot programs)

BuildingPlugins

• Plugins are built with Maven– Gradle wrapper coming soon™

• Syntax:– mvn package

• Prepare to have your screen bombarded with text

Installing LocalPlugins

• Install from build directory

• Restart Eclipse• Build robot

programs as usual• To revert, uninstall

the plugins, delete ~/wpilib, and install from the official repository

BuildingNTcoreNetworkTables CoreLibraryforC++andJava

NTcoreDependencies• As one of the “lowest layer” libraries, NTcore is designed to have no dependencies

other than compiler toolchains:– Platform native C++ compiler/linker

• Windows: MSVC, Mac: clang, Linux: GCC• C++11 support is required as NTcore extensively uses C++11 features including std::thread

– Platform native JDK (if building Java library)– FRC ARM toolchain (if building roboRio library)

• NTcore internally uses several LLVM utility classes but embeds simplified versions to avoid a LLVM library dependency– As WPILib depends on NTcore, it can also use these useful utility classes!– Classes include: ArrayRef, DenseMap, SmallPtrSet, SmallString, SmallVector,

StringMap, and StringRef (see http://llvm.org/docs/ProgrammersManual.html for docs on these)

NTcore CodeOrganizationLocation Purpose

doc/ Standalonedocumentation (e.g.protocols)

include/ External-facing(API) C andC++headers

java/ Javawrapper

java/lib/ JNIbridge C++sourcecode

java/src/ Java wrappersourcecode

java/test/ Javatests(exampleprograms,notautomated)

src/ C++source code

test/ C++standalonetests(exampleprograms, notautomated)

test/unit/ C++automatedunittests(usesgtest/gmock)

toolchains/,gradle/ Buildsupport files

C++UserCode

JavaUserCode

C++“NetworkTable”

class

Java“NetworkTable”

class

C++API

CAPI JNI

OtherLanguageWrappers

NTcoreUser-FacingInterfaces

NTcore KeyInternalModulesModule Purpose

C++API Function-oriented interfacetointernalclasses

Dispatcher Transmits periodicupdatesoflocalvaluechangestoeachremoteClientandserverthreadmanagementandinitialconnectionhandshakingClientautomaticreconnectionattemptstoserver

Storage Local storageofall“current”values(including user-facingset/get)PersistentfileloadingandsavingProcessingofincomingnetworkupdates

Log Centralizedand customizableerror/warning/info logreporting

NetworkConnection Transmits/receives messagesto/fromasingleremote

Notifier Managesentryandconnection listenersand handles thedetailsofactuallycallingthecorrectusercallbackwhenrequestedbyothermodules

RpcServer HandlesRPCincomingcallsbypassing themontousercode(server only)

BuildingNTcore• Building with Gradle:

– ./gradlew build – builds both FRC ARM and native libraries including Java support• This requires FRC ARM toolchain

– ./gradlew build -PskipArm – builds only native library• Useful if you don’t have FRC toolchain installed

– ./gradlew :arm:build -PcompilerPrefix=arm-linux-gnueabi- – builds only ARM library with the GCC cross compiler starting with “arm-linux-gnueabi-”

• Allows cross-compiler builds for other ARM platforms than FRC• In order to use a locally built NTcore when building other WPILib projects:

– ./gradlew publish – publishes ARM and current platform native libraries– ./gradlew publish -PmakeDesktop – publishes current platform only native library as “desktop”

library • The WPI build server builds Windows, Mac, and Linux and then merges the result as “desktop”• WPILib tools such as SmartDashboard use the “desktop” library so they are cross-platform when built by the WPI

build server, but for local testing a single platform library is sufficient• While building with Gradle is recommended, it’s also possible to use cmake to build

– This can be an easier path for a C++ only library if Java or Gradle is hard to get running natively

NTcoreDevelopmentAreasfor2017• Client round-robin server search (done)• More automated testing• Integrated standardized support for “array of struct”

– Intended to help with multi-value synchronization issue seen by GRIP

• Standardized support for communicating server connection information to clients

• Full support for RPC in C++ and Java• Low latency support

– Flag specific values to be updated more frequently and via UDP

ContributingtoWPILib

Contributors to2016WPILibSuiteSponsors AmazonRobotics

BAEOpenSourceRoboticsFoundation SolidWorksWPI

ContributingmembersfromtheFRCcommunity PeterJohnson (294)JerryMorrison (971)AustinSchuh (971)JoeRoss(330)DustinSpicuzza(1418)ManuelStoeckl(1511)TylerVeness(3512)

WPIstudentmembers ThomasClarkAlexHenningSamCarlbergJonathanLeitschuhJamesKuszmaulPeterMitranoPatrickPlenefischFredSilberberg

OurPolicyonWPILib SuiteChanges• Everything in the library has to work for the 3000+ teams that will

use it• We need to be able to support submitted changes even if the author

loses interest• Tool suite changes must be generally useful for a broad group of

teams• Changes in one language usually need corresponding changes in

the other language• Substantial changes often need to have corresponding LV changes• Library changes should have tests• Code should be well documented, often with ScreenSteps

WhatShouldYouContribute• Bug reports and fixes!• Improvements (large and small) to existing classes

– 2016 example: configurable SPI address for a sensor, making it more general purpose

• Generic reusable library components– 2016 example: Filter and LinearDigitalFilter classes

• Good rule of thumb: we'll almost always gratefully accept bug fixes. It's best to ask about new features before making a large time investment

• Particularly like features that make it easier to make teams with less experience be more successful

• Some focus areas for 2017: computer vision (GRIP, OpenCV), path planning

WhatShouldYouNotContribute• Game-specific code – the game changes each year!• Team-specific code – WPILib needs to be generic• New sensors

– Logistics of this is tricky—at the very least, we need to be able to test it in hardware!

– If you work for a company and want to get a sensor into the KOP, please contact FIRST

– We’re looking at establishing a “white pages” of unofficial (not officially supported) user-contributed modules

• Major restructuring of library classes or major rewrites– Ideas accepted, but please talk to us before putting in a lot of work– Backwards compatibility (and bitrot) are real concerns for teams

CodingStyleandLicense

• All WPILib projects are distributed under the 3-clause BSD license

• Coding style guides for WPILib are located in the styleguide/ directory of the allwpilib repository– Other tools/libraries (e.g. GRIP, ntcore) may use slightly

different style guides; please use prevailing style when contributing

WhoOwnsAllthePieces

WPIandteam

C++andJavalibraries

C++andJavaNetworkTables

SmartDashboard, SFX

RobotBuilder

Javainstaller

GRIP

NationalInstruments

LabVIEWlibraries

Driverstation

Defaultdashboard

FRCNetworkCommunications

FPGAcodeandinterfacelibrary

WPILibSuiteSchedule

Time Whathappens

Championship Startpairingdownthelistofimprovements forthefollowingFRCseason

MidMay Working full speedonupdates

September Betateamselectionhappens (30teamsperlanguage)

LateSeptember-October

Betatestingbegins,newprojectsbegintowinddown

December Betatestingends, codefreeze,andonlybug fixesgoing in

LateDecember Finalbuildsandkitsarecreated

Atkickoff Anygame-specificcodeispostedormerged

TheAutomatedBuildSystem• Pushed changes starts an automated build and test via

Jenkins– Suite component is checked out and built from scratch– Unit tests run where they exist (limited in the libraries)– Several hundred integration tests run on our test system– Notifications are posted to key developers with results– Build artifacts are stored locally, then pushed to a WPI server

• Promotions of build products– Development – each change merged to master (least stable)– Stable – used for testing out changes with a wider distribution (manual)– Release – code releases that we want everyone to have (manual)

SuiteRepositories

• Currently all of our projects are stored on usfirst.collab.net– Provides repositories, discussions, tracking, file releases, wiki, etc.– Also is a place for beta teams to post progress on their work

• New for this summer!!– We will be migrating source code and bug trackers to GitHub

• More familiarity in the community• Excellent documentation for teams• Easy to find

– Currently working out the details and workflows– More news when it happens

Questions?