Peter JohnsonBrad MillerFredric Silberberg
BuildingandContributingtoWPILib
Agenda
• What is the WPILib Suite?• Why build WPILib?• How to build the library• Building NetworkTables• How to contribute your changes• What’s next?
WPILibMission• Enable FIRST teams to focus on writing game-specific software rather than on
hardware details – “raise the floor, don’t lower the ceiling”– Enable teams with limited programming knowledge or mentor experience to do as much
as possible– Enable teams with intermediate programming knowledge to use powerful tools such as
PID to improve their robot performance– Enable teams with advanced programming knowledge to use the full power of the system
• Support the Kit of Parts control system hardware (e.g. controllers, sensors)• Provide parity across all officially supported languages (C++, Java, LabView)
– Enable teams to pick the language of their choice without worrying about supported features
• To this end, the library and associated tools need to be robust, reliable, maintainable, and understandable!
WPILib SuiteofProjects
Project PrimaryLanguage Purpose
WPILibC C++ UserlibraryforC++robot programs
WPILibJ Java UserlibraryforJavarobot programs
HAL C++ Low-levelhardwarelibraryforC++andJava
SmartDashboard Java Graphical dashboard
SFX JRuby Graphical dashboard
RobotBuilder Java Code generator
Eclipse Plugins Java Integrateddevelopmentenvironment, example programs
NTcore Library C++ NetworkTablescorelibraryfor C++andJava
FRCSim C++ andJava Robotprogramsimulationenvironment
GRIP Java Graphicalbuilder for imageprocessing
JavaInstaller Java Installsembedded Java runtimeonroboRIO
WhyBuildWPILib?
• Make local modifications for your robot projects– Fix issues that you find– Add extensions
• Contribute changes back to the project for all teams– Changes accepted to the library will continue to be maintained– Your improvements will benefit all FRC teams
BuildingWPILib
Robot Library for C++ and Java
NTcore
GRIPSmartDashboard
HAL
FRCARMToolchain
WPILibC
NIroboRioLibraries
WPILibJ
EclipsePlugins
RobotBuilder
Partof“allwpilib”repository
Separaterepositories
Legend
WPILib SuiteDependencyTree
JavaInstaller
Background:Gradle
• An open source build automation system• Uses a Groovy-based format rather than a declarative
project description (ant, maven, make)• Plugin based architecture• Great support for Java and native languages (C++)• Good multi-project support and scales to large projects• Integrations for all the popular IDEs
• Gradle Build System is used for (almost) all components– Exceptions are eclipse plugins and SFX– Build artifacts are published as Maven-style dependencies
apply plugin: 'java'apply plugin: 'maven-publish'
sourceSets {shared
}
dependencies {sharedCompile 'edu.wpi.first.wpilib.networktables.java:NetworkTables:3.0.0-SNAPSHOT:arm'sharedRuntime 'edu.wpi.first.wpilib.networktables.java:NetworkTables:3.0.0-SNAPSHOT:arm'
}
Background:Gradle
Background:Maven
• Build automation tool that focuses mostly on Java• Used in 3 places for WPILib Suite:
– Eclipse Plugins• Gradle support for building eclipse plugins isn't good
– SFX • Hasn't been converted
– Maven Dependency Repository• Used extensively through all projects• Gradle has robust support for reading and publishing
BuildingWPILib
• Obtain source• Open terminal in source directory• ./gradlew build
WhatDidthatDo?
1. Checked for dependency updates2. Built the HAL3. Built WPILibJ and WPILibC4. Built libraries are in the build directories of each
subproject• hal/build• wpilibj/build• wpilibc/build
BuildingaRobotProgramwithYourWPILib Build
• ~/wpilib/java/current/ant/build.properties• Edit wpilib.jar to point to your built version of WPILib
Gradle Tasks
Taskname Descriptiontasks Completelistofallavailabletasksbuild Buildslibrariesand/orprojectapplicationpublish Publishesmvn dependencies tolocalrepositoryclean Removesallbuildartifacts
• ./gradlew task-name• Many tasks for building each individual target, see tasks --all for a full list
Mavendependencies• Maven-style dependencies are used for interproject
dependencies• Hierarchy:
– Group ID: edu.wpi.first.wpilib– Artifact ID: wpilibj, wpilibc, networktables.cpp, networktables.java,
etc.– Version: Currently transitioning to semantic versioning. For more
info, see semver.org• Full format
– As a string : ‘group_id:artifact_id:version_id’– As a folder hierarchy: group/id/artifact_id/version_id
MavenDependencyResolution
• Hierarchical Resolution• Tries the first level, then falls back to the next level, and so on• We have 2 levels
– Local repository - ~/releases/maven/development• Local builds publish to here
– Global Repository –http://first.wpi.edu/FRC/roborio/maven/development
• Will be transitioning to an development repository (for building WPILib for developers) and a release repository (for teams who want to use Gradle or Maven to build robot programs)
BuildingPlugins
• Plugins are built with Maven– Gradle wrapper coming soon™
• Syntax:– mvn package
• Prepare to have your screen bombarded with text
Installing LocalPlugins
• Install from build directory
• Restart Eclipse• Build robot
programs as usual• To revert, uninstall
the plugins, delete ~/wpilib, and install from the official repository
BuildingNTcoreNetworkTables CoreLibraryforC++andJava
NTcoreDependencies• As one of the “lowest layer” libraries, NTcore is designed to have no dependencies
other than compiler toolchains:– Platform native C++ compiler/linker
• Windows: MSVC, Mac: clang, Linux: GCC• C++11 support is required as NTcore extensively uses C++11 features including std::thread
– Platform native JDK (if building Java library)– FRC ARM toolchain (if building roboRio library)
• NTcore internally uses several LLVM utility classes but embeds simplified versions to avoid a LLVM library dependency– As WPILib depends on NTcore, it can also use these useful utility classes!– Classes include: ArrayRef, DenseMap, SmallPtrSet, SmallString, SmallVector,
StringMap, and StringRef (see http://llvm.org/docs/ProgrammersManual.html for docs on these)
NTcore CodeOrganizationLocation Purpose
doc/ Standalonedocumentation (e.g.protocols)
include/ External-facing(API) C andC++headers
java/ Javawrapper
java/lib/ JNIbridge C++sourcecode
java/src/ Java wrappersourcecode
java/test/ Javatests(exampleprograms,notautomated)
src/ C++source code
test/ C++standalonetests(exampleprograms, notautomated)
test/unit/ C++automatedunittests(usesgtest/gmock)
toolchains/,gradle/ Buildsupport files
C++UserCode
JavaUserCode
C++“NetworkTable”
class
Java“NetworkTable”
class
C++API
CAPI JNI
OtherLanguageWrappers
NTcoreUser-FacingInterfaces
NTcore KeyInternalModulesModule Purpose
C++API Function-oriented interfacetointernalclasses
Dispatcher Transmits periodicupdatesoflocalvaluechangestoeachremoteClientandserverthreadmanagementandinitialconnectionhandshakingClientautomaticreconnectionattemptstoserver
Storage Local storageofall“current”values(including user-facingset/get)PersistentfileloadingandsavingProcessingofincomingnetworkupdates
Log Centralizedand customizableerror/warning/info logreporting
NetworkConnection Transmits/receives messagesto/fromasingleremote
Notifier Managesentryandconnection listenersand handles thedetailsofactuallycallingthecorrectusercallbackwhenrequestedbyothermodules
RpcServer HandlesRPCincomingcallsbypassing themontousercode(server only)
BuildingNTcore• Building with Gradle:
– ./gradlew build – builds both FRC ARM and native libraries including Java support• This requires FRC ARM toolchain
– ./gradlew build -PskipArm – builds only native library• Useful if you don’t have FRC toolchain installed
– ./gradlew :arm:build -PcompilerPrefix=arm-linux-gnueabi- – builds only ARM library with the GCC cross compiler starting with “arm-linux-gnueabi-”
• Allows cross-compiler builds for other ARM platforms than FRC• In order to use a locally built NTcore when building other WPILib projects:
– ./gradlew publish – publishes ARM and current platform native libraries– ./gradlew publish -PmakeDesktop – publishes current platform only native library as “desktop”
library • The WPI build server builds Windows, Mac, and Linux and then merges the result as “desktop”• WPILib tools such as SmartDashboard use the “desktop” library so they are cross-platform when built by the WPI
build server, but for local testing a single platform library is sufficient• While building with Gradle is recommended, it’s also possible to use cmake to build
– This can be an easier path for a C++ only library if Java or Gradle is hard to get running natively
NTcoreDevelopmentAreasfor2017• Client round-robin server search (done)• More automated testing• Integrated standardized support for “array of struct”
– Intended to help with multi-value synchronization issue seen by GRIP
• Standardized support for communicating server connection information to clients
• Full support for RPC in C++ and Java• Low latency support
– Flag specific values to be updated more frequently and via UDP
ContributingtoWPILib
Contributors to2016WPILibSuiteSponsors AmazonRobotics
BAEOpenSourceRoboticsFoundation SolidWorksWPI
ContributingmembersfromtheFRCcommunity PeterJohnson (294)JerryMorrison (971)AustinSchuh (971)JoeRoss(330)DustinSpicuzza(1418)ManuelStoeckl(1511)TylerVeness(3512)
WPIstudentmembers ThomasClarkAlexHenningSamCarlbergJonathanLeitschuhJamesKuszmaulPeterMitranoPatrickPlenefischFredSilberberg
OurPolicyonWPILib SuiteChanges• Everything in the library has to work for the 3000+ teams that will
use it• We need to be able to support submitted changes even if the author
loses interest• Tool suite changes must be generally useful for a broad group of
teams• Changes in one language usually need corresponding changes in
the other language• Substantial changes often need to have corresponding LV changes• Library changes should have tests• Code should be well documented, often with ScreenSteps
WhatShouldYouContribute• Bug reports and fixes!• Improvements (large and small) to existing classes
– 2016 example: configurable SPI address for a sensor, making it more general purpose
• Generic reusable library components– 2016 example: Filter and LinearDigitalFilter classes
• Good rule of thumb: we'll almost always gratefully accept bug fixes. It's best to ask about new features before making a large time investment
• Particularly like features that make it easier to make teams with less experience be more successful
• Some focus areas for 2017: computer vision (GRIP, OpenCV), path planning
WhatShouldYouNotContribute• Game-specific code – the game changes each year!• Team-specific code – WPILib needs to be generic• New sensors
– Logistics of this is tricky—at the very least, we need to be able to test it in hardware!
– If you work for a company and want to get a sensor into the KOP, please contact FIRST
– We’re looking at establishing a “white pages” of unofficial (not officially supported) user-contributed modules
• Major restructuring of library classes or major rewrites– Ideas accepted, but please talk to us before putting in a lot of work– Backwards compatibility (and bitrot) are real concerns for teams
CodingStyleandLicense
• All WPILib projects are distributed under the 3-clause BSD license
• Coding style guides for WPILib are located in the styleguide/ directory of the allwpilib repository– Other tools/libraries (e.g. GRIP, ntcore) may use slightly
different style guides; please use prevailing style when contributing
WhoOwnsAllthePieces
WPIandteam
C++andJavalibraries
C++andJavaNetworkTables
SmartDashboard, SFX
RobotBuilder
Javainstaller
GRIP
NationalInstruments
LabVIEWlibraries
Driverstation
Defaultdashboard
FRCNetworkCommunications
FPGAcodeandinterfacelibrary
WPILibSuiteSchedule
Time Whathappens
Championship Startpairingdownthelistofimprovements forthefollowingFRCseason
MidMay Working full speedonupdates
September Betateamselectionhappens (30teamsperlanguage)
LateSeptember-October
Betatestingbegins,newprojectsbegintowinddown
December Betatestingends, codefreeze,andonlybug fixesgoing in
LateDecember Finalbuildsandkitsarecreated
Atkickoff Anygame-specificcodeispostedormerged
TheAutomatedBuildSystem• Pushed changes starts an automated build and test via
Jenkins– Suite component is checked out and built from scratch– Unit tests run where they exist (limited in the libraries)– Several hundred integration tests run on our test system– Notifications are posted to key developers with results– Build artifacts are stored locally, then pushed to a WPI server
• Promotions of build products– Development – each change merged to master (least stable)– Stable – used for testing out changes with a wider distribution (manual)– Release – code releases that we want everyone to have (manual)
SuiteRepositories
• Currently all of our projects are stored on usfirst.collab.net– Provides repositories, discussions, tracking, file releases, wiki, etc.– Also is a place for beta teams to post progress on their work
• New for this summer!!– We will be migrating source code and bug trackers to GitHub
• More familiarity in the community• Excellent documentation for teams• Easy to find
– Currently working out the details and workflows– More news when it happens
Questions?