Executive Summary

Development groups depend on source control management systems to maintain and scale their software projects. But traditional methods and workflows may not work as developers shift to the complex world of Linux platform development, forcing a move to new protocols.

This paper highlights the difficulties of applying traditional methods to Linux platform development, provides the strategies to resolve these issues, and describes the tools that are available with Wind River Linux to make these strategies work for your environment.

Issues with Traditional Management Strategies

Projects are traditionally managed entirely within the configuration management system. The source and development files are registered with the system, the generated object and binary files are ignored as private data, and the user can generally check out the same project in a different location or at a later time and recreate the project as needed.

There are several problems with this traditional methodology for Linux platform projects. First, these platform projects are generally far bigger than traditional projects and can include an entire file system of several hundred packages in addition to the Linux kernel. A fully expanded project can easily have more than 40,000 files and occupy several gigabytes of space.

Second, the very nature of Linux packages makes projects difficult to manage. The packages start in a tarball or SRPM form, generally together with a set of patches. To capture the source, the packages must be unpacked and patched. However, the build directories that hold the flattened source are generally transient and may be destroyed and remade several times in the course of development. This is hostile to management systems.

Third, most packages use automake to configure themselves. The very nature of automake is to configure the files specifically for the particular host and the absolute location of the package source, so the configuration management capture and registration must occur after patching but before configuration. This can be done, but it is difficult to do correctly and consistently and in a way that scales. Also, the project itself is managed by automake, so already there are site-specific files in the project tree, in other words, hard-coded paths in the core project file that may not work for any other location.

Wind River Linux Management Strategies

Wind River Linux installations can be directly captured into a source control system for archiving and sharing. The custom content of Linux projects can be exported into standalone and portable layer directories, which in turn can be source controlled and shared. User applications that may already be in a source control system can be imported into projects as packages and included into the exported layer, where a second user can either utilize that captured application package content or reconnect it to the source control system from the new project location.

Here is the recommended strategy to overcome the issues with managing a Linux platform project.

Table of Contents

Executive Summary ............................................................ 1

Issues with Traditional Management Strategies ................. 1

Wind River Linux Management Strategies ......................... 1

Preserving the Installation .............................................. 2

Using Layers to Manage Custom Project Content ......... 2

Using Layers Now or 10 Years from Now ....................... 2

Simple Single-User Strategy ............................................... 2

Wind River Linux Tools ....................................................... 3

export-layer .................................................................... 3

export-patch ................................................................... 4

Import Application ......................................................... 4

Import Workbench Project ............................................. 4

Eclipse Source Control Plug-Ins ..................................... 4

Conclusion .......................................................................... 4

Wind River Linux and Source Control Management ToolsDavid Reyna, Senior Member Technical Services Wind River

2 | Wind River Linux and Source Control Management Tools

Preserving the Installation

You can check the entire Wind River Linux installation into your source control system, to preserve the original source content. The installation directory is designed to be portable, so it can be extracted to new locations now or in the future. Additionally, you will have all of the user space and kernel in their original compressed form.

You can optionally skip the directory of prebuilt package content (in the form of RPM binary files) that are included as part of the installation, if space is a concern. The prebuilt RPM files can take up to 30% to 80% of the installation’s space, depending on how many unique board-specific packages you have installed; plus this content can always be rebuilt as needed from the included original package sources.

Using Layers to Manage Custom Project Content

Since directly managing the actual project directory is problematic, the strategy is to use Wind River Linux layer technology to separate out and manage the changed and local project content in a safe, manageable, and scalable manner.

Wind River Linux layer technology is the crucial feature for extracting and sharing project changes between projects and developer groups. Layers can capture any and all changes to a project and can be included in any new project to reassert the captured changes. They also capture the state of the original project so it is easy to re-instantiate any project from scratch.

Specific to source control management, a layer directory is easy to both check in to a source control system and to manage. Most importantly, a layer directory is completely portable and re-locatable so it can be checked out and reused in any environment.

Using Layers Now or 10 Years from Now

If you wish to share the project changes immediately with other developers, all they need to do is the following:

1. Check out the captured layer directory. 2. Create a local version of the project using the settings

captured in the layer, using the local installation path and the layer path.

3. Build the project.

Once the second project is set up, a simple refresh of the source control managed layer directory and a reconfig command will assert any new changes to the second project.

If you wish to recreate the project several years later, there are some additional steps you should consider. Specifically, you must preserve a copy of the development host computer (or any of the Wind River Linux release’s supported hosts). This is because there are many package dependencies on the host’s tools, and while many host tools are included with the Wind River Linux installation, not all are. These dependencies can range from obvious requirements such as the host compiler and core utilities as well as non-obvious requirements such as asciidoc and htmlview. You will, of course, also need to preserve a computer that can run this host operating system. Given the rapid and tightly coupled Linux community development, supported hosts have a lifetime of only one to two years, so it is important to be prepared.

Simple Single-User Strategy

Not every project needs to be shared or preserved over time. Let us explore the simple scenario where a single user wishes to source manage a project but does not need the full ability to share that project with other users and other environments.

Source Control Repository

Project 2Project 1

Wind RiverLinux

Installation Wind RiverLinux

Installation

Wind RiverLinux

Installation

Layers

UserApplications

Build

LocalPackages

Build

LocalPackages

Figure 1: Integration of user projects and a source control repository

3 | Wind River Linux and Source Control Management Tools

In this case, the embedded paths for the project and installation are not an issue because the project will be checked in and out of the same location and in the same environment. This developer can check in and check out the project tree.

Here are a few suggestions for optimizing this arrangement.

Before doing the initial check-in, perform the clean rule for all packages that you have locally instantiated, those packages that you have built locally and that are not using the prebuilt intermediate RPM files from the installation. This will help reduce the burden on the source management tool to identify which files are “source” and which are “derived,” which is particularly difficult to accomplish with the intermediate generated tools and help content.

You should also perform the distclean rule for all packages that have been locally built but are not actually different from the original installation versions. For example, if you have performed a full build-all (or a build for debug or profiling support), you will have locally instantiated every single package but will not have added any new information beyond what is in the installation and will bloat the repository image by an order of magnitude.

Alternately, if your goal is to exactly capture all of the expanded source to all of the packages, perform the patch rule (after the distclean) for all packages, to flatten their source and patches into pure source trees.

Now check the project directory into the source control system. Optionally, check in the installation if you have not already done so because the project will have many symlinks in that installation tree.

There are some caveats to the single-user strategy. For Clear Case users, the check-in will normally convert all of the files

to read-only. While most packages can tolerate this, there are some that require that certain files remain read-write enabled. An example is the file build/linux/wrs/meta-series, which helps control the project’s local git tree for the kernel. You will need to either perform a non-reserved check-out or manually reset such files back to read-write.

For CVS users, there will be CVS directories introduced throughout the managed tree. Unfortunately, there may be packages that will fail to build with these extra directories, for example, packages that perform recursive makefile rules, where the make will fail because the CVS directory has no makefile. For these packages, you will need to either patch the makefiles to fix these build issues or manage only the package’s extracted patch files; in other words, explicitly use the export-patch tool to capture patches to the project’s dist directory, and distclean the package’s build directory prior to checking in the project directory so that it is not added to source control. This is not an issue for SVN, git, or Clear Case because the metadata is either in hidden files or is managed outside of the project directory.

Wind River Linux Tools

Here are some of the workflow tools provided with Wind River Linux for helping manage platform projects for source control systems.

export-layer

The export-layer build command will create a self-contained layer directory either under the project’s export directory (which you can move later) or at a user-specified location. This command will capture the project’s configuration data plus the high-level changes (such as the package list and the file system layout changes). This command can be used to create a second user’s project to receive the changes.

Source Control RepositoryImport

OptionalCheck-in/out

Import

SourceExternal

Project

Package 1

Package RulesSCM Rules

Source

Application 1

Package 2

Package RulesSCM Rules

Source

External

Application 1

Application 2

Figure 2: Applications under source control can be imported as packages in projects

This command will also explicitly include any kernel configure changes and any locally added or imported packages or applications. Finally, this command will also include any extracted package or kernel patches.

export-patch

The export-patch tool can be used to identify and extract changes to the user space packages, both SRPMs or classic package types. These patches are placed in the local project’s dist directory, in a form that the export-layer command can access.

Import Application

If you have an application source tree, the Import Application tool (available with Wind River Linux 4) will allow you to include this source tree in your project as a formal Wind River Linux package. You can leave your source in its external location and have the tool provide a wrapper package. In this manner, there is a single source, and changes are immediately visible to and from that location (though if you export this wrapper package, you will need to manually reconnect to its external source).

Alternately, you can have the tool copy the source tree into a local package so your application package is completely portable with the layer export tool (the import tool will generate optional push and pull rules so that you can re-sync on demand with that external source location). Additionally, if this application tree is under your source control management for CVS, SVN, git, or Clear Case, the generated package can optionally generate flexible rules to check out and check in changes from your development cycle to your repository.

If the export-patch tool is used with these special packages, it can perform an intelligent check-in with the specific files that were changed, using the patch description for the check-in comment.

In all cases, these imported applications will be included by the export-layer rule.

Import Workbench Project

You can use the Import Workbench Project tool (available with Wind River Linux 4) to import a Wind River Linux application or user-defined projects as a package into your project. This allows you to use the full application development project facilities in Wind River Workbench to create and develop your application and provide a way to capture and share that work as a formal Wind River Linux package for your platform maintainers, at any frequency that is appropriate.

Eclipse Source Control Plug-Ins

If you are using source control plug-ins with Wind River Workbench/Eclipse, you can also use them with Wind River Linux. For example, if you are directly managing a Wind River Linux application or user-defined project, you can use the Import Workbench Application tool to bring that application into a project and later have any captured changes pushed back to the original source. Also, you can use the export-layer tool to lace the extracted layer directly into a generic Eclipse project and then use Team > Share to add this “layer” into your source control system.

Conclusion

Wind River recognizes the importance of configuration management as part of a company’s strategy to control the life cycle and scalability of its software investment. With the tools and methodologies outlined in this paper, you can extend and match those systems to the complexity, ability, and life cycle of Wind River Linux projects. These tools, provided as part of Wind River Linux, are comprehensive enough to get the job done yet customizable enough to meet the particular needs and requirements of developers.

You can find more details on these tools, together with use cases, in several papers posted at the Wind River Online Support site at www.windriver.com/support for the Wind River Linux 4.0 product.

Wind River is a leader in embedded and mobile software. We enable companies to develop, run, and manage device software faster, better, at lower cost, and more reliably. www.windriver.com

© 2010 Wind River Systems, Inc. The Wind River logo is a trademark of Wind River Systems, Inc., and Wind River and VxWorks are registered trademarks of Wind River Systems, Inc. Other marks used herein are the property of their respective owners. For more information, see www.windriver.com/company/terms/trademark.html. Rev. 11/2010