UUWL aka Unix to Unix Wrapper Library

UUWL Shanghai Solution Center TeamCheng Yan-Zhang, HP RISC Migrations Solutions LeadBruno Cornec HP EMEA Open Source Profession LeadArnaud Meurant HP WW OSSI & RISC Migrations

Unix to Unix Wrapper Library (B64)

Bruno Cornec2

Introducing Myself● Software engineering and Unices since 1988

● Mostly Configuration Management Systems (CMS), Build systems, quality tools, on multiple commercial Unix systems

● Discover Open Source & Linux (OSL) & first contributions in 1993

● Full time on OSL since 1995, first as HP reseller then @HP

● Currently:● Master Technology Architect on OSL for the HP/Intel Solution Center, Grenoble

● OSL HP Advocate

● EMEA OSL HP Profession Lead

● Solutions Linux Conference and OWF board member

● MondoRescue, Dploy.org, Project-Builder.org project lead

● LinuxCOE, mrepo, tellico, rinse, fossology, collectl contributor

● FOSSBazaar and OSL Governance enthusiast

● Mandriva, Mageia, Fedora packager

©2009 HP Confidential3 ©2010 HP / Intel / Red Hat restricted3

Agenda

HP / OSSI / Bruno Cornec4

Agenda

• UUWL Introduction• Project Outline• Goals and phased approach

• Architecture• Global Architecture• Implementation details

• Demo• Project Deliverables• Wrap-up / Next steps


HP Intel RISC Migrations Program

Context & Status

Context: ● Work done as part of the joint HP Intel Solution Centers and Open Source Solutions

Initiative work on RISC to Linux migrations

Goal: ● Accelerate and ease customer transitions from competition RISC/Unix to HP IA.● Allow for easy integration inside their own developments

Timeline: ● 2011-Oct-17: Solution Stack approved by HP/Intel - First phase targets Solaris to Linux

migration, with a global architecture to support more migrations

● 2011-12 2012-04: First phase Development→

● 2012-04-19: Second phase targets AIX to Linux migration, approved by HP TS WW

● 2012-05 2012-11: Second phase Development→

● 2012-12 2013-03: Finalize tests, documentation and packaging→

● More information available on http://www.hpintelco.net

http://www.hpintelco.net/


Project Goals

UUWL is:• A porting tool for speeding up the Unix to

Unix/Linux migration of C/C++ applications.

• Reducing the number of changes to be made to the original source code while migrating Unix-based applications, to cut down the time and effort it takes to be up and running on Linux.

• Designed to support porting from any Unix to any Unix, with phase 1 focussing on Solaris (9 and upper) to Linux and phase 2 from AIX to Linux.

• NOT a tool to detect porting issues nor to help with porting the build environment as other tools like SLPK.

Definition and Targets

UUWL is targeted at:• Reducing the cost of migration by saving 40%

of total migration effort.

• covering >70% of all incompatibility issues in Linux porting, which may take about 60% of porting effort

• taking 5% of the porting effort in its study

• Taking out 15% of the porting effort of each project by leveraging code in it

• Improving the stability of the migrated system with a tested layer

• RHEL 6 x86_64 as the test platform (gcc 4.1.x and above), with support of 32 bits apps as well.


Architecture & Realization


Conceptual Approaches

Technical Architecture on Solaris

Two migration approaches :• modifying the code of the original

application to make it use a set of new portable system calls providence by the UUWL, themselves being identical from an interface perspective from one Unix to another

• keeping the original application untouched. In that case, the adaptation required should be performed with an additional piece of software, called the adapter, which will in turn call the UUWL, and should exist for each pair of source/target OS


Conceptual Approaches

Technical Architecture on AIX

Two migration approaches :• modifying the code of the original

application to make it use a set of new portable system calls providence by the UUWL, themselves being identical from an interface perspective from one Unix to another

• keeping the original application untouched. In that case, the adaptation required should be performed with an additional piece of software, called the adapter, which will in turn call the UUWL, and should exist for each pair of source/target OS


Design and Implementation of UUWLfor Approach 1

Technical Details Approach 1

An incompatible call is found

A new entry is created in the UUWL for this function and realized based on its specification on the source OS. It just calls the source function on the source OS.

Change the source file to include the related UUWL .h file•new function names corresponds to the

original one prefixed by “uuwl_” as much as possible, and similarly for include files

Application to be linked with the UUWL library (with -luuwl) to work correctly

The application is now portable by calling uuwl_ consistent and available system calls across OSes.


Design and Implementation of UUWLfor Approach 2

Technical Details Approach 2


UUWL characteristics

• Delivered:● UUWL Source code

●User Guide

●Function List

●Tested sample programs ported with UUWL

• Coverage Summary:●Functions covered: 321

●Macros covered: 537

●Structures covered: 51

• Sloccount Results Code:●Include ansic=2002

●src_top_dir ansic=1367

●Total Physical Source Lines of Code (SLOC) ansic 3369 (99.62%)

sh 13 (0.38%)

Sloccount Results Effort:Development Effort Estimate, Person-Years (Person-Months) = 0.72 (8.63)

(Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))

Schedule Estimate, Years (Months) = 0.47 (5.67)

(Basic COCOMO model, Months = 2.5 * (person-months**0.38))

Estimated Average Number of Developers (Effort/Schedule) = 1.52

Total Estimated Cost to Develop = $97112

(average salary = $56,286/year, overhead = 2.40).

Summary of the Development


UUWL usage

To Implement:1. Find the most commonly-used header file (ask

to the program owner, search for keywords e.g. “common.h”)

2. Adds a #Include ''uuwl/UUW.h'' line in the identified commonly-used header files or in all souce files if it doesn' t exist

3. Update the makefile by adapting compile options and referencing the UUWL library (-luuwl)

4. Update the original source code to replace non-portable system calls by the corresponding uuwl_ ones.

5. Compile the source code with new makefile

6. Fix other errors according to the compilation output

To Verify:• Is the program using the UUWL ?

$ ldd <executable_file>

• Does the program run on the target OS ?

Generic Steps to Use UUWL in Porting Approach 1 :


UUWL usage

To Implement:1. Find the most commonly-used header file (ask

to the program owner, search for keywords e.g. “common.h”)

2. Adds a #Include ''uuwl/ UUWLSolaristoLinuxAdapter.h'' line in the identified commonly-used header files or in all souce files if it doesn' t exist

3. Update the makefile by adapting compile options and referencing the UUWL library (-luuwl) and the related adapter --lUUWLSolarisToLinux)

4. Compile the source code with new makefile

5. Fix other errors according to the compilation output

To Verify:• Is the program using the UUWL ?

$ ldd <executable_file>

• Does the program run on the target OS ?

Generic Steps to Use UUWL in Porting Approach 2 :


Sample UUWL Implementation

Sample Programs:• sps on Solaris

• apcd on Solaris • lpr on Solaris

Targets:• Work on Linux after adding the UUWL layer

into the Solaris source-codes• Functions kept• Changes to the source code reduced

compared to a direct porting• Simple implementation process

Sample Sourcecode Compilation:• $ gcc -DLINUX -DNFS -DVPRINTF -DKVM -DSVR4 -DSUNOS=55

-DMAXUSERS=30000 -DMAXTTYS=2500 -D_GNU_SOURCE -I/usr/include -I. -lm -lrt -lcrypt -c -g -O2 -Wall filecount.c

• Before adding UUWL header to the common header-sps.h• sps.h:151:9: warning: extra tokens at end of #endif

directive• In file included from filecount.c:1:• sps.h:273: error: field 'pr_pi' has incomplete type• sps.h:276: error: field 'pr_p' has incomplete type• sps.h:282: error: field 'pr_pu' has incomplete type• sps.h:300:9: warning: extra tokens at end of #endif

directive

• After adding UUWL header to the common header-sps.h• sps.h:151:9: warning: extra tokens at end of #endif

directive• sps.h:300:9: warning: extra tokens at end of #endif

directive

To prove the design and part of the functions


Demo of portingsps from Solaristo Linux using the UUWL


Wrap-up / Next steps


Project URL and Resources

Deliverables

Project home URL• http://uuwl.project-builder.org/• Source code viewer• Reference documents, a User's guide and links

• Test code

• The project is an HP OSRB approved project since 2012-02-17 (#10686 at http://pts.corp.hp.com/proposal_editor/?10686).

• The code is delivered under the dual licences MIT and LGPLv2.1

To checkout source code from SVN:• svn co svn://svn.project-builder.org/uuwl/devel devel

To submit issues:• http://uuwl.project-builder.org/newticket

http://uuwl.project-builder.org/

http://pts.corp.hp.com/proposal_editor/?10686

http://uuwl.project-builder.org/newticket


Project usage in PoCs• Large plane manufacturer interested to use it in 3 migration projects (US). Shanghai team

asked to contribute to the setup and extend the UUWL.• 2 Banks in UK are also in the funnel list.

Next phases :• Funding is under discussion for an HP-UX to Linux proprietary module (Phase 3) with HP WW

TS

Other developments• Discussions are held with the SLPK team to see what commonalities and synergies can be

setup.• Project has been internally announced (5000 persons) and also externally.• 2 webinars done for phase 1 in May 2012 – 1 new webinar in Nov. 2012 for phase 1&2.• Use project-builder.org to produce packages for 120+ distribution tuples.• Official Mageia/Fedora packaging is a next step.

Next steps


Thank you !

Q&A (B64)