The Condor “RoadMap” Condor Week 2003

1www.cs.wisc.edu/condor

Todd TannenbaumDepartment of Computer SciencesUniversity of Wisconsin-Madisonhttp://www.cs.wisc.edu/condor

condor-admin@cs.wisc.edu

The Condor “RoadMap”

Condor Week 2003

2www.cs.wisc.edu/condor

Overview› Impact of the new license› What to expect over the next

year(*) Research thrusts Software Release plans

* Must-Forget-Disclaimer-Clause Rigorously Enforced

3www.cs.wisc.edu/condor

License Change:Old License

“Condor Internal Use License” Right to install & use Condor

at your organization Can create derivative works,

but… May not redistribute Long and complex “Draconian” clauses

4www.cs.wisc.edu/condor

License Change:New License

“Condor Public License”Changed Feb 03 – Permits:

Installation and use Reproduction Modification Redistribution All with or without

modification In source and binary forms ! Very similar to Globus

License

5www.cs.wisc.edu/condor

Open Source, but…WHERE’S THE BEEF^H^H

SOURCE?!?› Readying the code› Making code base

“self-contained”› Expect public

source releases in summer

› Note: License already changed!

6www.cs.wisc.edu/condor

How much support we can give How much support we can give open source contributors is an open source contributors is an

open question.open question.

Some Strategies...

7www.cs.wisc.edu/condor

Contribution Strategies

› Bug fixes, no problem…

› Significant Enhancements Layer logic on top of Condor whenever

possible If changes are required to core Condor,

open a dialog w/ the Condor Team before starting to work on your implementation

8www.cs.wisc.edu/condor

Sending Contributions to Condor

› Send us: Source Documentation Testing strategy utilized

› License issues; contribution must be Licensed to the Condor Project, or Assigned to the Condor Project, or In the public domain w/ a compatible license

• NOTE: GPL is not a compatible license!

9www.cs.wisc.edu/condor

Issues: Open Source to Open Community

› Discussion List

› Online Source Code Repository Do not need to wait for developer

releases

› RUST (or bugzilla?) move to online

› Better developer documentation

10www.cs.wisc.edu/condor

Release Schedule› Current Developer Release on the

web: Condor v6.5.1

› What we have in-house: Condor v6.5.3

› Condor v6.5.3 is our next stable release candidate, and is in “code freeze”.

› Condor v6.6.0 is ~4 weeks away.

11www.cs.wisc.edu/condor

› What’s New in Condor v6.6? Covered by Zach.

› What’s coming down the road in the next developer series? Already heard about: “Self-contained” open source friendly

build process Closer integration w/ NMI / VDT build

and test process HawkEye COD improvements

12www.cs.wisc.edu/condor

Already heard about, cont.

› Grid Shell Generic “wrapper” for grid jobs Monitor resource utilization of job Monitor progress and completion

status of job Online I/O access independent of grid

middleware Perhaps some FTSH components

13www.cs.wisc.edu/condor

Already heard about, cont.

› Condor-G MyProxy integration Job Policy Expressions Improved (easier) GlideIn New Job Types

•GT3 (OGSA), Oracle, NorduGrid

14www.cs.wisc.edu/condor

What’s Next?Future Directions

› Increased integration of data management and other Condor Project technology into the Condor HTC Software

• NeST• PFS• Stork• Diskrouter• FTSH

• ByPass• NMI/VDT• HawkEye• GCB• Condor-G

And collaborator data management technology:• SRM• IBP • SAM• GridFTP• SRB

15www.cs.wisc.edu/condor

What’s Next?Big ItemBig Item: More focus on being a service provider

than just an end-user tool XML representations of user logs, ClassAds,

accounting info, etc. Developer APIs / librariesDeveloper APIs / libraries

• DRMAA – (Distributed Resource Management Application API)

• SOAP access to services• Grid Service (OGSI)

16www.cs.wisc.edu/condor

Distributed Resource Management Application

API› Specification for the submission and control of jobs to one or more DRM systems

› An API : not a protocol› Scope

Does: job submission, monitoring, control, final status

Does not: file staging, reservations, security, …

› Proposed specification announced at GGF 7 Committee co-chaired by Intel and Sun

17www.cs.wisc.edu/condor

SOAP/Grid Service

condor_schedd

Cedar

OGSI:SOAPHTTPG

Web Service:SOAPHTTPS

18www.cs.wisc.edu/condor

More what’s next…› Condor on Windows

Pass user credential to the execution site• Run the job as the user• Complete Shared Filesystem Support

Remote System Calls• No relink required• Note: Remote I/O Socket is already available

Assist with ACL management Weds: Condor on Win32 BoF

› Ports “Clipped” Full

• Clipped means everything except the STANDARD universe

19www.cs.wisc.edu/condor

Jan-02

Feb

-02

Mar-02

Apr-02

May-02

Jun-02

Jul-0

2

Aug

-02

Sep

-02

Oct-02

Nov-02

050

100

150

200

250

300

350

400

450

Condor Downloads by Platform (monthly)

Other

Windows

Linux

20www.cs.wisc.edu/condor

Ports available Now (v6.6)› Full :

Linux RedHat 6.x, 7.x [x86] HPUX 10.20 Tru64 4.x Solaris 2.6,2.7,2.8,2.9 [SPARC] IRIX 6.5

› “Clipped” Linux RedHat 8.0 [x86] Linux SUSE and RH 7.2 [IA64] Tru64 5.1 HPUX 11.11 Windows NT/2K/XP Max OS X

21www.cs.wisc.edu/condor

Ports in the Hopper

› Full versions of everything clipped

› And: RedHat 9 [x86 and IA64] AIX 5.1 “clipped” HPUX 11.11 [PA2.0 and IA64]

“clipped”

22www.cs.wisc.edu/condor

More what’s next…› New version of ClassAds into Condor

Conditionals !! • if/then/else

Aggregates (lists, nested classads) Built-in functions

• String operations, pattern matching, time operators, unit conversions

Clean implementations in C++ and Java ClassAd collections

› Just this + open source changes may become ver 6.8.0.

23www.cs.wisc.edu/condor

Thank you!