Campus Grids ReportCampus Grids Report

OSG Area Coordinator’s MeetingOSG Area Coordinator’s MeetingDec 15, 2010Dec 15, 2010

Dan FraserDan Fraser(Derek Weitzel, Brian Bockelman)(Derek Weitzel, Brian Bockelman)

Mini Campus Grid HistoryMini Campus Grid HistoryLots of interest in Campus GridsLots of interest in Campus Grids

Workshops, meetings, a few campus engagements over past few yearsWorkshops, meetings, a few campus engagements over past few years Not much traction (Clemson, …)Not much traction (Clemson, …)

Some sites are building Campus Grids, but the effort is often unrelated to Some sites are building Campus Grids, but the effort is often unrelated to OSG activity.OSG activity.

Early 2010 -- A new approach (Dan, Miron)Early 2010 -- A new approach (Dan, Miron) Can we offer campuses more than a plan ?Can we offer campuses more than a plan ? Can we package up a technology set to help new campuses build a Can we package up a technology set to help new campuses build a

Campus grid ? Campus grid ? Started piecing together a technology packageStarted piecing together a technology package

Draft architecture documentDraft architecture document

Blueprint meeting in June (led to a refined whiteboard architecture)Blueprint meeting in June (led to a refined whiteboard architecture)

Derek agreed to work on this for his Masters ThesisDerek agreed to work on this for his Masters Thesis

Campus Grid ConceptsCampus Grid Concepts

Integrate different batch systems togetherIntegrate different batch systems together PBS, LSF, Condor, …PBS, LSF, Condor, … Users should not need to know details of eachUsers should not need to know details of each

Use the Glide-in modelUse the Glide-in model Proving to be a huge success on the broader OSGProving to be a huge success on the broader OSG Nebraska already using this for their Campus submissionsNebraska already using this for their Campus submissions Easy for users to use more resources such as the OSGEasy for users to use more resources such as the OSG

Don’t require Grid CertificatesDon’t require Grid Certificates Except possibly when using external campus resourcesExcept possibly when using external campus resources ““When you are already in the house, you don’t need a When you are already in the house, you don’t need a

passport to go to the bathroom” - M. Livnypassport to go to the bathroom” - M. Livny

ImplementationImplementation

Consider a typical Consider a typical Condor cluster.Condor cluster. User submits to a User submits to a

scheduler (schedd)scheduler (schedd) Schedd negotiates Schedd negotiates

slots from the slots from the negotiator.negotiator.

Schedd contacts Schedd contacts WN and runs jobWN and runs job

ImplementationImplementation

Next, flock between Condor clusters.Next, flock between Condor clusters.

My scheduler contacts other negotiators My scheduler contacts other negotiators for slotsfor slots

Condor GlideinCondor Glidein

From the grid world, Condor has developed From the grid world, Condor has developed glidein’s – the ability to configure and start a glidein’s – the ability to configure and start a condor_startd through one script.condor_startd through one script. Meant to be executed as the payload job Meant to be executed as the payload job

inside other batch systems – a pilot native inside other batch systems – a pilot native to Condor.to Condor.

Include non-Condor sitesInclude non-Condor sites

Derek created the Factory Derek created the Factory codecodeFactory process runs on Factory process runs on cluster login node.cluster login node.Queries known schedd’s to Queries known schedd’s to see if there are idle jobs.see if there are idle jobs.If so, creates an Condor-G If so, creates an Condor-G job (universe=PBS) and job (universe=PBS) and submits the job directly to submits the job directly to the local PBS scheduler via the local PBS scheduler via BLAHp.BLAHp.The condor_starters join The condor_starters join the virtual Condor pool the virtual Condor pool local to PBS cluster.local to PBS cluster.Then jobs flock from the Then jobs flock from the submit node to the PBS submit node to the PBS cluster. cluster.

Inter-Campus GridsInter-Campus Grids

Flocking doesn’t need to respect the campus Flocking doesn’t need to respect the campus boundaryboundaryCondor NAT transversal helps here (CCB)Condor NAT transversal helps here (CCB)Nebraska flocks to Purdue currentlyNebraska flocks to Purdue currently

Expand to use the OSGExpand to use the OSG

Further expand with GlideinWMSFurther expand with GlideinWMS

Current grid at NebraskaCurrent grid at Nebraska

SecuritySecurity

GSI/X509 works great for offsite computing as a GSI/X509 works great for offsite computing as a way to do identity management.way to do identity management.Use campus based security inside the campus.Use campus based security inside the campus. I.e., Physics sysadmins trust the math sysadmins.I.e., Physics sysadmins trust the math sysadmins.

Between campuses, Condor daemons can Between campuses, Condor daemons can negotiate the security protocol in order to make negotiate the security protocol in order to make both sides happy.both sides happy. This happens automaticallyThis happens automatically Sites can enforce local auth{z,n} policies at the Sites can enforce local auth{z,n} policies at the

daemon level.daemon level.

Issues to overcomeIssues to overcome

DataData

AccountingAccounting

……

More reading & DownloadsMore reading & Downloads

Campus grid weekly meetingsCampus grid weekly meetings https://twiki.grid.iu.edu/bin/view/CampusGrids/https://twiki.grid.iu.edu/bin/view/CampusGrids/

CampusGridMeetingsCampusGridMeetings

Release downloads, documentation, & Release downloads, documentation, & install guideinstall guide http://sourceforge.net/apps/trac/campusfactoryhttp://sourceforge.net/apps/trac/campusfactory

Info on Offline Class AdsInfo on Offline Class Ads https://twiki.grid.iu.edu/bin/viewauth/https://twiki.grid.iu.edu/bin/viewauth/

CampusGrids/OfflineClassAdFactory CampusGrids/OfflineClassAdFactory

Looking for Interested Looking for Interested CampusesCampuses

If you know of a Campus that may be If you know of a Campus that may be interested in building a campus grid, interested in building a campus grid, please contact us:please contact us: fraser@anl.govfraser@anl.gov dweitzel@cse.unl.edudweitzel@cse.unl.edu bbockelm@math.unl.edubbockelm@math.unl.edu