View
215
Download
0
Category
Preview:
Citation preview
www.cs.wisc.edu/~mironCondor Team 2008
www.cs.wisc.edu/Condor
Welcome toCondor Week
#10(year #25 for the project)
www.cs.wisc.edu/Condor
Goodbye GridsWelcome Clouds!
we survived grids, weare ready for clouds and
prepare for what will follow
www.cs.wisc.edu/Condor
Subject: PSFrom: Ian Foster <foster@anl.gov>Date: Thu, 4 Dec 2008 19:40:28 -0600To: Miron Livny miron@cs.wisc.edu
Miron:
I visited CSIRO today and they told me that they are deploying a "desktop cloud." I asked them what "cloud software" they are using, and I was pleased to hear that they were using Condor. So Condor is officially cloud software, at least in Australia. I thought you'd like to know.
Ian.
www.cs.wisc.edu/Condor
Stay away from
the hype and stick to the
principals!
www.cs.wisc.edu/Condor
High ThroughputComputing
became a well established and widely used paradigm
www.cs.wisc.edu/Condor
Condor-GGlide-ins
Matchmaking
carry the load of EGEE and OSG
www.cs.wisc.edu/Condor
Virtual Machines
fit our principals
(they look like a job, they ‘come and go’, they
checkpoint, they have many attributes,
the can be matched, … )
www.cs.wisc.edu/Condor
Innovation through widely
adopted technology
partnerships with users in
academia and industry, and IT providers
Miron LivnyCenter for High Throughput Computing
Computer Sciences DepartmentUniversity of Wisconsin-Madison
The Condor Project – an Experiment in Experimental
Computer Science
www.cs.wisc.edu/~miron
Why am I here today?
Because I believe that: the Condor project is ‘different’, we do not have many (any?) other such Computer
Science projects, it is important for the future of our science to have more
(many?) such projects, we can and should learn from what we have done so far
what it takes to build and sustain such projects and CISE can and should play a leadership role in changing
the dynamics of our field so that more Computer Science departments will have such projects
www.cs.wisc.edu/~miron
“Why are you leaving academia and taking a job
in industry?”
“I want to have
impact!”
www.cs.wisc.edu/~miron
In the words of Mike Carey
“I left academia for industry because I was drawn to the idea of getting more direct access to real problems - from customers and challenges encountered while building commercial-grade software - because I felt like I was in somewhat of a mode of inventing and solving problems, at least w.r.t. some of the things I'd been working on. Sure, that was leading to many written/submitted/accepted papers, but it was somehow less than satisfying after awhile.”
www.cs.wisc.edu/~miron
It is all about Software
I argue that ‘we’ do not know how to develop, maintain, support and evolve
(dependable) software, to integrate software components into end-
to-end (dependable) capabilities, to evaluate the quality of software, to estimate the cost (effort) related to
software and to treat software as infrastructure
www.cs.wisc.edu/~miron
(my) terminology
› Experiment - an act or operation for the purpose of discovering something unknown or of testing a principle, supposition, etc.:
› Technology Adoption – to select a technology as a means to meet an ends of significant importance/value
› Real users – individuals or groups who adopt (and use) a computing technology
› Experimental Computer Science – advance the state of the art of computing (new frameworks, new technologies, new abstractions) through experiments that involve real users
Two new Institutes on the UW Campus - MIR & WID
16
The MIR Three-Plane Model
17
Societal impact through education, research and technology
Human, physical and cyberinfrastructure
Summit Plane
Integration Plane
Foundation PlaneScientific Research Areas
Delivery avenues
Intellectual environment
Knowledge Discovery Base
Figure 1 The MIR Three-Plane Model
Computer Systems and Operations
Clemson Computing and Information Technology
Using Condor with BlackboardSam Hoover
IT Systems ArchitectComputer Systems and Operations
CCIT, Clemson Universityshoover@clemson.edu
Computer Systems and Operations
Clemson Computing and Information Technology
Problem Statement
• Courses in Blackboard need to be archived
• Blackboard provides a batch archive script• Over 6,200 courses are currently active• The course list was split into equal
numbers across 5 servers• Archives took 60 hours to complete
Computer Systems and Operations
Clemson Computing and Information Technology
Implementing Condor with Blackboard
• Blackboard is a “heavy” Java Enterprise application
• We wanted to take advantage of the multiple cores per server
• Security is a requirement• Performance of the application during
processing MUST be maintained
Sam Hoover: shoover@clemson.edu Sam Hoover: shoover@clemson.edu
Computer Systems and Operations
Clemson Computing and Information Technology
Benefits of using Condor with Blackboard
• Job scheduling ensures load balancing and enables higher throughput through use of all available hardware resources
• Post processing with DAGMan streamlines operation
• Jobs are suspended and resumed automatically to protect user performance
• Archive time has been reduced by 65%• End of Semester “full” archive is now feasible
Computer Systems and Operations
Clemson Computing and Information Technology
Credits
• Clemson could not have achieved this innovation without the contributions of the following people and groups:• Sam Hoover, CSO - Principal Innovator• Randy Martin and Matt Garrett, CSO• NSF CI-TEAM award (RENCI principal)• Sebastien Goasguen, School of Computing• Blackboard Operations Team (CCIT-wide
representation)
•
www.cs.wisc.edu/~miron
Thank you for building such
a wonderful community
Recommended