Transcript What’s New in Condor - Istituto Nazionale di Fisica Nucleare

What’s New in Condor
Alain Roy
Computer Sciences Department
University of Wisconsin-Madison
[email protected]
http://www.cs.wisc.edu/condor
Condor & INFN
› Feedback from INFN users has been
very useful and has given us new
features
Bologna Batch System & new policies
› I will be in Bologna, helping with
Condor, June 16-July 11
www.cs.wisc.edu/condor
Condor Development Process
› We maintain two different releases
at all times
Stable Series
• Second digit is even: e.g. 6.2.2, 6.4.7, 6.6.0
Development Series
• Second digit is odd: e.g. 6.3.1, 6.5.2
www.cs.wisc.edu/condor
Stable Series
› Heavily tested
› Runs on our production pool of nearly 1,000 CPUs
› No new features, only bugfixes, are allowed into a
›
stable series
A given stable release is always
compatible with other releases from
the same series
 6.4.X is compatible with 6.4.Y
› Recommended for production pools
www.cs.wisc.edu/condor
Development Series
› Less heavily tested
› New features are added frequently
› Compatibility in series is not
guaranteed
www.cs.wisc.edu/condor
Computing on Demand
› Run executable now
› Suspends job while running
› Useful for short-running important
work
www.cs.wisc.edu/condor
Improved Negotiation
› The condor_schedd (the job queue
manager) sends “classes” of jobs for
matching
› Previously, jobs were sent one at a
time.
› 1000 similar jobs will take the same
time to negotiate as one job
www.cs.wisc.edu/condor
Smarter File Transfer
› New file transfer mechanism:
 IF_NEEDED : will automatically transfer the
files if the submit and execute machine do not
share file system
 Users used to have to decide
www.cs.wisc.edu/condor
Better Security
› GSI (X.509 Certificates) implementation more
complete and customizable
 Each Condor daemon can have its own certificate
 You can run a “Personal Condor” with your user proxy
› Easier configuration
 Particularly if you already use Globus
› Improved error messages
www.cs.wisc.edu/condor
Central Manager
› Redundant central managers allowed
› If the main central manager goes down, you may
›
›
still run administrative commands
Central manager daemons can now run on any port
Can use TCP instead of UDP
www.cs.wisc.edu/condor
Avoiding Black Holes
› Condor can keep track of the last few
›
›
resource matches
This can be used to prefer the same
machine if restarted
Can also be used to avoid a machine if
restarted, which is a first step towards
avoiding “Black Holes”
www.cs.wisc.edu/condor
New Installer
› For UNIX
 Version 6.6.0 will be available as RPMs
 Command line options specify the installation
parameters, and no questions are asked
 Easier to automate
› For Windows
 Based on MSI (Microsoft Software Installer)
 Batch Install option
www.cs.wisc.edu/condor
ClassAds
› ClassAd attributes can be linked to
external functions
Rank = complex_function(TARGET.Owner)
www.cs.wisc.edu/condor
New Features in DAGMan
› Each job can now have it’s own log file
› XML userlogs
› Can produce pretty graphs
www.cs.wisc.edu/condor
Condor for Windows
› Ability to run scheduler universe jobs
 DAGMan now works on Windows
› JAVA universe support
 Better error management
 Ability to use CHIRP (Remote I/O)
www.cs.wisc.edu/condor
Improvements in Windows
(cont)
› Support for:
 Windows XP
 Foreign Language versions of Windows
 Legacy 16-bit app
› Improved Windows-to-UNIX job
›
submission and vice versa.
BirdWatcher, a system tray icon which
gives basic status and control of Condor
www.cs.wisc.edu/condor
More information
› Information: www.cs.wisc.edu/condor
› Support: [email protected]
› Ask me questions!
www.cs.wisc.edu/condor