I am looking into a Job Scheduler to replace extensive cron use on HP-UX and
Linux systems. Last I worked with this type of product was at least 5 years
ago, when the big 3 commercial products were Maestro, Control-M and AutoSys.

I would love to find an open source tool to do this, as I am not running
production batch scheduling (primarily monitoring and administrative scripts). 

Features I think are minimum:

1) date/time based scheduling
2) event-based (file creation, external trigger, etc) scheduling
3) notification on success/failure if desired (prefer a script/command trigger
   to SNMP)
4) job chaining/dependencies

Features I'd like to have:

1) ability to run jobs against servers in NIS or LDAP groups
2) ability to script changes to the schedules

If anyone wants to share their opinions of software they've used and liked (or
not), or other features they regularly use (or not), I would greatly appreciate
it.

And again, if you can point me to a decent open source tool or two, that would
be great.

Thanks,

Kevin

Kevin Collins wrote:

> I am looking into a Job Scheduler to replace extensive cron use on HP-UX
> and Linux systems. Last I worked with this type of product was at least 5
> years ago, when the big 3 commercial products were Maestro, Control-M and
> AutoSys.
> 
> I would love to find an open source tool to do this, as I am not running
> production batch scheduling (primarily monitoring and administrative
> scripts).
> 
> Features I think are minimum:
> 
> 1) date/time based scheduling
> 2) event-based (file creation, external trigger, etc) scheduling
> 3) notification on success/failure if desired (prefer a script/command
> trigger
>    to SNMP)
> 4) job chaining/dependencies
> 
> Features I'd like to have:
> 
> 1) ability to run jobs against servers in NIS or LDAP groups
> 2) ability to script changes to the schedules
> 
> If anyone wants to share their opinions of software they've used and liked
> (or not), or other features they regularly use (or not), I would greatly
> appreciate it.
> 
> And again, if you can point me to a decent open source tool or two, that
> would be great.
> 
> Thanks,
> 
> Kevin

We are about to publish some software to the open source community. That is
we have not done that yet.

One of that is a scheduler that may meet your needs. 
It is written in perl.
It is basically event based. (It was written to react on input files for
long running batch jobs)
It has internal job chaining facilities.
It does not support date/time based scheduling out of the box because I
thought cron can do that better. However this can easily implemented.

The open source archive shall be available somewhere in the beginning of the
next year.
But if you are interested it's no problem to send you a tar ball of the
software as it is to look into it.

Did you browse sourceforge.net? 

Wolfgang

In comp.unix.admin Kevin Collins <spamtotrash@toomuchfiction.com> wrote:
> I am looking into a Job Scheduler to replace extensive cron use on HP-UX and
> Linux systems. Last I worked with this type of product was at least 5 years
> ago, when the big 3 commercial products were Maestro, Control-M and AutoSys.

	I was waiting to see if anyone else would respond, either positively
or negatively, but as there's been little response, I thought I'd ask:
what's wrong with cron?  Since you say you've been using it, presumably it
has some failings that you're trying to remedy.  Since you list a number of
features which cron doesn't have, I guess those are what you're looking 
for...but the "Unix way" to solve these problems would simply be a combination
of cron and shell scripting.

> Features I think are minimum:
> 1) date/time based scheduling
	
	Cron, obviously.

> 2) event-based (file creation, external trigger, etc) scheduling

	This is a little unclear.  Event-based is kind of the opposite of
scheduled.  So when you say "file creation", do you mean "a scheduled check
every X minutes to see if file Y has been created", or "execution as soon
as file Y is created"?  Cron could handle the first, and as for the second
you might want to look at something like FAM: http://oss.sgi.com/projects/fam

> 3) notification on success/failure if desired (prefer a script/command trigger
>    to SNMP)

	This is also easily done with cron/shell/sendmail.

> 4) job chaining/dependencies

	Same here with cron/shell.


> Features I'd like to have:
> 1) ability to run jobs against servers in NIS or LDAP groups

	That's a pretty good one, and I'd be curious to know of any tools
for centralizing scheduling, beyond cron/ssh.


> 2) ability to script changes to the schedules

	Could you explain this one a little more?  You mean something like
"at 9am on April 5, change the logrotate job to run at 5am"?  Or am I 
misinterpreting this?


> And again, if you can point me to a decent open source tool or two, that 
> would be great.

	I realize that "cron + shell" is probably not the answer you were 
looking for, but I feel that it might still be a pretty good solution,
simply because
-it's open source
-you probably already have all the parts
-even if you don't, they're free
-they're simpler, more single-purpose tools.
As I mentioned earlier, the "Unix way" of doing things is generally to
have smaller, more single-purpose tools.  Cron exists to do scheduling; it
doesn't do email or checking for file existence or checking for the exit
status of other programs because we already have tools to do all those other
things, and duplicating their functionality within cron would only bloat 
cron and be a source of bugs.  Besides all this, I suspect there is little
in the open source world, because cron takes up 99% of that ecological niche,
so to speak.

	Anyway, only you can decide what you really need - it would take
a lot to make me move away from cron, but if you feel that the marginal
improvement of another scheduling client is worth the marginal cost, then
that choice may be right for you.

JDW

On Mon, 20 Dec 2004 16:25:40 +0000 (UTC), Jeremiah DeWitt Weiner <jdw@panix.com> wrote:
> In comp.unix.admin Kevin Collins <spamtotrash@toomuchfiction.com> wrote:
>> I am looking into a Job Scheduler to replace extensive cron use on HP-UX and
>> Linux systems. Last I worked with this type of product was at least 5 years
>> ago, when the big 3 commercial products were Maestro, Control-M and AutoSys.
> 
> 	I was waiting to see if anyone else would respond, either positively
> or negatively, but as there's been little response, I thought I'd ask:
> what's wrong with cron?  Since you say you've been using it, presumably it
> has some failings that you're trying to remedy.  

I'm not the OP, but I'm also watching the thread with interest.

For me, the "problem" with cron is that I've got 80 or 100 servers to
watch.  Because my employer is a publically traded US corporation, we
get to comply with "SOX" regulations, which basically enforce control
and verification of systems (thank you ENRON for doubling my work load).
Some of it is things we were doing already, some of it was things we
_should have_ been doing and now are, and some of it is just a big PITA.

Another problem is that when someone leaves, either for vacation or forever,
their cron jobs may or may not be important, may or may not keep running,
and probably arne't having their output checked for success or failure.

Anyway.  80 *nix boxes.  Hundreds of crontabs.  Lots of places for stuff
to hide, and not trivial to manage.  Also, let's say we need to move the
periodic job up or back for whatever reason.  Bit of a pain in the ass
to log in to each & change the crontab.

If I had a centralized system, I could group 'em, and say "This job
runs on this group of servers at this time, after this other job
finishes (or not), and notify (group) when it's done".

> Since you list a number of
> features which cron doesn't have, I guess those are what you're looking 
> for...but the "Unix way" to solve these problems would simply be a combination
> of cron and shell scripting.

I can home-grow something, but then I have _another_ home-grown app to
support.  We're short 2 people right now (Unix/Linix/Weblogic/Openview/
Apache shop), so if we could have something already written, even if it
took a while to implement, it'd be welcome.
 
>> Features I think are minimum:
> 
>> 2) event-based (file creation, external trigger, etc) scheduling
> 
> 	This is a little unclear.  Event-based is kind of the opposite of
> scheduled.  So when you say "file creation", do you mean "a scheduled check
> every X minutes to see if file Y has been created", or "execution as soon
> as file Y is created"?  

I think what he means is "Don't run the data export job until the data
extraction job finishes" type of interactions.  Eliminates the "Oh crap,
I gave it an hour, and it took 61 minutes to finish" race conditions.

> Cron could handle the first, and as for the second
> you might want to look at something like FAM: http://oss.sgi.com/projects/fam
 
Maybe that's what he's looking for, but I'm not seeing it.

>> 3) notification on success/failure if desired (prefer a script/command trigger
>>    to SNMP)
 
> 	This is also easily done with cron/shell/sendmail.
 
Yes, on a server-by-server-by-server-by-server-by-server basis.

>> Features I'd like to have:
 
>> 2) ability to script changes to the schedules
> 
> 	Could you explain this one a little more?  You mean something like
> "at 9am on April 5, change the logrotate job to run at 5am"?  Or am I 
> misinterpreting this?

"Oh shit, we need to run the financials this week on thursday instead
of friday.  I get to go change 30 crontabs.  Yay."  ...as opposed to
select "monthly financial jobs", change start date forward a day, and
be done.

>> And again, if you can point me to a decent open source tool or two, that 
>> would be great.
> 
> 	I realize that "cron + shell" is probably not the answer you were 
> looking for, but I feel that it might still be a pretty good solution,
> simply because
> -it's open source
> -you probably already have all the parts
> -even if you don't, they're free
> -they're simpler, more single-purpose tools.

And they become heavy and unwieldy for an enterprise-sized understaffed
organization.

> As I mentioned earlier, the "Unix way" of doing things is generally to
> have smaller, more single-purpose tools.  Cron exists to do scheduling; it
> doesn't do email or checking for file existence or checking for the exit
> status of other programs because we already have tools to do all those other
> things, and duplicating their functionality within cron would only bloat 
> cron and be a source of bugs.  Besides all this, I suspect there is little
> in the open source world, because cron takes up 99% of that ecological niche,
> so to speak.

I think you're understimating the usefulness of a true scheduling system.
Cron is great on each server it's on, but it doesn't have the "glue" to
talk nice between users, between boxes, or give flexability for 
scheduling, reporting, and dependancy changes.  It's one part of the
toolbox, but not the whole thing.

Dave Hinz

In article <cq1qpk$1mb$00$1@news.t-online.com>, Wolfgang Gr�per wrote:
> Kevin Collins wrote:
> 
>> I am looking into a Job Scheduler to replace extensive cron use on HP-UX
>> and Linux systems. Last I worked with this type of product was at least 5
>> years ago, when the big 3 commercial products were Maestro, Control-M and
>> AutoSys.
>> 
>> I would love to find an open source tool to do this, as I am not running
>> production batch scheduling (primarily monitoring and administrative
>> scripts).
>> 
>> Features I think are minimum:
>> 
>> 1) date/time based scheduling
>> 2) event-based (file creation, external trigger, etc) scheduling
>> 3) notification on success/failure if desired (prefer a script/command
>> trigger
>>    to SNMP)
>> 4) job chaining/dependencies
>> 
>> Features I'd like to have:
>> 
>> 1) ability to run jobs against servers in NIS or LDAP groups
>> 2) ability to script changes to the schedules
>> 
>> If anyone wants to share their opinions of software they've used and liked
>> (or not), or other features they regularly use (or not), I would greatly
>> appreciate it.
>> 
>> And again, if you can point me to a decent open source tool or two, that
>> would be great.
>> 
>> Thanks,
>> 
>> Kevin
> 
> We are about to publish some software to the open source community. That is
> we have not done that yet.
> 
> One of that is a scheduler that may meet your needs. 
> It is written in perl.
> It is basically event based. (It was written to react on input files for
> long running batch jobs)
> It has internal job chaining facilities.
> It does not support date/time based scheduling out of the box because I
> thought cron can do that better. However this can easily implemented.
> 
> The open source archive shall be available somewhere in the beginning of the
> next year.
> But if you are interested it's no problem to send you a tar ball of the
> software as it is to look into it.
> 
> Did you browse sourceforge.net? 
> 
> Wolfgang

Thanks for the response, Wolfgang. I would be interested in seeing your code
once it is released. I do browse sourceforge - in fact, I searched it for
schedulers before making this post and didn't find anything that would work.

What is the name of your project?

Kevin

In article <32oee7F3mstjiU1@individual.net>, Dave Hinz wrote:
> On Mon, 20 Dec 2004 16:25:40 +0000 (UTC), Jeremiah DeWitt Weiner <jdw@panix.com> wrote:
>> In comp.unix.admin Kevin Collins <spamtotrash@toomuchfiction.com> wrote:
>>> I am looking into a Job Scheduler to replace extensive cron use on HP-UX and
>>> Linux systems. Last I worked with this type of product was at least 5 years
>>> ago, when the big 3 commercial products were Maestro, Control-M and AutoSys.
>> 
>> 	I was waiting to see if anyone else would respond, either positively
>> or negatively, but as there's been little response, I thought I'd ask:
>> what's wrong with cron?  Since you say you've been using it, presumably it
>> has some failings that you're trying to remedy.  
> 
> I'm not the OP, but I'm also watching the thread with interest.

I am :)

> For me, the "problem" with cron is that I've got 80 or 100 servers to
> watch.  Because my employer is a publically traded US corporation, we
> get to comply with "SOX" regulations, which basically enforce control
> and verification of systems (thank you ENRON for doubling my work load).
> Some of it is things we were doing already, some of it was things we
> _should have_ been doing and now are, and some of it is just a big PITA.

Yeah, I have to say SOX SUX! We have spent countless wasted hours documenting
useless info.

> Another problem is that when someone leaves, either for vacation or forever,
> their cron jobs may or may not be important, may or may not keep running,
> and probably arne't having their output checked for success or failure.
> 
> Anyway.  80 *nix boxes.  Hundreds of crontabs.  Lots of places for stuff
> to hide, and not trivial to manage.  Also, let's say we need to move the
> periodic job up or back for whatever reason.  Bit of a pain in the ass
> to log in to each & change the crontab.
>
> If I had a centralized system, I could group 'em, and say "This job
> runs on this group of servers at this time, after this other job
> finishes (or not), and notify (group) when it's done".

In our case, we are running 99% of our cron jobs from a central, trusted server
via a series of scripts that use remsh/rsh. This simplifies the management
some, but adds complexity in other areas.

>> Since you list a number of
>> features which cron doesn't have, I guess those are what you're looking 
>> for...but the "Unix way" to solve these problems would simply be a combination
>> of cron and shell scripting.
> 
> I can home-grow something, but then I have _another_ home-grown app to
> support.  We're short 2 people right now (Unix/Linix/Weblogic/Openview/
> Apache shop), so if we could have something already written, even if it
> took a while to implement, it'd be welcome.

Yes, exactly. I well know the "Unix way" - I specialize in scripting and
automation, but a real job scheduler is a big beast with a lot of intricacies.
And that involves a lot of time, testing, documenting, etc.

>>> Features I think are minimum:
>> 
>>> 2) event-based (file creation, external trigger, etc) scheduling
>> 
>> 	This is a little unclear.  Event-based is kind of the opposite of
>> scheduled.  So when you say "file creation", do you mean "a scheduled check
>> every X minutes to see if file Y has been created", or "execution as soon
>> as file Y is created"?  

Yes, that is correct.

> I think what he means is "Don't run the data export job until the data
> extraction job finishes" type of interactions.  Eliminates the "Oh crap,
> I gave it an hour, and it took 61 minutes to finish" race conditions.

Actually, waiting for another job to finish is job chaining. The previous
poster had it right on the file creation (this is a feature many job schedulers
have).

>> Cron could handle the first, and as for the second
>> you might want to look at something like FAM: http://oss.sgi.com/projects/fam
>  
> Maybe that's what he's looking for, but I'm not seeing it.
> 
>>> 3) notification on success/failure if desired (prefer a script/command trigger
>>>    to SNMP)
>  
>> 	This is also easily done with cron/shell/sendmail.
>  
> Yes, on a server-by-server-by-server-by-server-by-server basis.
> 
>>> Features I'd like to have:
>  
>>> 2) ability to script changes to the schedules
>> 
>> 	Could you explain this one a little more?  You mean something like
>> "at 9am on April 5, change the logrotate job to run at 5am"?  Or am I 
>> misinterpreting this?

Not quite right... Mostly, what I'm looking for is a command line interface, so
that I can change a job or group of jobs in an automated fashion. For example,
we have groups of scripts that we run against various NIS netgroup maps. These
maps change frequently and I'd like to be able to script changing the run list
to the current systems for a given day/hour/etc...

> "Oh shit, we need to run the financials this week on thursday instead
> of friday.  I get to go change 30 crontabs.  Yay."  ...as opposed to
> select "monthly financial jobs", change start date forward a day, and
> be done.

Or that...

>>> And again, if you can point me to a decent open source tool or two, that 
>>> would be great.
>> 
>> 	I realize that "cron + shell" is probably not the answer you were 
>> looking for, but I feel that it might still be a pretty good solution,
>> simply because
>> -it's open source
>> -you probably already have all the parts
>> -even if you don't, they're free
>> -they're simpler, more single-purpose tools.
> 
> And they become heavy and unwieldy for an enterprise-sized understaffed
> organization.

Yes, that is certainly true.

>> As I mentioned earlier, the "Unix way" of doing things is generally to
>> have smaller, more single-purpose tools.  Cron exists to do scheduling; it
>> doesn't do email or checking for file existence or checking for the exit
>> status of other programs because we already have tools to do all those other
>> things, and duplicating their functionality within cron would only bloat 
>> cron and be a source of bugs.  Besides all this, I suspect there is little
>> in the open source world, because cron takes up 99% of that ecological niche,
>> so to speak.
> 
> I think you're understimating the usefulness of a true scheduling system.
> Cron is great on each server it's on, but it doesn't have the "glue" to
> talk nice between users, between boxes, or give flexability for 
> scheduling, reporting, and dependancy changes.  It's one part of the
> toolbox, but not the whole thing.

Agreed. Additionally, introducing job-chaining (for example) now introduces
things like possible circular depnedencies (Job-A runs after Job-B which runs
after Job-A), and other issues. Also, a lot of commercial schedulers also have
interfaces into large ERP systems such as SAP, which could be useful, too.

I didn't get the answer I was hoping for, but thanks to both of you for your
replies.

Also, just to give you an idea of our scope, I have a perl script with parses a
crontab file and reports each job run (think about * * *... entries) - the
current count for 1 week is 23053 cron jobs to be run. And many of those single
jobs run against 50-100 servers!

Thanks,

Kevin

On Mon, 20 Dec 2004 17:42:00 GMT, Kevin Collins <spamtotrash@toomuchfiction.com> wrote:
> In article <32oee7F3mstjiU1@individual.net>, Dave Hinz wrote:
>> 
>> I'm not the OP, but I'm also watching the thread with interest.
> 
> I am :)
 
>> and some of it is just a big PITA.
> 
> Yeah, I have to say SOX SUX! We have spent countless wasted hours documenting
> useless info.

Yes.

>> If I had a centralized system, I could group 'em, and say "This job
>> runs on this group of servers at this time, after this other job
>> finishes (or not), and notify (group) when it's done".
> 
> In our case, we are running 99% of our cron jobs from a central, trusted server
> via a series of scripts that use remsh/rsh. This simplifies the management
> some, but adds complexity in other areas.

You've still got r-services turned on??!!?!?  ssh and keys is the way to go.

>> I can home-grow something, but then I have _another_ home-grown app to
>> support.  We're short 2 people right now (Unix/Linix/Weblogic/Openview/
>> Apache shop), so if we could have something already written, even if it
>> took a while to implement, it'd be welcome.
 
> Yes, exactly. I well know the "Unix way" - I specialize in scripting and
> automation, but a real job scheduler is a big beast with a lot of intricacies.
> And that involves a lot of time, testing, documenting, etc.

Yes, time I don't have because I'm doing _other_ stuff, half of which is
SOX-driven.

>> I think what he means is "Don't run the data export job until the data
>> extraction job finishes" type of interactions.  Eliminates the "Oh crap,
>> I gave it an hour, and it took 61 minutes to finish" race conditions.
> 
> Actually, waiting for another job to finish is job chaining. The previous
> poster had it right on the file creation (this is a feature many job schedulers
> have).

Gotcha.  I'm not up on the terminology (yet), thanks for that.

>> "Oh shit, we need to run the financials this week on thursday instead
>> of friday.  I get to go change 30 crontabs.  Yay."  ...as opposed to
>> select "monthly financial jobs", change start date forward a day, and
>> be done.
> 
> Or that...

Real-life example from this week, y'see.

>> I think you're understimating the usefulness of a true scheduling system.
>> Cron is great on each server it's on, but it doesn't have the "glue" to
>> talk nice between users, between boxes, or give flexability for 
>> scheduling, reporting, and dependancy changes.  It's one part of the
>> toolbox, but not the whole thing.
> 
> Agreed. Additionally, introducing job-chaining (for example) now introduces
> things like possible circular depnedencies (Job-A runs after Job-B which runs
> after Job-A), and other issues. Also, a lot of commercial schedulers also have
> interfaces into large ERP systems such as SAP, which could be useful, too.
 
We've got Tivoli scheduling in-house, it's just a matter of getting it
set up for the Unix environment.  Guess who gets to do it?

> I didn't get the answer I was hoping for, but thanks to both of you for your
> replies.
 
Well, I learned something anyway, so thanks for asking.

> Also, just to give you an idea of our scope, I have a perl script with parses a
> crontab file and reports each job run (think about * * *... entries) - the
> current count for 1 week is 23053 cron jobs to be run. And many of those single
> jobs run against 50-100 servers!
 
I wrote a "go fetch all the crontabs from all the servers" set of scripts,
and we've got a metric boatload of them.  Now I get to go through, see
which ones are SOX-ish enough to require going to a centralized control
system, and go from there.  Of course, this is in a community where
the developers still resist change of "use ssh instead of rlogin...
NO, you can't have the root passwords, here's how to use sudo for what
you really need to do...I'm sorry but you really really need to migrate
this crap off of Solaris 2.6...", so it'll be an interesting series
of conversations.

Dave Hinz

Kevin Collins wrote:

> In article <cq1qpk$1mb$00$1@news.t-online.com>, Wolfgang Gr�per wrote:
>> Kevin Collins wrote:
>> 
>>> I am looking into a Job Scheduler to replace extensive cron use on HP-UX
>>> and Linux systems. Last I worked with this type of product was at least
>>> 5 years ago, when the big 3 commercial products were Maestro, Control-M
>>> and AutoSys.
>>> 
>>> I would love to find an open source tool to do this, as I am not running
>>> production batch scheduling (primarily monitoring and administrative
>>> scripts).
>>> 
>>> Features I think are minimum:
>>> 
>>> 1) date/time based scheduling
>>> 2) event-based (file creation, external trigger, etc) scheduling
>>> 3) notification on success/failure if desired (prefer a script/command
>>> trigger
>>>    to SNMP)
>>> 4) job chaining/dependencies
>>> 
>>> Features I'd like to have:
>>> 
>>> 1) ability to run jobs against servers in NIS or LDAP groups
>>> 2) ability to script changes to the schedules
>>> 
>>> If anyone wants to share their opinions of software they've used and
>>> liked (or not), or other features they regularly use (or not), I would
>>> greatly appreciate it.
>>> 
>>> And again, if you can point me to a decent open source tool or two, that
>>> would be great.
>>> 
>>> Thanks,
>>> 
>>> Kevin
>> 
>> We are about to publish some software to the open source community. That
>> is we have not done that yet.
>> 
>> One of that is a scheduler that may meet your needs.
>> It is written in perl.
>> It is basically event based. (It was written to react on input files for
>> long running batch jobs)
>> It has internal job chaining facilities.
>> It does not support date/time based scheduling out of the box because I
>> thought cron can do that better. However this can easily implemented.
>> 
>> The open source archive shall be available somewhere in the beginning of
>> the next year.
>> But if you are interested it's no problem to send you a tar ball of the
>> software as it is to look into it.
>> 
>> Did you browse sourceforge.net?
>> 
>> Wolfgang
> 
> Thanks for the response, Wolfgang. I would be interested in seeing your
> code once it is released. I do browse sourceforge - in fact, I searched it
> for schedulers before making this post and didn't find anything that would
> work.
> 
> What is the name of your project?
> 
> Kevin

This probably will be either EBCS (Event Batch Control System) 
or RBBC (Rule Based Batch Control).

Wolfgang
 

Jeremiah DeWitt Weiner wrote:

> In comp.unix.admin Kevin Collins <spamtotrash@toomuchfiction.com> wrote:
>> I am looking into a Job Scheduler to replace extensive cron use on HP-UX
>> and Linux systems. Last I worked with this type of product was at least 5
>> years ago, when the big 3 commercial products were Maestro, Control-M and
>> AutoSys.
> 
> I was waiting to see if anyone else would respond, either positively
> or negatively, but as there's been little response, I thought I'd ask:
> what's wrong with cron?  Since you say you've been using it, presumably it
> has some failings that you're trying to remedy.  Since you list a number
> of features which cron doesn't have, I guess those are what you're looking
> for...but the "Unix way" to solve these problems would simply be a
> combination of cron and shell scripting.
> 
>> Features I think are minimum:
>> 1) date/time based scheduling
> 
> Cron, obviously.
> 
>> 2) event-based (file creation, external trigger, etc) scheduling
> 
> This is a little unclear.  Event-based is kind of the opposite of
> scheduled.  So when you say "file creation", do you mean "a scheduled
> check every X minutes to see if file Y has been created", or "execution as
> soon
> as file Y is created"?  Cron could handle the first, and as for the second
> you might want to look at something like FAM:
> http://oss.sgi.com/projects/fam
> 
>> 3) notification on success/failure if desired (prefer a script/command
>> trigger
>>    to SNMP)
> 
> This is also easily done with cron/shell/sendmail.
> 
>> 4) job chaining/dependencies
> 
> Same here with cron/shell.
> 
> 
>> Features I'd like to have:
>> 1) ability to run jobs against servers in NIS or LDAP groups
> 
> That's a pretty good one, and I'd be curious to know of any tools
> for centralizing scheduling, beyond cron/ssh.
> 
> 
>> 2) ability to script changes to the schedules
> 
> Could you explain this one a little more?  You mean something like
> "at 9am on April 5, change the logrotate job to run at 5am"?  Or am I
> misinterpreting this?
> 
> 
>> And again, if you can point me to a decent open source tool or two, that
>> would be great.
> 
> I realize that "cron + shell" is probably not the answer you were
> looking for, but I feel that it might still be a pretty good solution,
> simply because
> -it's open source
> -you probably already have all the parts
> -even if you don't, they're free
> -they're simpler, more single-purpose tools.
> As I mentioned earlier, the "Unix way" of doing things is generally to
> have smaller, more single-purpose tools.  Cron exists to do scheduling; it
> doesn't do email or checking for file existence or checking for the exit
> status of other programs because we already have tools to do all those
> other things, and duplicating their functionality within cron would only
> bloat
> cron and be a source of bugs.  Besides all this, I suspect there is little
> in the open source world, because cron takes up 99% of that ecological
> niche, so to speak.
> 
> Anyway, only you can decide what you really need - it would take
> a lot to make me move away from cron, but if you feel that the marginal
> improvement of another scheduling client is worth the marginal cost, then
> that choice may be right for you.
> 
> JDW

The "UNIX way to do it" is allwayes the first choice. 
(This is what I learned and this is still my oppinion)
And in most situations it is the best choice too.

But there are limits.
Consider following situation:

<Scenario>
----------------
There is an ERP and some legacy sytems. A third party software - lets say a
CRM - will need to interact.

Batch:
Wait for new "order" data from ERP .
Wait for updated "item" data from legacy-X.
If both data files are then there merge them using some clever software
Import data into CRM
Do computation1 inside CRM
Do computation2 inside CRM after computation1 has finished
Do computation3 inside CRM after computation2 has finished
Wait for updated "customer" data from legacy-Y
Import this into CRM after computation3 is ready
Do some more calculation inside CRM
Export "sales" data to get imported into ERP

Do each step only if no errors occured in one of the previous steps.
---------
</Scenario>

If you go and try to solve this with cron/shell you are lost.
You start up with an UNIX-Admin job and end up inside a software development
project that has no end.

Cron is basically time based, but time is just one thing that can trigger
things to start. 

Wolfgang

In article <32oiu1F3pm2a0U2@individual.net>, Dave Hinz wrote:
> On Mon, 20 Dec 2004 17:42:00 GMT, Kevin Collins
> <spamtotrash@toomuchfiction.com> wrote:
>> In article <32oee7F3mstjiU1@individual.net>, Dave Hinz wrote:
>>> 
>>> I'm not the OP, but I'm also watching the thread with interest.
>> 
>> I am :)
>  
>>> and some of it is just a big PITA.
>> 
>> Yeah, I have to say SOX SUX! We have spent countless wasted hours
>> documenting useless info.
> 
> Yes.
> 
>>> If I had a centralized system, I could group 'em, and say "This job runs on
>>> this group of servers at this time, after this other job finishes (or not),
>>> and notify (group) when it's done".
>> 
>> In our case, we are running 99% of our cron jobs from a central, trusted
>> server via a series of scripts that use remsh/rsh. This simplifies the
>> management some, but adds complexity in other areas.
> 
> You've still got r-services turned on??!!?!?  ssh and keys is the way to go.

Yes, in a secure, non-internet connected environment.  I'd love to have ssh and
keys, but that is a ways down the road. There is so much infastructure here
built on remsh that it would be a very major undertaking to convert to ssh.
Plus, with the number of commands we run this way, overhead (in time) of using
ssh vs remsh is considerable - although its getting better.

>>> I can home-grow something, but then I have _another_ home-grown app to
>>> support.  We're short 2 people right now (Unix/Linix/Weblogic/Openview/
>>> Apache shop), so if we could have something already written, even if it
>>> took a while to implement, it'd be welcome.
>  
>> Yes, exactly. I well know the "Unix way" - I specialize in scripting and
>> automation, but a real job scheduler is a big beast with a lot of
>> intricacies.  And that involves a lot of time, testing, documenting, etc.
> 
> Yes, time I don't have because I'm doing _other_ stuff, half of which is
> SOX-driven.
> 
>>> I think what he means is "Don't run the data export job until the data
>>> extraction job finishes" type of interactions.  Eliminates the "Oh crap, I
>>> gave it an hour, and it took 61 minutes to finish" race conditions.
>> 
>> Actually, waiting for another job to finish is job chaining. The previous
>> poster had it right on the file creation (this is a feature many job
>> schedulers have).
> 
> Gotcha.  I'm not up on the terminology (yet), thanks for that.
> 
>>> "Oh shit, we need to run the financials this week on thursday instead of
>>> friday.  I get to go change 30 crontabs.  Yay."  ...as opposed to select
>>> "monthly financial jobs", change start date forward a day, and be done.
>> 
>> Or that...
> 
> Real-life example from this week, y'see.
> 
>>> I think you're understimating the usefulness of a true scheduling system.
>>> Cron is great on each server it's on, but it doesn't have the "glue" to
>>> talk nice between users, between boxes, or give flexability for scheduling,
>>> reporting, and dependancy changes.  It's one part of the toolbox, but not
>>> the whole thing.
>> 
>> Agreed. Additionally, introducing job-chaining (for example) now introduces
>> things like possible circular depnedencies (Job-A runs after Job-B which
>> runs after Job-A), and other issues. Also, a lot of commercial schedulers
>> also have interfaces into large ERP systems such as SAP, which could be
>> useful, too.
>  
> We've got Tivoli scheduling in-house, it's just a matter of getting it set up
> for the Unix environment.  Guess who gets to do it?

I used to work for Apple and we were using the Tivoli job scheduler on AIX
before it was part of Tivoli (I think it was called AIX Job Scheduler - ported
from mainframe). It was pretty nice as far as interface and features, but it
was not stable. We actually had DAILY conference calls with IBM development
staff in Italy for a couple months to work out the kinks. Eventually, they
ditched it for another product (Control-M, I think). Thats why I know about
circular dependencies, and other gotchas...

>> I didn't get the answer I was hoping for, but thanks to both of you for your
>> replies.
>  
> Well, I learned something anyway, so thanks for asking.
> 
>> Also, just to give you an idea of our scope, I have a perl script with
>> parses a crontab file and reports each job run (think about * * *...
>> entries) - the current count for 1 week is 23053 cron jobs to be run. And
>> many of those single jobs run against 50-100 servers!
>  
> I wrote a "go fetch all the crontabs from all the servers" set of scripts,
> and we've got a metric boatload of them.  Now I get to go through, see which
> ones are SOX-ish enough to require going to a centralized control system, and
> go from there.  Of course, this is in a community where the developers still
> resist change of "use ssh instead of rlogin...  NO, you can't have the root
> passwords, here's how to use sudo for what you really need to do...I'm sorry
> but you really really need to migrate this crap off of Solaris 2.6...", so
> it'll be an interesting series of conversations.

Fortunately for me, there is a smallish group (~30 including sys-admins) that
have shell access, and the whole group is fairly open for change. When I
started ~4.5 years ago, there was no Perl, Linux, PHP, Apache, sendmail, ssh,
sudo, expect, etc - I've been the instigator and implementor of much of that,
and its been well embraced for the most part.

But I've been in that shop where everyone's using emacs, csh and telnet and
won't change to save their life no matter how many times better alternatives
are given... :)

Thanks,

Kevin

Dave Hinz <DaveHinz@spamcop.net> wrote:
> Anyway.  80 *nix boxes.  Hundreds of crontabs.  Lots of places for stuff
> to hide, and not trivial to manage.  Also, let's say we need to move the
> periodic job up or back for whatever reason.  Bit of a pain in the ass
> to log in to each & change the crontab.
> 
> If I had a centralized system, ...

Is there reason why those 80 servers can't be consolidated into one (or
two) server, and replaced by 80 thin-clients?  That would clean up lots
of infrastructure problem.

-- 
William Park <opengeometry@yahoo.ca>
Open Geometry Consulting, Toronto, Canada
Linux solution for data processing. 

On 21 Dec 2004 02:32:03 GMT, William Park <opengeometry@yahoo.ca> wrote:
> Dave Hinz <DaveHinz@spamcop.net> wrote:
>> Anyway.  80 *nix boxes.  Hundreds of crontabs.  Lots of places for stuff
>> to hide, and not trivial to manage.  
 
> Is there reason why those 80 servers can't be consolidated into one (or
> two) server, and replaced by 80 thin-clients?  That would clean up lots
> of infrastructure problem.

More than a couple reasons, yes.  First, they're in two separate data 
centers.  Second, they're on physically different networks, with firewalls
between.  Additionally, for regulatory compliance reasons (either real or
as they have been interpreted by management and therfore real), we have 
3 distinct hardware groups - dev, QA, and Production.  We're working towards
a fourth, "staging/Disaster Recovery", in the other data center.

Also, our application and infrastructure here are built on the "need more
capacity?  Add another box to the custer and/or load balanced farm"
approach.

Further, many of our applications are licensed on a cost-per-cpu basis.
Particularly Oracle.  If we put it onto a 64 CPU box, our license costs
would quadruple, and we'd be paying for CPU that Oracle wasn't running on.

Then there's the fact that I work for an insurance company in the financial
(mortgage) business, so they are _very_ conservative in their approach.
I'm happy I've been able to introduce open-source to the world here,
trying to wedge a "let's get a really really big Unix box" approach in
is neither viable nor does it fit our architecture very well.  Besides,
if I proposed something like that, they'd just want to stick more stuff on
the mainframe here.

Other than those, no particular reason, why do you ask?

Dave Hinz

Dave Hinz <DaveHinz@spamcop.net> wrote:
> On 21 Dec 2004 02:32:03 GMT, William Park <opengeometry@yahoo.ca>
> wrote:
> > Dave Hinz <DaveHinz@spamcop.net> wrote:
> >> Anyway.  80 *nix boxes.  Hundreds of crontabs.  Lots of places for
> >> stuff to hide, and not trivial to manage.  
>  
> > Is there reason why those 80 servers can't be consolidated into one
> > (or two) server, and replaced by 80 thin-clients?  That would clean
> > up lots of infrastructure problem.
> 
> More than a couple reasons, yes.  First, they're in two separate data
> centers.  Second, they're on physically different networks, with
> firewalls between.  Additionally, for regulatory compliance reasons
> (either real or as they have been interpreted by management and
> therfore real), we have 3 distinct hardware groups - dev, QA, and
> Production.  We're working towards a fourth, "staging/Disaster
> Recovery", in the other data center.
> 
> Also, our application and infrastructure here are built on the "need
> more capacity?  Add another box to the custer and/or load balanced
> farm" approach.
> 
> Further, many of our applications are licensed on a cost-per-cpu
> basis.  Particularly Oracle.  If we put it onto a 64 CPU box, our
> license costs would quadruple, and we'd be paying for CPU that Oracle
> wasn't running on.
> 
> Then there's the fact that I work for an insurance company in the
> financial (mortgage) business, so they are _very_ conservative in
> their approach.  I'm happy I've been able to introduce open-source to
> the world here, trying to wedge a "let's get a really really big Unix
> box" approach in is neither viable nor does it fit our architecture
> very well.  Besides, if I proposed something like that, they'd just
> want to stick more stuff on the mainframe here.
> 
> Other than those, no particular reason, why do you ask?

Dual Opteron (with gobs of memory) should be able to crunch data /and/
service all the thin-clients you want.  Commodity PC is always changing
the equation.  What made sense for 9GB SCSI harddisks, no longer make
sense for 500GB USB harddisks.  Same for memory and cpu.

If dev/QA/Production groups have one "main" server each, then you just
have to babysit 3 servers, instead of 80 servers.  But, then again,
whatever management says goes. :-)

-- 
William Park <opengeometry@yahoo.ca>
Open Geometry Consulting, Toronto, Canada
Linux solution for data processing.