Unix Shell - how to avoid multiple script startup overhead

This is Interesting: Free IT Magazines  
Home > Archive > Unix Shell > October 2006 > how to avoid multiple script startup overhead





You are viewing an archived Text-only version of the thread. To view this thread in it's original format and/or if you want to reply to this thread please [click here]

Author how to avoid multiple script startup overhead
inetquestion

2006-10-20, 7:23 pm

I'd like to get some opinions on possible new solutions to handle a
problem we are seeing on our solaris servers due to various cron jobs
aligning at various times throughout the hour. The
obvious answer of changing the times so they do not line up really
isn't feasible and wont really buy us much in the long term. What
we've see is the spawning of new PERL and ksh interpreters to run
various scripts is what is consuming the bulk of the cpu. One idea
I've been contimplating for PERL is having one master script running to
avoid starting the interpreter up each minute. What I'm unclear about
is the ability to fork off entire scripts from there. Will this use an
instance of the existing interpreter, or just spawn another instance.
I'm not sure if this approach even begins to address the same concerns
with ksh scripts. Any suggestions along these lines or completely new
ideas altogether?

Regards,

-Inet

base60

2006-10-20, 7:23 pm

inetquestion wrote:
> I'd like to get some opinions on possible new solutions to handle a
> problem we are seeing on our solaris servers due to various cron jobs
> aligning at various times throughout the hour. The
> obvious answer of changing the times so they do not line up really
> isn't feasible and wont really buy us much in the long term. What
> we've see is the spawning of new PERL and ksh interpreters to run
> various scripts is what is consuming the bulk of the cpu.


Even given the issues of interpreted languages, what you seem to be
saying is difficult to picture.

It doesn't take substantial amounts of CPU to start PERL and even less
for ksh.

> One idea
> I've been contimplating for PERL is having one master script running to
> avoid starting the interpreter up each minute. What I'm unclear about
> is the ability to fork off entire scripts from there. Will this use an
> instance of the existing interpreter, or just spawn another instance.
> I'm not sure if this approach even begins to address the same concerns
> with ksh scripts. Any suggestions along these lines or completely new
> ideas altogether?


Python. You can compile it.
bsh

2006-10-21, 1:27 am


inetquestion wrote:
> ... spawning of new PERL and ksh interpreters to run various scripts
> is consuming the bulk of the cpu.
> ... Any suggestions along these lines or completely new ideas altogether?


For an enterprise environment, the trouble of writing (and debugging!)
a serialization system for cron(1) jobs is problematic. May I suggest
that you instead need a batch queuing system? Perhaps:

"Portable Batch System"
http://www.mta.ca/torch/doc_pbs.html

"GNU Queue"
http://www.gnuqueue.org/home.html

A ksh-only solution:

"qjob.ksh"
http://www.unixreview.com/documents...0508h/0508h.htm
http://www.samag.com/documents/s=98...8h/0508h_l1.htm

There are many others, for varying degrees of sophistication. Do a
search on the Internet and previous C.U.S. threads that have already
discussed this.

=Brian

Sponsored Links






Free braindumps | Software forum | Database administration forum

Copyright 2003 - 2008 webservertalk.com