Lech,<div><br></div><div>That is a bug. Does this patch fix the issue for you?</div><div><br></div><div>David<br><br><div class="gmail_quote">On Thu, Nov 15, 2012 at 7:09 AM, Lech Nieroda <span dir="ltr"><<a href="mailto:nieroda.lech@uni-koeln.de" target="_blank">nieroda.lech@uni-koeln.de</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Dear list,<br>
<br>
we've upgraded our torque 2.5.11 to torque 4.1.3 and have run into<br>
several problems. The most annoying one is the no longer working<br>
job_nanny feature. We are using torque together with maui 3.3.1.<br>
<br>
To reproduce the problem we submit a job and as soon as it is running,<br>
kill the pbs_mom on the appropriate node with "kill -9" (the idea here<br>
is to simulate a node crash). Some time after the walltime is exceeded<br>
maui sends deletion requests to the pbs_server and a mail is spawned to<br>
the user. Considering that maui does this every 60 seconds, this amounts<br>
to a sizeable amount of emails.<br>
<br>
On torque 2.5.x this was inhibited by the "job_nanny" feature: any<br>
further deletion requests of maui were met with a rejection ("job cancel<br>
in progress"). However, this doesn't work on torque 4.3.1 anymore. The<br>
feature is set to "true" on the pbs_server but each one of maui's<br>
deletion requests triggers an email.<br>
<br>
We've tried to set "$ignwalltime true" on the clients, to no avail.<br>
<br>
Here the relevant pbs_server logs with log_level 3:<br>
<br>
[snip]<br>
11/15/2012 13:18:20 S Job deleted at request of<br>
maui@localhost.localdomain<br>
11/15/2012 13:18:20 S preparing to send 'd' mail for job<br>
670947.cheops10 to nierodal@cheops10 (Job deleted at request of<br>
maui@localhost.localdomain<br>
11/15/2012 13:18:20 S Job sent signal SIGTERM on delete<br>
11/15/2012 13:19:27 S Job deleted at request of<br>
maui@localhost.localdomain<br>
11/15/2012 13:19:27 S preparing to send 'd' mail for job<br>
670947.cheops10 to nierodal@cheops10 (Job deleted at request of<br>
maui@localhost.localdomain<br>
11/15/2012 13:19:27 S Job sent signal SIGTERM on delete<br>
11/15/2012 13:20:30 S Job deleted at request of<br>
maui@localhost.localdomain<br>
11/15/2012 13:20:30 S preparing to send 'd' mail for job<br>
670947.cheops10 to nierodal@cheops10 (Job deleted at request of<br>
maui@localhost.localdomain<br>
11/15/2012 13:20:30 S Job sent signal SIGTERM on delete<br>
[snap]<br>
<br>
Is this a bug? Can a parameter change this behaviour?<br>
So far, we'd had to disable the mail functionality.<br>
<br>
Regards,<br>
Lech Nieroda<br>
<br>
PS: I've resent this mail since it didn't appear to have hit the list.<br>
<br>
--<br>
Dipl.-Wirt.-Inf. Lech Nieroda<br>
Regionales Rechenzentrum der Universität zu Köln (RRZK)<br>
Universität zu Köln<br>
Weyertal 121<br>
Raum 309 (3. Etage)<br>
D-50931 Köln<br>
Deutschland<br>
<br>
Tel.: <a href="tel:%2B49%20%28221%29%20470-89606" value="+4922147089606">+49 (221) 470-89606</a><br>
E-Mail: nieroda.lech at <a href="http://uni-koeln.de" target="_blank">uni-koeln.de</a><br>
_______________________________________________<br>
torqueusers mailing list<br>
<a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a><br>
<a href="http://www.supercluster.org/mailman/listinfo/torqueusers" target="_blank">http://www.supercluster.org/mailman/listinfo/torqueusers</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div>David Beer | Senior Software Engineer</div><div>Adaptive Computing</div><br>
</div>