<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
On 09/28/2010 08:57 AM, Abraham Zamudio wrote:
<blockquote
cite="mid:AANLkTi=t1V88vq3BVBJqeOMsT0QjfgWfnN7cMhvOt1QD@mail.gmail.com"
type="cite">
<meta http-equiv="content-type"
content="text/html; charset=ISO-8859-1">
<font class="Apple-style-span" color="#eeffe2"
face="arial, sans-serif"><span class="Apple-style-span"
style="border-collapse: collapse;"><font class="Apple-style-span"
color="#000000"><span class="Apple-style-span"
style="font-size: large;">Hi everybody , </span></font></span></font>
<div><font class="Apple-style-span" color="#eeffe2"
face="arial, sans-serif"><span class="Apple-style-span"
style="border-collapse: collapse;"><font class="Apple-style-span"
color="#000000"><span class="Apple-style-span"
style="font-size: large;"><br>
</span></font></span></font></div>
<div><font class="Apple-style-span" color="#eeffe2"
face="arial, sans-serif" size="4"><span class="Apple-style-span"
style="border-collapse: collapse; font-size: 15px;"><font
class="Apple-style-span" color="#000000"><span class="Apple-style-span"
style="border-collapse: separate; font-size: 13.3333px; color: rgb(136, 136, 136);">
<div id="gt-res-content" class="almost_half_cell"
style="padding-top: 9px; padding-right: 16px;">
<div dir="ltr" style=""><span id="result_box" class="short_text"
style="display: block;"><span title=""><span class="Apple-style-span"
style="background-color: rgb(255, 255, 255);"><font
class="Apple-style-span" color="#000000"><span class="Apple-style-span"
style="font-size: large;">I have a problem with one of my nodes : </span></font></span></span></span><span
id="result_box" class="short_text" style="display: block;"><span
title=""><span class="Apple-style-span"
style="background-color: rgb(255, 255, 255);"><font
class="Apple-style-span" color="#000000"><span class="Apple-style-span"
style="font-size: large;"><br>
</span></font></span></span></span><span id="result_box"
class="short_text" style="display: block;"><span title=""><span
class="Apple-style-span" style="background-color: rgb(255, 255, 255);"><font
class="Apple-style-span" color="#000000"><span class="Apple-style-span"
style="font-size: large;"><br>
</span></font></span></span></span><span id="result_box"
class="short_text" style="display: block;"><span title=""><span
class="Apple-style-span" style="background-color: rgb(255, 255, 255);"><font
class="Apple-style-span" color="#000000"><span class="Apple-style-span"
style="font-size: large;"><span id="result_box" class="short_text"
style="display: block;"><span class="Apple-style-span"
style="font-size: x-small;"><b>[mpiX@quad2 ~]$ cat
/var/spool/torque/mom_logs/20100928 | grep 46.master</b></span></span><span
id="result_box" class="short_text" style="display: block;"><span
class="Apple-style-span" style="font-size: x-small;">09/28/2010
09:29:29;0008; pbs_mom;Job;46.master;JOIN JOB as node 1</span></span><span
id="result_box" class="short_text" style="display: block;"><span
class="Apple-style-span" style="font-size: x-small;">09/28/2010
09:29:29;0001; pbs_mom;Job;46.master;task not started, '/bin/sh',
stdio setup failed (see syslog)</span></span><span id="result_box"
class="short_text" style="display: block;"><span
class="Apple-style-span" style="font-size: x-small;">09/28/2010
09:29:29;0008; pbs_mom;Job;46.master;ERROR: received request
'SPAWN_TASK' from <a moz-do-not-send="true"
href="http://10.10.10.3:1023">10.10.10.3:1023</a> for job '46.master'
(cannot start task)</span></span><span id="result_box"
class="short_text" style="display: block;"><span
class="Apple-style-span" style="font-size: x-small;">09/28/2010
09:29:29;0001; pbs_mom;Job;46.master;task not started, '/bin/sh',
stdio setup failed (see syslog)</span></span><span id="result_box"
class="short_text" style="display: block;"><span
class="Apple-style-span" style="font-size: x-small;">09/28/2010
09:29:29;0008; pbs_mom;Job;46.master;ERROR: received request
'SPAWN_TASK' from <a moz-do-not-send="true"
href="http://10.10.10.3:1023">10.10.10.3:1023</a> for job '46.master'
(cannot start task)</span></span><span id="result_box"
class="short_text" style="display: block;"><span
class="Apple-style-span" style="font-size: x-small;">09/28/2010
09:29:29;0001; pbs_mom;Job;46.master;task not started, '/bin/sh',
stdio setup failed (see syslog)</span></span><span id="result_box"
class="short_text" style="display: block;"><span
class="Apple-style-span" style="font-size: x-small;">09/28/2010
09:29:29;0008; pbs_mom;Job;46.master;ERROR: received request
'SPAWN_TASK' from <a moz-do-not-send="true"
href="http://10.10.10.3:1023">10.10.10.3:1023</a> for job '46.master'
(cannot start task)</span></span><span id="result_box"
class="short_text" style="display: block;"><span
class="Apple-style-span" style="font-size: x-small;">09/28/2010
09:29:29;0001; pbs_mom;Job;46.master;task not started, '/bin/sh',
stdio setup failed (see syslog)</span></span><span id="result_box"
class="short_text" style="display: block;"><span
class="Apple-style-span" style="font-size: x-small;">09/28/2010
09:29:29;0008; pbs_mom;Job;46.master;ERROR: received request
'SPAWN_TASK' from <a moz-do-not-send="true"
href="http://10.10.10.3:1023">10.10.10.3:1023</a> for job '46.master'
(cannot start task)</span></span>
<div><br>
</div>
<div>The status of job is active </div>
<div><br>
</div>
<div>
<div><span class="Apple-style-span" style="font-size: x-small;"><b>[mpiX@master
mpi_fitting]$ showq</b></span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">ACTIVE
JOBS--------------------</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">JOBNAME
USERNAME STATE PROC REMAINING STARTTIME</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;"><br>
</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">46
mpiX Running 12 00:35:52 Tue Sep 28 09:32:56</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;"><br>
</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">
1 Active Job 12 of 12 Processors Active (100.00%)</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">
2 of 2 Nodes Active (100.00%)</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;"><br>
</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">IDLE
JOBS----------------------</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">JOBNAME
USERNAME STATE PROC WCLIMIT QUEUETIME</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;"><br>
</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;"><br>
</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">0
Idle Jobs</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;"><br>
</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">BLOCKED
JOBS----------------</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">JOBNAME
USERNAME STATE PROC WCLIMIT QUEUETIME</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;"><br>
</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;"><br>
</span></div>
<div><span class="Apple-style-span" style="font-size: x-small;">Total
Jobs: 1 Active Jobs: 1 Idle Jobs: 0 Blocked Jobs: 0</span></div>
<div><br>
</div>
<div>The same software (mpich2+gsl) run on a single node of 8
cores, This problem occurs when two nodes use . </div>
<span class="Apple-style-span" style="color: rgb(136, 136, 136);">
<div id="gt-res-tools" class="g-section"
style="width: 686px; vertical-align: top; display: inline-block; margin-top: 8px;"></div>
</span></div>
<div><br>
</div>
</span></font></span></span></span></div>
</div>
<div id="gt-res-tools" class="g-section"
style="width: 686px; vertical-align: top; display: inline-block; margin-top: 8px;">
<div id="gt-res-listen" tabindex="0" class="gt-icon-c"
style="color: rgb(17, 17, 204); text-decoration: none; cursor: pointer; float: left; margin-right: 1em; outline-style: none;"></div>
</div>
</span></font></span></font></div>
<div><font class="Apple-style-span" color="#eeffe2"
face="arial, sans-serif" size="4"><span class="Apple-style-span"
style="border-collapse: collapse; font-size: 15px;"><font
class="Apple-style-span" color="#000000"><br>
</font></span></font></div>
<div><font class="Apple-style-span" color="#eeffe2"
face="arial, sans-serif" size="4"><span class="Apple-style-span"
style="border-collapse: collapse; font-size: 15px;"><font
class="Apple-style-span" color="#000000"><br>
</font></span></font>-- <br>
Abraham Zamudio Ch.<br>
<br>
</div>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
torqueusers mailing list
<a class="moz-txt-link-abbreviated" href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.supercluster.org/mailman/listinfo/torqueusers">http://www.supercluster.org/mailman/listinfo/torqueusers</a>
</pre>
</blockquote>
What does qstat show? Did you look at syslog?<br>
<br>
Ken Nielson<br>
Adaptive Computing<br>
</body>
</html>