<font face="verdana,sans-serif">Hi Gus,</font><div><font face="verdana,sans-serif"><br></font></div><div><font face="verdana,sans-serif">I have unset </font></div><div><font face="verdana,sans-serif"><span class="Apple-style-span" style="font-family: arial, sans-serif; font-size: 13px; border-collapse: collapse; color: rgb(80, 0, 80); ">&gt; set server resources_default.neednodes = 1<br>

&gt; set server resources_default.nodect = 1</span></font></div><div><font face="verdana,sans-serif"><span class="Apple-style-span" style="font-family: arial, sans-serif; font-size: 13px; border-collapse: collapse; color: rgb(80, 0, 80); "><br>

</span></font></div><div><font face="verdana,sans-serif"><span class="Apple-style-span" style="font-family: arial, sans-serif; font-size: 13px; border-collapse: collapse; color: rgb(80, 0, 80); ">I have 30 nodes listed in cluster and each one has 4 core</span></font></div>

<div><font face="verdana,sans-serif"><span class="Apple-style-span" style="font-family: arial, sans-serif; font-size: 13px; border-collapse: collapse; color: rgb(80, 0, 80); ">But still the job is running on one node.</span></font></div>

<div><font face="verdana,sans-serif"><span class="Apple-style-span" style="font-family: arial, sans-serif; font-size: 13px; border-collapse: collapse; color: rgb(80, 0, 80); "><br></span></font></div><div><font face="verdana,sans-serif"><span class="Apple-style-span" style="font-family: arial, sans-serif; font-size: 13px; border-collapse: collapse; color: rgb(80, 0, 80); ">Thanks</span></font></div>

<div><font face="verdana,sans-serif"><span class="Apple-style-span" style="font-family: arial, sans-serif; font-size: 13px; border-collapse: collapse; color: rgb(80, 0, 80); ">Govind</span></font></div><div><font face="verdana,sans-serif"><font class="Apple-style-span" color="#500050" face="arial, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse;"><br>

</span></font></font><br><div class="gmail_quote">On 15 June 2010 18:50, Gus Correa <span dir="ltr">&lt;<a href="mailto:gus@ldeo.columbia.edu">gus@ldeo.columbia.edu</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">

Hi Govind<br>
<br>
Please, read what Garrick said about your server configuration.<br>
To unset stuff do:<br>
qmgr -c &#39;unset server resources_default.neednodes&#39;<br>
and so on ...<br>
<br>
Besides, your *Torque* (not OpenMPI) nodes file should list<br>
*all* nodes on your cluster with the respective<br>
number of processors/cores.<br>
If I understood what you said right,<br>
you only have one node listed there.<br>
<br>
Something like this:<br>
<br>
node01 np=4<br>
node02 np=4<br>
... (and so on ...)<br>
<div class="im"><br>
I hope this helps,<br>
Gus Correa<br>
<br>
<br>
</div><div class="im">Govind Songara wrote:<br>
&gt; Hi Gus,<br>
&gt;<br>
&gt; Thanks for your reply, with your advise I build openMPI.<br>
&gt; I have  standard nodes list as<br>
&gt; node01 np=4<br>
&gt;<br>
&gt; Thanks<br>
&gt; Govind<br>
&gt;<br>
&gt; On 15 June 2010 17:20, Gus Correa &lt;<a href="mailto:gus@ldeo.columbia.edu">gus@ldeo.columbia.edu</a><br>
</div><div><div></div><div class="h5">&gt; &lt;mailto:<a href="mailto:gus@ldeo.columbia.edu">gus@ldeo.columbia.edu</a>&gt;&gt; wrote:<br>
&gt;<br>
&gt;     Hi Govind<br>
&gt;<br>
&gt;     What is the content of your Torque nodes file?<br>
&gt;     (located at /var/torque/server_priv/nodes or equivalent)<br>
&gt;     It should list all nodes and the number of processors/cores on each,<br>
&gt;     Something like this:<br>
&gt;<br>
&gt;     node01 np=2<br>
&gt;     node02 np=8<br>
&gt;     ...<br>
&gt;     node47 np=4<br>
&gt;     ...<br>
&gt;<br>
&gt;     Maybe I asked you this before in the OpenMPI list, not sure.<br>
&gt;<br>
&gt;     I hope this helps,<br>
&gt;     Gus Correa<br>
&gt;     ---------------------------------------------------------------------<br>
&gt;     Gustavo Correa<br>
&gt;     Lamont-Doherty Earth Observatory - Columbia University<br>
&gt;     Palisades, NY, 10964-8000 - USA<br>
&gt;     ---------------------------------------------------------------------<br>
&gt;<br>
&gt;<br>
&gt;     Govind wrote:<br>
&gt;      &gt; Hi,<br>
&gt;      &gt;<br>
&gt;      &gt; I have am openmpi build with tm support<br>
&gt;      &gt; When i run the mpi job requesting for two nodes and 4 core it run<br>
&gt;     only<br>
&gt;      &gt; on single node.<br>
&gt;      &gt;<br>
&gt;      &gt;  &gt;cat mpipbs-script.sh<br>
&gt;      &gt; #PBS -N mpipbs-script<br>
&gt;      &gt; #PBS -q short<br>
&gt;      &gt; ### Number of nodes: resources per node<br>
&gt;      &gt; ### (4 cores/node, so ppn=4 is ALL resources on the node)<br>
&gt;      &gt; #PBS -l nodes=2:ppn=4<br>
&gt;      &gt; echo `cat $PBS_NODEFILE`<br>
&gt;      &gt; NPROCS=`wc -l &lt; $PBS_NODEFILE`<br>
&gt;      &gt; echo This job has allocated $NPROCS processors<br>
&gt;      &gt; /opt/openmpi-1.4.2/bin/mpirun /scratch0/gsongara/mpitest/hello<br>
&gt;      &gt;<br>
&gt;      &gt; It   show only one node here is the output<br>
&gt;      &gt; ===============<br>
&gt;      &gt; node47.beowulf.cluster node47.beowulf.cluster node47.beowulf.cluster<br>
&gt;      &gt; node47.beowulf.cluster<br>
&gt;      &gt; This job has allocated 4 processors<br>
&gt;      &gt; Hello World! from process 1 out of 4 on node47.beowulf.cluster<br>
&gt;      &gt; Hello World! from process 2 out of 4 on node47.beowulf.cluster<br>
&gt;      &gt; Hello World! from process 3 out of 4 on node47.beowulf.cluster<br>
&gt;      &gt; Hello World! from process 0 out of 4 on node47.beowulf.cluster<br>
&gt;      &gt; ===============<br>
&gt;      &gt;<br>
&gt;      &gt; torque config<br>
&gt;      &gt; set queue short resources_max.nodes = 4<br>
&gt;      &gt; set queue short resources_default.nodes = 1<br>
&gt;      &gt; set server resources_default.neednodes = 1<br>
&gt;      &gt; set server resources_default.nodect = 1<br>
&gt;      &gt; set server resources_default.nodes = 1<br>
&gt;      &gt;<br>
&gt;      &gt; I also trying adding this parameter to maui as advised in some  old<br>
&gt;      &gt; thread but does not help<br>
&gt;      &gt; JOBNODEMATCHPOLICY     EXACTNODE<br>
&gt;      &gt; ENABLEMULTINODEJOBS   TRUE<br>
&gt;      &gt; NODEACCESSPOLICY         SHARED<br>
&gt;      &gt;<br>
&gt;      &gt;<br>
&gt;      &gt; Can someone please advise if i missing anything here.<br>
&gt;      &gt;<br>
&gt;      &gt; Regards<br>
&gt;      &gt; Govind<br>
&gt;      &gt;<br>
&gt;      &gt;<br>
&gt;      &gt;<br>
&gt;     ------------------------------------------------------------------------<br>
&gt;      &gt;<br>
&gt;      &gt; _______________________________________________<br>
&gt;      &gt; torqueusers mailing list<br>
</div></div>&gt;      &gt; <a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a> &lt;mailto:<a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a>&gt;<br>
<div class="im">&gt;      &gt; <a href="http://www.supercluster.org/mailman/listinfo/torqueusers" target="_blank">http://www.supercluster.org/mailman/listinfo/torqueusers</a><br>
&gt;<br>
&gt;     _______________________________________________<br>
&gt;     torqueusers mailing list<br>
</div>&gt;     <a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a> &lt;mailto:<a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a>&gt;<br>
<div><div></div><div class="h5">&gt;     <a href="http://www.supercluster.org/mailman/listinfo/torqueusers" target="_blank">http://www.supercluster.org/mailman/listinfo/torqueusers</a><br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; ------------------------------------------------------------------------<br>
&gt;<br>
&gt; _______________________________________________<br>
&gt; torqueusers mailing list<br>
&gt; <a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a><br>
&gt; <a href="http://www.supercluster.org/mailman/listinfo/torqueusers" target="_blank">http://www.supercluster.org/mailman/listinfo/torqueusers</a><br>
<br>
_______________________________________________<br>
torqueusers mailing list<br>
<a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a><br>
<a href="http://www.supercluster.org/mailman/listinfo/torqueusers" target="_blank">http://www.supercluster.org/mailman/listinfo/torqueusers</a><br>
</div></div></blockquote></div><br></div>