<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
<tt>If you have built mpiexec:<br>
<a class="moz-txt-link-freetext" href="http://www.osc.edu/~djohnson/mpiexec/index.php">http://www.osc.edu/~djohnson/mpiexec/index.php</a><br>
It is aware of $PBS_NODEFILE, and will do "the right thing", when used
similarly to mpirun as mentioned by Mr. Croyle.<br>
<br>
Jerry</tt><br>
<br>
Coyle, James J [ITACD] wrote:
<blockquote
cite="mid:D1D950C0853848438D74D2EB6EED082AA41C06247F@EXITS711.its.iastate.edu"
type="cite">
<pre wrap="">Torque and PBS give you a file named
PBS_NODEFILE
For example with MPIPCH you could use
mpirun -np 28 -machinefile ${PBS_NODEFILE} ./prog
Then 28 copies of ./prog will be started on
the 28 machines listed in ${PBS_NODEFILE}
Other programs like Fluent need you to specify something like:
fluent 3ddp -t28 -pib -g -i Case.jou -cnf=${PBS_NODEFILE}
again here you need to specify a file containing the
machines on which to run each process. If you leave off the
-cnf above, fluent will start all the processes on
the first node that the jobs got assigned to.
-----Original Message-----
</pre>
<blockquote type="cite">
<pre wrap="">From: <a class="moz-txt-link-abbreviated" href="mailto:torqueusers-bounces@supercluster.org">torqueusers-bounces@supercluster.org</a> [<a class="moz-txt-link-freetext" href="mailto:torqueusers">mailto:torqueusers</a>-
<a class="moz-txt-link-abbreviated" href="mailto:bounces@supercluster.org">bounces@supercluster.org</a>] On Behalf Of Lenox, Billy AMRDEC/Sentient
Corp.
Sent: Thursday, October 06, 2011 12:10 PM
To: Torque Users Mailing List
Subject: Re: [torqueusers] Need help with NCPUS not working in QSUB
Ok I tried PBS -l procs=28 and it still runs on one NODE seed001
I notice that if I put in the script on the EXEC line the location
of a
HOSTFILE it runs and bypasses TORQUE PBS. I just have the Default
Scheduler
on the System. I know I can not specify PBS -l nodes=5.
I have tried different ways and still it will only run on ONE NODE
seed001.
Billy
</pre>
<blockquote type="cite">
<pre wrap="">From: Troy Baer <a class="moz-txt-link-rfc2396E" href="mailto:tbaer@utk.edu"><tbaer@utk.edu></a>
Organization: National Institute for Computational Sciences,
</pre>
</blockquote>
<pre wrap="">University of
</pre>
<blockquote type="cite">
<pre wrap="">Tennessee
Reply-To: Torque Users Mailing List <a class="moz-txt-link-rfc2396E" href="mailto:torqueusers@supercluster.org"><torqueusers@supercluster.org></a>
Date: Thu, 6 Oct 2011 12:07:45 -0400
To: Torque Users Mailing List <a class="moz-txt-link-rfc2396E" href="mailto:torqueusers@supercluster.org"><torqueusers@supercluster.org></a>
Subject: Re: [torqueusers] Need help with NCPUS not working in
</pre>
</blockquote>
<pre wrap="">QSUB
</pre>
<blockquote type="cite">
<pre wrap="">On Thu, 2011-10-06 at 09:55 -0500, Lenox, Billy AMRDEC/Sentient
</pre>
</blockquote>
<pre wrap="">Corp.
</pre>
<blockquote type="cite">
<pre wrap="">wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I have torque setup on a head node system with 5 compute nodes
Two have 8 cores and 3 have 4 cores setup into on queue called
</pre>
</blockquote>
</blockquote>
<pre wrap="">batch
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">When I use a submit script
#!/bin/bash
#PBS -l ncpus=28
#PBS -l walltime=72:00:00
#PBS -o output.out
#PBS -e ie.error
Here /var/spool/torque/server_priv/nodes
seed001 np=8 batch
seed002 np=8 batch
seed003 np=8 batch
seed004 np=8 batch
seed005 np=8 batch
When I submit the script it only runs on one node SEED001
I don't know why it only runs on one node.
</pre>
</blockquote>
<pre wrap="">Which scheduler are you using? In most of the TORQUE-compatible
schedulers I've seen, the ncpus= resource is interpreted as how
</pre>
</blockquote>
<pre wrap="">many
</pre>
<blockquote type="cite">
<pre wrap="">processors you want on a single shared memory system. (If you
</pre>
</blockquote>
<pre wrap="">want X
</pre>
<blockquote type="cite">
<pre wrap="">processors and you don't care where they are, I think the
</pre>
</blockquote>
<pre wrap="">preferred way
</pre>
<blockquote type="cite">
<pre wrap="">of requesting it is procs=X.)
--Troy
--
Troy Baer, HPC System Administrator
National Institute for Computational Sciences, University of
</pre>
</blockquote>
<pre wrap="">Tennessee
</pre>
<blockquote type="cite">
<pre wrap=""><a class="moz-txt-link-freetext" href="http://www.nics.tennessee.edu/">http://www.nics.tennessee.edu/</a>
Phone: 865-241-4233
_______________________________________________
torqueusers mailing list
<a class="moz-txt-link-abbreviated" href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.supercluster.org/mailman/listinfo/torqueusers">http://www.supercluster.org/mailman/listinfo/torqueusers</a>
</pre>
</blockquote>
<pre wrap="">_______________________________________________
torqueusers mailing list
<a class="moz-txt-link-abbreviated" href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.supercluster.org/mailman/listinfo/torqueusers">http://www.supercluster.org/mailman/listinfo/torqueusers</a>
</pre>
</blockquote>
<pre wrap=""><!---->_______________________________________________
torqueusers mailing list
<a class="moz-txt-link-abbreviated" href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.supercluster.org/mailman/listinfo/torqueusers">http://www.supercluster.org/mailman/listinfo/torqueusers</a>
</pre>
</blockquote>
</body>
</html>