<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Hello</div><div><br></div><div>Currently we are lacking the ability to&nbsp;request appropriate&nbsp;</div><div>resources for a shared-memory, multi-threaded program.</div><div><br></div><div>We would like to be able to request one process with n&nbsp;</div><div>(cores/threads/cpus) &nbsp;with any mem, pmem, vmem, or pvmem&nbsp;</div><div>specification apply to that one process.</div><div><br></div><div>ex) nodes=1:ppn=8:vmempn=16gb:single_process</div><div><br></div><div>1 node, 8 processors per node , 16gb virtual memory per node, applied to a single process</div><div><br></div><div>Thanks&nbsp;</div><div><br></div><div>Kamil</div><div><br></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><div>Kamil Marcinkowski &nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;Westgrid&nbsp;System Administrator&nbsp;<div><a href="mailto:kamil@ualberta.ca">kamil@ualberta.ca</a>&nbsp; &nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;University of Alberta site&nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;</div><div>&nbsp;Tel.780 492-0354&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;Research Computing Support&nbsp;&nbsp;&nbsp;</div><div>Fax.780 492-1729&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;Academic ICT&nbsp;&nbsp;</div><div>Edmonton, Alberta, CANADA &nbsp; &nbsp;University of Alberta&nbsp;&nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;</div><div><br class="khtml-block-placeholder"></div><div><br class="khtml-block-placeholder"></div><div>"This communication is intended for the use of the recipient to which it is</div><div>addressed, and may contain confidential, personal, and/or privileged</div><div>information.&nbsp; Please contact us immediately if you are not the intended</div><div>recipient of this communication.&nbsp; If you are not the intended recipient of</div><div>this communication, do not copy, distribute, or take action on it. Any</div><div>communication received in error, or subsequent reply, should be deleted or</div><div>destroyed."</div><div><br class="khtml-block-placeholder"></div></div><br class="Apple-interchange-newline"></span></span></span>
</div>
<br><div><div>On 2010-03-02, at 12:12 PM, David Beer wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>The whole procs vs. ncpus thing is one of the ambiguities we're hoping to get rid of. At least to me, I don't read procs and think it can be spread out and read ncpus and think that it has to be on one node. <br><br>We also are looking at moving towards some kind of node-agnostic form of specification that would allow similar like chunks where you would specify a desired amount of cores and memory that are advantageously close together, without worrying about nodes. This seems like a more flexible way to specify things, and that it would be more compatible with SMP and similar systems. <br><br>Another enhancement that we're considering is locking memory close to the CPUs (where relevant). I know that on some systems (NUMA, for example) this is necessary in order to run jobs efficiently.<br><br>David<br><br><br>----- "Dr. Stephan Raub" &lt;<a href="mailto:raub@uni-duesseldorf.de">raub@uni-duesseldorf.de</a>&gt; wrote:<br><br><blockquote type="cite">Hi,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">our Institution had been using PBSPro for some while. We dumped it and<br></blockquote><blockquote type="cite">are now using Torque/Maui for several reasons, that don’t belong in<br></blockquote><blockquote type="cite">this forum (no, it was not because of money), BUT: I liked the idea of<br></blockquote><blockquote type="cite">their “chunks”. For example: as a quantum chemist I’m using TurboMole<br></blockquote><blockquote type="cite">a lot. For parallel runs with n compute processes it requires an<br></blockquote><blockquote type="cite">additional master-process which is exactly on the same node as the<br></blockquote><blockquote type="cite">first compute-prozess. Alas, this master process doesn’t need a lot of<br></blockquote><blockquote type="cite">memory. So I used a statement like<br></blockquote><blockquote type="cite">select=1:ncpus=2:mem=15gb+15:ncpus=1:mem=15gb. Up to now I haven’t<br></blockquote><blockquote type="cite">figured out a equivalent statement for torque/maui.<br></blockquote><blockquote type="cite">nodes=1:ppn=2+15:ppn=1 and pmem=8gb is not the same, as I am not able<br></blockquote><blockquote type="cite">to allocate ALL memory of a node for the job.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">The same with Jobs using heterogeneous mpi topologies (e.g. itanium<br></blockquote><blockquote type="cite">and xeon in the same mpi topology).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I don’t want to say, that PBSPro is better than Torque/Maui (as we<br></blockquote><blockquote type="cite">find out the opposite in the hard way), but the pure theoretical<br></blockquote><blockquote type="cite">concept of these resource chunks was quite useful.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Stephan<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">--<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">---------------------------------------------------------<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">| | Dr. rer. nat. Stephan Raub<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">| | Dipl. Chem.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">| | Lehrstuhl für IT-Management / ZIM<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">| | Heinrich-Heine-Universität Düsseldorf Universitätsstr. 1 /<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">| | 25.41.O2.25-2<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">| | 40225 Düsseldorf / Germany<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">| |<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">| | Tel: +49-211-811-3911<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">---------------------------------------------------------<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Wichtiger Hinweis: Diese E-Mail kann Betriebs- oder<br></blockquote><blockquote type="cite">Geschäftsgeheimnisse, bzw.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">sonstige vertrauliche Informationen enthalten. Sollten Sie diese<br></blockquote><blockquote type="cite">E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des<br></blockquote><blockquote type="cite">Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich<br></blockquote><blockquote type="cite">untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die<br></blockquote><blockquote type="cite">empfangene E-Mail. Vielen Dank.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Important Note: This e-mail may contain trade secrets or privileged,<br></blockquote><blockquote type="cite">undisclosed or otherwise confidential information. If you have<br></blockquote><blockquote type="cite">received this e-mail in error, you are hereby notified that any<br></blockquote><blockquote type="cite">review, copying or distribution of it is strictly prohibited. Please<br></blockquote><blockquote type="cite">inform us immediately and destroy the original transmittal. Thank you<br></blockquote><blockquote type="cite">for your cooperation.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Von: <a href="mailto:torqueusers-bounces@supercluster.org">torqueusers-bounces@supercluster.org</a><br></blockquote><blockquote type="cite">[mailto:torqueusers-bounces@supercluster.org] Im Auftrag von kamil<br></blockquote><blockquote type="cite">Marcinkowski<br></blockquote><blockquote type="cite">Gesendet: Dienstag, 2. März 2010 19:26<br></blockquote><blockquote type="cite">An: Josh Bernstein<br></blockquote><blockquote type="cite">Cc: torqueusers<br></blockquote><blockquote type="cite">Betreff: Re: [torqueusers] Fwd: ncpus anyone?<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Hello Josh<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">You should use the (procs=32) specification for parallel jobs<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">that don't care where they run.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">npus used to have 2 different and opposite meanings on<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">SMPs(nodes=1:ppn=32) and clusters (nodes=32:ppn=1).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I vote for defining -lncpus=32 to -lnodes=1:ppn=32.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Cheers,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Kamil<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Kamil Marcinkowski Westgrid System Administrator<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="mailto:kamil@ualberta.ca">kamil@ualberta.ca</a> University of Alberta site<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Tel.780 492-0354 Research Computing Support<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Fax.780 492-1729 Academic ICT<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Edmonton, Alberta, CANADA University of Alberta<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">"This communication is intended for the use of the recipient to which<br></blockquote><blockquote type="cite">it is<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">addressed, and may contain confidential, personal, and/or privileged<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">information. Please contact us immediately if you are not the intended<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">recipient of this communication. If you are not the intended recipient<br></blockquote><blockquote type="cite">of<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">this communication, do not copy, distribute, or take action on it. Any<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">communication received in error, or subsequent reply, should be<br></blockquote><blockquote type="cite">deleted or<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">destroyed."<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">On 2010-03-02, at 10:58 AM, Josh Bernstein wrote:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I vote for maintaing ncpus. It's very helpful for embarrssingly<br></blockquote><blockquote type="cite">parallel jobs that just need 32 CPUs but don't care where they come<br></blockquote><blockquote type="cite">from.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-Josh<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">On Mar 2, 2010, at 9:53 AM, "David Beer" &lt; <a href="mailto:dbeer@adaptivecomputing.com">dbeer@adaptivecomputing.com</a><br></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite">wrote:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Just to let everyone know, the qstat -a output has been changed to<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">read both the value stored in nodes and ncpus, using nodes when both<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">are specified.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Changing the code so that qstat -a displays correctly the number of<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">tasks with -lnodes=1:ppn=32 would be great. Then, you could also make<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">sure that -lncpus=32 is a complete synonymous of -lnodes=1:ppn=32.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Is this the behavior that everyone expects/hopes for? If so, we can<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">look at working on it. At the same time, TORQUE 3.0 is likely to<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">include much superior specification for how we are requesting<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">resources, which may end up including ncpus and may not. We're<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">looking to remove a lot of ambiguity and enhance capability. By the<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">way. we're still open to input as to how all that will work, but<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">maybe we'll send out some ideas shortly if nobody has any input yet.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Cheers,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">David<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">----- "Michel Béland" &lt; <a href="mailto:michel.beland@rqchp.qc.ca">michel.beland@rqchp.qc.ca</a> &gt; wrote:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">David Beer wrote:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">So, if I understand correctly, ncpus really only works for people<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">that are running SMP or similar systems? It seems like we definitely<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">need to update our documentation as I feel it is misleading on the<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">matter. Among other things, it seems that a clarification needs to be<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">made that ncpus isn't compatible with the nodes attribute.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">It is possible to specify both. In fact, at our site we have a qsub<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">wrapper script that makes sure, among other things, that everybody<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">specifies both on our Altix systems.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">On a related note, in the qstat -a output we have the TSK field,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">which I believe is meant to mean task (I couldn't find anything about<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">it in the man page, the variable in the code is named tasks). I<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">noticed that in the implementation we're just writing whatever value<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">is stored in ncpus for this field. It seems like this could be made<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">more accurate by checking the nodes attribute as well and using that<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">value where it is defined, since it seems to override ncpus when both<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">are present. What are you're thoughts on this?<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I agree. This is exactly why we make sure that all the jobs have both<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">resource requests. If one specifies -lnodes=1:ppn=32, the output of<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">qstat -a does not show how many cores you really use. On the other<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">hand,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">if one specifies -lncpus=32, Torque does not create cpusets correctly<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">(they always contain only processor 0). So if I specify -lncpus=32<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-lnodes=1:ppn=32, cpusets are created correctly and qstat -a shows<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">correctly how many cores the job is using. Maui, does not have any<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">problem dealing with this job.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">--<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Michel Béland, analyste en calcul scientifique<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="mailto:michel.beland@rqchp.qc.ca">michel.beland@rqchp.qc.ca</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">bureau S-250, pavillon Roger-Gaudry (principal), Université de<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Montréal<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">téléphone : 514 343-6111 poste 3892 télécopieur : 514 343-2<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">155<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">RQCHP (Réseau québécois de calcul de haute performance)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="http://www.rqchp.qc.ca">www.rqchp.qc.ca</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">--<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">David Beer | Senior Software Engineer<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Adaptive Computing<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">--<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">David Beer | Senior Software Engineer<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Adaptive Computing<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">torqueusers mailing list<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="http://www.supercluster.org/mailman/listinfo/torqueusers">http://www.supercluster.org/mailman/listinfo/torqueusers</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">torqueusers mailing list<br></blockquote><blockquote type="cite"><a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a><br></blockquote><blockquote type="cite"><a href="http://www.supercluster.org/mailman/listinfo/torqueusers">http://www.supercluster.org/mailman/listinfo/torqueusers</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">torqueusers mailing list<br></blockquote><blockquote type="cite"><a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a><br></blockquote><blockquote type="cite"><a href="http://www.supercluster.org/mailman/listinfo/torqueusers">http://www.supercluster.org/mailman/listinfo/torqueusers</a><br></blockquote><br>-- <br>David Beer | Senior Software Engineer<br>Adaptive Computing<br><br>_______________________________________________<br>torqueusers mailing list<br><a href="mailto:torqueusers@supercluster.org">torqueusers@supercluster.org</a><br>http://www.supercluster.org/mailman/listinfo/torqueusers<br><br></div></blockquote></div><br></body></html>