[Mauiusers] Repost: maui configuration: multiple jobs per node

[Donald E Tripp]

Quote...

I am spawning jobs using:
 qsub -l walltime=15:00,nodes=1:ppn=1 script.pbs

... end quote

You have here ppn=1. That means its only going to use 1 processor per node (ppn), so if another user requests the same thing, they can get the other 6 cpus free. Do this:

 qsub -l walltime=15:00,nodes=1:ppn=2 script.pbs

Also, why you can only run on 6 nodes at a time is the way pbs is inter-node communicating using ssh. If you were running more than 1 job per node as yourself, your user account would be logged in twice, and thus which one is job1 and which one is job2? PBS can't tell. 

- Don
UH Hilo

----- Original Message -----
From: Jeff Castura <jcastura at site.uottawa.ca>
Date: Thursday, July 6, 2006 4:02 pm
Subject: [Mauiusers] Repost: maui configuration: multiple jobs per node
To: mauiusers at supercluster.org

> I am reposting a message I originally posted a couple of months ago as
> there was no response and I have yet to solve this.  Any help is
> appreciated.
> 
> On Mon, 24 Apr 2006 09:24:50 -0400
> Jeff Castura <jcastura at site.uottawa.ca> wrote:
> 
> I am trying to configure maui (3.2.6p14) to allow a user to use all
> 12 of the processors in a small 6-node cluster.  Currently a user may
> use all of the nodes, but for some reason cannot use both processors
> in a single node.  (Note all jobs are serial)
> 
> Cluster configuration:
> /var/spool/pbs/server_priv/nodes:
> node001 np=2 all compute opteron
> node002 np=2 all compute opteron
> node003 np=2 all compute opteron
> node004 np=2 all compute opteron
> node005 np=2 all compute opteron
> node006 np=2 all compute opteron
> 
> When idle, pbstop reports:
> Usage Totals: 0/12 Procs, 0/6 Nodes, 0/0 Jobs Running
> Node States:     6 free
> 
> I am spawning jobs using:
> qsub -l walltime=15:00,nodes=1:ppn=1 script.pbs
> 
> If I spwan more than 6 jobs as a single user, only 6 will run at any
> time, the remainder stay queued.  However, another user can run jobs
> on the cluster using the remaining 6 processors.  So it seems that
> users are being prevented from using more than a single processor on
> a node.
> 
> Running 7 jobs, pbstop reports:
> Usage Totals: 6/12 Procs, 6/6 Nodes, 6/7 Jobs Running
> Node States:     6 free
> 
> Relevant lines from /usr/local/maui/maui.cfg
> NODELOADPOLICY              ADJUSTSTATE
> NODEALLOCATIONPOLICY        CPULOAD
> NODEACCESSPOLICY            SHARED
> NODEMAXLOAD                 1.75
> NODECFG[DEFAULT]            MAXJOB=3 MAXJOBPERUSER=2
> USERCFG[DEFAULT]            MAXJOB=8,12
> 
> 
> How can I configure (or use?) maui to allow a user to run more than 6
> jobs at a time?
> 
> Thanks very much,
> 
> Jeff
> 
> _______________________________________________
> mauiusers mailing list
> mauiusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/mauiusers
>