[Moabusers] Torque/Moab node allocation issue

[Gareth.Williams at csiro.au]

I think you don't want:
> set queue batch resources_max.nodes = 132:ppn=4
As documented, the value in not numeric so max/min will not apply as you
might be hoping.  This may not actually impact on your current problem.
Chris' suggestion is good.  Also/alternatively you could change
JOBNODEMATCHPOLICY.

Gareth Williams


> -----Original Message-----
> From: chris [mailto:chris at clusterresources.com]
> Sent: Friday, 7 December 2007 3:29 AM
> To: VS Ang
> Cc: moabusers at supercluster.org
> Subject: Re: [Moabusers] Torque/Moab node allocation issue
> 
> VS Ang,
> 
> You need to specify the processors per node, try msub -l
nodes=1:ppn=4.
> 
> Regards,
> 
> VS Ang wrote:
> > Hello,
> >
> > I am looking for some help with Torque/Moab configuration. We are
> > using Torque and Moab on a cluster with nodes having 4 cores each.
We
> > are trying to schedule jobs such that the nodes run multiple jobs in
> > shared mode. However, the scheduler is not allocating nodes based on
> > the processors. We specified the following policies in the moab.cfg
> file:
> >
> > NODEACCESSPOLICY            SHARED
> > JOBNODEMATCHPOLICY        EXACTNODE
> >
> > The Torque queue is configured as below:
> >
> > create queue batch
> > set queue batch queue_type = Execution
> > set queue batch resources_max.nodect = 132
> > set queue batch resources_max.nodes = 132:ppn=4
> > set queue batch resources_default.nodect = 1
> > set queue batch resources_default.nodes = 1:ppn=1
> > set queue batch resources_default.walltime = 01:00:00
> > set queue batch enabled = True
> > set queue batch started = True
> >
> > Also I configured:
> > set server node_pack = True
> >
> > The "server_priv/nodes" file has node entries as below:
> > blade1 np=4
> > blade2 np=4
> > blade3 np=4
> > blade4 np=4
> > blade5 np=4
> > ..
> >
> > However, no matter how the job is submitted, users always get 4
> > different nodes, as opposed to a single node with 4 CPUs.
> >
> > >msub -I -l nodes=4
> > >cat $PBS_NODEFILE
> > blade1
> > blade2
> > blade3
> > blade4
> >
> > >msub -l nodes=4,taskdistpolicy=pack
> >
> > Also, another odd behavior is, if I specify:
> > >msub -l ncpus=4
> >
> > I am actually getting all nodes in the cluster, as opposed to 4.
> >
> > So, what is wrong here? Am I missing any other policy parameters in
> > Torque or Moab?
> >
> > Thanks in advance!
> >
> > Srihari
> >
> >
------------------------------------------------------------------------
> >
> > _______________________________________________
> > moabusers mailing list
> > moabusers at supercluster.org
> > http://www.supercluster.org/mailman/listinfo/moabusers
> >
> 
> 
> --
> Chris Vaughan
> EMEA Systems Engineer
> Cluster Resources, Ltd.
> Direct - UK Office:  +44 (0)1223 437 132
> Mobile - +44 (0)7800 973 062
> US Headquarters:  +1 801 717 3700
> Skype: supercomputer1
> www.clusterresources.co.uk
> 
> 
>