[Moabusers] mapping node features to a class
Gareth.Williams at csiro.au
Gareth.Williams at csiro.au
Tue May 15 22:43:50 MDT 2007
We use a combination of a SR to prevent jobs from accessing designated
test nodes and a class 'mapping' to allow a simple mechanism for
choosing test nodes, something like this:
SRCFG[testsr] CLASSLIST=test HOSTLIST=ALL NODEFEATURES=intest
CLASSCFG[test] DEFAULT.FEATURES=intest
NODECFG[node10] FEATURES+=intest
NODECFG[node11] FEATURES+=intest
This way you have a minimal number of SR's to do the job (1), and that
means that each SR you define is easier to understand, because you don't
need to consistently define a whole bunch of them just to get a
relatively simple test policy implemented.
-- Gareth
Btw. there is a 'problem' with torque, in that if you define node
features in moab.cfg's NODECFG, then you can't request such node
properties at job submission because torque does not know about them and
will reject the job. ie qsub -l nodes=1:test a.sh
Will get rejected, at least with our configuration:
qsub: Job rejected by all possible destinations
To work around this, the node feature needs to be defined in torque -
but that means keeping the torque and moab configurations consistent
which is not desirable from a managability point of view.
An implication is that some node features are ones that users may need
to specify and others should be only used internally. Perhaps I should
use a QOS instead of a FEATURE for the purely internal usage - but that
is not possible as a QOS can't be defined in NODECFG. So make the QOS
the user specified thing. The test facility is then configured:
#WARNING THIS DID NOT WORK
# all intest nodes will have a SR only available to testqos
SRCFG[testsr] QOSLIST=testqos HOSTLIST=ALL NODEFEATURES=intest
# jobs to qtest will get testqos but jobs can also explicitly request
qsub -l qos=testqos
CLASSCFG[qtest] QDEF=testqos
# nodes still need intest to setup the SR
NODECFG[node10] FEATURES+=intest
NODECFG[node11] FEATURES+=intest
Note that qtest would be optional, but you would need to configure ACLs
for the QOS.
Jobs with qos=testqos may still go elsewhere (but will have affinity to
the testsr SR). This could (in principle) be fixed with the CredLock
ACL modifier, as long as that's all that testqos is used for:
SRCFG[testsr] QOSLIST=&testqos HOSTLIST=ALL NODEFEATURES=intest
But... I could not get this to work.
> -----Original Message-----
> From: Douglas Wightman [mailto:wightman at clusterresources.com]
> Sent: Tuesday, 15 May 2007 5:36 PM
> To: Lloyd Brown
> Cc: moabusers at supercluster.org
> Subject: Re: [Moabusers] mapping node features to a class
>
>
> Unfortunately there is no "|" feature available for node features. We
> have code that does this for jobs but it would have to be extended to
> reservations.
>
> - Douglas
>
> On Mon, 2007-05-14 at 16:33 -0600, Lloyd Brown wrote:
> > That looks like it will work. Just one additional
> question, though.
> > Some of the SRCFG and CLASSCFG parameters allow you to
> specify comma
> > separated (meaning require all of these), and pipe ("|") separated
> > (meaning require one of these) in lists. Does nodefeatures
> do this?
> > You see, I oversimplified a little in my earlier
> description: what I
> > actually have is a set of four mutually-exclusive
> attributes, (featureA,
> > featureB, featureC, and featureD), and I want to add an
> additional one
> > as featureE. I don't care which set of nodes a
> non-featureE job goes
> > to, as long as it's not featureE, but a job that requests
> featureE must
> > get featureE.
> >
> > On our staging area, this did work:
> >
> > > NODETOJOBATTRMAP featureE
> > >
> > > SRCFG[batch_sr_a] CLASSLIST=batch HOSTLIST=ALL
> > > NODEFEATURES=featureA
> > > SRCFG[batch_sr_b] CLASSLIST=batch HOSTLIST=ALL
> > > NODEFEATURES=featureB
> > > SRCFG[batch_sr_c] CLASSLIST=batch HOSTLIST=ALL
> > > NODEFEATURES=featureC
> > > SRCFG[batch_sr_c] CLASSLIST=batch HOSTLIST=ALL
> > > NODEFEATURES=featureD
> > > SRCFG[test_sr] CLASSLIST=test HOSTLIST=ALL
> > > NODEFEATURES=featureE JOBATTRLIST=featureE
> >
> > Is there any way I can combine all those "batch_sr_?"
> SRCFGs into one?
> > I tried both of the following:
> >
> > > NODEFEATURES=featureA,featureB,featureC,featureD
> >
> > and
> > > NODEFEATURES=featureA|featureB|featureC|featureD
> >
> > but neither worked. The log looked like neither one could
> create the
> > combined standing reservation, the first time because no
> node had all
> > the features listed, and the second time because no node
> had the feature
> > "featureA|featureB|featureC|featureD". I'm just
> speculating here, though.
> >
> > Thanks,
> > Lloyd
> >
> >
> >
> >
> > Dave Jackson wrote:
> > > Lloyd,
> > >
> > > You could use a standing reservation, ie
> > >
> > > SRCFG[A] classlist=classA hostlist=ALL nodefeatures=featureA
> > > SRCFG[B] classlist=classB hostlist=ALL nodefeatures=featureB
> > >
> > > Please let us know if this will work for you.
> > >
> > > Dave
> > >
> > > On Mon, 2007-05-14 at 15:52 -0600, Lloyd Brown wrote:
> > >
> > >> Hey, all. In Moab, is there an easy way to map nodes
> with a specific
> > >> feature (like specified in Torque) into a class? If,
> for example, all
> > >> my nodes have either "featureA" or "featureB", but never
> both, and I
> > >> want all the featureA nodes to be in one class/queue,
> and all the
> > >> featureB nodes to be in another class/queue, can I do
> that without using
> > >> the hostlist? We would have a lot of hosts to put into
> the hostlist if
> > >> we had to list them all out; pattern matching would even
> be a lot of
> > >> trouble.
> > >>
> > >> Basically the situation is this: we have two nodes out
> of a total of
> > >> 600+ that we are using to evaluate new hardware. We
> want to allow our
> > >> users to request that a job use these two nodes, but if
> they don't
> > >> request it, the job shouldn't run on those two nodes,
> only on the other
> > >> 600 or so.
> > >>
> > >> Any suggestions or alternative approaches would be helpful.
> > >>
> > >>
> > >> Thanks,
> > >> Lloyd Brown
> > >> _______________________________________________
> > >> moabusers mailing list
> > >> moabusers at supercluster.org
> > >> http://www.supercluster.org/mailman/listinfo/moabusers
> > >>
> > >
> > >
> > >
> >
> > _______________________________________________
> > moabusers mailing list
> > moabusers at supercluster.org
> > http://www.supercluster.org/mailman/listinfo/moabusers
>
>
>
More information about the moabusers
mailing list