[Moabusers] Kudos and a question (long!).

[Charles Johnson]

A good first conference. Thanks for the opportunity to be there. I am  
amazed, though, that hotels so often under power their wifi offerings!

Now for the question. We have a cluster that is a roughly equal mix of  
x86_64 and ppc64. From time to time the x86's experience very high use  
while the ppc's are grossly underutilized. We have a couple of users  
trying to be good cluster citizens, compiling their jobs to run on  
both architectures and wanting to have the scheduler do its best (all  
things considered) to put their jobs (all single core jobs) on a  
ppc64, but if there isn't a suitable one available, then use an x86 if  
it is available. What they have in mind is "if both ppc and x86 are  
available, then please select the ppc. If no ppc is available, then  
please select an x86."

My question: what is a "best practice" in trying to accomplish this?

I thought that nodesets might accomplish this. On my test cluster with  
no jobs running, I added

NODESETATTRIBUTE FEATURE
NODESETPRIORITYTYE WORSTFIT

to the maob.cfg file, restarted moab and then submitted 4 single-cpu  
jobs that included the line:

#PBS -l nodeset=FIRSTOF:FEATURE:ppc64:x86

My unwarranted expectation was that the ppc would be favored in  
running the jobs, but this doesn't appear to be the case. On some runs  
the jobs alternated between ppc and x86, some runs were x86 dominated  
and on other runs the ppc was the processor of choice.

So, for a user who can run a job on either x86, or ppc64, who wants to  
have her or his job preferentially run on the ppc64, but not have the  
job placed in the idle queue if a suitable x86 is available, what is  
the best way to accomplish this?

Any pointers, especially to documentation, would be appreciated.

Again, thanks for a solid con. I am looking forward to next year.

Cheers--

Charles
---
Charles Johnson
Advanced Computing Center for Research and Education
Vanderbilt University
charles.johnson at accre.vanderbilt.edu
Office: 615-343-2776
Cell: 615-478-8799