[Mauiusers] per-queue group priority setting

[Corey Ferrier]

On Wed, Jul 23, 2008 at 09:29:53AM -0600, Peter Ruprecht wrote:
>Corey, thanks for your suggestion - I'm just now getting the chance
>to test with it.  In fact, my situation is a little more complicated
>than I had let on in my original post.  Actually, I have two queues,
>xeon and opteron, and about 20 different groups of users.  One the
>xeon queue, the rey and ce groups should have a priority of 100000, the
>greene group should get 10000, and everyone else 1.  On the opteron queue,
>the nesbitt group should get priority of 20000, 16 other groups in an
>"amo" subdivision should get 10000, and everyone else 1.  Below is my
>maui.cfg which I hoped would set that up.
>
[..]
>QOSCFG[reyxeon]      PRIORITY=100000
>QOSCFG[greenexeon]   PRIORITY=10000
>QOSCFG[normalxeon]   PRIORITY=1
>QOSCFG[nesbittopt]   PRIORITY=20000
>QOSCFG[amoopt]       PRIORITY=10000
>QOSCFG[normalopt]    PRIORITY=1
>
># Creds: http://supercluster.org/mauidocs/6.1fairnessoverview.html
>
>CLASSCFG[xeon]       QLIST=reyxeon:greenexeon:normalxeon
>CLASSCFG[opteron]    QLIST=nesbittopt:amoopt:normalopt
>
>GROUPWEIGHT             1
[..]
>GROUPCFG[ce]         QDEF=reyxeon       QLIST=reyxeon:amoopt
[..]
>However, it doesn't seem to be assigning group priorities right.  When I
>submit a job as a member of the ce group to the opteron and xeon queues,
>requesting 10 nodes with 4 processors each, here's what happens:
>
>[root at yotta ~]# qstat -an
>yotta.colorado.edu:
>                                                                   Req'd  Req'd   Elap
>Job ID               Username Queue    Jobname    SessID NDS   TSK Memory Time  S Time
>-------------------- -------- -------- ---------- ------ ----- --- ------ ----- - -----
>9682.yotta.colorado. ruprech  opteron  test          --     10  --    --    --  Q   --
>    --
>9685.yotta.colorado. ruprech  xeon     test          --     10  --    --    --  Q   --
>    --
>9686.yotta.colorado. ruprech  xeon     test          --     10  --    --    --  Q   --
>    --
>
>(Other jobs edited out for clarity.)
>
>
>[root at yotta ~]# diagnose -p
>diagnosing job priority information (partition: ALL)
>Job                    PRIORITY*   Cred(Group)  Serv(QTime:Bypas)   Res( Proc)
>             Weights   --------       1(    1)     1(   50: 1000)     1(  500)
>9682                      22138     0.0(  0.0)   9.7(138.3:2000.)  90.3(20000)
>9685                      22054     0.0(  0.0)   9.3( 54.2:2000.)  90.7(20000)
>9686                      22050     0.0(  0.0)   9.3( 50.0:2000.)  90.7(20000)
>Percent Contribution   --------     0.0(  0.0)   9.4(  0.4:  9.1)  90.6( 90.6)
>
[..]

Maybe try using a ^ in the GROUPCFG lines for the QLIST definition
to force this group to only use QOS reyxeon or amoopt, eg:

  GROUPCFG[ce]         QDEF=reyxeon       QLIST=reyxeon^:amoopt^

Then when a member of group ce sends a job to either xeon or opteron
queues, that job is forced to use the group's qlist.  For example, when 
submitting to queue xeon, group ce must use qos reyxeon.  At least
that is my interpretation of the note on this page:

  http://www.clusterresources.com/products/maui/docs/jobflagoverview.shtml

- Corey

-- 
Corey Ferrier                               coreyf at clemson.edu
HPC Group, CCIT, Clemson University               864-656-2790
340 Computer Court, Anderson, SC, USA 29625