<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7638.1">
<TITLE>Standing Reservation Problem</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<P><FONT SIZE=2 FACE="Verdana">Mauiusers,</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">I seem to be having trouble understanding the behavior of Maui. We are running Maui on Torque. I have set up queues via Torque and two Standing Reservations via Maui to direct jobs to a small cluster containing 1 Master node and 2 compute nodes. All nodes have a single cpu and 1 GB of RAM.</FONT></P>
<P><FONT SIZE=2 FACE="Verdana">The intent of my test is to execute prime jobs on mylnxc1-n001 and glide jobs on mylnxc1-n002 anytime. Additionally, I would like to run ghts, test, any, and all jobs anytime on either node mylnxc1-n001 or mylnxc1-n002. However, when submitting jobs to the prime or glide queues, they get stuck in the queue and never execute. Checkjob shows they are waiting for resources but there is nothing running on the system (see below). Jobs sent to the other queues execute properly. If I comment out the 3<SUP>rd</SUP> standing reservation, then the prime and glide jobs execute properly but all other jobs now get stuck in the queues with the same message from checkjob. It would appear that maui won’t let me map multiple queues onto the nodes. Is anyone else experiencing this behavior?</FONT></P>
<P><FONT SIZE=2 FACE="Verdana">Is this a function of the policy? I’ve tried a few different node policy options with the same result for all. It doesn't seem to matter if I change it or not. And, I have the same problem using Maui 3.2.6p11 on Torque 1.2.0p1 as well as on Maui 3.2.6p14 on Torque 2.0.0p4.</FONT></P>
<P><FONT SIZE=2 FACE="Verdana">I also have the maui log set to 9 but it essentially confirms the same deferred message as checkjob. I haven’t included it in this set of data because of the volume, but I can provide it if required.</FONT></P>
<P><FONT SIZE=2 FACE="Verdana">Any help would be greatly appreciated.</FONT>
</P>
<P> <FONT SIZE=2 FACE="Verdana">Stewart</FONT>
</P>
<BR>
<P><FONT SIZE=2 FACE="Verdana">---------------------------------------------------------------------------------------------</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">[root@mylnxc1-a log]# qmgr -c 'p s'</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana"># Create queues and set their attributes.</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana"># Create and define queue glide</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana">create queue glide</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue glide queue_type = Execution</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue glide resources_max.nodect = 1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue glide enabled = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue glide started = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana"># Create and define queue prime</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana">create queue prime</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue prime queue_type = Execution</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue prime resources_max.nodect = 1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue prime enabled = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue prime started = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana"># Create and define queue test</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana">create queue test</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue test queue_type = Execution</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue test resources_max.nodect = 2</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue test enabled = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue test started = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana"># Create and define queue ghts</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana">create queue ghts</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue ghts queue_type = Execution</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue ghts resources_max.nodect = 2</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue ghts enabled = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue ghts started = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana"># Create and define queue any</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana">create queue any</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue any queue_type = Execution</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue any resources_max.nodect = 2</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue any enabled = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue any started = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana"># Create and define queue all</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana">create queue all</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue all queue_type = Execution</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue all resources_max.nodect = 2</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue all enabled = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set queue all started = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana"># Set server attributes.</FONT>
<BR><FONT SIZE=2 FACE="Verdana">#</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server scheduling = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server default_queue = ghts</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server log_events = 511</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server mail_from = adm</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server query_other_jobs = True</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server resources_default.neednodes = 1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server resources_default.nodect = 1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server resources_default.nodes = 1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server scheduler_iteration = 600</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server node_ping_rate = 300</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server node_check_rate = 600</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server tcp_timeout = 6</FONT>
<BR><FONT SIZE=2 FACE="Verdana">set server node_pack = False</FONT>
<BR><FONT SIZE=2 FACE="Verdana">[root@mylnxc1-a log]#</FONT>
</P>
<BR>
<P><FONT SIZE=2 FACE="Verdana">------------------------------------------------------------------------------------------</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">[root@mylnxc1-a log]# My maui.cfg</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">QUEUETIMEWEIGHT 10</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">BACKFILLPOLICY FIRSTFIT</FONT>
<BR><FONT SIZE=2 FACE="Verdana">RESERVATIONPOLICY CURRENTHIGHEST</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">#NODEALLOCATIONPOLICY MINRESOURCE</FONT>
<BR><FONT SIZE=2 FACE="Verdana">JOBNODEMATCHPOLICY EXACTNODE</FONT>
<BR><FONT SIZE=2 FACE="Verdana">NODEACCESSPOLICY SHARED</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">CLASSCFG[glide] MAXPROC=1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">CLASSCFG[prime] MAXPROC=1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">CLASSCFG[test] MAXPROC=2</FONT>
<BR><FONT SIZE=2 FACE="Verdana">CLASSCFG[ghts] MAXPROC=2</FONT>
<BR><FONT SIZE=2 FACE="Verdana">CLASSCFG[all] MAXPROC=2</FONT>
<BR><FONT SIZE=2 FACE="Verdana">CLASSCFG[any] MAXPROC=2</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">CREDWEIGHT 1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">CLASSWEIGHT 1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">QOSWEIGHT 1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">XFACTORWEIGHT 1</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">SRCFG[prime] CLASSLIST=prime</FONT>
<BR><FONT SIZE=2 FACE="Verdana">SRCFG[prime] PERIOD=INFINITY</FONT>
<BR><FONT SIZE=2 FACE="Verdana">SRCFG[prime] HOSTLIST=mylnxc1-n001</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">SRCFG[glide] CLASSLIST=glide</FONT>
<BR><FONT SIZE=2 FACE="Verdana">SRCFG[glide] PERIOD=INFINITY</FONT>
<BR><FONT SIZE=2 FACE="Verdana">SRCFG[glide] HOSTLIST=mylnxc1-n002</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">SRCFG[ghts] CLASSLIST=ghts,test,any,all</FONT>
<BR><FONT SIZE=2 FACE="Verdana">SRCFG[ghts] PERIOD=INFINITY</FONT>
<BR><FONT SIZE=2 FACE="Verdana">SRCFG[ghts] HOSTLIST=mylnxc1-n00[1-2]</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">[nm67109@mylnxc1-a nm67109]$ checkjob 108</FONT>
</P>
<BR>
<P><FONT SIZE=2 FACE="Verdana">checking job 108</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">State: Idle EState: Deferred</FONT>
<BR><FONT SIZE=2 FACE="Verdana">Creds: user:nm67109 group:lgdgis class:prime qos:DEFAULT</FONT>
<BR><FONT SIZE=2 FACE="Verdana">WallTime: 00:00:00 of 99:23:59:59</FONT>
<BR><FONT SIZE=2 FACE="Verdana">SubmitTime: Fri Feb 10 17:04:49</FONT>
<BR><FONT SIZE=2 FACE="Verdana"> (Time Queued Total: 00:00:45 Eligible: 00:00:01)</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">Total Tasks: 1</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">Req[0] TaskCount: 1 Partition: ALL</FONT>
<BR><FONT SIZE=2 FACE="Verdana">Network: [NONE] Memory >= 0 Disk >= 0 Swap >= 0</FONT>
<BR><FONT SIZE=2 FACE="Verdana">Opsys: [NONE] Arch: [NONE] Features: [NONE]</FONT>
</P>
<BR>
<P><FONT SIZE=2 FACE="Verdana">IWD: [NONE] Executable: [NONE]</FONT>
<BR><FONT SIZE=2 FACE="Verdana">Bypass: 0 StartCount: 0</FONT>
<BR><FONT SIZE=2 FACE="Verdana">PartitionMask: [ALL]</FONT>
<BR><FONT SIZE=2 FACE="Verdana">Flags: RESTARTABLE</FONT>
</P>
<P><FONT SIZE=2 FACE="Verdana">job is deferred. Reason: NoResources (cannot create reservation for job '108'</FONT>
<BR><FONT SIZE=2 FACE="Verdana"> (intital reservation attempt)</FONT>
<BR><FONT SIZE=2 FACE="Verdana">)</FONT>
<BR><FONT SIZE=2 FACE="Verdana">Holds: Defer (hold reason: NoResources)</FONT>
<BR><FONT SIZE=2 FACE="Verdana">PE: 1.00 StartPriority: 1</FONT>
<BR><FONT SIZE=2 FACE="Verdana">cannot select job 108 for partition DEFAULT (job hold active)</FONT>
</P>
<BR>
<P><FONT SIZE=2 FACE="Verdana">[nm67109@mylnxc1-a nm67109]$</FONT>
</P>
<BR>
<BR>
<P><B><I><FONT FACE="Script MT Bold"></FONT></I> <FONT FACE="Script MT Bold"> <U> </U></FONT><U><FONT COLOR="#0000FF" FACE="Script MT Bold">Stewart Samuels</FONT></U></B>
<BR><B><FONT FACE="Script MT Bold"> Infrastructure Evolution and Integration</FONT></B>
<BR><B><FONT FACE="Script MT Bold"> Scientific and Medical Affairs </FONT></B>
<BR><B><FONT FACE="Script MT Bold"> Sanofi-Aventis Pharmaceutical </FONT></B> <FONT SIZE=2 FACE="Arial"> </FONT><B></B><B></B><B> </B>
<BR><B><FONT FACE="Script MT Bold"> 1041 Route 202-206 </FONT></B>
<BR><B><FONT FACE="Script MT Bold"> Bridgewater, NJ 08807</FONT></B>
</P>
<P><B><FONT FACE="Script MT Bold"> </FONT><FONT COLOR="#0000FF" FACE="Script MT Bold">Phone: (908) 231-4762</FONT></B>
<BR><B><FONT FACE="Script MT Bold"> </FONT><FONT COLOR="#0000FF" FACE="Script MT Bold">Fax: (908) 231-3488</FONT></B>
<BR><B><FONT COLOR="#0000FF" FACE="Script MT Bold"> email: Stewart.Samuels@Sanofi-Aventis.com</FONT></B>
</P>
</BODY>
</HTML>