> Idle limits are mostly good. This mostly limits the number of each<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class="im">
> users jobs that maui will consider in any scheduling cycle so it make<br>
> the scheduling cycle shorter/faster. It also limits the priority<br>
> accumulated by queued jobs and alleviates 'queue stuffing'. I'd<br>
> recommend idle limits given that maui does not contain a better<br>
> facility to handle such issues.<br>
<br>
</div>Yep, a short queue reduces maui stress. I completely agree that.<br>
Seting a limit of 100 jobs per user leaves a 1k idle queue in normal<br>
behaviour, when many user are running jobs. That's the limit I'd use.<br>
<br>
<br>
But, as I've never tried this before, let me ask how maui will behave<br>
in this situation:<br>
<br>
<br>
if the farm is 70%, and I have only two users who have submited jobs<br>
(user A and B). User A has much more priority than user B, so let's say<br>
that the 30% must be filled with 25% of jobs from user A and 5% jobs<br>
from user B, if I have 1000 jobs in queue (500 from A and 500 from B)<br>
IDLE queue will contain 100 jobs of each user, so each scheduling<br>
cycle is going to schedule 200 jobs, is maui going to fill up the farm<br>
respecting our policies (25/5)? or is it going to start 100 jobs from<br>
each user on each scheduling cycle filling up the farm 15% and 15%?<br></blockquote><div><br><br>I've worked in an example:<br><br>QoS Target:<br><br> QOS<br>-------------<br>lhcatlas* -- 47.42 (user atprd)<br>
magic* -- 3.80 (user maprd)<br><br>I set a limit of 5 jobs per user. I use only one node with 10/20/40 "cpus". 80 jobs each user.<br><br>np=10)<br><br># qstat |grep R|grep -c map<br>5<br># qstat |grep R|grep -c at<br>
5<br><br>QOS<br>-------------<br>lhcatlas 50.00 47.42 50.00<br>magic* 50.00 3.80 50.00<br><br>np=20)<br># qmgr -c "s n <a href="http://pbs01-test.pic.es">pbs01-test.pic.es</a> np=20"<br>
# qstat |grep R|grep -c map<br>10<br># qstat |grep R|grep -c at<br>10<br>QOS<br>-------------<br>lhcatlas 50.00 47.42 50.00<br>magic* 50.00 3.80 50.00<br><br>np=40)<br># qmgr -c "s n <a href="http://pbs01-test.pic.es">pbs01-test.pic.es</a> np=40"<br>
# qstat |grep R|grep -c ma<br>20<br># qstat |grep R|grep -c at<br>20<br>QOS<br>-------------<br>lhcatlas 50.00 47.42 50.00 50.00 50.00<br>magic* 50.00 3.80 50.00 50.00 50.00<br><br><br>Without the limit FS is respected.<br>
<br>Maybe my conf is wrong, but this behaviour is why I'm afraid of limitng idle queue.<br><br>Cheers,<br>Arnau<br></div></div>