[torqueusers] vmem and pvmem

["Mgr. Šimon Tóth"]

On 25.2.2012 00:50, David Singleton wrote:
> On 02/25/2012 09:00 AM, Martin Siegert wrote:
>> On Fri, Feb 24, 2012 at 11:19:37AM +0100, "Mgr. Šimon Tóth" wrote:
>>>> Core_req       vmem  pvmem ulimit-v RPT
>>>> =========================================
>>>> nodes=1:ppn=2  1gb   256mb 256mb    512mb
>>>> procs=2        1gb   256mb 256mb    1gb
>>>> nodes=1:ppn=2  1gb   4gb   1gb      4gb
>>>> procs=2        1gb   4gb   1gb      4gb
>>>> nodes=1:ppn=2  1gb   -     1gb      512mb
>>>> procs=2        1gb   -     1gb      1gb
>>>>
>>>> So the ulimit value that influences whether a task can allocate
>>>> memory, is set as the lower of the vmem and pvmem values. That
>>>> makes some sense - at least more sense than taking the larger
>>>> value.  What doesn't make sense is allowing pvmem to be higher
>>>> than vmem in the first place - in that case torque should probably
>>>> reject the job or 'fix' one of the settings but leaving it as is
>>>> might not be so bad, except for moab's behaviour (keep reading).
>>>
>>> No. The logic is as follows:
>>>
>>> * if pvmem (or pmem) is set
>>>      then set the corresponding ulimit to pvmem (pmem) value
>>>
>>> * if pvmem (or pmem) isn't set
>>>      then set the corresponding ulimit to vmem (mem) value
>>>
>>> Note that using pvmem is mostly pointless. On Linux this represents
>>> address space, not virtual memory.
>>>
>>> You can use vmem as virtual memory, but even that is extremely confusing.
>>
>> I do not understand this comment. Both pvmem and vmem requests will
>> result in RLIMIT_AS getting set.
>
> I disagree with vmem setting RLIMIT_AS if that is what is happening.

Yes it does. It is logical, but also pointless, since this value is 
checked by the node itself. Therefore if the process is not killed by 
the ulimit it will be killed by the node daemon.

The issue here is that RLIMIT_AS is limiting address space (on Linux). 
That's not a resource. If you map a 1GB file to memory your are not 
using up anything.

The correct resource to be limited is mem (pmem) not vmem (pvmem).

-- 
Mgr. Simon Toth