[torqueusers] torque tmpdir on Lustre filesystem

[Lukasz Flis]

Hello,

I would like to ask Lustre users if anyone has pbs_mom variable $tmpdir 
set to a directory located on Lustre filesystem?

Such kind of setup in our case (Lustre 2.1.0 and Lustre 1.8.6) results 
in Torque being unable to create temprorary directory. Not all attempts 
are unsuccessful but it is still significant amount.


Feb 14 13:56:02 n6-4-16 pbs_mom: LOG_ERROR::Permission denied (13) in 
TMakeTmpDir, Unable to make job transient directory: 
/mnt/lustre/scratch/jobs/18555591.batch.grid
Feb 14 13:56:23 n6-4-16 pbs_mom: LOG_ERROR::Permission denied (13) in 
TMakeTmpDir, Unable to make job transient directory: 
/mnt/lustre/scratch/jobs/18555647.batch.grid
Feb 14 14:37:35 n6-4-16 pbs_mom: LOG_ERROR::Permission denied (13) in 
TMakeTmpDir, Unable to make job transient directory: 
/mnt/lustre/scratch/jobs/18557701.batch.grid
Feb 14 14:38:17 n6-4-16 pbs_mom: LOG_ERROR::Permission denied (13) in 
TMakeTmpDir, Unable to make job transient directory: 
/mnt/lustre/scratch/jobs/18557716.batch.grid

This is not a bug in torque but problem with lustre itself.

I'm just curious if we're the only site in the world facing the problem.

Here is a bug report in jira at whamcloud for more details:

http://jira.whamcloud.com/browse/LU-1101


Cheers,
--
Lukasz Flis
ACC Cyfronet AGH,
Cracow, POLAND