<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7226.0">
<TITLE>File descriptor limits with mpiexec and pbs/torque</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<P><FONT SIZE=2 FACE="Arial">Hello,</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">We recently discovered when launching > 504 processes on a system you hit a file descriptor limit.</FONT>
<BR><FONT SIZE=2 FACE="Arial">If you add the following to the startup script of your pbs_mom:</FONT>
<BR><FONT SIZE=2 FACE="Arial">----</FONT>
<BR><FONT SIZE=2 FACE="Arial">ulimit -n 4096</FONT>
<BR><FONT SIZE=2 FACE="Arial">----</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">And ensure that the fs.file-max parameter is high enough in sysctl.conf</FONT>
<BR><FONT SIZE=2 FACE="Arial">---</FONT>
<BR><FONT SIZE=2 FACE="Arial">fs.file-max = 65536</FONT>
<BR><FONT SIZE=2 FACE="Arial">---</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">You can launch job sizes > 504. This is tested with torque 1.1.0p4 and mpiexec-0.77 ….</FONT>
<BR><FONT SIZE=2 FACE="Arial">We modified the pbs_mom startup script from a modified Nico rpm for 1.1.0p4. :-)</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">-- Chris</FONT>
</P>
</BODY>
</HTML>