<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=ks_c_5601-1987">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:"¸¼Àº °íµñ";
panose-1:2 11 5 3 2 0 0 2 0 4;}
@font-face
{font-family:"\@¸¼Àº °íµñ";
panose-1:2 11 5 3 2 0 0 2 0 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
text-autospace:none;
word-break:break-hangul;
font-size:10.0pt;
font-family:"¸¼Àº °íµñ";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:40.0pt;
margin-bottom:.0001pt;
mso-para-margin-top:0cm;
mso-para-margin-right:0cm;
mso-para-margin-bottom:0cm;
mso-para-margin-left:4.0gd;
mso-para-margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
text-autospace:none;
word-break:break-hangul;
font-size:10.0pt;
font-family:"¸¼Àº °íµñ";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"¸¼Àº °íµñ";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
/* Page Definitions */
@page Section1
{size:612.0pt 792.0pt;
margin:3.0cm 72.0pt 72.0pt 72.0pt;}
div.Section1
{page:Section1;}
/* List Definitions */
@list l0
{mso-list-id:434181388;
mso-list-type:hybrid;
mso-list-template-ids:-1094542454 456546686 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:18.0pt;
text-indent:-18.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=KO link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span lang=EN-US>Hi all.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>I have little (but serious) problem when
submitting a job using mpirun.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>There¡¯s no problem with just ¡°1¡± node (many
processors) like below.<o:p></o:p></span></p>
<div style='mso-element:para-border-div;border:none;border-bottom:solid windowtext 1.0pt;
padding:0cm 0cm 1.0pt 0cm'>
<p class=MsoNormal style='border:none;padding:0cm'><span lang=EN-US><o:p> </o:p></span></p>
</div>
<p class=MsoNormal><span lang=EN-US>(job script)<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>#!/bin/sh<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>#PBS -l nodes=1:ppn=2<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>#PBS -j oe<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>echo "HOSTNAME : $HOSTNAME"<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>echo "PBS_NODEFILE =
$PBS_NODEFILE"<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>cat $PBS_NODEFILE<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>mpirun /home/jhlee/test_program<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>echo "finish : $(date)"<o:p></o:p></span></p>
<div style='mso-element:para-border-div;border-top:solid windowtext 1.0pt;
border-left:none;border-bottom:solid windowtext 1.0pt;border-right:none;
padding:1.0pt 0cm 1.0pt 0cm'>
<p class=MsoNormal style='border:none;padding:0cm'><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal style='border:none;padding:0cm'><span lang=EN-US><o:p> </o:p></span></p>
</div>
<p class=MsoNormal><span lang=EN-US>(result) – test_program just prints message
whether it is executed by mpirun or not. <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>start : Sun Jan 24 07:46:27 KST 2010<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>HOSTNAME : simulation01<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>PBS_NODEFILE = /var/spool/torque/aux//31.simulation00<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>simulation01<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>simulation01<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Detected OpenMPI Runtime Environment<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Detected OpenMPI Runtime Environment<o:p></o:p></span></p>
<div style='mso-element:para-border-div;border:none;border-bottom:solid windowtext 1.0pt;
padding:0cm 0cm 1.0pt 0cm'>
<p class=MsoNormal style='border:none;padding:0cm'><span lang=EN-US>finish :
Sun Jan 24 07:46:29 KST 2010<o:p></o:p></span></p>
</div>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>But with many nodes like below, mpirun cannot
make test_program start.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>#PBS -l nodes=2:ppn=2 (other things are
same)<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>I can¡¯t find any process. There¡¯s only
mpirun, no ¡®test_program¡¯. Please check the ¡®ps¡¯ result below.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>21680
? S 0:00
mpirun /home/jhlee/test_program<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>21684
? Ss 0:00
bash -c ps ax | grep test<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>21712
? R
0:00 grep test<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoListParagraph style='margin-left:18.0pt;mso-para-margin-left:0gd;
text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span
lang=EN-US><span style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span lang=EN-US>mpirun(not via TORQUE) works
correctly.<o:p></o:p></span></p>
<p class=MsoListParagraph style='margin-left:18.0pt;mso-para-margin-left:0gd;
text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span
lang=EN-US><span style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span lang=EN-US>OpenMPI was built with –with-tm
option.<o:p></o:p></span></p>
<p class=MsoListParagraph style='margin-left:18.0pt;mso-para-margin-left:0gd;
text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span
lang=EN-US><span style='mso-list:Ignore'>3.<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span lang=EN-US>iptables, selinux has been
shutdown already. And no password is required to connect other nodes using ssh.<o:p></o:p></span></p>
<p class=MsoListParagraph style='margin-left:18.0pt;mso-para-margin-left:0gd;
text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span
lang=EN-US><span style='mso-list:Ignore'>4.<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span lang=EN-US>OpenMPI 1.4.1, TORQUE 2.4.4<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>What can I check to solve this ?<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Thanks.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-family:"Tahoma","sans-serif";
color:black'>-------------------------------------------------------------------------------------------<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-family:"Tahoma","sans-serif";
color:black'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-family:"Tahoma","sans-serif";
color:black'>Jeong-hyun Lee<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-family:"Tahoma","sans-serif";
color:black'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-family:"Tahoma","sans-serif";
color:black'>Visual Simulation Laboratory <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-family:"Tahoma","sans-serif";
color:black'>Department of Computer Science and Engineering <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-family:"Tahoma","sans-serif";
color:black'>Dongguk University, Seoul, Korea <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
</div>
</body>
</html>