06/09/2010 09:44:40;0002;PBS_Server;Svr;Log;Log opened
06/09/2010 09:44:40;0006;PBS_Server;Svr;PBS_Server;Server kmn started, initialization type = 1
06/09/2010 09:44:40;0002;PBS_Server;Svr;Act;Account file /var/spool/torque/server_priv/accounting/20100609 opened
06/09/2010 09:44:40;0040;PBS_Server;Req;setup_nodes;setup_nodes()
06/09/2010 09:44:40;0086;PBS_Server;Svr;PBS_Server;Recovered queue batch
06/09/2010 09:44:40;0002;PBS_Server;Svr;PBS_Server;Expected 1, recovered 1 queues
06/09/2010 09:44:40;0002;PBS_Server;Svr;PBS_Server;Expected 0, recovered 0 jobs
06/09/2010 09:44:40;0006;PBS_Server;Svr;PBS_Server;Using ports Server:15001  Scheduler:15004  MOM:15002 (server: 'kmn')
06/09/2010 09:44:40;0002;PBS_Server;Svr;daemonize_server;INFO:      parent is exiting
06/09/2010 09:44:40;0002;PBS_Server;Svr;daemonize_server;INFO:      parent is exiting
06/09/2010 09:44:40;0002;PBS_Server;Svr;daemonize_server;INFO:      child process in background
06/09/2010 09:44:40;0002;PBS_Server;Svr;PBS_Server;Server Ready, pid = 2983, loglevel=0
06/09/2010 09:44:40;0004;PBS_Server;Svr;WARNING;ALERT: unable to contact node kmn
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;starting
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;ping attempting to contact 8 nodes
06/09/2010 09:44:45;0040;PBS_Server;Req;update_node_state;adjusting state for node kmn - state=514, newstate=2
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;sending ping to kmn (new stream 0)
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;successful ping to node kmn (stream 0)
06/09/2010 09:44:45;0040;PBS_Server;Req;update_node_state;adjusting state for node L18 - state=514, newstate=2
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;sending ping to L18 (new stream 1)
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;successful ping to node L18 (stream 1)
06/09/2010 09:44:45;0040;PBS_Server;Req;update_node_state;adjusting state for node L11 - state=514, newstate=2
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;sending ping to L11 (new stream 2)
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;successful ping to node L11 (stream 2)
06/09/2010 09:44:45;0040;PBS_Server;Req;update_node_state;adjusting state for node a01 - state=514, newstate=2
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;sending ping to a01 (new stream 3)
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;successful ping to node a01 (stream 3)
06/09/2010 09:44:45;0040;PBS_Server;Req;update_node_state;adjusting state for node a02 - state=514, newstate=2
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;sending ping to a02 (new stream 4)
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;successful ping to node a02 (stream 4)
06/09/2010 09:44:45;0040;PBS_Server;Req;update_node_state;adjusting state for node a03 - state=514, newstate=2
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;sending ping to a03 (new stream 5)
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;successful ping to node a03 (stream 5)
06/09/2010 09:44:45;0040;PBS_Server;Req;update_node_state;adjusting state for node a04 - state=514, newstate=2
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;sending ping to a04 (new stream 6)
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;successful ping to node a04 (stream 6)
06/09/2010 09:44:45;0040;PBS_Server;Req;update_node_state;adjusting state for node a05 - state=514, newstate=2
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;sending ping to a05 (new stream 7)
06/09/2010 09:44:45;0040;PBS_Server;Req;ping_nodes;successful ping to node a05 (stream 7)
06/09/2010 09:44:45;0002;PBS_Server;Svr;PBS_Server;Torque Server Version = 2.5.0, loglevel = 6
06/09/2010 09:44:45;0040;PBS_Server;Req;do_rpp;rpp request received on stream 1
06/09/2010 09:44:45;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message received from stream 1 (version 1)
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message received from stream 192.168.10.18:15003
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message HELLO (1) received from mom on host L18 (192.168.10.18:15003) (stream 1)
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;HELLO received from L18
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;sending cluster-addrs to node L18
06/09/2010 09:44:45;0040;PBS_Server;Req;do_rpp;rpp request received on stream 1
06/09/2010 09:44:45;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message received from stream 1 (version 1)
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message received from stream 192.168.10.18:15003
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message HELLO (1) received from mom on host L18 (192.168.10.18:15003) (stream 1)
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;HELLO received from L18
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;sending cluster-addrs to node L18
06/09/2010 09:44:45;0040;PBS_Server;Req;do_rpp;rpp request received on stream 2
06/09/2010 09:44:45;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message received from stream 2 (version 1)
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message received from stream 192.168.10.11:15003
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message HELLO (1) received from mom on host L11 (192.168.10.11:15003) (stream 2)
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;HELLO received from L11
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;sending cluster-addrs to node L11
06/09/2010 09:44:45;0040;PBS_Server;Req;do_rpp;rpp request received on stream 2
06/09/2010 09:44:45;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message received from stream 2 (version 1)
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message received from stream 192.168.10.11:15003
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;message HELLO (1) received from mom on host L11 (192.168.10.11:15003) (stream 2)
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;HELLO received from L11
06/09/2010 09:44:45;0004;PBS_Server;Svr;is_request;sending cluster-addrs to node L11
06/09/2010 09:44:47;0040;PBS_Server;Req;do_rpp;rpp request received on stream 1
06/09/2010 09:44:47;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:44:47;0004;PBS_Server;Svr;is_request;message received from stream 1 (version 1)
06/09/2010 09:44:47;0004;PBS_Server;Svr;is_request;message received from stream 192.168.10.18:15003
06/09/2010 09:44:47;0004;PBS_Server;Svr;is_request;message STATUS (4) received from mom on host L18 (192.168.10.18:15003) (stream 1)
06/09/2010 09:44:47;0004;PBS_Server;Svr;is_request;IS_STATUS received from L18
06/09/2010 09:44:47;0040;PBS_Server;Req;is_stat_get;received status from node L18
06/09/2010 09:44:47;0040;PBS_Server;Req;update_node_state;adjusting state for node L18 - state=2, newstate=0
06/09/2010 09:44:47;0040;PBS_Server;Req;update_node_state;node L18 marked free
06/09/2010 09:44:47;0040;PBS_Server;Req;do_rpp;rpp request received on stream 2
06/09/2010 09:44:47;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:44:47;0004;PBS_Server;Svr;is_request;message received from stream 2 (version 1)
06/09/2010 09:44:47;0004;PBS_Server;Svr;is_request;message received from stream 192.168.10.11:15003
06/09/2010 09:44:47;0004;PBS_Server;Svr;is_request;message STATUS (4) received from mom on host L11 (192.168.10.11:15003) (stream 2)
06/09/2010 09:44:47;0004;PBS_Server;Svr;is_request;IS_STATUS received from L11
06/09/2010 09:44:47;0040;PBS_Server;Req;is_stat_get;received status from node L11
06/09/2010 09:44:47;0040;PBS_Server;Req;update_node_state;adjusting state for node L11 - state=2, newstate=0
06/09/2010 09:44:47;0040;PBS_Server;Req;update_node_state;node L11 marked free
06/09/2010 09:44:49;0040;PBS_Server;Req;do_rpp;rpp request received on stream 3
06/09/2010 09:44:49;0040;PBS_Server;Req;do_rpp;corrupt rpp request received on stream 3 (node: "a01", 192.168.0.201:15003) - invalid protocol - rc=7 (Premature end of message)
06/09/2010 09:44:49;0001;PBS_Server;Svr;PBS_Server;LOG_ERROR::stream_eof, connection to a01 is bad, remote service may be down, message may be corrupt, or connection may have been dropped remotely (Premature end of message).  setting node state to down
06/09/2010 09:44:49;0040;PBS_Server;Req;update_node_state;adjusting state for node a01 - state=514, newstate=2
06/09/2010 09:44:49;0040;PBS_Server;Req;tdelete;deleting key 3
06/09/2010 09:44:49;0040;PBS_Server;Req;do_rpp;rpp request received on stream 4
06/09/2010 09:44:49;0040;PBS_Server;Req;do_rpp;corrupt rpp request received on stream 4 (node: "a02", 192.168.0.202:15003) - invalid protocol - rc=7 (Premature end of message)
06/09/2010 09:44:49;0001;PBS_Server;Svr;PBS_Server;LOG_ERROR::stream_eof, connection to a02 is bad, remote service may be down, message may be corrupt, or connection may have been dropped remotely (Premature end of message).  setting node state to down
06/09/2010 09:44:49;0040;PBS_Server;Req;update_node_state;adjusting state for node a02 - state=514, newstate=2
06/09/2010 09:44:49;0040;PBS_Server;Req;tdelete;deleting key 4
06/09/2010 09:44:49;0040;PBS_Server;Req;do_rpp;rpp request received on stream 5
06/09/2010 09:44:49;0040;PBS_Server;Req;do_rpp;corrupt rpp request received on stream 5 (node: "a03", 192.168.0.203:15003) - invalid protocol - rc=7 (Premature end of message)
06/09/2010 09:44:49;0001;PBS_Server;Svr;PBS_Server;LOG_ERROR::stream_eof, connection to a03 is bad, remote service may be down, message may be corrupt, or connection may have been dropped remotely (Premature end of message).  setting node state to down
06/09/2010 09:44:49;0040;PBS_Server;Req;update_node_state;adjusting state for node a03 - state=514, newstate=2
06/09/2010 09:44:49;0040;PBS_Server;Req;tdelete;deleting key 5
06/09/2010 09:44:49;0040;PBS_Server;Req;do_rpp;rpp request received on stream 6
06/09/2010 09:44:49;0040;PBS_Server;Req;do_rpp;corrupt rpp request received on stream 6 (node: "a04", 192.168.0.204:15003) - invalid protocol - rc=7 (Premature end of message)
06/09/2010 09:44:49;0001;PBS_Server;Svr;PBS_Server;LOG_ERROR::stream_eof, connection to a04 is bad, remote service may be down, message may be corrupt, or connection may have been dropped remotely (Premature end of message).  setting node state to down
06/09/2010 09:44:49;0040;PBS_Server;Req;update_node_state;adjusting state for node a04 - state=514, newstate=2
06/09/2010 09:44:49;0040;PBS_Server;Req;tdelete;deleting key 6
06/09/2010 09:44:49;0040;PBS_Server;Req;do_rpp;rpp request received on stream 7
06/09/2010 09:44:49;0040;PBS_Server;Req;do_rpp;corrupt rpp request received on stream 7 (node: "a05", 192.168.0.205:15003) - invalid protocol - rc=7 (Premature end of message)
06/09/2010 09:44:49;0001;PBS_Server;Svr;PBS_Server;LOG_ERROR::stream_eof, connection to a05 is bad, remote service may be down, message may be corrupt, or connection may have been dropped remotely (Premature end of message).  setting node state to down
06/09/2010 09:44:49;0040;PBS_Server;Req;update_node_state;adjusting state for node a05 - state=514, newstate=2
06/09/2010 09:44:49;0040;PBS_Server;Req;tdelete;deleting key 7
06/09/2010 09:44:59;0040;PBS_Server;Req;do_rpp;rpp request received on stream 3
06/09/2010 09:44:59;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:44:59;0004;PBS_Server;Svr;is_request;message received from stream 3 (version 1)
06/09/2010 09:44:59;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.73:1022
06/09/2010 09:44:59;0004;PBS_Server;Svr;is_request;stream 3 from node kmn already open on 0 (marking node state 'unknown')
06/09/2010 09:44:59;0040;PBS_Server;Req;tdelete;deleting key 0
06/09/2010 09:45:01;0080;PBS_Server;Req;dis_request_read;decoding command AuthenticateUser from root
06/09/2010 09:45:01;0100;PBS_Server;Req;;Type AuthenticateUser request received from root@kmn, sock=11
06/09/2010 09:45:01;0008;PBS_Server;Job;dispatch_request;dispatching request AuthenticateUser on sd=11
06/09/2010 09:45:01;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from PBS_Server
06/09/2010 09:45:01;0080;PBS_Server;Req;dis_request_read;decoding command StatusNode from root
06/09/2010 09:45:01;0100;PBS_Server;Req;;Type StatusNode request received from root@kmn, sock=10
06/09/2010 09:45:01;0008;PBS_Server;Job;dispatch_request;dispatching request StatusNode on sd=10
06/09/2010 09:45:01;0040;PBS_Server;Req;req_stat_node;entered
06/09/2010 09:45:01;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from root
06/09/2010 09:45:01;0040;PBS_Server;Req;do_rpp;rpp request received on stream 3
06/09/2010 09:45:01;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;message received from stream 3 (version 1)
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.73:1022
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;message HELLO (1) received from mom on host kmn (192.168.0.73:1022) (stream 3)
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;HELLO received from kmn
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;sending cluster-addrs to node kmn
06/09/2010 09:45:01;0040;PBS_Server;Req;do_rpp;rpp request received on stream 4
06/09/2010 09:45:01;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;message received from stream 4 (version 1)
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.201:1023
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;message HELLO (1) received from mom on host a01 (192.168.0.201:1023) (stream 4)
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;HELLO received from a01
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;sending cluster-addrs to node a01
06/09/2010 09:45:01;0040;PBS_Server;Req;do_rpp;rpp request received on stream 5
06/09/2010 09:45:01;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;message received from stream 5 (version 1)
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.202:1023
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;message HELLO (1) received from mom on host a02 (192.168.0.202:1023) (stream 5)
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;HELLO received from a02
06/09/2010 09:45:01;0004;PBS_Server;Svr;is_request;sending cluster-addrs to node a02
06/09/2010 09:45:02;0040;PBS_Server;Req;do_rpp;rpp request received on stream 6
06/09/2010 09:45:02;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:02;0004;PBS_Server;Svr;is_request;message received from stream 6 (version 1)
06/09/2010 09:45:02;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.203:1023
06/09/2010 09:45:02;0004;PBS_Server;Svr;is_request;message HELLO (1) received from mom on host a03 (192.168.0.203:1023) (stream 6)
06/09/2010 09:45:02;0004;PBS_Server;Svr;is_request;HELLO received from a03
06/09/2010 09:45:02;0004;PBS_Server;Svr;is_request;sending cluster-addrs to node a03
06/09/2010 09:45:02;0040;PBS_Server;Req;do_rpp;rpp request received on stream 7
06/09/2010 09:45:02;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:02;0004;PBS_Server;Svr;is_request;message received from stream 7 (version 1)
06/09/2010 09:45:02;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.204:1023
06/09/2010 09:45:02;0004;PBS_Server;Svr;is_request;message HELLO (1) received from mom on host a04 (192.168.0.204:1023) (stream 7)
06/09/2010 09:45:02;0004;PBS_Server;Svr;is_request;HELLO received from a04
06/09/2010 09:45:02;0004;PBS_Server;Svr;is_request;sending cluster-addrs to node a04
06/09/2010 09:45:03;0040;PBS_Server;Req;do_rpp;rpp request received on stream 3
06/09/2010 09:45:03;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;message received from stream 3 (version 1)
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.73:1022
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;message STATUS (4) received from mom on host kmn (192.168.0.73:1022) (stream 3)
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;IS_STATUS received from kmn
06/09/2010 09:45:03;0040;PBS_Server;Req;is_stat_get;received status from node kmn
06/09/2010 09:45:03;0040;PBS_Server;Req;update_node_state;adjusting state for node kmn - state=256, newstate=0
06/09/2010 09:45:03;0040;PBS_Server;Req;do_rpp;rpp request received on stream 4
06/09/2010 09:45:03;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;message received from stream 4 (version 1)
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.201:1023
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;message STATUS (4) received from mom on host a01 (192.168.0.201:1023) (stream 4)
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;IS_STATUS received from a01
06/09/2010 09:45:03;0040;PBS_Server;Req;is_stat_get;received status from node a01
06/09/2010 09:45:03;0040;PBS_Server;Req;update_node_state;adjusting state for node a01 - state=2, newstate=0
06/09/2010 09:45:03;0040;PBS_Server;Req;update_node_state;node a01 marked free
06/09/2010 09:45:03;0040;PBS_Server;Req;do_rpp;rpp request received on stream 5
06/09/2010 09:45:03;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;message received from stream 5 (version 1)
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.202:1023
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;message STATUS (4) received from mom on host a02 (192.168.0.202:1023) (stream 5)
06/09/2010 09:45:03;0004;PBS_Server;Svr;is_request;IS_STATUS received from a02
06/09/2010 09:45:03;0040;PBS_Server;Req;is_stat_get;received status from node a02
06/09/2010 09:45:03;0040;PBS_Server;Req;update_node_state;adjusting state for node a02 - state=2, newstate=0
06/09/2010 09:45:03;0040;PBS_Server;Req;update_node_state;node a02 marked free
06/09/2010 09:45:04;0040;PBS_Server;Req;do_rpp;rpp request received on stream 6
06/09/2010 09:45:04;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:04;0004;PBS_Server;Svr;is_request;message received from stream 6 (version 1)
06/09/2010 09:45:04;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.203:1023
06/09/2010 09:45:04;0004;PBS_Server;Svr;is_request;message STATUS (4) received from mom on host a03 (192.168.0.203:1023) (stream 6)
06/09/2010 09:45:04;0004;PBS_Server;Svr;is_request;IS_STATUS received from a03
06/09/2010 09:45:04;0040;PBS_Server;Req;is_stat_get;received status from node a03
06/09/2010 09:45:04;0040;PBS_Server;Req;update_node_state;adjusting state for node a03 - state=2, newstate=0
06/09/2010 09:45:04;0040;PBS_Server;Req;update_node_state;node a03 marked free
06/09/2010 09:45:04;0040;PBS_Server;Req;do_rpp;rpp request received on stream 7
06/09/2010 09:45:04;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:04;0004;PBS_Server;Svr;is_request;message received from stream 7 (version 1)
06/09/2010 09:45:04;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.204:1023
06/09/2010 09:45:04;0004;PBS_Server;Svr;is_request;message STATUS (4) received from mom on host a04 (192.168.0.204:1023) (stream 7)
06/09/2010 09:45:04;0004;PBS_Server;Svr;is_request;IS_STATUS received from a04
06/09/2010 09:45:04;0040;PBS_Server;Req;is_stat_get;received status from node a04
06/09/2010 09:45:04;0040;PBS_Server;Req;update_node_state;adjusting state for node a04 - state=2, newstate=0
06/09/2010 09:45:04;0040;PBS_Server;Req;update_node_state;node a04 marked free
06/09/2010 09:45:08;0040;PBS_Server;Req;do_rpp;rpp request received on stream 0
06/09/2010 09:45:08;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:08;0004;PBS_Server;Svr;is_request;message received from stream 0 (version 1)
06/09/2010 09:45:08;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.205:1023
06/09/2010 09:45:08;0004;PBS_Server;Svr;is_request;message HELLO (1) received from mom on host a05 (192.168.0.205:1023) (stream 0)
06/09/2010 09:45:08;0004;PBS_Server;Svr;is_request;HELLO received from a05
06/09/2010 09:45:08;0004;PBS_Server;Svr;is_request;sending cluster-addrs to node a05
06/09/2010 09:45:10;0040;PBS_Server;Req;do_rpp;rpp request received on stream 0
06/09/2010 09:45:10;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:10;0004;PBS_Server;Svr;is_request;message received from stream 0 (version 1)
06/09/2010 09:45:10;0004;PBS_Server;Svr;is_request;message received from stream 192.168.0.205:1023
06/09/2010 09:45:10;0004;PBS_Server;Svr;is_request;message STATUS (4) received from mom on host a05 (192.168.0.205:1023) (stream 0)
06/09/2010 09:45:10;0004;PBS_Server;Svr;is_request;IS_STATUS received from a05
06/09/2010 09:45:10;0040;PBS_Server;Req;is_stat_get;received status from node a05
06/09/2010 09:45:10;0040;PBS_Server;Req;update_node_state;adjusting state for node a05 - state=2, newstate=0
06/09/2010 09:45:10;0040;PBS_Server;Req;update_node_state;node a05 marked free
06/09/2010 09:45:11;0080;PBS_Server;Req;dis_request_read;decoding command AuthenticateUser from ken
06/09/2010 09:45:11;0100;PBS_Server;Req;;Type AuthenticateUser request received from ken@kmn, sock=11
06/09/2010 09:45:11;0008;PBS_Server;Job;dispatch_request;dispatching request AuthenticateUser on sd=11
06/09/2010 09:45:11;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from PBS_Server
06/09/2010 09:45:11;0080;PBS_Server;Req;dis_request_read;decoding command QueueJob from ken
06/09/2010 09:45:11;0100;PBS_Server;Req;;Type QueueJob request received from ken@kmn, sock=10
06/09/2010 09:45:11;0008;PBS_Server;Job;dispatch_request;dispatching request QueueJob on sd=10
06/09/2010 09:45:11;0080;PBS_Server;Req;dis_request_read;decoding command JobScript from ken
06/09/2010 09:45:11;0100;PBS_Server;Req;;Type JobScript request received from ken@kmn, sock=10
06/09/2010 09:45:11;0008;PBS_Server;Job;dispatch_request;dispatching request JobScript on sd=10
06/09/2010 09:45:11;0080;PBS_Server;Req;dis_request_read;decoding command ReadyToCommit from ken
06/09/2010 09:45:11;0100;PBS_Server;Req;;Type ReadyToCommit request received from ken@kmn, sock=10
06/09/2010 09:45:11;0008;PBS_Server;Job;dispatch_request;dispatching request ReadyToCommit on sd=10
06/09/2010 09:45:11;0008;PBS_Server;Job;102.kmn;ready to commit job
06/09/2010 09:45:11;0008;PBS_Server;Job;102.kmn;ready to commit job completed
06/09/2010 09:45:11;0080;PBS_Server;Req;dis_request_read;decoding command Commit from ken
06/09/2010 09:45:11;0100;PBS_Server;Req;;Type Commit request received from ken@kmn, sock=10
06/09/2010 09:45:11;0008;PBS_Server;Job;dispatch_request;dispatching request Commit on sd=10
06/09/2010 09:45:11;0008;PBS_Server;Job;102.kmn;committing job
06/09/2010 09:45:11;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 102.kmn state from TRANSIT-TRANSICM to QUEUED-QUEUED (1-10)
06/09/2010 09:45:11;0100;PBS_Server;Job;102.kmn;enqueuing into batch, state 1 hop 1
06/09/2010 09:45:11;0008;PBS_Server;Job;102.kmn;Job Queued at request of ken@kmn, owner = ken@kmn, job name = psaux, queue = batch
06/09/2010 09:45:11;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from ken
06/09/2010 09:45:16;0080;PBS_Server;Req;dis_request_read;decoding command AuthenticateUser from ken
06/09/2010 09:45:16;0100;PBS_Server;Req;;Type AuthenticateUser request received from ken@kmn, sock=11
06/09/2010 09:45:16;0008;PBS_Server;Job;dispatch_request;dispatching request AuthenticateUser on sd=11
06/09/2010 09:45:16;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from PBS_Server
06/09/2010 09:45:16;0080;PBS_Server;Req;dis_request_read;decoding command RunJob from ken
06/09/2010 09:45:16;0100;PBS_Server;Req;;Type RunJob request received from ken@kmn, sock=10
06/09/2010 09:45:16;0008;PBS_Server;Job;dispatch_request;dispatching request RunJob on sd=10
06/09/2010 09:45:16;0040;PBS_Server;Req;set_nodes;allocating nodes for job 102.kmn with node expression '3'
06/09/2010 09:45:16;0040;PBS_Server;Req;node_spec;entered spec=3
06/09/2010 09:45:16;0040;PBS_Server;Req;node_spec;job allocation debug: 3 requested, 17 svr_clnodes, 8 svr_totnodes
06/09/2010 09:45:16;0040;PBS_Server;Req;node_spec;job allocation debug(2): 3 requested, 8 svr_numnodes
06/09/2010 09:45:16;0040;PBS_Server;Req;node_spec;job allocation debug(3): returning 3 requested
06/09/2010 09:45:16;0040;PBS_Server;Req;add_job_to_node;allocated node kmn/0 to job 102.kmn (nsnfree=4)
06/09/2010 09:45:16;0040;PBS_Server;Req;set_nodes;job 102.kmn allocated 1 nodes (nodelist=kmn/0)
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;Job Run at request of ken@kmn
06/09/2010 09:45:16;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 102.kmn state from QUEUED-QUEUED to RUNNING-PRERUN (4-40)
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;forking in send_job
06/09/2010 09:45:16;0004;PBS_Server;Svr;svr_connect;attempting connect to server 192.168.0.73 port 15002
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;entering post_sendmom
06/09/2010 09:45:16;0002;PBS_Server;Job;102.kmn;child reported success for job after 0 seconds (dest=kmn), rc=0
06/09/2010 09:45:16;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 102.kmn state from RUNNING-PRERUN to RUNNING-RUNNING (4-42)
06/09/2010 09:45:16;0004;PBS_Server;Svr;svr_connect;attempting connect to server 192.168.0.73 port 15002
06/09/2010 09:45:16;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from ken
06/09/2010 09:45:16;000d;PBS_Server;Job;102.kmn;sending 'b' mail for job 102.kmn to ken@kmn (---)
06/09/2010 09:45:16;000d;PBS_Server;Job;102.kmn;Not sending email: User does not want mail of this type.
06/09/2010 09:45:16;0080;PBS_Server;Req;dis_request_read;decoding command StatusJob from pbs_mom
06/09/2010 09:45:16;0100;PBS_Server;Req;;Type StatusJob request received from pbs_mom@kmn, sock=10
06/09/2010 09:45:16;0008;PBS_Server;Job;dispatch_request;dispatching request StatusJob on sd=10
06/09/2010 09:45:16;0080;PBS_Server;Req;dis_request_read;decoding command JobObituary from pbs_mom
06/09/2010 09:45:16;0100;PBS_Server;Req;;Type JobObituary request received from pbs_mom@kmn, sock=10
06/09/2010 09:45:16;0008;PBS_Server;Job;dispatch_request;dispatching request JobObituary on sd=10
06/09/2010 09:45:16;0009;PBS_Server;Job;102.kmn;obit received - updating final job usage info
06/09/2010 09:45:16;0009;PBS_Server;Job;102.kmn;job exit status 0 handled
06/09/2010 09:45:16;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 102.kmn state from RUNNING-RUNNING to EXITING-EXITING (5-50)
06/09/2010 09:45:16;000d;PBS_Server;Job;102.kmn;sending 'e' mail for job 102.kmn to ken@kmn (Exit_status=0
06/09/2010 09:45:16;000d;PBS_Server;Job;102.kmn;Not sending email: User does not want mail of this type.
06/09/2010 09:45:16;0010;PBS_Server;Job;102.kmn;Exit_status=0 resources_used.cput=00:00:00 resources_used.mem=0kb resources_used.vmem=0kb resources_used.walltime=00:00:00
06/09/2010 09:45:16;0009;PBS_Server;Job;102.kmn;on_job_exit task assigned to job
06/09/2010 09:45:16;0009;PBS_Server;Job;102.kmn;req_jobobit completed
06/09/2010 09:45:16;0004;PBS_Server;Svr;svr_connect;attempting connect to server 192.168.0.73 port 15002
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;JOB_SUBSTATE_EXITING
06/09/2010 09:45:16;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 102.kmn state from EXITING-EXITING to EXITING-RETURNSTD (5-70)
06/09/2010 09:45:16;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 102.kmn state from EXITING-RETURNSTD to EXITING-STAGEOUT (5-51)
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;no spool files to return
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;JOB_SUBSTATE_STAGEOUT
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;about to copy stdout/stderr/stageout files
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;JOB_SUBSTATE_STAGEOUT
06/09/2010 09:45:16;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 102.kmn state from EXITING-STAGEOUT to EXITING-STAGEDEL (5-52)
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;JOB_SUBSTATE_STAGEDEL
06/09/2010 09:45:16;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 102.kmn state from EXITING-STAGEDEL to EXITING-EXITED (5-53)
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;JOB_SUBSTATE_EXITED
06/09/2010 09:45:16;0040;PBS_Server;Req;free_nodes;freeing nodes for job 102.kmn
06/09/2010 09:45:16;0040;PBS_Server;Req;free_nodes;freeing node kmn/0 from job 102.kmn (nsnfree=3)
06/09/2010 09:45:16;0040;PBS_Server;Req;free_nodes;increased sub-node free count to 4 of 4
06/09/2010 09:45:16;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 102.kmn state from EXITING-EXITED to COMPLETE-COMPLETE (6-59)
06/09/2010 09:45:16;0008;PBS_Server;Job;102.kmn;JOB_SUBSTATE_COMPLETE
06/09/2010 09:45:19;0080;PBS_Server;Req;dis_request_read;decoding command AuthenticateUser from ken
06/09/2010 09:45:19;0100;PBS_Server;Req;;Type AuthenticateUser request received from ken@kmn, sock=11
06/09/2010 09:45:19;0008;PBS_Server;Job;dispatch_request;dispatching request AuthenticateUser on sd=11
06/09/2010 09:45:19;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from PBS_Server
06/09/2010 09:45:19;0080;PBS_Server;Req;dis_request_read;decoding command StatusJob from ken
06/09/2010 09:45:19;0100;PBS_Server;Req;;Type StatusJob request received from ken@kmn, sock=10
06/09/2010 09:45:19;0008;PBS_Server;Job;dispatch_request;dispatching request StatusJob on sd=10
06/09/2010 09:45:19;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from ken
06/09/2010 09:45:32;0040;PBS_Server;Req;do_rpp;rpp request received on stream 1
06/09/2010 09:45:32;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:32;0004;PBS_Server;Svr;is_request;message received from stream 1 (version 1)
06/09/2010 09:45:32;0004;PBS_Server;Svr;is_request;message received from stream 192.168.10.18:15003
06/09/2010 09:45:32;0004;PBS_Server;Svr;is_request;message STATUS (4) received from mom on host L18 (192.168.10.18:15003) (stream 1)
06/09/2010 09:45:32;0004;PBS_Server;Svr;is_request;IS_STATUS received from L18
06/09/2010 09:45:32;0040;PBS_Server;Req;is_stat_get;received status from node L18
06/09/2010 09:45:32;0040;PBS_Server;Req;update_node_state;adjusting state for node L18 - state=0, newstate=0
06/09/2010 09:45:32;0040;PBS_Server;Req;do_rpp;rpp request received on stream 2
06/09/2010 09:45:32;0040;PBS_Server;Req;do_rpp;inter-server request received
06/09/2010 09:45:32;0004;PBS_Server;Svr;is_request;message received from stream 2 (version 1)
06/09/2010 09:45:32;0004;PBS_Server;Svr;is_request;message received from stream 192.168.10.11:15003
06/09/2010 09:45:32;0004;PBS_Server;Svr;is_request;message STATUS (4) received from mom on host L11 (192.168.10.11:15003) (stream 2)
06/09/2010 09:45:32;0004;PBS_Server;Svr;is_request;IS_STATUS received from L11
06/09/2010 09:45:32;0040;PBS_Server;Req;is_stat_get;received status from node L11
06/09/2010 09:45:32;0040;PBS_Server;Req;update_node_state;adjusting state for node L11 - state=0, newstate=0
06/09/2010 09:45:39;0080;PBS_Server;Req;dis_request_read;decoding command AuthenticateUser from root
06/09/2010 09:45:39;0100;PBS_Server;Req;;Type AuthenticateUser request received from root@kmn, sock=11
06/09/2010 09:45:39;0008;PBS_Server;Job;dispatch_request;dispatching request AuthenticateUser on sd=11
06/09/2010 09:45:39;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from PBS_Server
06/09/2010 09:45:39;0080;PBS_Server;Req;dis_request_read;decoding command StatusJob from root
06/09/2010 09:45:39;0100;PBS_Server;Req;;Type StatusJob request received from root@kmn, sock=10
06/09/2010 09:45:39;0008;PBS_Server;Job;dispatch_request;dispatching request StatusJob on sd=10
06/09/2010 09:45:39;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from root
06/09/2010 09:45:42;0080;PBS_Server;Req;dis_request_read;decoding command AuthenticateUser from root
06/09/2010 09:45:42;0100;PBS_Server;Req;;Type AuthenticateUser request received from root@kmn, sock=11
06/09/2010 09:45:42;0008;PBS_Server;Job;dispatch_request;dispatching request AuthenticateUser on sd=11
06/09/2010 09:45:42;0080;PBS_Server;Req;dis_request_read;decoding command Disconnect from PBS_Server
06/09/2010 09:45:42;0080;PBS_Server;Req;dis_request_read;decoding command Shutdown from root
06/09/2010 09:45:42;0100;PBS_Server;Req;;Type Shutdown request received from root@kmn, sock=10
06/09/2010 09:45:42;0008;PBS_Server;Job;dispatch_request;dispatching request Shutdown on sd=10
06/09/2010 09:45:42;0086;PBS_Server;Svr;PBS_Server;Shutdown request from root@kmn 
06/09/2010 09:45:42;0086;PBS_Server;Svr;PBS_Server;Starting to shutdown the server, type is Quick
06/09/2010 09:45:46;0002;PBS_Server;Svr;PBS_Server;Server shutdown completed
06/09/2010 09:45:46;0002;PBS_Server;Svr;Log;Log closed