[torquedev] processor affinity

[Toni L. Harbaugh-Blackford [Contr]]

On Sat, 2 Jun 2007, Prakash Velayutham wrote:

  > I think linux currently supports cpusets. At least, I can see the option
  > in the Kernel source.

Yes, recent 2.6 kernels do support cpusets, and you can create cpusets *by hand*
as long as you mount /dev/cpuset.

HOWEVER, Torque uses SGI's libcpuset library functions to manage cpusets.  So
on non-SGI linux systems, to use cpusets you would have to create a library to
do all the things that SGI's library does, or else re-write the cpuset code
in Torque.

Toni

  >
  > Prakash
  >
  > >>> "Toni L. Harbaugh-Blackford [Contr]" <harbaugh at ncifcrf.gov> 06/02/07
  > 3:41 PM >>>
  >
  >
  > Torque does not assign specific cpus.
  >
  > On some architectures (Altix, maybe IRIX still) cpusets are implemented,
  > and these do restrict jobs to a given set of cpus.  You many want to
  > take
  > a look at the cpuset code and see if it could be generalized to other
  > linux-based systems.
  >
  > On Sat, 2 Jun 2007, Menshutin Anton wrote:
  >
  >   > Hi list,
  >   >
  >   > I found that there is no processor affinity in torque. Jobs assigned
  > to run
  >   > on some cpu's selected by scheduler, could also run on other cpus on
  > this
  >   > node. This means that if user lunches more processes or threads on a
  > node
  >   > than he has requested, his processes will span over other cpus
  > according to
  >   > the operating system scheduling policy. Thus he can get more cpu
  > resources
  >   > than he should. Its clear that this is undesired behavior.
  >   > Nevertheless, starting from linux kernel 2.6 there is a system call
  >   > sched_setaffinity() which enables programmer to tell OS scheduler
  > not to use
  >   > some cpus. So this is thing one have to add to torque.
  >   >
  >   > This property is inherited by child process. It is obvious that
  > setting it
  >   > after fork() and before exec() will be enough. The only thing I
  > don't know -
  >   > where can I get info about cpus assigned to me by scheduler.
  >   > Qstat shows this info in exec_host attribute, and I suppose I can
  > get this
  >   > string, parse it, find out localhostname and get CPUs numbers. But
  > may be
  >   > there is a better way for getting this info?
  >   >
  >   > I'm asking for help from torque-dev mailing list :) Given an advice,
  > I could
  >   > try to implement and test it myself or may be anybody could send me
  > a patch?
  >   >
  >   >
  >   >
  >   >
  >   >
  >   >
  >   >
  >   >
  >   > _______________________________________________
  >   > torquedev mailing list
  >   > torquedev at supercluster.org
  >   > http://www.supercluster.org/mailman/listinfo/torquedev
  >   >
  >
  > -------------------------------------------------------------------
  > Toni Harbaugh-Blackford                       harbaugh at ncifcrf.gov
  > System Administrator
  > Advanced Biomedical Computing Center (ABCC)
  > National Cancer Institute
  > Contractor - SAIC/Frederick
  > _______________________________________________
  > torquedev mailing list
  > torquedev at supercluster.org
  > http://www.supercluster.org/mailman/listinfo/torquedev
  >

-------------------------------------------------------------------
Toni Harbaugh-Blackford                       harbaugh at ncifcrf.gov
System Administrator
Advanced Biomedical Computing Center (ABCC)
National Cancer Institute
Contractor - SAIC/Frederick