
Using Full Disclosure:

http://wiki.mcs.anl.gov/FullDisclosure/index.php/Main_Page

logon: sfsiegel@fd-login.mcs.anl.gov

running batch jobs: create shell script like runme with the srun command

sfsiegel@scx-m23n6 ~/372/mpi/hello $ cat runme
#!/bin/bash
srun -p scx-comp -N 6 -n 36 /home/sfsiegel/372/mpi/hello/hello

sfsiegel@scx-m23n6 sbatch -p scx-comp -N 6 -n 36 /home/sfsiegel/372/mpi/hello/runme
sbatch: Submitted batch job 2981

sfsiegel@scx-m23n6 ~/372/mpi/hello $ squeue
  JOBID PARTITION     NAME     USER  ST       TIME  NODES NODELIST(REASON)
   2981  scx-comp    runme sfsiegel  PD       0:00      6 (Resources)
   2982  scx-comp tst_old-  spieper  PD       0:00     20 (Priority)

sfsiegel@scx-m23n6 ~/372/mpi/hello $ more slurm-2981.out 
Hello from process 0 running on scx-m32n10.scsystem on host scx-m32n10.scsystem.
Hello from process 6 running on scx-m32n11.scsystem on host scx-m32n11.scsystem.
Hello from process 18 running on scx-m32n13.scsystem on host scx-m32n13.scsystem.
Hello from process 30 running on scx-m32n15.scsystem on host scx-m32n15.scsystem.
.
.
.


I created script in bin directory called fdrun.pl for batch running jobs
on FD.  Makes life a little easier.  

Note: before running jobs set the following environment variable:
export SCMPI_BDT_ENTRIES=32768

Else too many request will block things.