X11 Logo

I have an example program that came with BCCD called Pandemic which I wanted to run on our local HPCC. Unfortunately Pandemic requires X11 to run and I would get a segmentation fault every time I ran it in the batch system.

This blog post shows how I used the X11 Virtual Frame Buffer (Xvfb) to enable X11 in batch mode. This example uses SLURM running in CentOS7. This trick can come in really handing when you are using MATLAB to because last I checked it needed X11 to run in order to generate and save figures. This will let you do that even in batch mode.

First, here is the batch script for the OpenMP version (Should work for serial jobs as well).

#!/bin/bash
#SBATCH --time=00:10:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=10G
#SBATCH --job-name Pandemic-OpenMP

cd ${SLURM_SUBMIT_DIR}

# Have X11 find an open display number and communicate the number
# though a temporary file to set the environment variable.
display_file=.tmp_display.txt
Xvfb -displayfd 1 -auth /dev/null  1>$display_file 2> /dev/null &
sleep 5
export DISPLAY=:`cat $display_file`
echo "DISPLAY set to $DISPLAY"
rm $display_file

# Benchmark program with different numbers of processes
export OMP_NUM_THREADS=16
time ./Pandemic.c-openmp
export OMP_NUM_THREADS=8
time ./Pandemic.c-openmp
export OMP_NUM_THREADS=4
time ./Pandemic.c-openmp
export OMP_NUM_THREADS=2
time ./Pandemic.c-openmp
export OMP_NUM_THREADS=1
time ./Pandemic.c-openmp
echo done

# Report job statistics
scontrol show job $SLURM_JOB_ID

And here is the job I got working with MPI. Notice it is basically the same but I need to pass the DISPLAY variable though MPI for it to work.

#!/bin/bash
#SBATCH --time=00:10:00
#SBATCH --ntasks=16
#SBATCH --cpus-per-task=1
#SBATCH --mem=10G
#SBATCH --job-name Pandemic-MPI

cd ${SLURM_SUBMIT_DIR}
module swap GNU Intel
module load OpenMPI

# Have X11 find an open display number and communicate the number
# though a temporary file to set the environment variable.
display_file=.tmp_display.txt
Xvfb -displayfd 1 -auth /dev/null  1>$display_file 2> /dev/null &
sleep 5
export DISPLAY=:`cat $display_file`
echo "DISPLAY set to $DISPLAY"
rm $display_file

# Benchmark program with different numbers of processes
time mpirun -n 16 -x DISPLAY=$HOSTNAME:$DISPLAY ./Pandemic.c-mpi
time mpirun -n 8 -x DISPLAY=$HOSTNAME$DISPLAY ./Pandemic.c-mpi
time mpirun -n 4 -x DISPLAY=$HOSTNAME$DISPLAY ./Pandemic.c-mpi
time mpirun -n 2 -x DISPLAY=$HOSTNAME$DISPLAY ./Pandemic.c-mpi
time mpirun -n 1 -x DISPLAY=$HOSTNAME$DISPLAY ./Pandemic.c-mpi

# Report job statistics
scontrol show job $SLURM_JOB_ID

Hope you find this useful.


Emerging Technologies (FPGAs) @ 2018 ACI-REF VR

Fri 10 August 2018 by Dr. Dirk Colbry

Picture from Panel

Today is the last day of the 2018 ACI-REF Virtual residency workshop here in Norman Ok. It was a great week with a lot of great people. I highly recommend the workshop for anyone in the area of Advanced Computing Instruction especially those do some Research and Education Facilitation in …

read more

Deciding Which Technologies to Adopt, and When @ 2018 ACI-REF VR

Wed 08 August 2018 by Dr. Dirk Colbry

Image used in presentation representing the technology adoption wave

I had fun leading a discussion on Deciding Which Technologies to Adopt, and When at the 2018 ACI-REF Virtual residency event in Norman Oklahoma. Since many of the participants were virtual I experimented with doing a Brainstorming exercise over zoom. We had local volunteers on typing in comments from our …

read more

Teams of CI Professionals: Recruitment & Retention, Management, Team-building, and Motivation Panel @ 2018 ACI-REF VR

Tue 07 August 2018 by Dr. Dirk Colbry

Image from Panel

Thank you Aaron Bergstrom (U North Dakota) for facilitating the panel discussion on Teams of CI Professionals: Recruitment & Retention, Management, Team-building, and Motivation at the 2018 ACI-REF Virtual residency. My fellow panel members included:

  • Jerry Perez, U Texas Dallas
  • Derek Leydig, Pennsylvania State U
  • Claire Mizumoto, U California San Diego …
read more

Installing RStudio on the MSU HPCC

Fri 06 July 2018 by Dr. Dirk Colbry

Rstudio

These instructions are for installing RStudio on the HPCC. RStudio provides a variety of installers on their website https://www.rstudio.com/products/rstudio/download2/. However, the HPCC uses an older version of Linux and the precompiled binaries are not compatible. Trying to install RStudio from the source code is …

read more

Managing Inconsistent Runtimes on the HPCC

Fri 29 June 2018 by Dr. Dirk Colbry

One of our postdocs stopped by my office today to talk about a problem with inconsistant runtimes on our HPCC. This user is running a lot of jobs with 512 cores (current cpu limit is 520). The problem is that it is really difficult for the user to estimate the …

read more

Inverse Problems Seminar

Mon 04 June 2018 by Dr. Dirk Colbry

I was invited to give a talk at the inverse seminars symposium:

https://inverseproblems2018.org/

At first I was a little nervous since I don't consider myself an expert in inverse problems so I did not fully understand what they wanted me to present. However after talking to the organizers …

read more