Communication-aware Job Scheduling using SLURM

I worked with Prof. Preeti Malakar on improving the performance of communication-intensive jobs on high performance clusters or supercomputers. Jobs submitted to clusters or supercomputers vary in resource requirements and execution times. Increasingly, they also vary in other characteristics such as communication and I/O behavior. Most of the current job-schedulers do not consider the communication characteristics of a job which leads to sub-optimal allocations. We implemented three new node-allocation algorithms, greedy, balanced and adaptive which consider the communication characteristics of the submitted jobs and proposed a novel way of optimizing node-allocation based on underlying algorithms of MPI collectives.

Additional details can be found in the paper: Communication-aware Job Scheduling using SLURM