README

Arrhenius Storage Benchmark
===========================

elbencho 3.0.19 <https://github.com/breuner/elbencho/releases/tag/v3.0-19>
will be used for the benchmark.

The elbencho service will be started on all client nodes participating
in the test by running

    elbencho --service --foreground

on each client node.


Sequential read and write
-------------------------

After the elbencho service has been started on all client nodes
participating in the benchmark, the sequential read and write part of
the benchmark will be started by running

    elbencho --hosts <hosts> --rotatehosts=1 -t <threads per node> \
        -w -r -b <blocksize> -s <bytes per thread> -n 0 -F <directory>

where

 - <hosts> is a comma-separated list of the participating client nodes'
   hostnames,

 - <threads per node> is the number of threads (tasks) per client node
   performing I/O,

 - <blocksize> is the number of bytes written or read in a single
   operation,

 - <bytes per thread> is the number of bytes written and read per
   thread (task),

 - <directory> is the path to an empty directory on the filesystem being
   benchmarked.

<threads per node> and <blocksize> will be chosen optimally for best
performance.  <bytes per thread> will be chosen so that the write and
read phases run for at least 60 minutes each or until 95% of the
filesystem capacity is reached by the write phase.

Results will be evaluated by looking at the "WRITE" and "READ" sections
in the table output from elbencho on stdout.  The row "Elapsed time" in
column "LAST DONE" in each section will be used to evaluate how long a
phase ran for.  The row "Throughput MiB/s" in column "LAST DONE" in each
section will be used to evaluate the sequential write and read
performance.


IOPS
----

After the elbencho service has been started on all client nodes
participating in the benchmark, the IOPS part of the benchmark will be
started by running

    elbencho --hosts <hosts> --rotatehosts=1 -t <threads per node> \
	-w -r -b 4K -s <bytes per thread> -n 0 -F --rand <directory>

where

 - <hosts> is a comma-separated list of the participating client nodes'
   hostnames,

 - <threads per node> is the number of threads (tasks) per client node
   performing I/O,

 - <bytes per thread> is the number of bytes written and read per
   thread (task),

 - <directory> is the path to an empty directory on the filesystem being
   benchmarked.

<threads per node> will be chosen optimally for best performance.
<bytes per thread> will be chosen so that the write and read phases runs
for at least 60 minutes each or until 95% of the filesystem capacity is
reached by the write phase.

Results will be evaluated by looking at the "WRITE" and "READ" sections
in the table output from elbencho on stdout.  The row "Elapsed time" in
column "LAST DONE" in each section will be used to evaluate how long a
phase ran for.  The row "IOPS" in column "LAST DONE" in each section
will be used to evaluate the write and read IOPS.


Metadata
--------

After the elbencho service has been started on all client nodes
participating in the benchmark, the metadata part of the benchmark will
be started by running

    elbencho --hosts <hosts> --rotatehosts=1 -t <threads per node> \
	-d -w --stat -F -N <files per thread> -D <directory>

where

 - <hosts> is a comma-separated list of the participating client nodes'
   hostnames,

 - <threads per node> is the number of threads (tasks) per client node
   performing I/O,

 - <files per thread> is the number of files created, stat, and deleted
   per thread (task),

 - <directory> is the path to an empty directory on the filesystem being
   benchmarked.

<threads per node> will be chosen optimally for best performance.
<number of files> will be chosen so that each phase; create, stat, and
delete; runs for at least 60 minutes or until 95% of the filesystem's
maximum number of files is reached by the create phase.

Results will be evaluated by looking at the "WRITE", "STAT", and
"RMFILES" sections in the table output from elbencho on stdout.  The
"WRITE" section shows the "create" performance.  The "STAT" section
shows the "stat" performance".  The "RMFILES" shows the "delete"
performance.

The row "Elapsed time" in column "LAST DONE" in each section will be
used to evaluate how long a phase ran for.  The row "Files/s" in column
"LAST DONE" in each section will be used to evaluate the number of
creates, stats, and deletes per second.


sbatch script
=============

For convenience, a Slurm sbatch script is included in this repository.
This section describes how to run the different parts of the benchmark
using the sbatch script.

Sequential read and write:

    sbatch -N <number of nodes> --cpus-per-task=<threads per node> \
        storage-benchmark.sbatch \
            sequential <blocksize> <bytes per thread> <directory>

IOPS:

    sbatch -N <number of nodes> --cpus-per-task=<threads per node> \
        storage-benchmark.sbatch \
            iops <bytes per thread> <directory>

Metadata:

    sbatch -N <number of nodes> --cpus-per-task=<threads per node> \
        storage-benchmark.sbatch \
            meta <files per thread> <directory>