Reproduce the literature head-to-head¶
A path under benchmarks/ on this page names a record of the d-SEAMS 2.0
benchmark campaign. The records and the Snakemake rules that write them
are in the private paper repository and the Hugging Face dataset
d-SEAMS/dseams2-benchmark-records (docs/records/README.md names the
revision); the scripts under scripts/ that time a run stay here.
scripts/h2h_rc3_sitting.py writes benchmarks/rc3-sitting.csv.
One warm-up, then six timed calls. The median and the type-7
interquartile range are the columns on
the sitting page.
This page is how each guest code is given a thread width, and what
that width did to the median.
The driver starts every row under taskset. One thread is
taskset -c 0. Eight threads are taskset -c 0-7. It also sets
OMP_NUM_THREADS to that width and OMP_DYNAMIC=FALSE. Each code
then receives the same width through its own control.
code |
what is timed |
thread control |
|---|---|---|
freud |
local q4 and q6 |
|
OVITO |
CHILL+, primitive rings, polyhedral template matching |
|
mdapy |
CHILL+, q4, q6, rings |
|
vesin |
full cell-list neighbour pairs |
|
PLUMED |
symfunc q4 and q6 |
|
rings-statistics-matter |
primitive rings, Rust backend |
|
Polyhedral template matching is the OVITO modifier. Its row is
PTM. rings-statistics-matter is the rsm package.
python3 scripts/h2h_rc3_sitting.py
What the medians do¶
On ideal ice Ic at 110592 and 884736 molecules, freud, OVITO, mdapy,
vesin, and rings-statistics-matter are faster at eight threads than at
one. OVITO’s template matching and its ring finder fall with the same
control as its CHILL+. meson test docs_h2h_threads checks those
inequalities against the CSV.
The PLUMED rows in this sitting were measured on one thread because
PLUMED_NUM_THREADS was unset. The driver set OMP_NUM_THREADS and
left PLUMED_NUM_THREADS alone. The installed header
plumed/tools/OpenMP.h exposes PLMD::OpenMP::getNumThreads, which
returns one thread when that variable is unset. That is why the
threads=1 column and the threads=8 column of these rows are the
same one-thread measurement.
benchmarks/paper-cells/sitting-large.csv sets
PLUMED_NUM_THREADS to the width of the row, and that q6 time falls
from one thread to eight. These rc3 rows are not that measurement.
#include "plumed/tools/OpenMP.h"
#include <cstdio>
int main() {
unsigned threads = PLMD::OpenMP::getNumThreads();
std::printf("plumed threads %u\n", threads);
return threads == 0 ? 1 : 0;
}
Rendered from |
One thread against eight¶
code |
operation |
molecules |
t1_ms |
t8_ms |
ratio |
note |
|---|---|---|---|---|---|---|
freud |
q4 |
110592 |
276.580 |
51.302 |
5.39 |
|
freud |
q4 |
884736 |
2358.091 |
431.376 |
5.47 |
|
freud |
q6 |
110592 |
362.326 |
68.760 |
5.27 |
|
freud |
q6 |
884736 |
2973.372 |
550.161 |
5.40 |
|
OVITO |
CHILL+ |
110592 |
622.424 |
88.685 |
7.02 |
|
OVITO |
CHILL+ |
884736 |
4890.789 |
666.288 |
7.34 |
|
OVITO |
PTM |
110592 |
2790.624 |
370.341 |
7.54 |
|
OVITO |
PTM |
884736 |
21304.364 |
2849.382 |
7.48 |
|
OVITO |
rings |
110592 |
583.334 |
354.605 |
1.65 |
|
OVITO |
rings |
884736 |
4883.899 |
2582.748 |
1.89 |
|
mdapy |
CHILL+ |
110592 |
61.938 |
12.379 |
5.00 |
|
mdapy |
CHILL+ |
884736 |
744.487 |
125.578 |
5.93 |
|
mdapy |
q4 |
110592 |
64.687 |
12.400 |
5.22 |
|
mdapy |
q4 |
884736 |
807.101 |
96.238 |
8.39 |
|
mdapy |
q6 |
110592 |
79.232 |
14.572 |
5.44 |
|
mdapy |
q6 |
884736 |
921.574 |
107.347 |
8.59 |
|
mdapy |
rings |
110592 |
2276.747 |
375.967 |
6.06 |
|
mdapy |
rings |
884736 |
19221.718 |
3361.913 |
5.72 |
|
vesin |
neighbours |
110592 |
51.194 |
22.269 |
2.30 |
|
vesin |
neighbours |
884736 |
1112.702 |
196.866 |
5.65 |
|
rings-statistics-matter |
rings |
110592 |
1151.862 |
264.953 |
4.35 |
|
rings-statistics-matter |
rings |
884736 |
11504.451 |
2665.750 |
4.32 |
|
PLUMED |
q4 |
110592 |
1310.025 |
1316.423 |
1.00 |
one thread; PLUMED_NUM_THREADS unset |
PLUMED |
q4 |
884736 |
10053.680 |
10122.158 |
0.99 |
one thread; PLUMED_NUM_THREADS unset |
PLUMED |
q6 |
110592 |
1685.454 |
1680.281 |
1.00 |
one thread; PLUMED_NUM_THREADS unset |
PLUMED |
q6 |
884736 |
12947.767 |
12818.837 |
1.01 |
one thread; PLUMED_NUM_THREADS unset |