================================ Signature-search thread controls ================================ A path under ``benchmarks/`` on this page names a record of the d-SEAMS 2.0 benchmark campaign. The records and the Snakemake rules that write them are in the private paper repository and the Hugging Face dataset ``d-SEAMS/dseams2-benchmark-records`` (``docs/records/README.md`` names the revision); the scripts under ``scripts/`` that time a run stay here. Commit ``9896a478`` on ``perf/trace-threads`` retimes the hydrate signature search at 1, 2, 4 and 8 threads, after the pool and the row placement changed. The census at every width is the one-thread census. The rows are ``benchmarks/trace-threads.csv`` in the benchmark records dataset; the tables below are rendered from it, with each median and type-7 interquartile range recomputed from ``samples_ms``. The process ----------- ``scripts/paper_stats.py`` builds the environment for one width :math:`n`: .. table:: +------------------------------------------------------------------------+-----------+ | variable | value | +========================================================================+===========+ | ``OMP_NUM_THREADS`` | :math:`n` | +------------------------------------------------------------------------+-----------+ | ``OMP_DYNAMIC`` | ``FALSE`` | +------------------------------------------------------------------------+-----------+ | ``OMP_PROC_BIND`` | ``close`` | +------------------------------------------------------------------------+-----------+ | ``OMP_PLACES`` | ``cores`` | +------------------------------------------------------------------------+-----------+ | ``RAYON_NUM_THREADS`` | :math:`n` | +------------------------------------------------------------------------+-----------+ | ``OPENBLAS_NUM_THREADS``, ``MKL_NUM_THREADS``, ``NUMEXPR_NUM_THREADS`` | ``1`` | +------------------------------------------------------------------------+-----------+ The process is pinned with ``taskset -c 0`` when :math:`n = 1`, and with ``taskset -c 0-(n-1)`` otherwise. One warm-up is dropped. Eight repeats remain. The median of an even count is the mean of the two central samples. The interquartile range is the type-7 upper quartile minus the lower one. ``cores`` is the size of the affinity set. ``affinity`` is ``Cpus_allowed_list``. The ``commit`` column is the source commit the sitting recorded. One pool, then the search ------------------------- Each sample is one linkcell neighbour list, one depth-6 primitive-ring network, and one combined search of :math:`5^{12}`, :math:`5^{12}6^{2}` and :math:`5^{12}6^{4}`. Linkcell keeps one Rayon pool for the process. A later search with the same width reuses it. A different width drops that pool before the new one starts, so two widths never run together. Between searches the workers sleep on Rayon's idle condvar. That is the passive wait for the neighbour list. ``lc_set_threads`` chooses the width. Zero or one runs on the calling thread. OpenMP is left on the active wait. ``seams::prepareCompute``, which the linkcell list calls, sets ``omp_set_dynamic(0)`` and does not set ``OMP_WAIT_POLICY``. Forcing OpenMP passive as well made the next region wake on a spread of tens of milliseconds, which was the slow eight-thread sample. Below 4096 molecules the linkcell team is capped at four. A caller that asked for eight threads still gets four workers on those cells, because the list and the cage walk do not fill a wider team and the extra workers show up as a slow sample. Crystals at or above 4096 molecules keep the width the caller asked for. The ``threads`` column is the width that was requested. On sI :math:`2\times2\times2` (368 molecules) and :math:`4\times4\times4` (2944) the eight-thread median therefore sits next to the four-thread median. Neighbour rows are placed on the calling thread until the pair list reaches :math:`2^{20}` pairs. The parallel sort forks a team that has just been idle through the linkcell search. Waking it costs more than writing the rows on the caller until the list is that large. ``kParallelRowsMinAtoms`` (4096) still has to hold as well. The cage index is merged from per-thread buckets in ring order. ``tests/test_trace_threads.cpp`` requires the closed-cage faces, and the hexagonal and double-diamond cages, to match the one-thread result. Membership is the sorted atom ids of each face, not the ring index. ``seams_set_num_threads`` is the installed way to set the OpenMP width the next search reads. Every kernel ------------ Commit ``848426f9`` sizes the team on every kernel. ``caeb71f8`` puts the one-thread carried update back on a direct copy and sort, binds a narrowed team to that many cores, and skips the periodic shift inside an interior cutoff cell. ``1ce3be3b`` retimes the same eight kernels after that change. The rows are ``benchmarks/pool-audit.csv``. The ``commit`` column is ``caeb71f8``. Eight kernels run on ideal ice Ic: primitive rings, local and cutoff :math:`q_4` and :math:`q_6`, CHILL+, the shell label, and one carried hop-ball update. The sizes are 4096, 32768, 110592 and 884736 molecules. Each size is 32 rows, the eight kernels at 1, 2, 4 and 8 threads. One warm-up, then eight timed calls. The fingerprint of a kernel is the same at every width. ``meson test docs_pool_audit`` recomputes each median and type-7 interquartile range from ``samples_ms``. The rings kernel is ``primitive::ringNetwork`` on a neighbour graph built before the timer starts. It is not the full cutoff call. The rings row of ``benchmarks/paper-cells/sitting-large.csv`` is ``seams_frame_rings``, which builds the cutoff list and then searches the rings. Those two rows are not the same measurement. The full cutoff ring call ------------------------- Several threads inside ``neighListLinkcell`` build an orthorhombic or triclinic cutoff list with the cell-list CSR. One thread still uses linkcell's buffered walk. The rows are the same sorted cutoff set, so the ring search matches across thread counts. ``cad887b2`` times that full call, the neighbour list plus the index conversion plus the depth-6 search, on ice Ic at 884736 molecules. It is interleaved with the ``1ce3be3b`` builder. Each sample is the last of three repeats in a fresh process. The table is ``benchmarks/cutoff-rings.csv``. ``link`` is the neighbour-list build. This sitting is not the ``ringNetwork`` kernel above, and it is not the rings row of the 884736-molecule sitting. .. include:: cutoff-rings-tables.rst ``seams::teamFor`` is the width of a per-atom walk. Below 2048 atoms it is one thread. Below 8192 atoms, when the caller asked for more than four, it is four. At or above 8192 atoms it is the OpenMP width. ``seams::TeamWidth`` sets that width for the rest of the call and restores the caller's width afterwards. When the width is narrower than the process, it binds the workers once to the first cores of the allowed set, one core for each worker, and sets ``OMP_PROC_BIND`` to ``false`` so OpenMP does not put them back on the whole mask. The bind is once for that width and is not undone when the call returns. Four workers asked for on an eight-core mask therefore stay on four cores. Bond order, published CHILL+, the k-nearest list, the shell label, and local :math:`q_l` use ``TeamWidth``. Those calls do not start a Rayon pool. ``seams::adoptThreads`` no longer pauses the OpenMP team and no longer sets ``OMP_WAIT_POLICY``. It calls ``prepareCompute``, then ``lc_set_threads``. The linkcell pool is still one per process, and its workers still sleep between searches. The OpenMP team is left running, so the next region does not pay the wake-up that used to move by tens of milliseconds. Enumerating the changed ring sources uses ``sourceTeam``, not ``teamFor``. That search stays on the calling thread until the sources fill the workers, about 80 sources per thread. Fewer than that is a serial enumeration, because the fork costs more than the search. ``teamFor`` is a different rule: it would also put a crystal of a few hundred molecules on one thread. On one thread the carried update copies the new rings and sorts them directly, then merges the rings that stay with a direct walk. The parallel index sort and the parallel merge are the several-thread path. They were slower than the direct copy when only one thread ran. Several threads still sort the indices, scan for rings that meet :math:`T`, and merge the rings that are kept, using ``teamFor``. Dropped ring buffers are kept and reused. They are not freed on the update. The orthorhombic cutoff list is one shared CSR. Each pair is counted once, from the atom with the lower slot, on the z layer that owns that atom. A layer writes its own atoms and the atoms in the next layer, so layers of one parity do not share an atom. Even layers run together. When the number of layers is odd, the last layer wraps onto layer 0 and runs only after the even layers have stopped. Odd layers run after that. An interior cell has no periodic image. Its thirteen forward neighbours are plain index steps and the shift is zero, so that cell skips the periodic-shift arithmetic. A cell on the boundary still applies the shift. One thread uses the same half stencil without a team. Published CHILL+ and cutoff :math:`q_6` both read this list. The one-thread and several-thread pair lists match. The linkcell cap below 4096 molecules, and the row placement below :math:`2^{20}` pairs, are the signature-search rules above. They are not replaced by ``teamFor``. :doc:`/stable/c` names ``seams_set_num_threads`` and ``seams_get_num_threads``. .. include:: trace-threads-tables.rst .. include:: pool-audit-tables.rst