SIMMSUS — Simulation of Magnetic Suspensions — is a research Fortran code for numerical simulations of suspensions of interacting spherical particles. The code was originally developed to study magnetic suspensions and ferrofluids, including Brownian motion, hydrodynamic interactions, long-range dipolar interactions, periodic Ewald-like summations, sedimentation, magnetic-field-driven dynamics, and microstructural evolution.
The current version of the code includes a major HPC-oriented refactoring:
- source files are now organized under
src/; - long-range periodic interactions were moved to a dedicated module,
periodic_interactions.f90; - CPU parallelism through OpenMP is supported;
- GPU acceleration through OpenACC is supported;
- validation cases are organized under
validation/; - reproducible HPC benchmarks are organized under
benchmarks_hpc/.
The validated OpenACC implementation preserves the physical behavior of the OpenMP implementation for hydrodynamic and dipolar periodic interactions while exposing additional GPU parallelism in the real-space and reciprocal-space periodic summations.
- SIMMSUS
- Overview
- Repository Structure
- Compilation
- Running a Simulation
- Source-Code Organization
- Periodic Interactions
- Validation
- HPC Benchmarks
- Physics and Numerics
- Gallery
- References
SIMMSUS simulates systems of interacting particles in suspension. The original motivation was the study of magnetic spherical particles in ferrofluids, but the code has evolved to support several physical mechanisms and numerical scenarios.
The code can simulate Brownian and non-Brownian suspensions, magnetic and non-magnetic particles, sedimentation, hydrodynamic interactions, dipole-dipole magnetic interactions, periodic long-range interactions, steady and oscillatory magnetic fields, rotating magnetic fields, shear flows, ordered and random initial conditions, and mono- or polydisperse particle distributions.
The project has been used in studies published in journals such as Physics of Fluids, Journal of Magnetism and Magnetic Materials, Powder Technology, and Mechanics Research Communications.
The repository is organized as follows:
.
├── src/
│ ├── simmsus.f90
│ ├── input.f90
│ ├── main.f90
│ ├── subroutines.f90
│ ├── periodic_interactions.f90
│ ├── statistics.f90
│ └── variables.f90
│
├── validation/
│ ├── dipolar_interactions/
│ │ └── README.md
│ └── hydrodynamic_interactions/
│ └── README.md
│
├── benchmarks_hpc/
│ ├── openmp/
│ │ ├── README.md
│ │ ├── simconfig.dat
│ │ └── scripts / results
│ └── openacc/
│ ├── README.md
│ ├── simconfig.dat
│ ├── run_openacc_benchmark.sh
│ └── Allclean.sh
│
├── gallery/
├── makefile
└── README.md
The source code is now concentrated in src/, while validation and performance tests are kept separate from the main implementation. This organization is intended to make the repository easier to maintain, validate, benchmark, and extend.
The build system is based on make. The current makefile supports different compilation targets for CPU and GPU execution.
Depending on the available compiler, SIMMSUS can be compiled with a standard Fortran compiler such as gfortran or Intel Fortran.
Typical examples:
make gfortranor:
make ifxTo compile the OpenMP version:
make openmpThe OpenMP implementation is intended for shared-memory CPU execution. It preserves the historical parallelization strategy of SIMMSUS, especially for the outer loops over realizations and particles.
A typical OpenMP execution uses:
export OMP_NUM_THREADS=32
./simmsus.exThe OpenMP benchmark suite in benchmarks_hpc/openmp/ provides scripts to run the code with different thread counts and compare performance.
To compile the OpenACC GPU version:
make openaccThe OpenACC version is intended for NVIDIA GPUs and has been tested with:
- NVIDIA HPC SDK 26.3
- CUDA 13.2
- NVIDIA GeForce RTX 5070
The OpenACC target uses NVIDIA Fortran (nvfortran) and flags consistent with GPU offloading, managed memory, and relocatable device code. A typical flag set is:
-acc -gpu=ccnative,mem:managed,rdc -Minfo=accel,ccff -MfreeThe OpenACC benchmark suite in benchmarks_hpc/openacc/ provides a reference case and an execution script.
To run a simulation, the user needs:
simmsus.ex
simconfig.dat
A typical execution is:
./simmsus.exThe physical and numerical configuration is controlled through simconfig.dat.
This file defines, among several other options:
- number of particles;
- number of realizations;
- volume fraction;
- Brownian effects;
- hydrodynamic interactions;
- periodic magnetic torques;
- periodic magnetic forces;
- external magnetic field;
- shear flow;
- statistical post-processing options;
- execution paradigm, such as serial, OpenMP, or OpenACC.
Main program. It controls the high-level execution flow:
- reads the input file;
- calls the main simulation routine;
- optionally calls statistical analysis routines;
- prints basic execution information.
Reads simconfig.dat and stores the selected physical and numerical parameters in global variables defined in variables.f90.
Controls the main simulation workflow. It allocates arrays, initializes the system, computes initial conditions, evaluates interactions, advances the simulation in time, writes output files, and deallocates memory.
The main loop includes:
- force calculation;
- velocity solution;
- position update;
- torque calculation;
- angular velocity solution;
- dipole orientation update.
Contains the general-purpose physical and numerical routines used by SIMMSUS. Historically, this module contained most of the computational logic. Long-range periodic interactions have now been moved to the dedicated periodic_interactions.f90 module.
Dedicated module for long-range periodic interactions.
This module computes real-space and reciprocal-space contributions associated with:
- hydrodynamic interactions;
- periodic magnetic torques;
- periodic magnetic forces;
- Ewald-type periodic sums.
It contains separate execution paths for serial, OpenMP, and OpenACC execution.
Performs statistical post-processing when enabled in simconfig.dat. This includes quantities such as average velocities, velocity fluctuations, Reynolds-stress-like quantities, time correlations, and correlation times.
Defines the global variables used throughout SIMMSUS. It also contains OpenACC declare create directives for variables that must be available on the device during GPU execution.
The most computationally expensive part of SIMMSUS is the calculation of long-range periodic interactions. These interactions are computed through real-space and reciprocal-space summations.
The current version of SIMMSUS introduces a dedicated module:
src/periodic_interactions.f90
This separation improves maintainability and allows different parallelization strategies for different hardware backends.
The OpenMP implementation preserves the efficient CPU-oriented strategy based on the outer loops:
(q, i)
where:
qis the realization index;iis the particle index.
This strategy works well on multicore CPUs because each thread receives a relatively large amount of work.
The initial OpenACC implementation also parallelized the outer (q,i) loops. Although physically correct, this approach left the inner periodic sums over (s,j) mostly sequential inside each GPU thread, limiting performance.
The current OpenACC refactoring introduces GPU-specific kernels that expose more parallelism by distributing work over the inner periodic interaction loops:
(s, j)
where:
sis the periodic image / reciprocal vector index;jis the interacting particle index.
The OpenACC implementation uses reductions to accumulate local hydrodynamic, force, and torque contributions safely before writing final results to:
U(q,i,:)
FORCAS(...)
TORQUES(...)
This design keeps the OpenMP path intact while enabling a GPU-oriented execution model.
Validation cases are organized under:
validation/
The repository currently includes validation material for:
validation/dipolar_interactions/
validation/hydrodynamic_interactions/
Each validation folder contains its own README with specific instructions, physical context, expected behavior, and reference results.
The validation suite is intended to verify that SIMMSUS correctly captures:
- long-range hydrodynamic effects;
- dipolar magnetic interactions;
- equilibrium magnetization;
- sedimentation velocity behavior;
- real-space and reciprocal-space periodic contributions.
The OpenACC implementation was validated against the OpenMP implementation for the periodic hydrodynamic and dipolar interaction cases.
Performance benchmarks are organized under:
benchmarks_hpc/
benchmarks_hpc/openmp/
The OpenMP benchmark evaluates CPU scaling by varying the number of OpenMP threads, typically using values such as:
1, 2, 4, 8, 16, 32
The benchmark scripts collect execution logs and may generate performance plots using tools such as gnuplot.
benchmarks_hpc/openacc/
The OpenACC benchmark provides a reproducible GPU execution case using the same physical configuration adopted in the OpenMP benchmark.
The folder includes:
simconfig.dat
run_openacc_benchmark.sh
Allclean.sh
README.md
The script:
./run_openacc_benchmark.shprints GPU information using nvidia-smi, runs the OpenACC executable, and stores detailed execution logs including OpenACC timing information.
The benchmark is useful for evaluating GPU execution of the refactored periodic interaction module.
Long-range particle interactions can decay slowly with distance. For interactions with slow decay, finite simulation boxes may lead to strong size effects. SIMMSUS uses a periodic lattice representation and Ewald-like summations to emulate an effectively infinite suspension.
Periodic calculations are activated when the user enables mechanisms such as:
ACCOUNT HYDRODYNAMIC INTERACTIONS
PERIODIC MAGNETIC TORQUES
PERIODIC MAGNETIC FORCES
The periodic calculation workflow includes:
- Green-function table construction;
- periodic lattice structure construction;
- real-space summation;
- reciprocal-space summation;
- accumulation of velocities, forces, and torques.
SIMMSUS includes Brownian effects through random forcing and random torques. Brownian motion is essential for magnetic suspensions at the microscale and is required for recovering equilibrium magnetization behavior consistent with Langevin theory in dilute conditions.
Random numbers are generated through a custom routine historically validated against Brownian diffusion benchmarks, including mean-square displacement behavior.
SIMMSUS can account for several force contributions:
- long-range dipolar forces;
- short-range repulsive forces;
- contact forces;
- Brownian forces;
- gravitational forces;
- hydrodynamic mobility effects.
It also includes several torque contributions:
- dipole-dipole magnetic torques;
- external-field torques;
- Brownian torques.
Periodic magnetic torques and forces are computed inside periodic_interactions.f90.
When hydrodynamic interactions are neglected, SIMMSUS can solve particle motion either with or without translational inertia.
When hydrodynamic interactions are enabled, the code assumes a mobility formulation appropriate for creeping-flow suspensions and neglects translational particle inertia. Rotational dynamics may still include a small rotational inertia for numerical regularization.
SIMMSUS supports different initial configurations, including:
- random particle distributions;
- ordered distributions;
- spherical aggregate distributions.
Dipole orientations may also be initialized in different ways depending on the physical scenario.
SIMMSUS supports several external magnetic field configurations, including:
- steady magnetic field;
- oscillatory magnetic field;
- double-frequency oscillatory field;
- rotating magnetic field;
- nonlinear Duffing-like excitation;
- frequency sweep excitations.
These excitations can be combined with shear flow in selected configurations.
[1] Ivanov, Alexey O., and Olga B. Kuznetsova. "Magnetic properties of dense ferrofluids: an influence of interparticle correlations." Physical Review E 64.4 (2001): 041405.
[2] Gontijo, R. G., and F. R. Cunha. "Numerical simulations of magnetic suspensions with hydrodynamic and dipole-dipole magnetic interactions." Physics of Fluids 29.6 (2017).
[3] Guimarães, A. B., F. R. Cunha, and R. G. Gontijo. "The influence of hydrodynamic effects on the complex susceptibility response of magnetic fluids undergoing oscillatory fields: New insights for magnetic hyperthermia." Physics of Fluids 32.1 (2020).
[4] de Carvalho, Douglas Daniel, and Rafael Gabler Gontijo. "Reconstructing a continuous magnetization field based on local vorticity cells, CFD and Langevin dynamics: A new numerical scheme." Journal of Magnetism and Magnetic Materials 514 (2020): 167135.
[5] Gontijo, R. G. "A numerical perspective on the relation between particle rotational inertia and the equilibrium magnetization of a ferrofluid." Journal of Magnetism and Magnetic Materials 434 (2017): 91-99.
[6] Gontijo, Rafael Gabler, and Andrey Barbosa Guimarães. "Langevin dynamic simulations of magnetic hyperthermia in rotating fields." Journal of Magnetism and Magnetic Materials 565 (2023): 170171.
[7] Gontijo, R. G., and F. R. Cunha. "Dynamic numerical simulations of magnetically interacting suspensions in creeping flow." Powder Technology 279 (2015): 146-165.
[8] Gontijo, R. G., S. Malvar, and F. R. Cunha. "Magnetic particulate suspensions from the perspective of a dynamical system." Powder Technology 297 (2016): 165-182.
[9] Gontijo, R. G., and S. Malvar. "Microstructural transition in an ordered set of magnetic spheres immersed in a carrier liquid." Mechanics Research Communications 83 (2017): 12-17.
[10] Gontijo, R. G. "Heat transfer increase for a laminar pipe flow of a magnetic fluid subjected to constant heat flux: An initial theoretical approach." Mechanics Research Communications 91 (2018): 27-32.
[11] Berkov, D. V., L. Yu Iskakova, and A. Yu Zubarev. "Theoretical study of the magnetization dynamics of nondilute ferrofluids." Physical Review E — Statistical, Nonlinear, and Soft Matter Physics 79.2 (2009): 021407.
[12] Ewald, P. "Die Berechnung optischer und elektrostatischer Gitterpotentiale". Annalen der Physik 369 (3): 253–287.
[13] Gontijo, R. G. "Micromechanics and microhydrodynamics of magnetic suspensions (in Portuguese)". PhD Thesis, Graduate Program in Mechanical Sciences, University of Brasília.
