openQCD

openQCD

Simulation programs for lattice QCD

Lattice QCD simulations are performed on high-performance parallel computers with hundreds and thousands of processing units. The openQCD package includes some of the most advanced simulation techniques, but keeps a high degree of flexibility so that the users can select the ones to use.

Lattice theory

Currently the common features of the supported lattice theories are the following:

4-dimensional hypercubic lattice with even sizes N₀ x N₁ x N₂x N₃. Open, Schrödinger functional (SF), open-SF or periodic boundary conditions in the time direction. Periodic boundary conditions in the space directions.

SU(3) gauge group, plaquette plus rectangle gauge action (Wilson, Symanzik, Iwasaki,...).

O(a)-improved Wilson quarks in the fundamental representation. Among the supported quark multiplets are the classical ones (pure gauge, two-flavour theory, 2+1 and 2+1+1 flavour QCD), but twisted mass doublets and theories with many doublets, for example, are also supported.

The O(a)-improvement includes the boundary counterterms required for the improvement of the correlation functions near the boundaries of the lattice if open, SF or open-SF boundary conditions are chosen. Either the traditional implementation of the O(a) Pauli term in the quark action or an "exponential" variant (which offers some technical advantages) may be used. For the quark fields phase-periodic boundary conditions in the space directions are implemented too.

Simulation algorithm

The simulation programs are based on the HMC or the SMD (Stochastic Molecular Dynamics) algorithm. For the strange and the heavier quarks, a rational approximation of the exact pseudo-fermion action is used as in the RHMC algorithm. Several advanced techniques are implemented that can be configured at run time:

Nested hierarchical integrators for the molecular-dynamics equations, based on any combination of the leapfrog, 2nd order Omelyan-Mryglod-Folk (OMF) and a 4th order OMF elementary integrators.

Twisted-mass Hasenbusch frequency splitting, with any number of factors and twisted masses. Optionally with even-odd preconditioning.

Twisted-mass determinant reweighting.

Deflation acceleration and chronological solver along the molecular-dynamics trajectories.

A choice of solvers (CGNE, MSCG, SAP+GCR, deflated SAP+GCR) for the Dirac equation, separately configurable for each force component and pseudo-fermion action.

Parallel I/O of gauge, momentum and quark fields.

All these depend on a number of parameters, whose values are passed to the simulation program together with those of the action parameters (coupling constants, quark masses, etc.) through a structured input parameter file.

Master-field simulations

Since version 2.0, openQCD includes various technical improvements (quadruple-precision accumulation of large sums, uniform-norm solver stopping criterion, etc.), which are expected to make master-field simulations of very large lattices numerically safe.

Program features

The programs parallelize in 0,1,2,3 or 4 dimensions, depending on what is specified at compilation time. They are highly optimized for machines with current Intel or AMD processors, but should run correctly on any system complying with the IEEE 754, ISO C89, MPI 1.2 and (since openQCD 2.4) OpenMP 4.5 standards.

For the purpose of testing and code development, the programs can also be run on a desktop or laptop computer. All what is needed for this is a compliant C compiler and a local MPI installation such as Open MPI.

Documentation

The simulation programs have a modular form, with strict prototyping and a minimal use of external variables. Each program file contains a small number of externally accessible functions whose functionality is described at the top of the file.

The data layout is explained in various README files and detailed instructions are given on how to run the main programs. Further documentation, specifying the normalization conventions and algorithms used, is included in the doc directory.

Authors

The initial release of the openQCD package was written by Martin Lüscher and Stefan Schaefer. Support for Schrödinger functional boundary conditions was added by John Bulava. Phase-periodic boundary conditions for the quark fields were introduced by Isabel Campos and the implementation of the "exponential" variant of the O(a) Pauli term was developed by Antonio Rago. Several modules were taken over from the DD-HMC program tree, which includes contributions from Luigi Del Debbio, Leonardo Giusti, Björn Leder and Filippo Palombi.

Acknowledgements

In the course of the development of the openQCD code, many people suggested corrections and improvements or tested preliminary versions of the programs. The authors are particularly grateful to Isabel Campos, Dalibor Djukanovic, Georg Engel, Leonardo Giusti, Björn Leder, Daniel Mohler, Carlos Pena and Hubert Simma for their communications and help.

Download

The publicly available openQCD releases are

    openQCD-2.4.2.tar.gz (latest stable release)
    openQCD-2.4.1.tar.gz
    openQCD-2.4.tar.gz
    openQCD-2.0.tar.gz
    openQCD-1.6.tar.gz
    openQCD-1.4.tar.gz
    openQCD-1.2.tar.gz
    openQCD-1.0.tar.gz

See the CHANGELOG file for a summary of the changes made from one version to the next and there is an up-to-date list of known bugs. The packages contain the source code, usage instructions and other documentation files. After unpacking, first read the README file in the top directory.

License

The software may be used under the terms of the GNU General Public Licence (GPL).

Alternative packages and complementary programs

There is a publicly available BG/Q version of openQCD that takes advantage of the machine-specific features of IBM BlueGene/Q computers.

As explained here, the most time-consuming parts of the programs can be further accelerated using AVX-512 intrinsics on machines supporting these instructions. A publicly available code implementing this in openQCD-1.6 may work with later versions of openQCD too.

The openQCD programs currently do not support reweighting in the quark masses, but a module providing this functionality can be downloaded from here.

Another extension of openQCD-1.6, openQCD-FASTSUM, supports simulations of anisotropic lattices and fermion actions in which the gauge-field variables are replaced by stout-smeared ones.

The inclusion of electro-magnetic effects in QCD simulations is conceptually non-trivial and requires important additions to (and modifications of) the code provided by openQCD. A complete QCD+QED program package derived from openQCD is available here and described in detail in this paper.

Full-fledged QCD simulation programs tend to have many adjustable parameters. In the case of openQCD, most parameters are passed to the programs through a human-readable structured file. Liam Keegan's sleek graphical editor for these parameter files offers some guidance and complains when inconsistent parameter values are entered.

Last updated 04 January 2024