KokkosECG

Sat, 21 Feb 2026 00:00:00 +0100

KokkosECG is a C++ implementation of the Enlarged Conjugate Gradient (ECG) method on top of Kokkos, so that the same solver code can run on CPUs and GPUs.

Why ECG?

The classical Conjugate Gradient (CG) method for a sparse symmetric positive definite system is dominated by sparse matrix-vector products and dot products. On current hardware these operations are limited by memory bandwidth, and every dot product is a global reduction, which is expensive in a distributed setting.

ECG, introduced by Grigori, Moufawad and Nataf, replaces the single search direction of CG by a block of t directions. The sparse matrix-vector product becomes a sparse matrix times a block of vectors, and the scalar coefficients of CG become small t x t dense matrices. This gives:

a higher arithmetic intensity, because the matrix is read once for t vectors,
fewer synchronisation points per iteration, because the global reductions are done on blocks,
a richer Krylov subspace, hence potentially fewer iterations.

With t = 1 the method is exactly CG. ECG is a method I already worked with during my PhD at INRIA, on linear solvers for the map-making problem (see B3DCMB).

What the code does

The implementation follows the Block CG Orthomin variant with A-CholQR orthogonalisation, wrapped in the ECG framework. The initial residual is split into t vectors with contiguous, disjoint supports, and the A-orthogonalisation is done through the t x t Gram matrix of the search directions, factorised once per iteration with a Cholesky decomposition.

The library is organised around a small class hierarchy, which makes it simple to add other Krylov variants:

IterativeMethod, the abstract base class (iteration limit, tolerance, convergence history),
ConjugateGradient, the BLAS-2 baseline,
EnlargedConjugateGradient, the block version with a sparse-matrix-times-block product and BLAS-3 kernels.

All vectors and matrices are Kokkos::View, and the sparse matrices are stored in CRS format and handled with KokkosKernels. The build uses CMake with C++20, and the repository enforces formatting, static analysis and tests through a pre-commit hook.

The preconditioned variants, a benchmark suite against plain CG and a CI pipeline are still on the roadmap. The details are in the repository.

Reference

L. Grigori, S. Moufawad, F. Nataf, Enlarged Krylov Subspace Conjugate Gradient Methods for Reducing Communication, SIAM Journal on Matrix Analysis and Applications, 37(2), 744-773, 2016.

Linear solvers | TC

KokkosECG

Why ECG?

What the code does

Reference