Deep Unrolling & Model-Based Learning
Deep unrolling turns a classical iterative reconstruction algorithm into a neural network by fixing the number of iterations and learning some of its components, such as step sizes, regularisers or whole proximal operators, from data. Each layer applies the CT forward and back-projection operators, so the network keeps data consistency by construction while learning a prior far richer than a hand-crafted one.
Typical examples unroll gradient descent (learned gradient descent), primal-dual algorithms (Learned Primal-Dual), ISTA/FISTA, ADMM or fields-of-experts schemes (LEARN). Related model-based approaches include plug-and-play and regularisation-by-denoising, which keep the iterative solver and plug in a trained denoiser, and dual-domain networks that alternate between sinogram and image.
Unrolled networks are among the most accurate supervised methods for sparse-view, limited-angle, low-dose and metal-artifact problems, and they tend to transfer across acquisition settings better than pure image-to-image networks. The cost is memory and training time, because every layer calls the projector; memory-efficient and 3D unrolling schemes address this.
Browse all 68 Deep Unrolling & Model-Based Learning papers in the searchable directory
Start here
If you are new to this topic, these are good first reads, chosen by the maintainers.
- Learned Primal-Dual Reconstruction (TMI, 2018)
Learned Primal-Dual, the reference unrolled network that learns the proximal steps of a primal-dual solver. - LEARN: Learned Experts’ Assessment-Based Reconstruction Network for Sparse-Data CT (TMI, 2018)
Unrolls a fields-of-experts iterative scheme for sparse-data CT; one of the first CT-specific unrolled networks. - FISTA-Net: Learning a Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging (TMI, 2021 · code)
Unrolls FISTA with learned thresholds and step sizes, keeping the solver's interpretability, with code. - An Interpretable Dual Domain Network for CT Metal Artifact Reduction (MICCAI, 2021 · code)
A model-driven dual-domain network for metal artifact reduction, a good example of unrolling beyond sparse-view, with code.
Papers
The 30 most recent of 68 papers, newest first.
Browse all 68 Deep Unrolling & Model-Based Learning papers in the searchable directory
Datasets, benchmarks and toolkits
Entries from the directory whose tasks or tags match this topic.
- LoDoPaB-CT (official page) · Dataset — Projection + image. 42,895 simulated human thorax slice pairs (LIDC/IDRI) at a single noise level (N0 = 4096) — other dose levels have to be re-simulated.
- DeepInverse benchmark suite (official page) · Benchmark — A unified PyTorch benchmark harness covering CT, MRI, deblurring and inpainting, with reproducible metrics and pretrained baselines — the most practical way to compare a new CT method against strong generic solvers.
- tomosipo · Toolkit — Painless specification of 2D/3D projection geometries; wraps ASTRA and integrates with PyTorch.
- DeepInverse · Toolkit — PyTorch library for imaging inverse problems — physics operators, learned denoisers, unrolled networks and benchmarks.
- torch-radon · Toolkit — Differentiable Radon transform and filtered backprojection in PyTorch.
- LION · Toolkit — Learned Iterative Optimization Networks — unrolled reconstruction with PyTorch, used in CT and PET.
- Learned Primal-Dual · Toolkit — Reference implementation of Learned Primal-Dual reconstruction — the canonical unrolled primal-dual scheme, originally applied to CT.
- Learned Gradient Tomography · Toolkit — Reference implementation of learned gradient descent for ill-posed tomography (Adler & Öktem); the origin of many later unrolled CT networks.
Guides
Guides that cover this topic.
Other topics
- Surveys & Reviews (24)
- Classical & Iterative Reconstruction (71)
- Sparse-View CT (157)
- Limited-Angle Tomography (46)
- Low-Dose CT Denoising (69)
- Metal Artifact Reduction (23)
- Spectral & Dual-Energy CT (19)
- Cone-Beam CT (77)
- Dynamic & 4D CT (24)
- Scatter Correction (11)
- Geometry Calibration & Motion Compensation (28)
- Self-Supervised & Untrained Methods (35)
- Diffusion & Generative Priors (68)
- Implicit Neural Representations (25)
- Gaussian Splatting (26)