In 2024, I read 99 papers and 21 non-technical books. 99 papers is slightly more than my previous record (87 papers in 2023), while 21 books is slighly less…
Selected Publications
Evaluating Computational Pathology Foundation Models for Prostate Cancer Grading under Distribution Shifts
Foundation models (FMs) are intended to be general-purpose feature extractors. Real-world pathology image data does however exhibit considerable variability. FMs should be robust to these variations and other distribution shifts which might be encountered in practice. We evaluate two computational pathology FMs (UNI and CONCH), by utilizing them as feature extractors within prostate cancer grading models. We find that while FMs perform well relative to baselines, the absolute performance can still be far from satisfactory. The fact that FMs have been trained on large and varied datasets does not guarantee that downstream models always will be robust to common distribution shifts.
Preprint, 2024
Evaluating Deep Regression Models for WSI-Based Gene-Expression Prediction
Prediction of mRNA gene-expression profiles directly from routine WSIs using deep learning models could potentially offer cost-effective and widely accessible molecular phenotyping. While such WSI-based gene-expression prediction models have recently emerged, the high-dimensional nature of the corresponding regression problem offers numerous design choices which remain to be analyzed in detail. We provide recommendations on how deep regression models should be trained for WSI-based gene-expression prediction. For example, we conclude that training a single model to simultaneously regress all 20530 genes is a computationally efficient yet very strong baseline.
Preprint, 2024
Taming Diffusion Models for Image Restoration: A Review
Diffusion models have achieved remarkable progress in generative modelling, and have recently also been applied to low-level computer vision for photo-realistic image restoration (IR). We introduce key constructions in diffusion models and survey contemporary techniques that make use of diffusion models in solving general IR tasks. Furthermore, we point out the main challenges and limitations of existing diffusion-based IR frameworks and provide potential directions for future work.
Philosophical Transactions of the Royal Society A, 2025
Evaluating Regression and Probabilistic Methods for ECG-Based Electrolyte Prediction
Imbalances in electrolyte concentrations can have severe consequences, but accurate and accessible measurements could improve patient outcomes. The current measurement method based on blood tests is accurate but invasive and time-consuming, and is often unavailable e.g. in remote locations or an ambulance setting. In contrast, an ECG is a widely adopted tool which is quick and simple to acquire. The problem of estimating continuous electrolyte concentrations directly from ECGs is however not well-studied. We therefore investigate if DNN regression models can be used for ECG-based prediction of electrolyte concentrations, utilizing a novel dataset of over 290,000 ECGs.
Scientific Reports, 2024
Controlling Vision-Language Models for Multi-Task Image Restoration
We present a degradation-aware vision-language model (DA-CLIP) as a multi-task framework for image restoration. DA-CLIP trains an additional controller that adapts the fixed CLIP image encoder to predict high-quality feature embeddings. By integrating the embedding into an image restoration network via cross-attention, we are able to pilot the model to learn a high-fidelity image reconstruction. The controller itself also outputs a degradation feature that matches the real corruptions of the input, yielding a natural classifier for different degradation types. Our approach advances state-of-the-art performance on both degradation-specific and unified image restoration tasks.
ICLR, 2024
How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts?
We propose a benchmark for testing the reliability of regression uncertainty estimation methods under real-world distribution shifts. It consists of 8 image-based regression datasets with different types of challenging distribution shifts. We use our benchmark to evaluate many of the most common uncertainty estimation methods, as well as two state-of-the-art uncertainty scores from OOD detection. While methods are well calibrated when there is no distribution shift, they all become highly overconfident on many of the benchmark datasets. This uncovers important limitations of current uncertainty estimation methods, and our benchmark thus serves as a challenge to the research community.
TMLR, 2023
Image Restoration with Mean-Reverting Stochastic Differential Equations
We present a stochastic differential equation (SDE) approach for general-purpose image restoration. The key construction is a mean-reverting SDE that models the degradation process from high-quality image to low-quality counterpart. By simulating the corresponding reverse-time SDE, high-quality images can then be restored. We also propose a maximum likelihood objective that stabilizes the training and improves the restoration results. Our method achieves highly competitive performance on the tasks of image deraining, deblurring and denoising. The general applicability is further demonstrated via qualitative results on image super-resolution, inpainting and dehazing.
ICML, 2023
Learning Proposals for Practical Energy-Based Regression
We derive an efficient and convenient objective that can be employed to train a parameterized distribution q(y|x; phi) by directly minimizing its KL divergence to a conditional EBM p(y|x; theta). We then employ the proposed objective to jointly learn an effective MDN proposal distribution during EBM training, thus addressing the main practical limitations of energy-based regression. Furthermore, we utilize our derived training objective to learn MDNs with a jointly trained energy-based teacher, consistently outperforming conventional MDN training on four real-world regression tasks within computer vision.
AISTATS, 2022
Accurate 3D Object Detection using Energy-Based Models
We apply energy-based models p(y|x; theta) to the task of 3D bounding box regression, extending the recent energy-based regression approach from 2D to 3D object detection. This is achieved by designing a differentiable pooling operator for 3D bounding boxes y, and adding an extra network branch to the state-of-the-art 3D object detector SA-SSD. We evaluate our proposed detector on the KITTI dataset and consistently outperform the SA-SSD baseline, demonstrating the potential of energy-based models for 3D object detection.
CVPR Workshops, 2021
How to Train Your Energy-Based Model for Regression
We propose a simple yet highly effective extension of noise contrastive estimation (NCE) to train energy-based models p(y|x; theta) for regression tasks. Our proposed method NCE+ can be understood as a direct generalization of NCE, accounting for noise in the annotation process of real-world datasets. We provide a detailed comparison of NCE+ and six popular methods from literature, the results of which suggest that NCE+ should be considered the go-to training method. We also apply NCE+ to the task of visual tracking, achieving state-of-the-art performance on five commonly used datasets. Notably, our tracker achieves 63.7% AUC on LaSOT and 78.7% Success on TrackingNet.
BMVC, 2020
Energy-Based Models for Deep Probabilistic Regression
We propose a general and conceptually simple regression method with a clear probabilistic interpretation. We create an energy-based model of the conditional target density p(y|x), using a deep neural network to predict the un-normalized density from the input-target pair (x,y). This model of p(y|x) is trained by directly minimizing the associated negative log-likelihood, approximated using Monte Carlo sampling. Notably, our model achieves a 2.2% AP improvement over Faster-RCNN for object detection on the COCO dataset, and sets a new state-of-the-art on visual tracking when applied for bounding box regression.
ECCV, 2020
Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision
We propose a comprehensive evaluation framework for scalable epistemic uncertainty estimation methods in deep learning. It is specifically designed to test the robustness required in real-world computer vision applications. We also apply our proposed framework to provide the first properly extensive and conclusive comparison of the two current state-of-the-art scalable methods: ensembling and MC-dropout. Our comparison demonstrates that ensembling consistently provides more reliable and practically useful uncertainty estimates.
CVPR Workshops, 2020
