Publications

We use RL with a dense reward to train a CTC-based encoder-only model to do scene text recognition, we use the plackett-luce distribution and ctc-loss to derive this reward, learning an explicit, inspectable reading order over the image patches.

Uses mechanistic interpretability to guide pruning and distillation of the Donut VLM, yielding Donut-MINT, a lighter variant that keeps DocVQA performance at a fraction of the compute and memory.

Presents visualisation methods that expose how multiple modalies interact in vision-language models, making modality collapse easier to read and diagnose.

Introduces an aggregation framework for federated learning that formalises and handles computational heterogeneity across clients with different data and local update budgets.

Proposes robust federated learning aggregation rules that weight clients by the value of their losses, with full convergence proofs validated on IID and non-IID benchmarks.

Reframes federated aggregation through control theory. Inspired by PID controllers, we adapt the server step from each client's local performance trajectory rather than averaging coordinates blindly.