Laura O'Mahony

About me...

Hey! I am a computer science and mathematics PhD student from Ireland. I am currently undertaking my PhD in the SFI foundations of data science PhD program. Here, I am a member of the BDARG group.

I initially focused on projects centred around statistical analysis, algorithms concerning graph drawing, visualisation of networks, moving on to more applied projects involving data collection and processing, and statistical machine learning. In the end, I became another maths nerd who got distracted by ML and LLMs… I guess I would say I am just following the gradient of interesting 🌞

Broadly speaking, I aim to tackle the issue that current AI systems lack the robustness, diversity and creativity of humans, and we have a poor understanding of them. One angle I have approached this is through mechanistic and concept-based interpretability, with the goal of understanding a model’s representations. I think it would be very cool to one day understand the rich internal structure of deep neural networks, or at least have better tools to reduce their the black-box mysterious nature. I am also interested in understanding and improving upon current model limitations such as robustness, quality and diversity, and open-endedness. My research spans both image models and LLMs, and I am open to studying multimodal architectures in the future.

I love to collaborate on interesting projects! I have been very fortunate to intern with MEDGIFT group in Switzerland as well as involve myself in various open science initiatives in my free time such as Aya led by Sara Hooker of CohereForAI and various fascinating projects with EleutherAI.

Email / Twitter / LinkedIn / Github / Medium

Research:

Here is an overview of a few of my projects:

	Attributing Mode Collapse in the Fine-Tuning of Large Language Models O’Mahony, L., and Grinsztajn, L., Schoelkopf, H., Biderman, S. ICLR 2024, Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) Openreview / Github Studies the effects of instruction tuning on the diversity of language models. Quantifies how each step in a typical RLHF or instruction-tuning pipeline changes a model’s diversity, for a series of models trained in a controlled fine-tuning setup.
	Uncovering unique concept vectors through latent space decomposition Graziani, M., O'Mahony, L., Nguyen A., Müller, H., Andrearczyk, V. TMLR Openreview / arXiv A novel post-hoc unsupervised method that automatically uncovers the concepts learned by deep models during training.
	Disentangling Neuron Representations with Concept Vectors O'Mahony, L., Andrearczyk, V. Müller, H., Graziani, M. In CVPR 2023 2nd Workshop on Explainable AI for Computer Vision (XAI4CV). arXiv / GitHub / Medium Shows that many neurons act as multiple feature detectors that can be disentangled into linear directions or concept vectors encapsulating distinct features.
	Concept discovery and Dataset exploration with Singular Value Decomposition Graziani, M., Nguyen, A. P., O'Mahony, L., Müller, H., & Andrearczyk, V. In ICLR 2023 Workshop on Pitfalls of limited data and computation for Trustworthy ML (TrustML-(un)Limited). OpenReview / GitHub Application of a post-hoc unsupervised method that automatically discovers high-level concepts learned by intermediate layers of vision models to identify anomalies in the data collection.
	On the Detection of Anomalous or Out-of-Distribution Data in Vision Models Using Statistical Techniques O’Mahony, L., O’Sullivan, D. J., & Nikolov, N. S. AICV 2023. Best paper award. Cham: Springer Nature Switzerland. Chapter / GitHub A filter for anomalous data points and for signalling out-of-distribution data.
	Ocean Box Models and their Tipping points O’Mahony, L., & Wieczorek, S. Report / GitHub Mathematical modelling of the Atlantic Ocean with interesting insights into previous bifurcations such as glacial periods (ice age cycles) and tipping point bifurcations and reversibility

See Google Scholar for my full list of published works.

Service:

Conference and Journal Reviewing:

Springer Big Data journal (2022)
BMVC (2024)
CVPR XAI4CV workshop (2023, 2024)

Previously:

Worked on a fascinating project on social media data for Met Eireann, Ireland's meteorological society.
Pain in Older European adults: Analysis of SHARE data.
Before my PhD I worked as a risk analyst in AIB, Dublin.
Graduated with a first class honours grade BSc in Mathmatical Sciences from University College Cork in 2019. Project work highlight: Undertook a research project on using mathematical modelling to model the Atlantic Ocean in order to study the distribution of heat in the world. Interesting insights into previous climate bifurcations such as the glacial periods (ice age cycles) as well as the stability of our current climate (tipping point reversibility).

Hosted on GitHub Pages — Thanks to Jon Barron for the theme.