Notation matters: chemistry language models and what is inside them

My paper has been published in Digital Discovery, Royal Society of Chemistry, as a Gold Open Access article: Notation matters: cross-representation inconsistency in chemistry language models and its mechanistic origins.

The short version: I investigate how large language models behave when applied to a real scientific task, molecular property prediction.

What I tested

The same molecule can be represented using SMILES, IUPAC, InChI or SELFIES. I tested these representations across 1,072 molecules and found that 88% showed inconsistent predictions depending on how the molecule was represented.

What happens inside the model

I then went beyond measuring the outputs and investigated what was happening inside the model. Using representation-level analysis with ChemBERTa-2, I found that chemically identical molecules progressively develop different internal representations as they move through the network.

Why it matters

What I find most interesting about this work is the broader application of AI to science. As LLMs are increasingly used for problems in chemistry and other scientific domains, it becomes important to understand not just whether an AI model can make a prediction, but what it is actually responding to when it does.

This work was a great opportunity for me to explore the intersection of AI, scientific applications, and understanding model behaviour.

Paper: Notation matters, Digital Discovery. Code: notation-matters on GitHub.