The short version. For public sector organizations, through the HMGCC Co-Creation challenge framework, Dot Square Lab developed and evaluated machine-learning approaches to detect changes in authorship within online communication such as emails, messages and documents, where multiple authors or impersonation may be involved. We compared two approaches, one based on stylometric features and one on contrastive deep learning, using public multilingual datasets covering 60 million posts and comments from about 1.29 million authors. The aim was to flag the stylistic shifts that can signal unauthorized access, impersonation or collaborative writing, with explainable reasons rather than a black-box score.
At a glance
- Sector: public sector, delivered through the HMGCC Co-Creation challenge framework
- Problem: detecting subtle changes in authorship in online communication (emails, messages, documents), where multiple authors or impersonation attempts might occur
- Approach: developed and compared two approaches, stylometric features and contrastive deep learning, for detecting authorship change
- Tech: pre-trained multilingual language models, encoder-based embeddings, contrastive learning for style distance, SHAP-based explainability; public multilingual datasets (60M posts and comments, ~1.29M authors)
- Outcome: a benchmarked, explainable comparison of the two approaches, on public data and multilingual by design
The challenge
Public sector organizations, through the HMGCC Co-Creation challenge framework, needed a robust way to detect subtle changes in authorship within online communication, where multiple authors or impersonation attempts might occur. The goal was to flag stylistic shifts that could indicate unauthorized access, impersonation or collaborative writing, reliably enough to act on and with a low false-positive rate, across different communication contexts and languages.
The approach
Dot Square Lab developed and evaluated two approaches to detecting authorship change, then compared them head to head:
- Stylometric features. The proven route: measure the linguistic patterns unique to a writer, such as word choice, sentence structure and rhythm, and detect when a text departs from an established style.
- Contrastive deep learning. Using pre-trained multilingual language models, encoder-based embeddings trained with contrastive learning to represent style distance, so texts in a similar style sit close together and different styles sit far apart.
We processed and cleaned diverse public multilingual datasets with multi-author, multi-language components (60 million posts and comments from about 1.29 million authors), then ran performance benchmarking, comparative testing, and SHAP-based feature-importance analysis so the results could be inspected rather than taken on trust.
What we delivered
A rigorous, explainable comparison of the two approaches for detecting authorship change in online communication, evaluated on large multilingual data. Because it is built on public datasets and multilingual models, the work is ethical and adaptable across languages, and its decisions can be inspected through feature importance rather than handed down as a black-box score.
|
|
| Initial latent structure, before training. | Learnt style representation, after contrastive training. |
Where this approach applies
The same problem recurs wherever the question is not just what was written but who wrote it, and the signal is a subtle shift in style: verifying authorship, spotting compromised or shared accounts, checking whether a document had one author or several, and content-integrity checks across languages. Each needs the same two ingredients: a model of a writer's style, and explainability, so a person can see why a text was flagged and keep the false-positive rate in check.
Frequently asked questions
Can AI detect when the author of a text changes? Within limits, yes. Writers leave consistent stylistic patterns, such as word choice, sentence structure and rhythm, known as stylometry. Modelling those patterns lets a system flag when a piece of communication departs from an established style, which can indicate unauthorized access, impersonation or more than one author. It is built to flag shifts for review with explainable reasons, not to hand down an unaccountable verdict.
Stylometric features or deep learning, which is better? It depends on the data and the constraints, which is why this project evaluated both rather than assuming one. Stylometric features are interpretable and reliable; contrastive deep learning can capture subtler style distinctions. Benchmarking and comparative testing show the trade-off.
How does it stay explainable? Through SHAP-based feature-importance analysis alongside the benchmarking, so a reviewer can see which features drove a given decision. That supports trust and helps keep false positives down.
Does it work across languages? Yes. It uses publicly available multilingual language models and was evaluated on multilingual data, so it is designed for multi-author, multi-language communication rather than a single language.
A hard problem hiding in your data?
This project is one example of the work we take on: a faint signal in large, messy data, and a decision that has to be explainable enough to act on. If that is the shape of your problem, in any domain, we should talk.
