Enhancing AI Text Detection with Frozen Pretrained Encoders and Ensemble Learning
Shushanta Pudasaini, Luis Miralles-Pechuán, David Lillis and Marisa Llorens Salvador
In Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2025), volume 4038 of CEUR Workshop Proceedings, pages 3889--3897, Madrid, Spain, 2025.
Abstract
As AI systems become increasingly capable of generating text, distinguishing it from human-written content remains an ongoing research challenge. This paper proposes a simple yet effective ensemble-based approach for detecting AI-generated text using pre-trained encoders. Six different Large Language Models (LLMs) were fine-tuned with the PAN CLEF 2025 training set, and six ensemble learning approaches were applied on top of the five best-performing LLMs. These models were evaluated on the PAN CLEF validation dataset and a subset of the COLING 2025 dataset to ensure the model's performance across multiple datasets and domains. Experiments on benchmark datasets show that ensemble approaches significantly outperform individual models, achieving improved F1 scores and robustness across diverse AI-generated text samples. The best configuration (Bagging with support vector classifier on top of the results achieved from the top 5 performing individual LLMs) was able to achieve an F1 score of 0.9886 on the PAN CLEF 2025 benchmark compared to the F1 score of 0.9767 from the individual Deberta-v3-large model on the same benchmark dataset. Likewise, the preservation of pre-trained knowledge through frozen encoder layers consistently improved detection performance, demonstrated by the Deberta-v3-large model's 2.67\% F1 scores improvement compared to its fully fine-tuned version. From this research, ensemble learning algorithms applied on top of LLMs were found to improve the performance of the AI-generated text detection task as experimented in the Voight-Kampff Generative AI Detection 2025, which was a part of the PAN at CLEF 2025 submission made through the TIRA platform. The research is publicly available on GitHub under https://github.com/ShushantaTUD/Ensemble-Based-AI-Generated-Text-Detection.