Automated title generation for research papers: A transformer-based approach
Date
2024
Authors
Hosseinzadeh, Maryam
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
The task of generating concise, informative, and relevant titles for research papers is a critical but challenging aspect of academic publishing. This project report explores the application of Transformer-based Large Language Models (LLMs) to automatically generate and suggest research paper titles from their abstracts. Building on the foundational Transformer architecture introduced by Vaswani et al., we evaluate the performance of different models, including a custom-built LLM, a pre-trained GPT-2 model, and a fine-tuned version of GPT-2. Through qualitative analysis, we demonstrate that fine-tuning GPT-2 on a specific dataset of research paper abstracts and titles significantly enhances the coherence, relevance, and contextual accuracy of the generated titles. We address challenges such as hallucinations in LLM-generated text and discuss the importance of high-quality datasets and task-specific fine-tuning. This work contributes to the broader understanding of the capabilities and limitations of LLMs in specialized NLP tasks, offering insights for future research and applications in academic publishing.
Description
Keywords
transformers, large language models, natural language processing