New Term Weighting Algorithm for Single Document Summarization

Keyword extraction plays a central role in single-document summarization, where the task is toidentify the most salient terms that capture the meaning of a text. Existing unsupervised keywordextraction approaches, such as RAKE, TextRank, and YAKE, rely primarily on frequency andstatistical co-occurrence. Although effective, they often overlook the structural and semanticcontributions of sentences, which are essential for preserving context, especially in long orcomplex documents. In this work, we propose Sentencer, a novel unsupervised term weightingalgorithm that integrates sentence-level features into keyword scoring. Unlike frequency-onlyapproaches, Sentencer leverages contextual relevance, sentence length, intra-sentence probability,and sentence position to refine keyword importance. The algorithm is evaluated against YAKE onthree benchmark datasets: SemEval (scientific papers), Inspec (abstracts), and a collection of newsreports. Results show that Sentencer performs particularly well on long, complex texts such asscientific papers, where it achieves superior precision and recall compared to YAKE, albeit at thecost of computational efficiency. Furthermore, Sentencer offers a secondary benefit as a diagnostictool for analyzing sentence behavior and word distribution dynamics within documents. For shortscientific abstracts, Sentencer out performed YAKE by 3.5% while for scientific articles,Sentencer out performed YAKE by 1%. However, for short news articles (100 to 400 words),YAKE outperforms Sentencer.