Automatic document summarization based on statistical information (Journal of Mathematics Mechanics and Computer Science, 2019)
Actual problem in nowadays is to efficiently process the large amount of data that pass throughour mind everyday. The object of study of this paper is automatic summarization algorithms.The main goal is to implement and make comparison of different summarization techniqueson corpora of news articles parsed from the web. This research work contains the descriptionof three summarization techniques based on TextRank algorithm: General TextRank, BM25,LongestCommonSubstring. It is specially noted the languages of used corpora: Russian and Kazakhlanguages. The results of summarization processes and their comparison are provided. It shouldbe emphasized that used algorithms are well-known, but the way of their evaluation on definedcorpora is different from those which usually used in summary evaluation. The method of summaryevaluation proposed use the special dictionary of extracted key-words on the topic of corpora. Asthe title implies the article describes applying statistical information. The semantic and syntacticfeatures of text are not examined.
Keywords: summarization; automatic extraction; key-words; N-gram; TextRank