Cognition: Accurate and Consistent Linear Log Parsing Using Template Correction

doi:10.1007/s11390-021-1691-3

	Cognition: Accurate and Consistent Linear Log Parsing Using Template Correction
	Tian, Ran 1,2; Diao, Zu-Long 2,3; Jiang, Hai-Yang 2; Xie, Gao-Gang 1,4
	2023-09-01
发表期刊	JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY
ISSN	1000-9000
卷号	38 期号:5 页码:1036-1050
摘要	Logs contain runtime information for both systems and users. As many of them use natural language, a typical log-based analysis needs to parse logs into the structured format first. Existing parsing approaches often take two steps. The first step is to find similar words (tokens) or sentences. Second, parsers extract log templates by replacing different tokens with variable placeholders. However, we observe that most parsers concentrate on precisely grouping similar tokens or logs. But they do not have a well-designed template extraction process, which leads to inconsistent accuracy on particular datasets. The root cause is the ambiguous definition of variable placeholders and similar templates. The consequences include abuse of variable placeholders, incorrectly divided templates, and an excessive number of templates over time. In this paper, we propose our online log parsing approach Cognition. It redefines variable placeholders via a strict lower bound to avoid ambiguity first. Then, it applies our template correction technique to merge and absorb similar templates. It eliminates the interference of commonly used parameters and thus isolates template quantity. Evaluation through 16 public datasets shows that Cognition has better accuracy and consistency than the state-of-the-art approaches. It also saves up to 52.1% of time cost on average than the others.
关键词	log analysis log parsing template correction
DOI	10.1007/s11390-021-1691-3
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2019YFB1802800] ; National Science Fund for Distinguished Young Scholars of China[61725206]
WOS研究方向	Computer Science
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Software Engineering
WOS记录号	WOS:001114345700007
出版者	SPRINGER SINGAPORE PTE LTD
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38474
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Xie, Gao-Gang
作者单位	1.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 3.Purple Mt Labs, Nanjing 211111, Peoples R China 4.Chinese Acad Sci, Comp Network Informat Ctr, Beijing 100083, Peoples R China
推荐引用方式 GB/T 7714	Tian, Ran,Diao, Zu-Long,Jiang, Hai-Yang,et al. Cognition: Accurate and Consistent Linear Log Parsing Using Template Correction[J]. JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY,2023,38(5):1036-1050.
APA	Tian, Ran,Diao, Zu-Long,Jiang, Hai-Yang,&Xie, Gao-Gang.(2023).Cognition: Accurate and Consistent Linear Log Parsing Using Template Correction.JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY,38(5),1036-1050.
MLA	Tian, Ran,et al."Cognition: Accurate and Consistent Linear Log Parsing Using Template Correction".JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 38.5(2023):1036-1050.