An Objective Effect Evaluation Framework for Vectorization Models on Patent Semantic Similarity Measurement

doi:10.3390/electronics14204056

	An Objective Effect Evaluation Framework for Vectorization Models on Patent Semantic Similarity Measurement
	Li, Jiazheng 1,2; Zhou, Jian 3; Cao, Mengyun 4
	2025-10-15
发表期刊	ELECTRONICS
ISSN	2079-9292
卷号	14 期号:20 页码:14
摘要	How to objectively evaluate the effect of different vectorization models in measuring similarity between patents is a fundamental issue, which can help to select high-performance vectorization models to support advanced patent services. Based on the rank consistency index and hypothesis testing approach, a framework for evaluating the effect of different vectorization models on patents' similarity is proposed based on whether the model can accurately predict the similarity ranking of patents. Integrating the factors of time and technical field, an empirical study is conducted under the proposed framework to objectively evaluate the effect of six mainstream text vectorization models for assessing the semantic similarity of patents, which is evaluated based on Chinese patents (English Translation) from 2010 to 2024. The results show that the performance of Llama 2 is the best among six compared models in all years and in all technical fields. The proposed framework can objectively evaluate the similarity measurement effect of different vectorization models and provides a basis for the selection of the vectorization model for patent semantic similarity measurement for advanced patent services.
关键词	comparative study language model patent semantic similarity measurement statistical hypothesis testing text vectorization
DOI	10.3390/electronics14204056
收录类别	SCI
语种	英语
资助项目	Natural Science Foundation of Fujian Province, China[2022J05157] ; Natural Science Foundation of Xiamen, China[3502Z20227049]
WOS研究方向	Computer Science ; Engineering ; Physics
WOS类目	Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Physics, Applied
WOS记录号	WOS:001601463600001
出版者	MDPI
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41617
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Cao, Mengyun
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 101408, Peoples R China 3.Chinese Acad Sci, Natl Sci Lib, Beijing 100190, Peoples R China 4.Jimei Univ, Coll Comp Engn, Xiamen 361021, Peoples R China
推荐引用方式 GB/T 7714	Li, Jiazheng,Zhou, Jian,Cao, Mengyun. An Objective Effect Evaluation Framework for Vectorization Models on Patent Semantic Similarity Measurement[J]. ELECTRONICS,2025,14(20):14.
APA	Li, Jiazheng,Zhou, Jian,&Cao, Mengyun.(2025).An Objective Effect Evaluation Framework for Vectorization Models on Patent Semantic Similarity Measurement.ELECTRONICS,14(20),14.
MLA	Li, Jiazheng,et al."An Objective Effect Evaluation Framework for Vectorization Models on Patent Semantic Similarity Measurement".ELECTRONICS 14.20(2025):14.