- Apply for Authority
- P-ISSN1013-0799
- E-ISSN2586-2073
- KCI
Article Contents
- 2024 (Vol.41)
- 2023 (Vol.40)
- 2022 (Vol.39)
- 2021 (Vol.38)
- 2020 (Vol.37)
- 2019 (Vol.36)
- 2018 (Vol.35)
- 2017 (Vol.34)
- 2016 (Vol.33)
- 2015 (Vol.32)
- 2014 (Vol.31)
- 2013 (Vol.30)
- 2012 (Vol.29)
- 2011 (Vol.28)
- 2010 (Vol.27)
- 2009 (Vol.26)
- 2008 (Vol.25)
- 2007 (Vol.24)
- 2006 (Vol.23)
- 2005 (Vol.22)
- 2004 (Vol.21)
- 2003 (Vol.20)
- 2002 (Vol.19)
Optimization of Number of Training Documents in Text Categorization
- Downloaded
- Viewed
Abstract
This paper examines a level of categorization performance in a reallife collection of abstract articles in the fields of science and technology, and tests the optimal size of documents per category in a training set using a kNN classifier. The corpus is built by choosing categories that hold more than 2,556 documents first, and then 2,556 documents per category are randomly selected. It is further divided into eight subsets of different size of training documents: each set is randomly selected to build training documents ranging from 20 documents (Tr20) to 2,000 documents (Tr2000) per category. The categorization performances of the 8 subsets are compared. The average performance of the eight subsets is 30% in F1 measure which is relatively poor compared to the findings of previous studies. The experimental results suggest that among the eight subsets the Tr100 appears to be the most optimal size for training a kNN classifier. In addition, the correctness of subject categories assigned to the training sets is probed by manually reclassifying the training sets in order to support the above conclusion by establishing a relation between and the correctness and categorization performance.
- keywords
- text categorization, KNN classifier, test collections, size of training documents, 문헌범주화, 텍스트 범주화, 실험문헌집단, 학습문헌집합의 규모, kNN 분류기
Reference
(1999). 문서범주화를 위한 선형분류기와 kNN의 결합모델.. , 225-231.
(2006). The effect of the quality of pre-assigned subject categories on the text categorization performance.. 23(2), 266-285.
(2003.). 복합분류기를 이용한 웹 문서 범주화에 관한 실험적 연구. , -.
(2005.). 정보검색연구. , -.
Automated learning of decision rules for text categorization ACM Transactions on Information Systems. , 233-251.
(2003). Using asymmetric distributions to improve text classifier probability estimates. , 111-118.
(2002). Feature selection using linear support vector machines. , -.
(2004). Hierarchical document categorization with support vector machine. , 2004-.
(2000). The effect of using hierarchical classifiers in text categorization. , 66-75.
(1998). Category levels in hierarchical text categorization Proceedings of the 3rd Conference on Empirical Methods in Natural Language Processing. , 1-3 18.
(2003). A maximal figure-of-merit learning approach to text categorization. , 174-181.
(2002). Natural Language Processing for Online Applications: Text Retrieval, Extraction and Categorization. , -.
(1999). Transductive inference for text classification using support vector machines. , -.
(1998). Text categorization with support vector machines learning with many relevant features.. , 137-142.
(2001). Summarization as feature selection for text categorization. , 365-370.
(1998). Using a generalized instance set for automatic text categorization. , 81-89.
(1996). Combining classifiers in text categorization. , 289-297.
(1996). Training algorithms for linear text classifiers. , 298-306.
(2002). Hierarchical text categorization using neural networks. 5, 87-118.
(1999a). Combining machine learning and hierarchical indexing structures for text categorization. , 107-124.
(1999b). Hierarchical neural networks for text categorization. , 281-282.
(2002). Machine learning in automated text categorization. 34(1), 1-47.
(2003). Boosting support vector machines for text classification through parameter-free threshold relaxation. , 247-254.
(1999). Exploiting hierarchy in text categories. 1(3), 193-216.
(1996). An evaluation of statistical approaches to MEDLINE indexing. , -.
(yiming.1994). effective and efficient learning from human decisions in text categorization and retrieval Proceedings of SIGIR-94 17th ACM International Conference on Research and Development in Information Retrieval. , 13-22.
(1999). and X. Liu. 1999. A re-examination of text categorization methods.. , 42-49.
(1999). A re-examination of text categorization methods.. , 42-49.
- Downloaded
- Viewed
- 0KCI Citations
- 0WOS Citations