Title

The Kappa Statistic: A Second Look

Authors

B. Di Eugenio, University of Illinois at Chicago
Michael Glass, Valparaiso University

Document Type

Article

Publication Date

3-2004

Abstract

In recent years, the kappa coefficient of agreement has become the de facto standard for evaluating intercoder agreement for tagging tasks. In this squib, we highlight issues that affect kappa and that the community has largely neglected. First, we discuss the assumptions underlying different computations of the expected agreement component of kappa. Second, we discuss how prevalence and bias affect the K measure.

Recommended Citation

Di Eugenio, B. and Glass, Michael, "The Kappa Statistic: A Second Look" (2004). Mathematics and Computer Science Faculty Publications. 18.
https://scholar.valpo.edu/mcs_fac_pubs/18