Abstracting essay evaluation structure by using multidimensional alpha coefficient

Research output: Contribution to journalArticle

Abstract

When we abstract the structure of three-way essay evaluation data, the aggregated matrix tends to show more unidimensional feature than that of each rater if the equal weights are used. Multidimensional Alpha Coefficient (MAC) proposed by Yanai (1994) gives independent sets of scoring weights which vary dimensionality of the aggregated matrices. The present study tried to extract the evaluation structure from three-way essay data by using the method for preservation of the original multidimensionality. The essay data of Taira (1995) evaluated by seven raters were used. Some criteria, including the factor analysis with Procrustes rotation method, were set to choose the most suitable weights. The result showed that the weights for the sixth solution of MAC was regarded as the best. The aggregated matrix yielded three factors, consisting of one more dimension compared with the former analysis. The revised path diagram showed much clearer causal relationships. The emotional factors on writing and reading had an effect on the ability of story making, whereas Writing Habits influenced only upon the Loyalty to the Task Condition.

Original languageEnglish
Pages (from-to)31-40
Number of pages10
JournalJapanese Journal of Educational Psychology
Volume46
Issue number1
DOIs
Publication statusPublished - 1998 Jan 1
Externally publishedYes

Keywords

  • Essay evaluation
  • Multidimensional Alpha Coefficient (MAC)
  • Multidimensional Intraclass Correlation Coefficient (MICC)
  • Procrustes rotation method
  • Structure

ASJC Scopus subject areas

  • Education
  • Developmental and Educational Psychology

Fingerprint Dive into the research topics of 'Abstracting essay evaluation structure by using multidimensional alpha coefficient'. Together they form a unique fingerprint.

  • Cite this