What you think and what I think: Studying intersubjectivity in knowledge artifacts evaluation

Dmytro Babik, Rahul Singh, Xia Zhao, Eric W. Ford

Research output: Contribution to journalArticle

Abstract

Miscalibration, the failure to accurately evaluate one’s own work relative to others' evaluation, is a common concern in social systems of knowledge creation where participants act as both creators and evaluators. Theories of social norming hold that individual’s self-evaluation miscalibration diminishes over multiple iterations of creator-evaluator interactions and shared understanding emerges. This paper explores intersubjectivity and the longitudinal dynamics of miscalibration between creators' and evaluators' assessments in IT-enabled social knowledge creation and refinement systems. Using Latent Growth Modeling, we investigated dynamics of creator’s assessments of their own knowledge artifacts compared to peer evaluators' to determine whether miscalibration attenuates over multiple interactions. Contrary to theory, we found that creator’s self-assessment miscalibration does not attenuate over repeated interactions. Moreover, depending on the degree of difference, we found self-assessment miscalibration to amplify over time with knowledge artifact creators' diverging farther from their peers' collective opinion. Deeper analysis found no significant evidence of the influence of bias and controversy on miscalibration. Therefore, relying on social norming to correct miscalibration in knowledge creation environments (e.g., social media interactions) may not function as expected.

Original languageEnglish (US)
JournalInformation Systems Frontiers
DOIs
StateAccepted/In press - Jul 19 2015

Keywords

  • Intersubjectivity
  • Knowledge artifacts
  • Latent classes
  • Longitudinal analysis
  • Miscalibration
  • Peer-evaluation

ASJC Scopus subject areas

  • Information Systems
  • Computer Networks and Communications
  • Software
  • Theoretical Computer Science

Fingerprint Dive into the research topics of 'What you think and what I think: Studying intersubjectivity in knowledge artifacts evaluation'. Together they form a unique fingerprint.

  • Cite this