What you think and what I think: Studying intersubjectivity in knowledge artifacts evaluation

Dmytro Babik, Rahul Singh, Xia Zhao, Eric W. Ford

Research output: Contribution to journalArticle

Abstract

Miscalibration, the failure to accurately evaluate one’s own work relative to others' evaluation, is a common concern in social systems of knowledge creation where participants act as both creators and evaluators. Theories of social norming hold that individual’s self-evaluation miscalibration diminishes over multiple iterations of creator-evaluator interactions and shared understanding emerges. This paper explores intersubjectivity and the longitudinal dynamics of miscalibration between creators' and evaluators' assessments in IT-enabled social knowledge creation and refinement systems. Using Latent Growth Modeling, we investigated dynamics of creator’s assessments of their own knowledge artifacts compared to peer evaluators' to determine whether miscalibration attenuates over multiple interactions. Contrary to theory, we found that creator’s self-assessment miscalibration does not attenuate over repeated interactions. Moreover, depending on the degree of difference, we found self-assessment miscalibration to amplify over time with knowledge artifact creators' diverging farther from their peers' collective opinion. Deeper analysis found no significant evidence of the influence of bias and controversy on miscalibration. Therefore, relying on social norming to correct miscalibration in knowledge creation environments (e.g., social media interactions) may not function as expected.

Original languageEnglish (US)
JournalInformation Systems Frontiers
DOIs
StateAccepted/In press - Jul 19 2015

Fingerprint

Self-assessment
Evaluation
Interaction
Social Systems
Social Media
Refinement
Knowledge
Iteration
Evaluate
Modeling
Influence
Evidence

Keywords

  • Intersubjectivity
  • Knowledge artifacts
  • Latent classes
  • Longitudinal analysis
  • Miscalibration
  • Peer-evaluation

ASJC Scopus subject areas

  • Information Systems
  • Computer Networks and Communications
  • Software
  • Theoretical Computer Science

Cite this

What you think and what I think : Studying intersubjectivity in knowledge artifacts evaluation. / Babik, Dmytro; Singh, Rahul; Zhao, Xia; Ford, Eric W.

In: Information Systems Frontiers, 19.07.2015.

Research output: Contribution to journalArticle

@article{933c6aafda384905b27b232689e7d973,
title = "What you think and what I think: Studying intersubjectivity in knowledge artifacts evaluation",
abstract = "Miscalibration, the failure to accurately evaluate one’s own work relative to others' evaluation, is a common concern in social systems of knowledge creation where participants act as both creators and evaluators. Theories of social norming hold that individual’s self-evaluation miscalibration diminishes over multiple iterations of creator-evaluator interactions and shared understanding emerges. This paper explores intersubjectivity and the longitudinal dynamics of miscalibration between creators' and evaluators' assessments in IT-enabled social knowledge creation and refinement systems. Using Latent Growth Modeling, we investigated dynamics of creator’s assessments of their own knowledge artifacts compared to peer evaluators' to determine whether miscalibration attenuates over multiple interactions. Contrary to theory, we found that creator’s self-assessment miscalibration does not attenuate over repeated interactions. Moreover, depending on the degree of difference, we found self-assessment miscalibration to amplify over time with knowledge artifact creators' diverging farther from their peers' collective opinion. Deeper analysis found no significant evidence of the influence of bias and controversy on miscalibration. Therefore, relying on social norming to correct miscalibration in knowledge creation environments (e.g., social media interactions) may not function as expected.",
keywords = "Intersubjectivity, Knowledge artifacts, Latent classes, Longitudinal analysis, Miscalibration, Peer-evaluation",
author = "Dmytro Babik and Rahul Singh and Xia Zhao and Ford, {Eric W.}",
year = "2015",
month = "7",
day = "19",
doi = "10.1007/s10796-015-9586-x",
language = "English (US)",
journal = "Information Systems Frontiers",
issn = "1387-3326",
publisher = "Springer Netherlands",

}

TY - JOUR

T1 - What you think and what I think

T2 - Studying intersubjectivity in knowledge artifacts evaluation

AU - Babik, Dmytro

AU - Singh, Rahul

AU - Zhao, Xia

AU - Ford, Eric W.

PY - 2015/7/19

Y1 - 2015/7/19

N2 - Miscalibration, the failure to accurately evaluate one’s own work relative to others' evaluation, is a common concern in social systems of knowledge creation where participants act as both creators and evaluators. Theories of social norming hold that individual’s self-evaluation miscalibration diminishes over multiple iterations of creator-evaluator interactions and shared understanding emerges. This paper explores intersubjectivity and the longitudinal dynamics of miscalibration between creators' and evaluators' assessments in IT-enabled social knowledge creation and refinement systems. Using Latent Growth Modeling, we investigated dynamics of creator’s assessments of their own knowledge artifacts compared to peer evaluators' to determine whether miscalibration attenuates over multiple interactions. Contrary to theory, we found that creator’s self-assessment miscalibration does not attenuate over repeated interactions. Moreover, depending on the degree of difference, we found self-assessment miscalibration to amplify over time with knowledge artifact creators' diverging farther from their peers' collective opinion. Deeper analysis found no significant evidence of the influence of bias and controversy on miscalibration. Therefore, relying on social norming to correct miscalibration in knowledge creation environments (e.g., social media interactions) may not function as expected.

AB - Miscalibration, the failure to accurately evaluate one’s own work relative to others' evaluation, is a common concern in social systems of knowledge creation where participants act as both creators and evaluators. Theories of social norming hold that individual’s self-evaluation miscalibration diminishes over multiple iterations of creator-evaluator interactions and shared understanding emerges. This paper explores intersubjectivity and the longitudinal dynamics of miscalibration between creators' and evaluators' assessments in IT-enabled social knowledge creation and refinement systems. Using Latent Growth Modeling, we investigated dynamics of creator’s assessments of their own knowledge artifacts compared to peer evaluators' to determine whether miscalibration attenuates over multiple interactions. Contrary to theory, we found that creator’s self-assessment miscalibration does not attenuate over repeated interactions. Moreover, depending on the degree of difference, we found self-assessment miscalibration to amplify over time with knowledge artifact creators' diverging farther from their peers' collective opinion. Deeper analysis found no significant evidence of the influence of bias and controversy on miscalibration. Therefore, relying on social norming to correct miscalibration in knowledge creation environments (e.g., social media interactions) may not function as expected.

KW - Intersubjectivity

KW - Knowledge artifacts

KW - Latent classes

KW - Longitudinal analysis

KW - Miscalibration

KW - Peer-evaluation

UR - http://www.scopus.com/inward/record.url?scp=84937154496&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84937154496&partnerID=8YFLogxK

U2 - 10.1007/s10796-015-9586-x

DO - 10.1007/s10796-015-9586-x

M3 - Article

AN - SCOPUS:84937154496

JO - Information Systems Frontiers

JF - Information Systems Frontiers

SN - 1387-3326

ER -