Towards assigning references using semantic, journal and citation relevance

Ding Cheng Li, Hongfang Liu, Christopher Chute, Siddhartha R. Jonnalagadda

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Clinical knowledge systems (CKS) facilitate healthcare practitioners to make the best decisions at the point of care by having access to most recent knowledge. However, because of the overabundance of clinical research articles, it is time consuming to add new content manually to any CKS such as UpToDate and to ensure that it is consistent with evidence. For this reason, many CKS such as Mayo Clinic's AskMayoExpert use expert-based (and not evidence-based) knowledge learned from peer-to-peer communication. The main motivation of this work is to explore methods for assigning references automatically to expert-written content. The system employs information retrieval (IR) techniques to retrieve related sentences from MEDLINE abstracts as evidence candidates and then uses a machine learning model trained with physician's evaluation score on certain journals for re-ranking. Finally, the system utilizes citation counts of individual articles for further refining the ranking. A preliminary evaluation using 59 UpToDate sentences and respective citations as gold standard showed that the median ranking improved 11 folds after adding the journal and citation relevance metrics on the top of baseline that only uses the semantic relevance metric. The system seems promising and ready for trials in real use-case scenarios with experts.

Original languageEnglish (US)
Title of host publicationProceedings - 2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013
Pages499-503
Number of pages5
DOIs
StatePublished - 2013
Externally publishedYes
Event2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013 - Shanghai, China
Duration: Dec 18 2013Dec 21 2013

Other

Other2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013
CountryChina
CityShanghai
Period12/18/1312/21/13

Fingerprint

Information retrieval systems
Refining
Learning systems
Semantics
Communication

Keywords

  • AskMayoExpert
  • citation relevance
  • journal
  • knowledge system
  • semantic
  • UpToDate

ASJC Scopus subject areas

  • Biomedical Engineering

Cite this

Li, D. C., Liu, H., Chute, C., & Jonnalagadda, S. R. (2013). Towards assigning references using semantic, journal and citation relevance. In Proceedings - 2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013 (pp. 499-503). [6732545] https://doi.org/10.1109/BIBM.2013.6732545

Towards assigning references using semantic, journal and citation relevance. / Li, Ding Cheng; Liu, Hongfang; Chute, Christopher; Jonnalagadda, Siddhartha R.

Proceedings - 2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013. 2013. p. 499-503 6732545.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Li, DC, Liu, H, Chute, C & Jonnalagadda, SR 2013, Towards assigning references using semantic, journal and citation relevance. in Proceedings - 2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013., 6732545, pp. 499-503, 2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013, Shanghai, China, 12/18/13. https://doi.org/10.1109/BIBM.2013.6732545
Li DC, Liu H, Chute C, Jonnalagadda SR. Towards assigning references using semantic, journal and citation relevance. In Proceedings - 2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013. 2013. p. 499-503. 6732545 https://doi.org/10.1109/BIBM.2013.6732545
Li, Ding Cheng ; Liu, Hongfang ; Chute, Christopher ; Jonnalagadda, Siddhartha R. / Towards assigning references using semantic, journal and citation relevance. Proceedings - 2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013. 2013. pp. 499-503
@inproceedings{1b9f1c9e592240de9e7d0560c0e14773,
title = "Towards assigning references using semantic, journal and citation relevance",
abstract = "Clinical knowledge systems (CKS) facilitate healthcare practitioners to make the best decisions at the point of care by having access to most recent knowledge. However, because of the overabundance of clinical research articles, it is time consuming to add new content manually to any CKS such as UpToDate and to ensure that it is consistent with evidence. For this reason, many CKS such as Mayo Clinic's AskMayoExpert use expert-based (and not evidence-based) knowledge learned from peer-to-peer communication. The main motivation of this work is to explore methods for assigning references automatically to expert-written content. The system employs information retrieval (IR) techniques to retrieve related sentences from MEDLINE abstracts as evidence candidates and then uses a machine learning model trained with physician's evaluation score on certain journals for re-ranking. Finally, the system utilizes citation counts of individual articles for further refining the ranking. A preliminary evaluation using 59 UpToDate sentences and respective citations as gold standard showed that the median ranking improved 11 folds after adding the journal and citation relevance metrics on the top of baseline that only uses the semantic relevance metric. The system seems promising and ready for trials in real use-case scenarios with experts.",
keywords = "AskMayoExpert, citation relevance, journal, knowledge system, semantic, UpToDate",
author = "Li, {Ding Cheng} and Hongfang Liu and Christopher Chute and Jonnalagadda, {Siddhartha R.}",
year = "2013",
doi = "10.1109/BIBM.2013.6732545",
language = "English (US)",
isbn = "9781479913091",
pages = "499--503",
booktitle = "Proceedings - 2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013",

}

TY - GEN

T1 - Towards assigning references using semantic, journal and citation relevance

AU - Li, Ding Cheng

AU - Liu, Hongfang

AU - Chute, Christopher

AU - Jonnalagadda, Siddhartha R.

PY - 2013

Y1 - 2013

N2 - Clinical knowledge systems (CKS) facilitate healthcare practitioners to make the best decisions at the point of care by having access to most recent knowledge. However, because of the overabundance of clinical research articles, it is time consuming to add new content manually to any CKS such as UpToDate and to ensure that it is consistent with evidence. For this reason, many CKS such as Mayo Clinic's AskMayoExpert use expert-based (and not evidence-based) knowledge learned from peer-to-peer communication. The main motivation of this work is to explore methods for assigning references automatically to expert-written content. The system employs information retrieval (IR) techniques to retrieve related sentences from MEDLINE abstracts as evidence candidates and then uses a machine learning model trained with physician's evaluation score on certain journals for re-ranking. Finally, the system utilizes citation counts of individual articles for further refining the ranking. A preliminary evaluation using 59 UpToDate sentences and respective citations as gold standard showed that the median ranking improved 11 folds after adding the journal and citation relevance metrics on the top of baseline that only uses the semantic relevance metric. The system seems promising and ready for trials in real use-case scenarios with experts.

AB - Clinical knowledge systems (CKS) facilitate healthcare practitioners to make the best decisions at the point of care by having access to most recent knowledge. However, because of the overabundance of clinical research articles, it is time consuming to add new content manually to any CKS such as UpToDate and to ensure that it is consistent with evidence. For this reason, many CKS such as Mayo Clinic's AskMayoExpert use expert-based (and not evidence-based) knowledge learned from peer-to-peer communication. The main motivation of this work is to explore methods for assigning references automatically to expert-written content. The system employs information retrieval (IR) techniques to retrieve related sentences from MEDLINE abstracts as evidence candidates and then uses a machine learning model trained with physician's evaluation score on certain journals for re-ranking. Finally, the system utilizes citation counts of individual articles for further refining the ranking. A preliminary evaluation using 59 UpToDate sentences and respective citations as gold standard showed that the median ranking improved 11 folds after adding the journal and citation relevance metrics on the top of baseline that only uses the semantic relevance metric. The system seems promising and ready for trials in real use-case scenarios with experts.

KW - AskMayoExpert

KW - citation relevance

KW - journal

KW - knowledge system

KW - semantic

KW - UpToDate

UR - http://www.scopus.com/inward/record.url?scp=84894582507&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84894582507&partnerID=8YFLogxK

U2 - 10.1109/BIBM.2013.6732545

DO - 10.1109/BIBM.2013.6732545

M3 - Conference contribution

SN - 9781479913091

SP - 499

EP - 503

BT - Proceedings - 2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE BIBM 2013

ER -