Enhancing biomedical word embeddings by retrofitting to verb clusters

Billy CHIU; Simon BAKER; Martha PALMER; Anna KORHONEN

doi:10.18653/v1/W19-5014

Enhancing biomedical word embeddings by retrofitting to verb clusters

Billy CHIU, Simon BAKER, Martha PALMER, Anna KORHONEN

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Research › peer-review

Abstract

Verbs play a fundamental role in many biomed-ical tasks and applications such as relation and event extraction. We hypothesize that performance on many downstream tasks can be improved by aligning the input pretrained embeddings according to semantic verb classes.In this work, we show that by using semantic clusters for verbs, a large lexicon of verbclasses derived from biomedical literature, weare able to improve the performance of common pretrained embeddings in downstream tasks by retrofitting them to verb classes. We present a simple and computationally efficient approach using a widely-available “off-the-shelf” retrofitting algorithm to align pretrained embeddings according to semantic verb clusters. We achieve state-of-the-art results on text classification and relation extraction tasks.

Original language	English
Title of host publication	Proceedings of the 18th BioNLP Workshop and Shared Task
Editors	Dina DEMNER-FUSHMAN, Kevin Bretonnel COHEN, Sophia ANANIADOU, Junichi TSUJII
Publisher	Association for Computational Linguistics (ACL)
Pages	125–134
Number of pages	10
ISBN (Electronic)	9781950737284
DOIs	https://doi.org/10.18653/v1/W19-5014
Publication status	Published - Aug 2019
Externally published	Yes
Event	18th BioNLP Workshop and Shared Task - Florence, Italy Duration: 1 Aug 2019 → 1 Aug 2019 https://aclanthology.org/volumes/W19-50/

Conference

Conference	18th BioNLP Workshop and Shared Task
Country/Territory	Italy
City	Florence
Period	1/08/19 → 1/08/19
Internet address	https://aclanthology.org/volumes/W19-50/

Bibliographical note

This work is supported by the Medical Research Council [grant number MR/M013049/1], the ERC Consolidator Grant LEXICAL [grant number:648909], the ESRC Doctoral Fellowship [grant number: ES/J500033/1] and the Defense Advanced Research Projects Agency [DARPA 15-18-CwCFP-032].

We would like to thank our reviewers for their constructive feedback. We are very grateful to Tyler Griffiths for helping with proofreading and typesetting this paper.

Access to Document

10.18653/v1/W19-5014Licence: CC BY

Cite this

@inproceedings{22212fd784874c12ab978b3b511b1386,

title = "Enhancing biomedical word embeddings by retrofitting to verb clusters",

abstract = "Verbs play a fundamental role in many biomed-ical tasks and applications such as relation and event extraction. We hypothesize that performance on many downstream tasks can be improved by aligning the input pretrained embeddings according to semantic verb classes.In this work, we show that by using semantic clusters for verbs, a large lexicon of verbclasses derived from biomedical literature, weare able to improve the performance of common pretrained embeddings in downstream tasks by retrofitting them to verb classes. We present a simple and computationally efficient approach using a widely-available “off-the-shelf” retrofitting algorithm to align pretrained embeddings according to semantic verb clusters. We achieve state-of-the-art results on text classification and relation extraction tasks.",

author = "Billy CHIU and Simon BAKER and Martha PALMER and Anna KORHONEN",

note = "This work is supported by the Medical Research Council [grant number MR/M013049/1], the ERC Consolidator Grant LEXICAL [grant number:648909], the ESRC Doctoral Fellowship [grant number: ES/J500033/1] and the Defense Advanced Research Projects Agency [DARPA 15-18-CwCFP-032]. We would like to thank our reviewers for their constructive feedback. We are very grateful to Tyler Griffiths for helping with proofreading and typesetting this paper.; 18th BioNLP Workshop and Shared Task ; Conference date: 01-08-2019 Through 01-08-2019",

year = "2019",

month = aug,

doi = "10.18653/v1/W19-5014",

language = "English",

pages = "125–134",

editor = "Dina DEMNER-FUSHMAN and COHEN, {Kevin Bretonnel} and Sophia ANANIADOU and Junichi TSUJII",

booktitle = "Proceedings of the 18th BioNLP Workshop and Shared Task",

publisher = "Association for Computational Linguistics (ACL)",

address = "United States",

url = "https://aclanthology.org/volumes/W19-50/",

}

CHIU, B, BAKER, S, PALMER, M & KORHONEN, A 2019, Enhancing biomedical word embeddings by retrofitting to verb clusters. in D DEMNER-FUSHMAN, KB COHEN, S ANANIADOU & J TSUJII (eds), Proceedings of the 18th BioNLP Workshop and Shared Task. Association for Computational Linguistics (ACL), pp. 125–134, 18th BioNLP Workshop and Shared Task, Florence, Italy, 1/08/19. https://doi.org/10.18653/v1/W19-5014

Enhancing biomedical word embeddings by retrofitting to verb clusters. / CHIU, Billy; BAKER, Simon; PALMER, Martha et al.
Proceedings of the 18th BioNLP Workshop and Shared Task. ed. / Dina DEMNER-FUSHMAN; Kevin Bretonnel COHEN; Sophia ANANIADOU; Junichi TSUJII. Association for Computational Linguistics (ACL), 2019. p. 125–134.

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Research › peer-review

TY - GEN

T1 - Enhancing biomedical word embeddings by retrofitting to verb clusters

AU - CHIU, Billy

AU - BAKER, Simon

AU - PALMER, Martha

AU - KORHONEN, Anna

N1 - This work is supported by the Medical Research Council [grant number MR/M013049/1], the ERC Consolidator Grant LEXICAL [grant number:648909], the ESRC Doctoral Fellowship [grant number: ES/J500033/1] and the Defense Advanced Research Projects Agency [DARPA 15-18-CwCFP-032]. We would like to thank our reviewers for their constructive feedback. We are very grateful to Tyler Griffiths for helping with proofreading and typesetting this paper.

PY - 2019/8

Y1 - 2019/8

N2 - Verbs play a fundamental role in many biomed-ical tasks and applications such as relation and event extraction. We hypothesize that performance on many downstream tasks can be improved by aligning the input pretrained embeddings according to semantic verb classes.In this work, we show that by using semantic clusters for verbs, a large lexicon of verbclasses derived from biomedical literature, weare able to improve the performance of common pretrained embeddings in downstream tasks by retrofitting them to verb classes. We present a simple and computationally efficient approach using a widely-available “off-the-shelf” retrofitting algorithm to align pretrained embeddings according to semantic verb clusters. We achieve state-of-the-art results on text classification and relation extraction tasks.

AB - Verbs play a fundamental role in many biomed-ical tasks and applications such as relation and event extraction. We hypothesize that performance on many downstream tasks can be improved by aligning the input pretrained embeddings according to semantic verb classes.In this work, we show that by using semantic clusters for verbs, a large lexicon of verbclasses derived from biomedical literature, weare able to improve the performance of common pretrained embeddings in downstream tasks by retrofitting them to verb classes. We present a simple and computationally efficient approach using a widely-available “off-the-shelf” retrofitting algorithm to align pretrained embeddings according to semantic verb clusters. We achieve state-of-the-art results on text classification and relation extraction tasks.

U2 - 10.18653/v1/W19-5014

DO - 10.18653/v1/W19-5014

M3 - Conference paper (refereed)

SP - 125

EP - 134

BT - Proceedings of the 18th BioNLP Workshop and Shared Task

A2 - DEMNER-FUSHMAN, Dina

A2 - COHEN, Kevin Bretonnel

A2 - ANANIADOU, Sophia

A2 - TSUJII, Junichi

PB - Association for Computational Linguistics (ACL)

T2 - 18th BioNLP Workshop and Shared Task

Y2 - 1 August 2019 through 1 August 2019

ER -

Enhancing biomedical word embeddings by retrofitting to verb clusters

Abstract

Conference

Bibliographical note

Access to Document

Fingerprint

Cite this