A Conjoint Analysis Framework for Evaluating User Preferences in Machine Translation

Despite much research on machine translation (MT) evaluation, there is surprisingly little work that directly measures users' intuitive or emotional preferences regarding different types of MT errors. However, the elicitation and modeling of user preferences is an important prerequisite for res...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:Machine translation : MT. - 1998. - 28(2014), 1 vom: 01. März, Seite 1-17
1. Verfasser: Kirchhoff, Katrin (VerfasserIn)
Weitere Verfasser: Capurro, Daniel, Turner, Anne M
Format: Aufsatz
Sprache:English
Veröffentlicht: 2014
Zugriff auf das übergeordnete Werk:Machine translation : MT
Schlagworte:Journal Article evaluation machine translation preference elicitation user modeling
LEADER 01000caa a22002652 4500
001 NLM236895737
003 DE-627
005 20250216205002.0
007 tu
008 231224s2014 xx ||||| 00| ||eng c
028 5 2 |a pubmed25n0789.xml 
035 |a (DE-627)NLM236895737 
035 |a (NLM)24683295 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Kirchhoff, Katrin  |e verfasserin  |4 aut 
245 1 2 |a A Conjoint Analysis Framework for Evaluating User Preferences in Machine Translation 
264 1 |c 2014 
336 |a Text  |b txt  |2 rdacontent 
337 |a ohne Hilfsmittel zu benutzen  |b n  |2 rdamedia 
338 |a Band  |b nc  |2 rdacarrier 
500 |a Date Revised 21.10.2021 
500 |a published: Print 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Despite much research on machine translation (MT) evaluation, there is surprisingly little work that directly measures users' intuitive or emotional preferences regarding different types of MT errors. However, the elicitation and modeling of user preferences is an important prerequisite for research on user adaptation and customization of MT engines. In this paper we explore the use of conjoint analysis as a formal quantitative framework to assess users' relative preferences for different types of translation errors. We apply our approach to the analysis of MT output from translating public health documents from English into Spanish. Our results indicate that word order errors are clearly the most dispreferred error type, followed by word sense, morphological, and function word errors. The conjoint analysis-based model is able to predict user preferences more accurately than a baseline model that chooses the translation with the fewest errors overall. Additionally we analyze the effect of using a crowd-sourced respondent population versus a sample of domain experts and observe that main preference effects are remarkably stable across the two samples 
650 4 |a Journal Article 
650 4 |a evaluation 
650 4 |a machine translation 
650 4 |a preference elicitation 
650 4 |a user modeling 
700 1 |a Capurro, Daniel  |e verfasserin  |4 aut 
700 1 |a Turner, Anne M  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t Machine translation : MT  |d 1998  |g 28(2014), 1 vom: 01. März, Seite 1-17  |w (DE-627)NLM098197967  |x 0922-6567  |7 nnns 
773 1 8 |g volume:28  |g year:2014  |g number:1  |g day:01  |g month:03  |g pages:1-17 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 28  |j 2014  |e 1  |b 01  |c 03  |h 1-17