AMTA 2018 | Tutorial | MQM-DQF: A Good Marriage (Translation Quality for the 21st Century)

In the past three years, the language industry has been converging on the use of the MQM-DQF framework for analytic quality evaluation. It emerged from two separate quality-evaluation approaches: the European Commission-funded Multidimensional Quality Metrics (MQM) and the Dynamic Quality Framework (DQF) from TAUS. Harmonized in 2015, the resulting shared hierarchy of error types allows implementers to classify common translation problems and perform comparative analysis of translation quality.

MQM-DQF is currently undergoing a formal standardization process in ASTM F43 and will remain a free and open framework.

Attendees will learn how to apply MQM-DQF to their particular needs, including use in typical MT research scenarios where it can bring consistency and clarity. They will be better prepared to select a quality assessment methodology that is appropriate to their needs and that can help connect the needs of technology developers, users, linguists, and information consumers.

The tutorial will focus on the following topics:

1. A typology of translation quality metrics. This discussion will enable participants to understand how MQM-DQF compares to other quality evaluation approaches and the comparative strengths and weaknesses of them.
2. Overview of MQM-DQF and key features. This detailed overview will highlight how the framework relates to existing standards, the role of translation specifications in evaluating quality, and the approach the specification takes to developing numerical quality scores.
3. Market adoption. This section will cover the tools that have already adopted MQM/DQF and how they apply it.
4. Detailed case studies. The presenters will discuss specific use cases submitted by tutorial participants to explore how they can create a customized MQM-DQF metric.
5. Validity and reliability. This section discusses the importance of determining validity and measuring reliability within a translation quality evaluation system.

Note: The presenters were two of the leads in the harmonization of MQM and DQF and are active in the ongoing standardization effort around the resulting combined approach.

Presenters:

Arle Lommel: (Senior analyst, CSA Research), Alan K. Melby (Chair, LTAC Global)

Target audience:

The target audience includes researchers, developers, and linguists interested in understanding translation quality, ways of assessing it, and the strengths and weaknesses of various approaches.