Machine Translation Post-editing Pricing Models

Our second event in our Machine Translation Workshop Series focused on discussing different approaches to pricing machine translation post-editing (MTPE). The event featured a diverse panel of industry experts and brought together over 400 live viewers from all over the world.

It’s not surprising that MTPE as a topic should attract so much attention.

Data provided by CSA Research shows that the volume of post-editing has increased by approximately 46% between 2016 and 2019. This is driven by strong business incentives, as customers demand cheaper and faster translations. For language service providers (LSPs) this development is especially critical: adopting MT can help them grow three times faster than LSPs without MT.

For linguists MTPE represents a considerable challenge. Many translators view MT skeptically and there is significant resistance to taking on MTPE jobs, which are often seen as less profitable and interesting compared to standard translation. Yet the growing demand for MTPE has made it a reality for many.

The question of how to create pricing models that work for clients, LSPs, and translators is now more important than ever.

To help provide a range of perspectives, our panel brought together speakers representing different parts of the industry.

Our panelists included:

Francesca Arca, Vendor Manager at Acolad
Silvia Ghiazza, Head of Product, Quality and Talent Management at STAR7
Arle Lommel, Senior Analyst at CSA Research
Anna Marjanovics, Freelance Translator & Co-Founder at Unicorn Translations

Throughout the event we engaged participants through interactive polls. Approximately a third of the attendees identified as translators, though other groups were represented as well, including Vendor and Project Managers, and Tech Specialists.

The recording is available on-demand here.

Here are a few highlights from the event:

What is “Machine Translation Post-Editing”?

At its core MTPE is essentially the practice of adapting machine translation output by a human translator.

Although MTPE is still a developing area, there has been some degree of standardization, notably with the approval of ISO 18587:2017 which established some of the key definitions. These include the well-known distinction between so-called “light” post-editing and “full” post-editing, which reflects the amount of effort a linguist is expected to put in the correction of the output.

Responses to our event poll suggested that the vast majority of linguists have encountered this distinction in their work, with only a fifth (22%) never having encountered it. However, exact definitions can differ from business to business, with some LSPs in the past offering as many as five different levels of post-editing. Event participants also noted encountered variants that included “rapid post-editing” or “publishing-grade post-editing”.

Exactly how you define MTPE is important, because different degrees of post-editing will inevitably result in different pricing models.

What makes it difficult to price MTPE?

The main challenge in creating post-editing pricing models is the sometimes unpredictable quality of MT. Although MT engines have improved significantly over the past few years, they can still struggle with specific language pairs and domains, or with poorly written source texts.

What is more, the quality can sometimes vary at segment level within a document. This can make it very difficult to estimate how much time and effort will be required to successfully post-edit the MT.

For LSPs one way of mitigating this unpredictability is by specializing. When STAR7 began offering machine translation post-editing they started by focusing on a specific document type. They created a specific data set to train their engine and then evaluated its performance using a combination of human and automated evaluation metrics. They retrained their engine several times before they were satisfied with its results. Only then were they confident in offering it to customers.

What are the most common pricing models?

For translators there are generally two different pricing models: per-word and per-hour.

A survey carried by Acolad in February 2019 found that among 4,000 translation vendors, 76% had already worked on MTPE projects, with a near even split between word rate (47%) and per-hour pricing (53%).

Our own event poll found that among linguists the per-word pricing model is the clear winner, with 79%. However when asked which model might be most suitable for vendors there was a clear shift away from per word pricing towards alternatives that reflected the actual effort or time taken to complete the post-editing.

CSA Research found that per-word is the preferred pricing model for most businesses. Although per-hour and other effort-based pricing models are attractive for both LSPs and translators, these can be a hard sell for many clients operating on a fixed budget. For businesses it is often preferable to pay a higher upfront cost than to be billed for more than they expected once the work is completed.

Acolad has developed its own pricing model, which was presented in a paper at the 22nd Annual Conference of the European Association for Machine Translation in November 2020. It relies on three tests carried out by machine translation specialists with input from post-editors. These include both automatic and manual evaluation, as well as a real conditional test. The sum of these scores are averaged to create a percentage discount that is applied to the translation word rate.

This process is used for each engine trained at Acolad, so the exact discounts may vary from engine to engine. The discount for any specific engine will however remain the same, unless the engine is retrained. This approach relies on continuous quality monitoring. After each MTPE project, post-editors are asked to fill in a feedback form, which is useful in providing information about the quality of the MT engine, but also can be used to revise the proposed discount with the Project Manager.

What is the future of MTPE?

The approach advocated by Acolad is just one of many. There was some debate on the topic of standardizing the pricing practice for MTPE across the industry. The benefits are clear: all parties involved in a translation project approach it with a clear set of expectations. There was however some doubt that this standardization is immediately desirable. When surveyed, nearly half of the viewers believed that there should be an industry standard for MTPE pricing, but approximately 25% disagreed.

One thing that our panel agreed on is the importance of transparency. As the industry continues to invest more into MTPE, new ideas and technologies are likely to emerge that will lead to creating more sophisticated pricing models.

Although MTPE is often viewed negatively by translators, this sentiment may change in the future. CSA Research anticipates that with the introduction of new MT-driven technologies and work processes the sum total of work in translation will increase. Although traditional written human language services are likely to reduce, much of that effort will shift towards other areas where technology plays a key role.

Arle Lommel compared the position of translators today to what was once true for accountants: in the pre-digital age an accountant was an extension of his calculator, mainly concerned with the addition and subtraction of appropriate values to create reports.

Computers have not made the accountant redundant, but rather freed them from repetitive work and allowed them to focus on more important tasks. It is likely that technological innovations in translation will do the same for translators.