A paper titled "Attention Is All You Need" was posted on arXiv on June 12. Its eight authors are researchers, most of them at Google. The paper is short: considerable mathematics, a few diagrams of boxes and arrows, and a focus on machine translation, from English to German and from English to French.
We read the abstract and reviewed the rest of the paper. The full mathematical detail is the domain of specialists, but the implications are significant, and very few people in marketing appear to be aware of the paper. This summer's marketing conversation centres on Facebook video, Instagram Stories and Messenger chatbots, not on a translation paper.
That gap deserves attention, and this article explains why in plain terms.
What does the Transformer paper actually propose?
A design that reads a whole sentence at once instead of word by word. The following is a non-specialist summary.
Until now, the best language systems have read text more or less one word at a time, in order, carrying a form of running memory from one word to the next. The approach works, but it is slow to train, because step ten cannot easily be completed before step nine. The memory also weakens over long sentences: by the end of a long paragraph, the beginning is partly lost.
The paper proposes a design called the Transformer, which abandons word-by-word reading. Instead, every word in a sentence examines every other word simultaneously and determines which ones matter for its meaning. That is the "attention" mechanism. In "the diver checked her gauge before she descended," the model can link "she" to "diver" directly, without carrying that link through every intervening word.
According to the paper, two results follow. The translations score better on the authors' tests than the previous best systems. And because the model processes everything at once, it can be trained in parallel on a large amount of hardware, much more quickly. The larger version was trained in a few days on eight graphics cards, which is fast for this type of work.
Better quality and lower cost at the same time: in our experience, that combination tends to spread.
Why does a translation paper matter to a small business?
Because translation is already part of daily operations for many small businesses. Guests at our dive centre on Nusa Ceningan write in English, German, French, Dutch, Chinese and Indonesian, and many small tourism businesses handle the same mix. In this context, translation is not an abstract research problem; it is the inbox.
Google switched Translate to a neural system last fall, and daily users noticed immediately. The output went from frequently amusing to largely usable, almost overnight. Errors remain, but the tool is now reliable enough to grasp the essence of an email in German before replying, which was not the case two years ago.
This paper appears to be the next step in the same direction. If it reaches Google's products, which is likely sooner or later, translation in browsers and on phones will improve again.
Could the approach extend beyond translation?
Possibly. The researchers claim results only in translation. In their conclusion, they state an intention to test the approach on other types of input, such as images and audio, without further promises.
Translation, however, consists of taking a text, understanding it and writing a new text with the same meaning. A system that excels at this task is developing a general competence in language: summarizing, answering questions and possibly writing. This is a projection rather than a certainty, but the trend suggests that the idea will spread well beyond translation benchmarks.
Why is the marketing industry overlooking this paper?
For three reasons. First, it is a research paper. Marketing professionals follow product launches, not arXiv, which is understandable; until recently, we did not follow arXiv either.
Second, this year's AI story in marketing is chatbots. Facebook opened Messenger to bots last year, and many agencies now sell decision-tree bots that require users to choose among four buttons. Most of them perform poorly, and anyone who has used one knows the experience of searching for a way to reach a person. These bots lead many small business owners to perceive "AI" as a gimmick, which risks causing them to dismiss genuine progress when it arrives.
Third, it is difficult to look at a translation score and imagine a tool that would be used at a desk. The paper does not ask readers to do so; the connection requires interpretation.
Where do major technological changes begin?
In research papers, not in product launches. This is the main point we would like small business owners to retain.
By the time a technology reaches a product launch with a polished logo, the change has already happened. It took place in a laboratory, often years earlier, in a document most people will never read. At the I/O conference in May, Google's CEO stated that the company is moving from "mobile first" to "AI first." Papers like this one show what that statement means in practice.
Most research leads nowhere, while some of it brings significant change. Distinguishing between the two is a skill, and nobody is consistently right. The ultimate impact of this paper remains to be seen; it is, however, one we will continue to monitor closely.
Is the paper worth reading?
At least the first page. The paper is available free of charge on arXiv. The abstract is a single readable paragraph; the remainder quickly becomes technical. A brief review is still worthwhile, if only to see what the beginning of an important development can look like: tables, equations and a diagram of stacked boxes, with nothing announcing its significance.
For a small business, the most useful habit is simpler than reading research papers: identify which part of the week consists of language work, such as answering emails, translating, writing descriptions and replying to reviews. That is where tools are most likely to improve first. When they do, businesses will need to know what to adopt and what to disregard, which is the purpose of the practical side of AI consulting.
Four-button chatbots are not required.