Two weeks ago, OpenAI announced a language model called GPT-2 and stated that it would not release the full version because of the risks involved. It published a much smaller version, a research paper and a set of sample texts, and kept the large model private.
In our long experience following technology companies, we do not recall another case of a company announcing a product and then declining to release it because it works too well.
The most widely shared sample is a news story about scientists discovering a herd of unicorns in a remote valley in the Andes, unicorns that speak perfect English. The premise is absurd. The text, however, is grammatical and assured, and it reads like a genuine wire story for several paragraphs before it loses coherence.
On first reading, the example seems amusing. On closer reading, its implications are serious.
What is GPT-2?
Beyond the headlines, GPT-2 is a model trained to predict the next word. OpenAI trained it on text from about eight million web pages and allowed it to learn the patterns. The largest version has 1.5 billion parameters. A parameter is, roughly, one of the internal settings the model adjusts as it learns, and 1.5 billion is a very large number of them.
Given an opening sentence, the model continues the text: news, fiction, reviews, recipes, or any other genre it is directed toward. It has no way of knowing whether any of it is true. It only knows what usually comes next.
OpenAI's own article, Better Language Models and Their Implications, is short and worth reading. It identifies the concerns: fake news articles, impersonation, automated spam and fake content on social media. It also states that the decision will be reviewed in about six months.
A number of people in the AI field have dismissed the decision as a publicity strategy. That may be partly accurate, and it has certainly generated publicity. Nevertheless, someone raised the question publicly before release, which is uncommon in this industry.
How can text be fluent and wrong at the same time?
The significant point is not that a machine can write. Spam bots have produced low-quality text for years, and it is easy to recognize.
GPT-2 is different. It writes well while being completely wrong in the same paragraph, with no change in tone. When a person bluffs, there are usually signs: the sentences become vague, hedged or defensive. GPT-2 shows no such signs. The unicorn story is as fluent when it is nonsensical as when it seems plausible.
Much of our professional work has involved building websites and helping businesses with their content. A large part of that work consisted of demonstrating that good writing builds trust. Clear, confident, well-organised text persuades readers that the author knows the subject.
A machine can now reproduce the appearance of good writing on demand. As a result, the appearance of good writing will soon carry much less meaning.
If anyone can generate words, judgement becomes the skill.
This is the central point for small business owners.
What could inexpensive text mean for small business content?
At present, none of this is available to the general public in a browser. The small model is available to people who can run code; the large one is not. For most business owners, it remains a headline and an unusual story about unicorns.
We do not expect this situation to last. Technologies of this kind tend to become less expensive and easier to use faster than anticipated. The following is our assessment of what will happen once they do.
Why will the first wave be low-quality content?
Fake reviews, blog posts with no substance written by the thousands for search engines, and comment sections filled with accounts that seem almost human. Content farms already pay writers very little to produce filler. A machine that does it at no cost will be very attractive to precisely the wrong users.
Reviews are the greatest concern. A small tourism business depends on its TripAdvisor and Google reviews. If it becomes easy to generate a hundred credible five-star reviews, or a hundred credible one-star reviews for a competitor further along the beach, the entire system becomes less reliable. These platforms will need to become much better at detecting such content.
Why will sources matter more than sentences?
If words are inexpensive, the facts behind them are what hold value. Where did this figure come from? Who said this? Can it be verified? A local business that shows real photos, names real people and links to the sources of its information is doing something a generator cannot imitate for long.
Why will a genuine voice stand out?
Generated text tends toward the average, because it works by predicting the most probable next word. The result is smooth, balanced and somewhat bland. A writer who holds a real opinion, who knows a particular region well, and who will write "I tried this and it didn't work" produces text that reads differently. Readers tend to perceive this difference, even if they cannot explain it.
The same applies to translation and multilingual work. A machine can convey the words of a menu from one language to another, but it does not know how local people talk about their own place. That local knowledge is the part of bilingual copywriting we do not expect a model to take over in the near future.
What practices do we recommend now?
There is nothing to purchase or install at this stage. However, several practices are worth adopting now.
Read more carefully. When online content is perfectly smooth and the source of its claims is unclear, treat it as an unverified account: possibly true, but requiring verification.
Include sources in what you write, even when it requires additional effort: links, names, dates, and photos you actually took.
Remain open to the benefits. If these tools become effective and affordable, some of the more routine parts of writing, such as first drafts, rewording and summaries, could become much faster. We would use them for that purpose, with the condition that a person reviews every word before it is published under a business's name.
A single fabricated story about unicorns is not the main concern. The greater risk is a gradual flood of plausible-looking text that makes all content somewhat harder to trust.