Two days ago, OpenAI released GPT-4, barely three and a half months after ChatGPT became available. Many expected a modest incremental update; the change is considerably more significant.
The most widely shared moment from the launch livestream shows Greg Brockman photographing a website sketched by hand on a notepad, and the model converting it into working code. The result was rudimentary, but functional. For a team that has spent years building websites for clients, the demonstration was particularly notable.
It was not the only AI announcement that Tuesday. Anthropic opened access to its Claude model the same day, and Google announced AI writing features for Gmail and Docs, making it an exceptionally active day for technology news.
How does GPT-4 improve on GPT-3.5?
The difference is apparent within the first ten minutes for anyone who has used ChatGPT on GPT-3.5 since December. GPT-4 follows long instructions without omitting half of them, maintains coherence across longer conversations, and can reason through a problem step by step, generally reaching the correct conclusion.
OpenAI's headline figure concerns exams. In its GPT-4 research announcement, the company reports that GPT-4 scored around the top 10% of test takers on a simulated bar exam, whereas GPT-3.5 scored around the bottom 10%. This result has attracted considerable attention. For a small business in Quebec, however, the bar exam matters far less than the model's ability to write a competent email in French. OpenAI also administered a large multiple-choice test translated into a couple of dozen languages, and in most of those languages GPT-4 outperformed GPT-3.5's results in English. This finding is the more relevant one for our clients.
Two further developments emerged this week. Microsoft confirmed that the new Bing chat has been running on GPT-4 from the outset. Within ChatGPT, GPT-4 is available only to paying Plus subscribers, at US$20 a month, with a limit on the number of messages that can be sent every few hours.
Most people trying ChatGPT this spring will therefore still be using the older model and will not see these improvements. Some confusion is to be expected: users will compare what ChatGPT can and cannot do without realizing they are using two different models, with one finding it ineffective for a task the other finds easy.
Before accepting any assessment of "AI," we recommend asking which model was used. This question is likely to become increasingly important as new versions accumulate.
What does multimodal AI mean in practice?
A model that can read images as well as text. "Multimodal" simply means that the model accepts more than one type of input; in GPT-4's case, text and images together.
At present, however, very few users can access the image capability. OpenAI has demonstrated it and is testing it with a single partner, Be My Eyes, an app for blind and low-vision people. All other users have access to text only, and no date has been announced for broader availability.
The direction is nonetheless clear. For a small business, the ability to show a model something rather than describe it opens many possibilities:
- a photo of a handwritten price list to be typed up
- a screenshot of a website error that is difficult to interpret
- a picture of a government form to be completed
- a chart from your analytics to be explained in plain language
If the capability proves reliable, it will likely seem remarkable for about a month and then become routine: users will simply take a photo and ask a question.
How well it will handle a blurry photo, small print or a crowded image is not yet known outside a small group of testers. We anticipate that it may share a tendency observed on the text side, occasionally describing what it expects to see rather than what is actually present. This remains to be confirmed.
Does GPT-4 still invent citations?
Yes. This is the most important point to retain.
GPT-4 is more convincing, which makes its errors harder to detect. OpenAI's own research page states clearly that it is still not fully reliable and that it can "hallucinate" facts and make reasoning errors. Most readers will focus on the exam scores and overlook that paragraph.
The most common failure involves sources. When asked for studies, articles or court decisions, ChatGPT produces a well-organized list with authors, years and titles. Some are real; others are entirely fabricated, formatted exactly like the real ones, with no visible indication of the difference. Nothing published this week indicates that GPT-4 has resolved this issue. There may be fewer fabricated sources, but not none.
We expect this to lead to serious consequences for some users, such as a student, a journalist or someone filing documents in court. When this occurs, it will be because the format was trusted without the content being verified.
Our rule is simple: when a model provides a citation, a statistic or a quotation, open the source yourself. If it cannot be found, it should be considered non-existent until proven otherwise.
What should a small business take from GPT-4?
In our assessment, this release marks the point at which these tools move from novelty to something around which real work can be organized. Many of the assistants and automations that small businesses adopt are likely to trace back to this release.
The more useful lesson, however, is a cautionary one. Each new model will be described as "better," and each is also likely to continue making errors that appear correct. Improvement encourages users to lower their guard, and that is when false information gets published under a business's name.
We therefore recommend adopting new capabilities as they become available: provide the model with unstructured documents, work through problems with it, and use images once that feature opens. For businesses unsure where these tools fit, a discussion with a provider of AI consulting services is worthwhile. A productive first conversation usually begins by identifying what should not be delegated to the model.