The Next Probable Word: Generative Text Tools
Image created by the author using the Stable Diffusion text-to-image generation tool.
Generative AI has touched a cultural nerve. Every day there’s a fresh wave of stories, articles, and opinion pieces on the latest advances in generative AI. The writers discuss what the current tools can and can't (yet) do, and what the emergence of generative AI means—for knowledge workers of all kinds, students, teachers, creatives, and everyone else who works with words, images, video, numbers, music or other types of digital content. It's impossible to keep up with it all...
In part, this often emotionally charged response to the emergence of generative AI is due to the fact that unlike previous advances in machine learning, such as robotics, generative AI technologies pose a potential threat to knowledge workers (aka white-collar workers)—the segment of our economy we've focused on for new job opportunities and overall economic growth.
While more and more of us are learning how to think and express ourselves with images, words are still the dominant medium for developing and expressing ideas in knowledge work. The latest text generation tools, such as OpenAI’s GPT-3 and ChatGPT platforms, are forcing us to rethink our assumptions about the processes we use to develop and express ideas and the value of the written content we produce. Even the meaning of the word "content" is increasingly ambiguous.*
This is the third article in my series on generative AI.** I can’t think of a more important current topic for a newsletter focused on “tools for a productive creative life.” The frenzy over generative AI will fade, as all stories do in our headline driven news environment, but generative AI is an emerging technology. We don’t know its full potential, its limitations, or the real dangers it poses. What we do know is that thoughtful discussion and debate on these important topics requires a shared understanding of the technologies themselves.
GPT-3
At the moment, the two most widely used and discussed text generation systems are OpenAI's Generative Pre-trained Transformer 3 (GPT-3) and ChatGPT platforms. These are not the only text generation systems currently available, but they are the systems in the spotlight. That may change when Google’s highly anticipated LaMDA platform is released later this year.
GPT-3 and ChatGPT are similar, but they are not the same. GPT-3 is the largest neural network released to date. It’s over 100 times larger than its predecessor GPT-2. The GPT-3 language model was trained on a massive amount of text data from the Internet, including petabytes of web pages collected over 8 years of web crawling, millions of Wikipedia pages, two large collections of digitized books in the public domain, and several other specialized repositories of text data.
Unlike symbolic (logical) AI systems that use knowledge bases and rule-sets to make intelligent decisions, GPT-3 (and other similar generative AI systems) use statistical probability*** techniques to recognize patterns, build clusters of related patterns, and create unique responses. GPT-3 has over 175 million trained parameters that control how the model processes input data and produces responses. Each parameter is an equation the system has trained to evaluate the strength of the connection between specific words. Using the text strings in the dataset (the phrases, sentences, and paragraphs) the system uses the feedback from the parameters to form billions of connections between trillions of words.
The massive size of the training dataset, the number of parameters in the model, and quality of the tools used to train the system all contribute to its ability to produce responses that read like they were written by a human. It’s important to understand that GPT-3’s responses are not canned. The system doesn’t have a database of pre-written responses and it’s not plagiarizing existing content in its training dataset. Instead, it uses statistical probability to conceptualize and to produce unique responses.
For example, GPT-3 doesn’t know what a poem is, it just knows about the language patterns that are associated with poems, such as image-rich phrases, emotionally charged words, and rhyming end-words in regular patterns—all of which it learned for the content in its training data that was identified as poetry. It doesn’t have a conceptual understanding of poetry or poems, just a cluster of statistically weighted attributes that taken together probably add up to a poem. When asked to write a poem, it continuously tests the string of words it’s putting together against the patterns and attributes in its cluster of things that are “poem-like.” A string of words with a high enough number of those highly weighted “poem” attributes is probably a poem.
If you want to try out GPT-3 for yourself, you can use the free OpenAI GPT-3 Playground. But you’ll probably have a better experience if you use it on site, or in an application that's integrated GPT-3 for a specific purpose. For example, the popular productivity tool Notion now includes an integrated GPT-3 powered writing assistant. Their AI assistant helps you brainstorm ideas, write a blog post, develop an outline, create a story and more:
Notion’s new note page design featuring their GPT-3 powered AI writing assist tool
Here’s the initial response I received when I clicked on “Outline” and asked AI assist to create an outline for an article on the invention of the ballpoint pen:
Sample outline produced by Notion’s AI assist feature
It’s a great start!
The pop-up menu at the bottom of the response enables you to continue writing with AI assist or try again if you’re not happy with the response.
Notion also uses GPT-3 to help you improve your writing, summarize your notes, find action items, and translate your text.
Notion’s robust implementation of GPT-3 offers a wide range of tools for creating, editing, and manipulating text
Notion has done a great job of integrating GPT-3 into its application. But they aren’t the only ones. Hundreds of other application developers, services, and organizations across a wide variety of domains have integrated GPT-3. That includes writing and note taking tools such as Craft, Lex, Jasper, Ryter, WordPlay, and Mem to name just a few. The popular graphic design tool Canvais also using GPT-3 to power their new Magic Write feature.
Chances are good that one or more of the applications or services you’re currently using has integrated GPT-3, possibly in the application or service itself, or in their various support functions. The productivity gains and cost savings GPT-3 enables are hard to resist.
GPT-3 has some limitations and issues that you should be aware of:
The dataset used to train GPT-3 doesn’t include material published after mid-2021. It can’t accurately answer questions about recent events.
The training dataset contains biases—some known and some that have become apparent as more people use the system.
The system occasionally “hallucinates”—it fails to properly interpret the prompt or goes down a rabbit hole and produces a response that includes some obviously incorrect information or is flat-out completely wrong.
It’s occasionally overwhelmed by the number of people trying to use it and is temporarily unavailable or very slow to respond.
OpenAI is aware of these limitations and issues and is addressing them.
As good as GPT-3 is, recognize that you’re interacting with an emerging technology. Its failings are as instructive as its successes.
ChatGPT
GPT-3 is a general-purpose language model designed to support a wide range of natural language tasks. ChatGPT has a narrower focus: it’s designed to enable and sustain natural feeling human-machine conversations. Released at the end of 2022, ChatGPT uses an updated version of GPT-3 (GPT-3.5) that includes a small amount of more recent content from the web and a new training dataset of conversational text. It does a better job of remembering the previous prompts in a conversation and produces more “human-like” responses than GPT-3. ChatGPT is also faster than GPT-3, which also helps sustain the illusion that you’re conversing with another human.
But the most significant innovation is ChatGPT’s user interface—it makes it easy and fun for users without any technical knowledge or expertise to interact with the system. It’s very similar to the text messaging apps most of us are already using:
The ChatGPT home page
The best way to learn about ChatGPT is to play with it! It's free. OpenAI has just announced ChatGPT Plus—a new paid subscription option that promises access to the system even during peak usage times, faster response times, and priority access to new features. ChatGPT Plus costs $20 a month.****
You’ll need to create an account on the OpenAI site to use ChatGPT. If you already created an OpenAI account to play with DALL·E, you can use the same account to play with ChatGPT. Once you’ve successfully created your account and logged in, you’ll see the following ChatGPT shown above.
To start a conversation with ChatGPT, type a question into the white prompt entry box at the bottom of the window and click on the paper airplane icon on the right side of the box to send your question to ChatGPT.
Type your question into the prompt entry box at the bottom of the window
In a few seconds ChatGPT will respond. The system is designed to produce natural sounding conversational responses.
If you’re satisfied with the response, you can continue the conversation by asking for more information on the same point, asking a related follow-up question, or redirecting the conversation to a new topic. If you’re not satisfied with ChatGPT’s response, you can ask it to regenerate its response.
Continue your chat by asking for more information on the same point, asking a related follow-up question, or asking a new question
The system remembers the previous prompts you submitted in the same conversion, so you can even go back to a topic you’ve drifted away from.
ChatGPT is designed for conversations, which means it doesn't disclose everything it knows at once—that would amount to monopolizing the conversation. Like a good conversation partner, it tells you enough to keep the conversation going, then waits for you to respond. But unlike your conversations with your fellow humans, ChatGPT can't detect your physical reactions, pauses in the conversation's flow, or any other non-verbal cues. It's only cues are in what you write—in your initial prompt and your follow-ups. Here are a few tips from ChatGPT that will help you begin and continue your conversations with the system:
Provide enough detail: The more information you give, the better I can understand what you're looking for.
Avoid repeating questions: Repeating the same question multiple times won't yield different results.
Use proper grammar and spelling: This helps me understand your question better.
Avoid making unrealistic requests: I'm trained to respond based on information available, not to speculate or make assumptions.
Provide context: Give relevant background information
Avoid slang and abbreviations
Check for errors: Proofread your questions before submitting
Be patient: Some questions may require research or take time to answer.
Avoid open-ended questions: Be specific about what you want to know.
Remember, if ChatGPT’s response is vague or not helpful, ask it to provide more detail or generate a new response. You won’t hurt its feelings! Also remember that ChatGPT has inherited most of GPT-3 limitations and issues. It’s more advanced than GPT-3 in some ways, but it’s still an emerging technology.
You can use ChatGPT to write email messages, social media posts, blog posts, essays, poetry, stories, song lyrics, presentations, and so much more. You’ll find links to articles and guides on crafting ChatGPT prompts in the Recommended Resources section below.
The next article in this series will focus on integrating text generation tools into your writing process. I’ll then bring the series to a close with an article on some of the issues raised by the emergence of generative AI (such as copyright concerns, AI biases, and potential job-loss) and what generative AI can teach us about what it means to be human.
Footnotes
* I’ll return to the meaning of the word “content” in the next article in this series.
** The first two articles in this series are:
*** Statistical probability is the branch of mathematics that’s focused on measuring uncertainty and quantifying the likelihood of an event or outcome. GPT-3 uses statistical probability to predict the next most likely word or phrase in a response. It assigns probabilities to each potential outcome based on its training data. The outcome with the highest probability is then selected as the prediction.
**** Click here to join the waitlist or ChatGPT Plus.
**** Here’s the transcript of my conversation with ChatGPT on the First Amendment:
Me: Explain the first amendment of the U.S. Constitution.
ChatGPT: The First Amendment guarantees the freedom of religion, speech, press, assembly, and petition in the United States. It prohibits the federal government from making laws that establish a religion or abridge the freedom of speech, religion, press, assembly, or petition.
Me: Tell me more
ChatGPT: The First Amendment protects individual rights such as the right to express oneself freely, to practice one's religion without government interference, to gather with others for peaceful purposes, and to petition the government for redress of grievances. It is considered a cornerstone of American democracy and has been interpreted by the courts to include freedom of the press, the right to receive information, and the right to associate with others. The amendment has been the basis for many landmark Supreme Court cases that have shaped American society and defined the limits of government power over individual rights.
Me: Who authored the amendment?
ChatGPT: The First Amendment was written by James Madison, one of the Founding Fathers of the United States, as part of the Bill of Rights. The Bill of Rights was added to the Constitution in 1791 to ensure that individual liberties were protected from the federal government. Madison proposed the amendments in response to calls from some states for a bill of rights to be added to the Constitution. The First Amendment, along with the other nine amendments in the Bill of Rights, has since become an integral part of the U.S. Constitution and is considered one of the most important protections of individual liberties in the world.
Me: What are some common misconceptions about the amendment?
ChatGPT:
The First Amendment applies only to government censorship: The amendment applies not only to government censorship but also to private censorship by corporations, universities, and other entities.
The First Amendment guarantees unlimited freedom of speech: The amendment guarantees freedom of speech, but it does not protect all forms of speech. For example, speech that incites violence, constitutes a true threat, or is considered obscenity is not protected.
The First Amendment applies only to American citizens: The amendment applies to all individuals within the United States, regardless of citizenship status.
The First Amendment guarantees the right to say anything, anywhere: The amendment guarantees the right to express oneself, but it does not protect speech in all contexts. For example, speech that creates a clear and present danger or interferes with the rights of others may be limited.
The First Amendment only applies to political speech: The amendment applies to all forms of speech, including political speech, artistic expression, and commercial speech.
These misconceptions are common, but it's important to understand the true scope and limitations of the First Amendment to ensure that individual liberties are protected in practice.
Me: Where can I find more information about the amendment?
ChatGPT:
Government Websites: You can find information about the First Amendment on websites such as the U.S. Constitution Online (usconstitution.net), the National Constitution Center (constitutioncenter.org), and the Library of Congress (loc.gov).
Books: You can find books about the First Amendment and the Bill of Rights at your local library or bookstore. Some popular books include "First Amendment Law in a Nutshell" by Jerome A. Barron and C. Thomas Dienes, and "The First Amendment: Freedom of Speech, Religion, and the Press" by Jamin B. Raskin.
Online Articles: You can find articles about the First Amendment and related topics on websites such as the American Civil Liberties Union (aclu.org), the Electronic Frontier Foundation (eff.org), and the First Amendment Center (firstamendmentcenter.org).
Courses and Workshops: You can also take courses or attend workshops on the First Amendment and related topics at universities, colleges, and community organizations.
Lawyers and Legal Organizations: You can contact a lawyer or a legal organization such as the American Bar Association (americanbar.org) for more information and guidance on the First Amendment and related legal issues.
These resources can help you learn more about the First Amendment, its history, and its impact on American society and the world.
Recommended Resources
The ChatGPT Prompt Book, Alan D. Thompson (Free PDF)
The Art of Writing ChatGPT Prompts for Any User Case, Sara Tamsin, sarahtamsin.com
How to use ChatGPT effectively: A guide to master generative AI, Eray Eliacik, Data Economy
The 100 Best ChatGPT Prompts to Power Your Workflow, gridfiti.com