Skip to Main Content

Generative AI and Large Language Models (LLMs)

Understanding LLMs (e.g., ChatGPT) and the concept, use, and ethics of Generative AI tools and platforms.

Generative AI Primer

What is generative AI?

Generative AI (GAI) is a type of artificial intelligence that can generate text, images, videos, and other content in response to a user prompt, based on its training data.

This guide focuses on text-based AI generators. For information on image and video AI generators, see the Machines and Society guide.

Text-based generative AI: LLMs

Large language model (LLMs) are the foundation of GAI. LLMs are trained on vast amounts of text to understand existing content and generate original content. They function as chatbots, responding to user prompts by processing natural language in a conversational, human-like way. They can perform a variety of language-based tasks, like generating, summarizing, and translating text.

ChatGPT web page with the question posed What are you and what do you do? and a brief answer from ChatGPT.What are some of the LLMs out there?

  • ChatGPT: (OpenAI) Has free and fee-based options

  • Copilot: (Microsoft) Free

  • Gemini: (Google) Free to anyone with a Google account

  • Llama2: (Meta) Free for download

What types of things can ChatGPT and other LLMs do?

When asked what it is capable of doing, this is one way ChatGPT has answered:

  • Text generation: This refers to my ability to generate new text based on a given prompt or topic. I can use my understanding of language and my training data to produce sentences, paragraphs, or even longer pieces of text.
  • Text completion: This involves predicting and generating the most likely words or phrases to complete a given sentence or paragraph based on context.
  • Question-answering: I can answer questions by identifying the most relevant information from my training data and generating an appropriate response.
  • Summarization: I can summarize longer pieces of text into shorter summaries while retaining the main ideas and important details.
  • Text translation: I can translate text from one language to another, with varying levels of accuracy depending on the language pairs.
  • Conversational AI: This refers to my ability to engage in natural language conversations with humans, responding to prompts and questions in a conversational manner.
  • Sentiment analysis: I can analyze text to determine the overall sentiment or emotion expressed within it.
  • Named entity recognition: I can identify and extract named entities such as people, places, and organizations from text.
  • Part-of-speech tagging: I can identify the part of speech (noun, verb, adjective, etc.) of each word in a given sentence.

(ChatGPT, personal communication, March 14, 2023)

How do LLMs work, for example ChatGPT?

What is behind ChatGPT's mystique? This is one way that ChatGPT has responded to the question of how it works:

ChatGPT is a language model that utilizes a range of concepts, architectures, processes, and data inputs to perform its tasks. Here are some of the key elements that enable ChatGPT to work:

  1. Large language model: ChatGPT is a large language model that is designed to process and generate large amounts of natural language text. It is trained on massive amounts of text data, including books, articles, websites, the Common Crawl, Wikipedia, and other sources. These datasets provide the foundational knowledge that ChatGPT draws on when processing input text and generating responses.

  2. Artificial Neural Networks (ANNs): ANNs are mathematical models that simulate the way that the human brain processes information. They are used extensively in ChatGPT to process input text and generate responses.

  3. Natural Language Processing (NLP): This is the use of computational techniques to analyze and understand human language. ChatGPT uses NLP to process input text and generate responses.

  4. Transformer architecture: The transformer architecture is a type of neural network that is used in GPT models. It enables ChatGPT to process input text and generate responses in a way that is both efficient and effective.

  5. Generative Pre-trained Transformer (GPT): ChatGPT is based on GPT, a type of deep learning model specifically designed for natural language processing. GPT is trained on vast amounts of text data to learn the patterns and structures of language, enabling it to generate human-like responses to input prompts.

  6. Transfer learning: ChatGPT is trained on a vast corpus of text data that covers a wide range of topics and styles. Transfer learning enables ChatGPT to transfer knowledge learned in one domain to another related domain, allowing it to generate responses to a diverse range of input prompts.

  7. Reinforcement learning: ChatGPT uses reinforcement learning to improve its performance over time. This involves training it on a dataset of prompts and human-generated responses, and then adjusting its behavior based on feedback from humans.

  8. Attention mechanism: The attention mechanism is a component of the transformer architecture that allows ChatGPT to focus on specific parts of the input text when generating a response. This helps it to generate more accurate and relevant responses.

Overall, ChatGPT utilizes a complex combination of technologies and processes to analyze and understand human language, enabling it to generate responses that are similar to those produced by humans.

(ChatGPT, personal communication, March 14, 2023)

ChatGPT Q&A Examples

Example 1

User question: Can you provide a diverse list of the best books of the 20th century?

ChatGPT answer:  The 20th century produced an incredible amount of great literature across a wide range of genres and styles. Here are some of the best books of the 20th century, in no particular order, that represent a diverse range of voices and perspectives:

  1. "To Kill a Mockingbird" by Harper Lee
  2. "1984" by George Orwell
  3. "One Hundred Years of Solitude" by Gabriel Garcia Marquez
  4. "Beloved" by Toni Morrison
  5. "The Great Gatsby" by F. Scott Fitzgerald
  6. "Invisible Man" by Ralph Ellison
  7. "The Catcher in the Rye" by J.D. Salinger
  8. "The Sound and the Fury" by William Faulkner
  9. "The Grapes of Wrath" by John Steinbeck
  10. "Their Eyes Were Watching God" by Zora Neale Hurston
  11. "Slaughterhouse-Five" by Kurt Vonnegut
  12. "The Bell Jar" by Sylvia Plath
  13. "Things Fall Apart" by Chinua Achebe
  14. "A Passage to India" by E.M. Forster
  15. "The Color Purple" by Alice Walker
  16. "The Sun Also Rises" by Ernest Hemingway
  17. "The Handmaid's Tale" by Margaret Atwood
  18. "Brave New World" by Aldous Huxley
  19. "The Lord of the Rings" by J.R.R. Tolkien
  20. "The Diary of Anne Frank" by Anne Frank

This is by no means an exhaustive list, but it includes some of the most influential and widely-read books of the 20th century.

(ChatGPT, personal communication, March 14, 2023)

Example 2

User question: What is the most efficient cardio-vascular exercise?

ChatGPT answer: There is no single "most efficient" cardio-vascular exercise as different exercises offer different benefits and are suited for different individuals and fitness goals. However, some cardio exercises are generally considered more effective than others for improving cardiovascular fitness and burning calories.

Here are a few examples of popular cardio exercises that are generally considered effective:

  1. Running: Running is a great way to get your heart rate up and improve your cardiovascular fitness. It also burns a significant number of calories per minute and can be done indoors or outdoors.
  2. Cycling: Cycling is a low-impact exercise that can be done indoors or outdoors. It's a great way to work your legs and get your heart rate up, and can be adjusted to suit different fitness levels.
  3. Swimming: Swimming is a low-impact exercise that works your whole body and can improve cardiovascular fitness. It's also a great option for people with joint problems or injuries.
  4. High-Intensity Interval Training (HIIT): HIIT workouts involve short bursts of high-intensity exercise followed by periods of rest. These workouts can be very effective for improving cardiovascular fitness and burning calories in a short amount of time.

Ultimately, the most efficient cardio exercise for you will depend on your individual fitness level, goals, and preferences. It's important to choose an exercise that you enjoy and can stick to in the long term, as consistency is key to seeing results.

(ChatGPT, personal communication, March 13, 2023)

Example 3

User question: What should I make for dinner tonight?

ChatGPT answer: I'm happy to help you come up with some ideas for dinner tonight! Here are a few suggestions based on different dietary preferences and ingredients you might have on hand:

  • Grilled chicken or tofu with a side of roasted vegetables, such as broccoli, carrots, and sweet potatoes.
  • Spaghetti with tomato sauce and sautéed vegetables, such as zucchini, mushrooms, and bell peppers. You could also add some ground turkey or lentils to the sauce for protein.
  • Black bean tacos with avocado, salsa, and a side of brown rice. You could also add some grilled vegetables, such as onions and bell peppers, for extra flavor.
  • Baked salmon with a side of steamed green beans and quinoa. You could season the salmon with lemon, garlic, and herbs for added flavor.

(ChatGPT, personal communication, March 13, 2023)

Accessibility (opens in new window)