A list of resources, articles, and opinion pieces relating to large language models

Teresa Berndtsson / Better Images of AI / Letter Word Text Taxonomy / Licenced by CC-BY 4.0.

We’ve collected some of the articles, opinion pieces, videos and resources relating to large language models. Some of these links also cover other generative models. We will periodically update this list to add any further resources of interest.

How they work

• What are Generative AI models?, Kate Soule, video from IBM Technology.
• What is GPT-4 and how does it differ from ChatGPT?, Alex Hern, The Guardian.
• What Is ChatGPT Doing … and Why Does It Work?, Stephen Wolfram.
• Understanding Large Language Models — A Transformative Reading List, Sebastian Raschka.
• How ChatGPT is Trained, video by Ari Seff.
• ChatGPT – what is it? How does it work? Should we be excited? Or scared?Deep Dhillon, The Radical AI podcast.

Journal, conference and arXiv articles

• Scientists’ Perspectives on the Potential for Generative AI in their Fields, Meredith Ringel Morris, arXiv.
• LaMDA: Language Models for Dialog Applications, Romal Thoppilan et al, arXiv.
• What Language Model to Train if You Have One Million GPU Hours?, Teven Le Scao et al, arXiv.
• Alpaca: A Strong, Replicable Instruction-Following Model, Rohan Taori et al.
• Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets, Irene Solaiman, Christy Dennison, NeurIPS 2021.
• On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜, Emily Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell, FAccT 2021.
• A Survey of Large Language Models, Wayne Xin Zhao et al, arXiv.
• A Watermark for Large Language Models, John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, Tom Goldstein, arXiv.
• Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision, Milagros Miceli, Martin Schuessler, Tianling Yang, Proceedings of the ACM on Human-Computer Interaction.
• AI classifier for indicating AI-written text, OpenAI.
• Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling, Stella Biderman et al, arXiv.
• GPT-4 Technical Report, OpenAI, arXiv.
• GPT-4 System Card, OpenAI.
• BloombergGPT: A Large Language Model for Finance, Shijie Wu et al, arXiv.

Newspaper, magazine, University website, and blogpost articles

• Why exams intended for humans might not be good benchmarks for LLMs like GPT-4, Ben Dickson, Venture Beat.
• Does GPT-4 Really Understand What We’re Saying?, David Krakauer, Nautilus.
• Large language models are biased. Can logic help save them?, Rachel Gordon, MIT News.
• Ecosystems graph for ML models and their relationships, researchers at Stanford University.
• ChatGPT struggles with Wordle puzzles, which says a lot about how it works, Michael G. Madden, The Conversation.
• AIhub coffee corner: Large language models for scientific writing, AIhub.
• ChatGPT Is a Blurry JPEG of the Web, Ted Chiang, The New Yorker.
• ChatGPT, Galactica, and the Progress Trap, Abeba Birhane and Deborah Raji, Wired.
• ChatGPT can’t lie to you, but you still shouldn’t trust it, Mackenzie Graham, The Conversation.
• AI information retrieval: A search engine researcher explains the promise and peril of letting ChatGPT and its cousins search the web for you, Chirag Shah, The Conversation.
• A small step for research but a giant leap for utility, Interview with Fredrik Heintz, Linköping University.
• ChatGPT threatens language diversity. More needs to be done to protect our differences in the age of AI, Collin Bjork, The Conversation.
• Column: Afraid of AI? The startups selling it want you to be, Brian Merchant, Los Angeles Times.
• Three ways AI chatbots are a security disaster, Melissa Heikkilä, MIT Tech Review.
• Time: OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic, Billy Perrigo, TIME.
• Misplaced fears of an ‘evil’ ChatGPT obscure the real harm being done, John Naughton, The Guardian.
• Darktrace warns of rise in AI-enhanced scams since ChatGPT release, Mark Sweney, The Guardian.
• Lawmakers struggle to differentiate AI and human emails, Kate Blackwood, Cornell Chronicle.
• Colombian judge says he used ChatGPT in ruling, Luke Taylor, The Guardian.
• Bhashini: At your service an Indian language chatbot powered by ChatGPT, video from The Economic Times.

Podcasts and video discussions

• The Limitations of ChatGPT with Emily M. Bender and Casey Fiesler, Radical AI Podcast.
• CLAIRE AQuA: “ChatGPT and Large Language Models”, CLAIRE.
• Su Lin Blodgett on Creating Just Language Technologies, The Good Robot Podcast.

Focus on LLMs and education

• Opinion: ChatGPT – what does it mean for academic integrity?, Giselle Byrnes, Massey University.
• Debate: ChatGPT offers unseen opportunities to sharpen students’ critical skills, Erika Darics, Lotte van Poppel, The Conversation.
• ChatGPT and cheating: 5 ways to change how students are graded, Louis Volante, Christopher DeLuca Don A. Klinger, The Conversation.
• ChatGPT: students could use AI to cheat, but it’s a chance to rethink assessment altogether, Sam Illingworth, The Conversation.
• A Teacher’s Prompt Guide to ChatGPT, @herfteducator.

Relating to art and other creative processes

• ‘ChatGPT said I did not exist’: how artists and writers are fighting back against AI, Vanessa Thorpe, The Guardian.
• AI and the future of work: 5 experts on what ChatGPT, DALL-E and other AI tools mean for artists and knowledge workers, Lynne Parker, Casey Greene, Daniel Acuña, Kentaro Toyama Mark Finlayson, The Conversation.
• Is there a way to pay content creators whose work is used to train AI? Yes, but it’s not foolproof, Brendan Paul Murphy, The Conversation.
• ChatGPT is the push higher education needs to rethink assessment, Sioux McKenna, Dan Dixon, Daniel Oppenheimer, Margaret Blackie, Sam Illingworth, The Conversation.
• AI Art: How artists are using and confronting machine learning, YouTube video from the Museum of Modern Art.

Misinformation, fake news and the impact on journalism

• Misinformation Monitor: March 2023, focus on GPT-4, NewsGuard.
• A fake news frenzy: why ChatGPT could be disastrous for truth in journalism, Emily Bell, The Guardian.
• Defending Against Neural Fake News, Rowan Zellers et al, arXiv.

Regulation and policy

• ‘Political propaganda’: China clamps down on access to ChatGPT, Helen Davidson, The Guardian.
• Chatbots, deepfakes, and voice clones: AI deception for sale, USA Federal Trade Commission blog post.

AIhub is dedicated to free high-quality information about AI.

AIhub is dedicated to free high-quality information about AI.