Society Needs Open Sourced Language Models

Last November was a time of phenomenal deployments in the world of generative AI. Google opened the month with Wordcraft and showed what professional writers could produce with generative AI in their Wordcraft Writers Workshop. Meta attempted to launch their own language model geared toward researchers and educators called Galactica but pulled it within a few days because it produced dangerous outputs. Midjourney released what was then one of the most powerful updates for image generation the public had yet seen, which was likely only possible because of the open-source Stable Diffusion breakthroughs. Less than ten days after their failure with Galactica, Meta announced their amazingly successful Cicero experiment, where a large language model defeated human players in the game of Diplomacy, using a combination of deception, empathy, strategy, and persuasion. Any one of these developments was worthy of our attention, but you likely didn’t hear about a single one of them because a scrappy upstart called OpenAI released a little-known app at the end of November called ChatGPT.

The fall of 2022 was indeed a remarkable time in generative AI, and November was its crescendo. I can’t think of a month-long period in technical deployments that matched the frenzied pace. What fueled many of those breakthroughs were open models from Stable Diffusion and Meta. It turns out OpenAI doesn’t quite live up to its name and isn’t exactly open with its model data since the release of GPT-2. OpenAI argues that these models are now too big, and too dangerous to open source, but this sort of warning doesn’t ring sincere when their company’s business model relies on closed models and opaque data practices.

Meta recently came under fire for using pirated books in the training data for their most recent model, Llama. To be sure, no company should use stolen works to train any system. But the dust-up this stirred masked something crucially important—the public would have never known what data Meta had used if they had not open-sourced their model. Certainly, this does not forgive the transgression, but let’s also be real about something—no one else is committed to releasing how they train their models. Not Google. Not OpenAI. Society needs open models, otherwise, our future will be controlled by a handful of companies that hold the keys to the technology.

The Moral Quagmire Posed By Open Models

Open models are by no means perfect, but they absolutely are what’s needed to democratize the increasingly privatized and siloed world-powering LLMs. Researchers need access to training data for obvious ethical reasons, but they also need access to how these models were trained and how the model weights impact performance and coherence. Society needs to know what this technology can do beyond rudimentary text generation. Meta’s Cicero was one such example that we all must pay attention to in education. The reason? If you can get a bot to win a game of strategy against humans that requires a certain degree of persuasion, even guile, then it isn’t hard to imagine what a system like this could do if you assign it a synthetic voice and instruct it to gamify student success; winning conditions = 75% of players graduating in four years.

The problem with open models is the age-old dilemma of knowledge being used wisely vs. abused. And trust me, there are some truly gross open models out there. Yannic Kilcher trained one such model on nothing but 4Chan’s most toxic content and then set it loose as a bot that posted thousands of horrible comments, mirroring the vile sexist, racist, and violent material it was trained on. GPT-4Chan was a stunt, but as Annette Vee eloquently noted in her postmortem analysis, the biggest takeaway is the duality represented by such open systems:

Writing is dangerous because it is open, transferrable, and scalable. But that’s where it can be powerful, too. Lawmakers who forbid teaching enslaved people to write knew that literacy could be transferred from plantation business to freedom passes (Cornelius, 1992). These passes were threatening to enslavers but liberating for the enslaved. While it’s impossible to consider GPT-4chan liberating, it represents an edge case about open systems that carry both danger and power. Writing, the Internet—and, increasingly, AI—present both the promise and peril of a “here comes everybody” system.

GPT-4Chan is timid compared to the dozens upon dozens of deep fake generative AI apps that use various open models to allow users to non-consensually take someone’s picture and upload it on a porn star’s body. The idea that millions of teens will now have this ability to harass women in public schools in real-time from apps on their phones is by far the best reason to regulate how these models are deployed publicly.

But what private companies have done instead of regulate themselves, is nerf the responses of their models through intense content filters. With so many safety features and content restrictions now present in private models, many users are left puzzled about what ideas they can explore without setting off automated filters that abruptly end their sessions. A person who is exploring their identity should be allowed to creatively query a language model about their gender, their sexuality, and be conservative or liberal without worrying if what they are asking will set off a content filter and shut down the conversation. I could not use Google’s NotebookLM as a research tool in my creative writing because some of my published stories contained characters using foul language, plots that were violent, and content that could have offended audiences. That’s a huge roadblock to wider adoption of this technology beyond banal office-space-type work. I imagine a pharmacist studying fentanyl overdoses or a social scientist studying harassment in the workplace would run into similar issues with content filters putting roadblocks in their research.

We clearly need nuance here and that isn’t going to come from ramping up content filters and may not even come from regulation. It may need to come from the hardest place of all—teaching ethical usage.

Assignment: Ethical Exploration of an Open Model

I developed a short assignment for my Digital Media Studies students to explore the ethical challenges posed by an open model. Perplexity Labs hosts several open models, including the newly released Mistral 7b. The model isn’t meant to have many safeguards—that’s what fine-tuning is for. But that doesn’t mean you can use it as an aide to help students develop their AI literacy skills. Now that so much of the bias and safety features in propriety models have been removed, it’s extremely difficult to make these closed models, like ChatGPT, generate offensive or unsafe content. Not so with Mistral. You can pull it up in class and generate quite a few things to educate students about the nature of red-teaming language models for bias and harm reduction.

For the assignment, I used Paul Rottger’s Unsafe Prompt List and asked students to develop their own prompts as they red-teamed the AI. I was also careful about allowing students to opt out of the assignment and only ask for prompts they felt comfortable with. You can use the assignment with your students: Ethical Exploration of Safety Features in Mistral 7-B.

The ethical challenges posed by generative AI will only grow as the technology becomes more powerful and accessible. While open models provide crucial transparency, they also lower the barriers to potential misuse. Companies developing these systems have a responsibility to implement thoughtful safeguards, but we as a society must also nurture wisdom and empathy in how we engage with AI. The conversation around ethics cannot be restricted to tech companies alone. We must have open and thoughtful discussions, guided by shared values of human dignity, justice, and compassion. Our technology mirrors our cultural habits and morality - by uplifting the best in ourselves, we uplift the best in our creations.