Chatbots Stink: Writers Need Better Interfaces

Generative AI Does Not = ChatGPT

We’re approaching an existential crisis in education and I’m not talking about generative AI for once. Millions of future students may not even bother applying for college because they no longer see the point in getting a degree without a guaranteed job after graduating. States are increasingly engaging in bizarro political stunts to pull books with any mention of gender identity or sex from library shelves. Teachers are overworked, underpaid, and completely fed up. Then you have Covid and its continual aftereffects. Trust me, higher education doesn’t need a new crisis. But boy are we looking for one with generative AI.

Hundreds of universities and thousands of educators have published often contradictory policies on generative AI’s use in the teaching of writing. But the landscape of generative AI is evolving and quickly creeping outside the ChatGPT box, as Microsoft, Google, and Amazon have all committed to deploying generative AI technology within their Office and Workplace tools. Now the American Federation of Teachers, one of the largest teacher’s unions in the nation, has partnered with GPTZero to bring unreliable AI detection to millions of teachers. I’ve posted about this again, and again, and again on this blog and on social media—AI detection is not reliable in an educational context. If we could move past the knee-jerk panic over students using ChatGPT to cheat we could actually have a critically discussion of how lousy ChatGPT is as an interface for people who want to use it for writing.

We Need Better Interfaces

Of all the academic fields that teach writing, creative writing will likely endure the coming storm and is also uniquely qualified to gauge what’s useful and what isn’t about generative AI. I think Vauhini Vara’s personal essay she coauthored with GPT-3 is a wonderful example of this. Ghosts is an amazing exercise in how a language model doesn’t understand grief or personal information, but can still be a valuable asset to a writer. Vara used the earlier version of generative AI as a writing aide to try and write about the death of her sister. While the LLM didn’t know anything about her sister, it was still capable of producing wonderful and shocking sentences that Vara laments in “Confessions of a Viral AI Writer” are not present in ChatGPT. The model may have improved, but what killed the magic was the introduction of a chatbot interface.

Put simply, chatbot interfaces suck. There’s no other graceful way to say it. Surly there are better ways to explore the subtleties of human language or bring AI into the writing or creation process as a partner. Chatbots simply generate once prompted, then generate some more, ad infinitum. I want something that adds to my writing process, not replaces it.

Few artists are keen to offload their creative impulses to an algorithm, so I don’t foresee chatbots becoming the go to interface for writers. Creativity will likely remain uniquely human ability; however, the rest of the writing process — from editing, submitting, and publishing, will very likely change, and not always for the better. We can push back and work with developers to help craft a vision of AI assistance that doesn’t offload our humanity, but this will be a monumental task. It’s going to require writers to form relationships with developers and build tools specifically tailored to what we’d like to use them for, not simply what they envision us using them for. We’re going to have to form bridges between our disciplines in order to shape how this technology impacts our future.

Part of the challenge is moving away from chatbots and encouraging the development of systems that enhance human creativity using generative AI instead of offloading it. There are a handful of developers working in the field of generative AI who are trying to build feedback systems to show writers stylized suggestions tailored to their work in real-time. These advanced AI-powered feedback systems put the human being in control of their writing, not an algorithm.

PenPal

Amelia Wattenberger’s PenPal app is an example of such a system. A user can toggle on and off what feedback features they’d like. Tone, writing style, genre, etc. are just a few of the elements Wattenberger programmed her tool to offer feedback on her writing and her process. Generating suggestions allows writers agency and control in the amount of influence they want from synthetic feedback. Wattenberger’s ethos on developing such systems challenges chatbot developers to move beyond generative text to “embrac[e] our humanity instead of blindly improving efficiency. And that involves using our new AI technology in more deft ways than generating more content for humans to evaluate.”

Language Model Sketchbook

Maggie Appleton, a developer at Ought, has also thoughtfully considered interfaces beyond chatbots as “epistemic rubber ducks—as things we can query for fuzzy answers, bounce ideas off, and think through problems with. They can help strengthen our own critical thinking and reasoning abilities in the same way a good debate partner does.”

Both Appleton and Wattenberger share a common critique of chatbots as the primary interface for language models, yet they propose different solutions that can be useful for creative writers. Appleton views chatbots as a "lazy solution" and envisions more innovative ways to interact with language models. She proposes a few alternative interfaces that could be helpful for writers. One of these is the concept of "Daemons," characters that hang out in the background of the user’s writing environment and offer suggestions, revisions, and ideas. These characters have different personalities and roles, such as playing devil's advocate, complimenting your writing, synthesizing ideas, or fetching research. Another concept is "Branches," which helps explore cause-and-effect chains in understanding an issue. Appleton also proposes "Epi," a more general-purpose reasoning assistant for research and non-fiction writing.

Wattenberger rightly criticizes chatbots for their lack of affordances, the burden they place on users to learn what works, and the isolation of their responses. She suggests that interfaces should make it easier for users to provide context and that responses should not be isolated but part of a "working buffer." She also proposes an AI writing interface that mimics a good writing tutor, suggesting improvements to your content based on the user's specified use case.

For creative writers, these arguments suggest that language models can be more than just tools for generating text. Language models with more humanistic interfaces can be partners in the creative process, making suggestions, challenging ideas, and helping to refine and improve a writer's work, not offloading the skill entirely. The interfaces proposed by both authors could help writers to engage more deeply with craft of writing, to explore different perspectives, and to improve their writing in real-time. These innovative interfaces could also help writers better understand and control the capabilities of language models, making them more effective tools for creative writing.

Spatial Canvases

There are others using the technology to move beyond generative text and explore generative AI as feedback systems. Max Drake built a number of features for a tool called Fermat, which I piloted with my first-year writing students in the fall of 2022. One such tool is a live feedback generator, offering writers real-time feedback on their writing. I used it to explore counterarguments with my composition students, but that is just one example—you can program it to offer feedback on character development, plotting, writing style, and more. Drake’s argument is “AI generated critiques can, if done well, allow writers to develop their writing as if being advised by another person.” There’s great potential in such a system, but we would need to employ it with care. The risk of an automated feedback system that is always on, always has a response, and is always providing feedback, is that it stifles the creative impulse with too much feedback. We also need to carefully evaluate responses to ensure the feedback is high quality.

We may try to resist by not adopting generative AI, but how long will it be until it is baked into our productivity apps or running in our background processes? Meeting with developers gives writers a seat at the table, a position to push back against deterministic visions of the future by advocating for what we want – or do not want – to see in the products we use.

Publishers and Editors will Use Generative AI

Another area where creative writers need to take a proactive approach is in publishing, both big and small.  AI’s reach has the potential to disrupt submission software, impacting how editors pick and craft books. I’ve written before about Anthropic’s Claude, a Large Language Model like the one that powers ChatGPT. Claude can read manuscript-sized prompts with a context window of 75,000 words. This is a massive leap compared to the 4,000 word context window offered by ChatGPT, and it has wide-ranging implications. Soon, submission systems will be able to vet prose and poetry submissions based on a literary journal’s aesthetic. All an editor needs to do is upload a few back issues or hand-picked selections and automate the process of identifying what submissions match the journal or publisher’s style. This is a prime example of where writers and artists must intervene and push back.

What’s lost in the process of offloading such labor is the human relationship between reader and writer. When a creative writer submits work to a journal, they know that a human being will receive it, even if it is only to reject it. And, trust me, most of your subs will be rejections—that’s just part of the publishing game. Automating such a process has benefits, yes, but at the potential loss of critical skills that the creative writing students who serve as journal readers gain in the process of selecting submissions.

The offloading of labor won’t be confined to literary journals and small presses. OpenAI anticipates raising their context window to nearly one million tokens by the end of the year, or roughly 750,000 words, and Google’s NotebookLM already has a combined context window of 500,000 words. That’s enough for an agency to load a small reading list into a model to allow them to vet submissions to their aesthetic. Imagine the consequences if dozens of literary agencies suddenly adopted such systems and trained algorithms to vet manuscripts based solely on the bestselling books from the past decade.

And what of the editors and publishers? Generative AI’s impact on their labor is much in line with literary journals and agencies, but the major difference is an editor will be able to help craft a manuscript into a book using feedback targeted to audiences the system identifies. The cognitive process of making critical judgments about how to direct the plot, style, or characters toward a particular audience gets similarly offloaded in the editing stage. All an editor needs to do is plug in the parameters from their target market and get automated suggestions during the editing phase from an AI.

If we want to keep the act of editing and publishing a book in human hands, then we absolutely need to advocate for features like Wattenberger and Appleton call for; otherwise, the very books we read may well be the result of human imagination, but virtually all moments from the point a book is submitted until it lands in our hands will be impacted by a synthetic process that is anything but human.

It’s up to each of us to arm ourselves with knowledge and become AI literate as quickly as we can, because big tech wants the creativity you possess, but only when it’s used in connection with the systems they control. When you reduce language to something a computer can synthesize, words stop being an expression of humanity and start being viewed as a problem in need of being solved. We can resist such things, and we should. And, if we join the conversation in the development stages, we’ll have more opportunities to be persuasive.