[This post was originally published on Cohost.]

I’ve previously said my piece that people who want to write, at least in an original manner, should just try doing that instead of trying to get a large language model to do it for them. My opinion applies even more strongly to the idea of prompting LLMs to write poetry, whether that be parlor tricks like “write my VC pitch in rhyming couplets” or more serious attempts to create poems that one might confuse with poetical works of past or present.

Like a dog trained to walk on its hind legs, these attempts may entertain for a moment but offer no lasting pleasure. And as with making a dog play the biped, they’re an insult to the true nature of the entity: LLMs are best at producing an average of human creativity and commentary directed along on a dimension or dimensions of choice. However, the best poetry is singular, not collective, the product of particular moments of a particular life.

But recommending poetry and poets—that’s a different task, one where a smoothed-out average of critical consensus is exactly what one might want. That’s the task I set out for ChatGPT in its GPT-4 mode.

My question was relatively simple; note that in the interest of brevity I’ve here combined two prompts into one, and omitted the response to the first prompt:

Here is a list of my favorite poets, in alphabetical order: Frank Bidart, Emily Dickinson, Louise Glück, Adrienne Rich, Muriel Rukeyser, Anne Sexton, Wallace Stevens, and Walt Whitman. Please provide a summary of the types of poetry that I would probably like.

The ChatGPT response was as follows:

Based on your preference for poets such as [repeats list], it is apparent that you have an affinity for poetry that encompasses the following characteristics:

Emotional Depth and Intimacy: You seem to appreciate poetry that delves deeply into emotions and presents an intimate, personal voice. This can include confessional poetry and poems that explore psychological states.

Innovative Form and Structure: Your interest in poets like Frank Bidart and Walt Whitman suggests that you value innovation in poetic form, including free verse and experimental structures.

Intellectual and Philosophical Themes: With poets like Wallace Stevens and Emily Dickinson on your list, it’s evident that you enjoy poems that engage with philosophical questions, explore the nature of reality, and are intellectually stimulating.

Rich Imagery and Language: Poets like Whitman and Stevens are known for their evocative imagery and lush language. It seems you appreciate the poetic craft, where words are meticulously chosen to create vivid mental pictures.

Cultural and Mythological Allusions: Your preference for Louise Glück indicates an appreciation for poems that incorporate classical references and mythology to address contemporary themes.

Social and Political Engagement: With Adrienne Rich and Muriel Rukeyser among your favorites, it appears that poetry with social and political themes, especially those related to justice, feminism, and social change, resonate with you.

Exploration of Identity and Human Relationships: Many of the poets on your list explore complexities of identity, gender, and human relationships. It seems you appreciate poetry that delves into these aspects of human experience, including themes of loss and desire.

Historical and Documentary Elements: Your inclusion of Muriel Rukeyser suggests an interest in poetry that incorporates historical events and documents, engaging with history through a poetic lens.

Adaptation of Traditional Stories: Anne Sexton’s use of fairy tales indicates that you might enjoy poetry that reinvents or reimagines traditional stories in new and intriguing ways.

In summary, you likely appreciate poetry that is emotionally resonant, intellectually stimulating, innovative in form, rich in imagery, and engages with social, cultural, and historical themes. It seems that both the aesthetic qualities of the language and the depth of the themes are important to you.

This summary is somewhat generic, but the particular items called out are pretty spot-on: a smoothed-out critical average, but a relatively accurate one.

As a follow-on, my next prompt was to ask for a list of five other contemporary poets whose work I might like. Here’s what ChatGPT came back with:

Based on the types of poetry you appreciate, which include emotional depth, innovative forms, intellectual themes, rich imagery, social engagement, cultural allusions, and exploration of identity, here are five contemporary poets whose work you might enjoy:

Ocean Vuong: Why You Might Like Him: Ocean Vuong’s work often involves emotional depth and intimacy. His language is lush and he explores themes of identity, particularly regarding his Vietnamese-American background. His debut collection, “Night Sky with Exit Wounds,” deals with family, the legacy of war, and finding one’s place in the world.

Natalie Diaz: Why You Might Like Her: Natalie Diaz’s poetry is known for its vivid imagery and exploration of cultural identity, particularly her Mojave American and Latinx heritage. Her collection “Postcolonial Love Poem” addresses themes of love, colonialism, and environmental injustice.

Ilya Kaminsky: Why You Might Like Him: Ilya Kaminsky is a Ukrainian-American poet whose work often deals with issues of displacement, social justice, and the human capacity for resilience. His poetry collection “Deaf Republic” is noted for its narrative innovation and poignant social commentary.

Tracy K. Smith: Why You Might Like Her: Tracy K. Smith’s poetry often combines intellectual depth with emotional resonance. Her collection “Life on Mars” tackles themes ranging from science fiction and the universe to human relationships and loss, showcasing a unique blend of the cosmic and the intimate.

Danez Smith: Why You Might Like Them: Danez Smith’s work often engages with social issues such as race, gender, and sexuality. Their poems are both raw and beautifully crafted, offering innovative forms and a powerful voice. Their collection “Don’t Call Us Dead” is a remarkable exploration of what it means to be Black, queer, and HIV-positive in America.

These poets share some thematic and stylistic elements with the poets on your list, and their work encompasses a range of contemporary perspectives.

Now, this being GPT-4, I had to account for the possibility that some of the works—or even some of the poets—were made up. So I looked them all up on the Poetry Foundation website and checked out the various poems included for each of them. Here’s how I would score the recommendations, on a 1-5 star basis, where 5 stars is reserved for a poem that can pass the Emily Dickinson test:

• Ocean Vuong. 3 stars. Overall his work is solid, and he has some great lines, but no individual poems really stood out to me.

• Natalie Diaz. 4 stars. As Adrienne Rich did in a lot of her late poems, Diaz has a tendency to let politics overwhelm the personal, so that one hears not the voice of the poet but the assumed Voice of a People (or in Rich’s case, a movement). But the poems about her brother—“My Brother at 3 A.M.,” “My Brother My Wound,” and “No More Cake Here”—are outstanding.

• Ilya Kaminsky. 3 stars. I admire his ambition in Deaf Republic, but the poems included from it just don’t do it for me. I prefer the personal poems, particularly “Author’s Prayer” and “I Ask That I Do Not Die.”

• Tracy K. Smith. 4 stars. I especially like the SF-tinged poems from Life on Mars. Highlights: “My God, It’s Full of Stars,” which features a guest appearance by Charles Heston, and “Declaration,” which excerpts the Declaration of Independence to devastating effect.

• Danez Smith. 5 stars, because “acknowledgments” aced the test. Other highlights: “alternate names for black boys,” “broke n rice” (a brilliant use of a technique I hadn’t seen before), and “dogs!” (the next time they do an anthology of poetry about dogs, this deserves to be in it).

Overall this was a pretty good exercise, and I might try this exercise again with another list of poets I like. I‘ll definitely be buying a couple books of poetry off of this exercise—print, of course, the only way poetry should be read.

P.S. I cancelled my ChatGPT subscription, because I didn’t see it providing enough value to be worth the money.

If you pay any attention at all to AI-related news, you know that today was the day that OpenAI.com released GPT-4, the newest version of its large language model (LLM), and made it available through the paid version of its ChatGPT web-based chatbot. I had already tried out the free version of ChatGPT (based on the previous GPT-3 LLM) and decided it was worth spending $20 for the paid version (ChatGPT Plus) to find out what all the hype was about.

Here are a few of my hype-free thoughts:

First, a minor comment: I really wish that people demoing LLMs would stop doing things like telling the LLM to make all words in the response start with the same letter, or having it produce the response in rhymed verse—both examples from the GPT-4 developer livestream today. These feel like fancy parlor tricks of minimal relevance for how LLMs might actually be used in real life, like showing off a border collie by having it dance on its hind legs.

Second, the most interesting application of GPT-4 I saw today was the Be My Eyes Virtual Volunteer, an app that leverages the new ability of GPT-4 to analyze images and provide a text description of what’s in them. The original Be My Eyes app allowed blind or otherwise visually impaired users to call upon the help of sighted human volunteers: the user would point their smartphone at something and the human volunteer would describe what was shown on camera. The Virtual Volunteer, as its name implies, substitutes GPT-4 for a human.

(Incidentally, this app illustrates an interesting point about how LLM-enabled apps will be tailored to the needs and expectations of users. In the brief demo video of Virtual Volunteer, the audio produced by the app seems very robotic and difficult to understand. This is a feature, not a bug. Experienced users of screen reader software can comprehend computer-generated text-to-speech “spoken” at speeds that would astound the typical sighted person, and in this context having the computer voices sound “natural” is a secondary consideration at best.)

Third, I tried out ChatGPT Plus with GPT-4 for what I would likely use it for, namely helping me write online essays like this one, and gave it a number of example tasks. (I’m skipping including the GPT-4 responses for reasons of length. If there’s real interest I’ll post the entire set of transcripts somewhere.)

In the first task I copied in the entire text of my post “I fought the power law and the power law won” and asked for it to be summarized. The resulting three-paragraph summary was pretty good, hitting all the main points I made. I could definitely see using this feature of GPT-4 as a way to summarize other people’s essays and news stories to see if it’s worth my reading them in full.

In the second task I asked the LLM to explain the meaning of the analogy “AI is to the world of ideas as index funds were to investing,” the topic of another of my posts. Here GPT-4 didn’t do much better than GPT-3 in explaining the analogy at more than a surface level.

I also tried asking about the meaning of a metaphor I was planning to feature in a future post. Here GPT-4 actually produced some useful and interesting meanings, but again not the particular meaning I intended.

As a final task I asked GPT-4 to explain the concept of a “fixation index,” a measure of distance between populations originally used in genetics and then adapted in the study of cultural evolution (a recent interest of mine). Here GPT-4 did very well, producing definitions of the fixation index that made sense in both contexts, and detailed explanations of how the measure might be calculated in both contexts.

But (and there’s often a “but” when it comes to LLMs), I can’t really use the GPT-4 output to help me learn about the concept, because based on others’ experiences with GPT-3 and similar LLMs I can’t trust that the output from GPT-4 is completely correct. I’d have to go back to the original source material and look up definitions and explanations there, and do some work to convince myself that the original source’s explanation matches what GPT-4 produced. But if I’m doing all that, why would I bother with GPT-4?

Some final thoughts:

Given the contempt heaped upon the heads of humanities majors by those promoting the “shape rotators vs. wordcels” meme, I find it amusing that GPT-4 does quite well on various physics and math tests and tests like the bar exam that test a combination of rote knowledge plus deductive ability, but has difficulties with advanced English language and literature tests. One speculation I’ve seen is that current LLMs are not that great at reasoning with abstract and fuzzy concepts, which sounds plausible based on my own experience.

But of course it’s possible that future versions of LLM, be it GPT-5 or others, will overcome this lack, which will only intensify further the confident, even giddy, pronouncements that LLMs will replace humans in most if not all professions involving mental activities. Certainly corporate executives across the business landscape will attempt to do just that—nobody ever got fired for pursuing reductions in labor costs as the primary if not only path to profitability—but how that scenario plays out remains to be seen.

As for me, I’ll pay my $20 a month for at least a little while longer to see how GPT-4 might help improve my own writing and research. But based on my experience thus far I suspect I may end up saving the money and using it either for more specialized LLM-enabled tools, or just buying more volumes of manga.

AndreL (@AndreL) - 2023-03-15 13:29

phind.com has an LLM which cites its sources. The catch is that it also generates plausible nonsense, complete with plausible citations.