JokeGPT: ChatGPT vs Standup Comedian. Are the robots coming for our mics?

As a comedian, Charlie Nadler is highly skeptical of a machine being funny. As an engineer, Dr. Kiar Holland knows technology makes anything possible. Below is their conversation around a recent experiment they conducted with AI and Stand-Up Comedy.

ENDANGERED COMEDIAN (Charlie) : The robots are coming for our mics. A comedian friend, who is incredibly funny, predicted we will be irrelevant as we know it in three years. But I don’t want to be like a lot of people who hate change so much they try to will it away and dismiss it, or be in denial about how technology might change my industry. 

EXCITED ENGINEER (Kiar):  We’ll definitely see a transformation in comedy.  Maybe more people rising to the top who are good joke tellers versus joke writers.  Possibly an emphasis on comedians who make off the cuff jokes, are amazing at crowd work, etc.

ENDANGERED COMEDIAN: My view is you should keep your enemies close, so Kiar and I decided we should use ChatGPT (using GPT-3.5) to punch up some material. I have some jokes in my longer act (40-60 minutes) that just aren’t good enough, and I haven’t been able to crack them after a few rewrites and retests. These are jokes that I would normally just slice from my act and replace with better stuff once I have it. But before I do that, let’s see if this AI can save them from the cutting room floor. I provided Kiar with 3 of these mediocre jokes to see what it could come back with. Here are the jokes:

1. I have a super old doctor. He’s so old I have to call him to remind him about my appointments. His diagnoses are ancient, last check up he told me I had scurvy. I thought I couldn’t read his handwriting, but it was hieroglyphics.
2. I miss being 18 when I could play sports for hours and barely stretch and be ok. I got injured the other day and my friend said the meanest thing. He told me I was “over the hill.” I realized that was a weird thing to tell me because I was never on the hill. I guess at some point I tunneled through the hill! I was never athletic enough to experience any hill. Being over the hill means you peaked. I like to think of it as I’m not over the hill, I’ve always been on flat ground!
3. I’m getting more harmless looking as I age. I went to this beer garden to meet up with a buddy recently, and they had a sign saying we search all bags. So I took off my backpack and the employee goes, “You don’t have anything in there!” But what’s messed up is I did have stuff in there, my chess board and my chess clock!

EXCITED ENGINEER:  Naturally, the best way to judge comedy is through spreadsheets, so I created a new workbook and went at it.  With each joke, I gave two prompts to our future overlords:

1. Make this joke funnier
2. Make this joke funnier in the style of Charlie Nadler

Regenerating each so that there were a total of five results for each prompt.  

Although Charlie should be a household name, the AI Gods were not familiar enough with his work to mimic his style, so I copied and pasted about a dozen of his best jokes, so I could reference him the rest of the session.  Prior, it actually pretended to know him until I asked very specific questions about him, and busted out with the AI version of  “ha… you got me!”).

Charlie came into the spreadsheet where I copied all the results, rated each, and crafted a new version of the joke (v2), taking pieces from all 10 of the results.  He broke v2 into 2-3 parts, and I repeated the same steps for those as individual parts.  Instead of “in the style of Charlie Nadler” for the v2 parts, I asked it to do in the style of three comedians we considered were approximate to Charlie’s style.

ENDANGERED COMEDIAN: After looking at the data that GPT spit back, it all feels like an open mic comic who is very new to comedy. All of the jokes are way too wordy. There are no huge punchlines and it doesn’t seem to understand some key elements of joke structure. But there are a fair amount of interesting ideas and phrases sprinkled around here and there, that feel like they could be additive to the jokes, and serve as great rewriting inspiration.

EXCITED ENGINEER:  I’m shocked that it’s able to create so many original ideas for changes at such a rapid pace.  We were able to get several spins on the joke in a matter of seconds.  Simply getting that quantity and taking pieces seems like the way to go. Below is Charlie’s rating data from this process.

Asking it to “make the joke funnier in the style of three comedians” created some hallucinations, and really bad results.  This DeepLearning-Diva would say things like “As John Mulaney would say [quote him]”.  I ended up getting frustrated with this prompt, and stopped using it all together. Below is a chart that compares average word count to average rating:

The first passes seemed to yield the most results (blue bar), where the AI increased the number or words by 157%, but the jokes were rated to be 157% better.  Charlie’s collaboration with the AI made the biggest changes (red bar), which led to a 266% increase in words and 244% funnier in his view.  Splitting each joke into parts (yellow bar) was probably a waste of time since it added the most filler words and was less than half as funny. 

When pressed for time, it’s probably best to just pick from the first 10 results, before you start to hit the wall of diminishing returns.

ENDANGERED COMEDIAN: When it came to using the AI’s ideas to rewrite my jokes, I wanted to ensure I had a high level of ethics and integrity around this so that the new jokes still felt true to who I am as a writer and performer. My approach was to use this like a thesaurus on steroids (I already use a thesaurus a lot for all types of creative writing). I took a few direct phrases I liked, while also being inspired by other phrases and turning them into completely original lines, and then used this combination of AI and my own new content to strengthen the overall bits. It was nice to have a writing partner of sorts who gave me different angles, ones that my brain hadn’t yet come up with. My comic friends and I do this all the time when we workshop material, giving each other lines we can add to jokes. For this experiment, only incorporating short lines like that is where I personally “drew the line”. I was always worried where the AI’s material and phrases were coming from. When a couple things felt too polished and too out of left field, I was very wary that they weren’t original. Whether that assessment was fair or not, I chose not to use that content. I empathize a ton with the recent outrage around parallel AIs like DALL-E. It seems to border on plagiarism in a lot of instances, and I feel we need much more formal licensing and plagiarism protections for the artists when it comes to this new AI era. This in my view needs to be solved and/or outright avoided by any large language model or equivalent technology.  

EXCITED ENGINEER:  I was impressed to see the AI Angels make the jokes at least 2 times funnier, based on Charlie’s rating (although he mentioned later that the AI was rated on a bit of a curve, mentally eliminating filler words).  But the real test was to see how these jokes did in front of HewMoNs!

ENDANGERED COMEDIAN: I had a scheduled show a few days after we ran the jokes through the AI, so I tested the new versions and got laughs with a lot of the new material I tried! It was exciting to do this experiment, but the glow quickly faded for me when I realized how primitive ChatGPT (using GPT-3.5) is. If an AI can be the best in the world at Go, it only makes sense that another could someday conceivably be the best in the world at joke writing. It feels like a foregone conclusion that these AIs in the near future will be able to get their “10,000 hours” of mastery in nanoseconds and become incredibly hilarious by absorbing all of the greats’ writing tactics, mannerisms, and structures. It’s hard to imagine this thing completely replacing people like me, but I could see it taking a huge chunk out of comedians’ market shares. Is this what it felt like to be a ship captain when overseas air travel first became a thing? Are humorists as original and exceptional as we like to think we are, or easily replicable?

EXCITED ENGINEER:  You are nothing but a human AI with a much smaller and more neurotic processor than these new epic AIs! Kidding, buddy, kidding… 

ENDANGERED COMEDIAN: All jokes aside, I’ve spent my entire adulthood chasing success as a screenwriter and comedian. While this basic tool helped me today, my industry now feels much more existential. I just have to trust that comedians can and will still have a place in the future and be able to speak their own truths in perpetuity. Fortunately, GPT4 agreed when I asked if AI will replace comedians. Its response:

While AI has made great strides in natural language processing and generation, it is unlikely that it will completely replace comedians. AI-generated humor often lacks the creativity, nuance, and spontaneity that are essential to great comedy. Additionally, much of comedy relies on human experiences, emotions, and social context, which may be difficult for AI to fully understand and replicate. However, AI can certainly aid comedians by generating ideas, providing feedback, and even creating some simple jokes. Ultimately, the best comedy comes from a combination of human creativity and AI assistance.

Despite this making me feel better about things, we have to figure out NOW how to partner with and utilize this new technology to evolve comedy into its new phase in our history. Because an AI doesn’t need to be put up at a Red Roof Inn or provided blue M&Ms in the green room. So we better level up, and quick…