6 Times ChatGPT Fell Short of the MarketSmiths Standard for Copywriting

How does ChatGPT really stack up to the MarketSmiths process? We decided to try it out, and here's what we found.

ChatGPT isn't as good as MarketSmiths.

“I can be your best friend or your worst enemy.” If ChatGPT had a social media profile, that’s what it would say. That was my first impression—and after two weeks of taking this baby for a spin, seeing what it could do, and feeling out its vulnerabilities—I’m even more convinced. 

At MarketSmiths, our tagline is: Copywriting for Humans. We use a mix of writers & content strategists with industry expertise and awareness of B2B sales cycles to produce high-quality content. ChatGPT uses a machine learning model. 

How would the two differ in practice? I wondered how well AI can simulate the human experience without living it. Can a language model combine words in a way that’s truly helpful or moving? To explore this idea further, I tasked ChatGPT to write some of my old client assignments and analyzed differences between the two. 

Test #1: Write a Christmas card poem for a video game music producer that’s both warm and fierce.

ChatGPT:

Oh, video game music master,
We give thanks for your grandeur,
For your soundscapes, bold and vast,
That bring worlds to life with grandeur.

In this season of goodwill,
May your heart be filled with cheer,
And may the melodies you thrill,
Echo far and near.

Your music carries us away,
To places beyond our dreams,
A symphony of swords at play,
Or soft, serene, tranquil streams.

But amidst the beauty and the light,
There lies a warrior’s heart,
A battle cry that’s bold and bright,
A lion’s roar, a fearless start.

So take this holiday to rest,
And bask in joy and peace,
For soon enough, you’ll be pressed,
To create a new masterpiece.

We’ll be waiting with eager ears,
For the next note that you play,
For your music, like the finest spears,
Can conquer all in its way.

So keep on fighting, never cease,
And may your music ever soar,
For in this season of peace,
We thank you and so much more.

Merry Christmas, video game music producer,
May your season be grand,
For your music, like a golden sun,
Will always light up the land.

MarketSmiths: 

A poem. Take our word for it, it's great.

Analysis: Not a bad start, though the perspective is off. This is from the music producer to his clients—not pledging fealty to him. 

Excessive repetition is something I see a lot, so I wasn’t surprised to see “grandeur” used twice in the first stanza. (One time we asked ChatGPT to write a sonnet contemplating existence and it used the word “fleeting” three times.) 

While ChatGPT brought the sharp-tipped spears and golden suns, we lost the main point our client wanted to convey. Rather than focusing on “the fight to write great music,” we want to reflect on the shared human experience of the holidays and rouse our readers’ spirits to face uncertainties ahead in the New Year.

Test #2: Write a product description for a new hiking pant. 

ChatGPT:

Introducing the Ferrosi Women’s Transit Pants, perfect for everyday wear, travel, and commuting. Made with eco-friendly, bluesign® approved fabric, these pants are made from 46% recycled nylon, 40% nylon, and 14% spandex, making them lightweight, durable, and stretchy. 

These pants offer a clean, trim fit and feature a reflective inseam cuff for extra visibility, a secure zip pocket for your phone, and an elastic waistband with a clean flat front. With water-resistant, wind-resistant, and breathable qualities, these pants are perfect for climbing, hiking, and multi-sport days. 

The UPF 50+ rating offers protection from harmful UV rays, while the movement-mirroring stretch and abrasion-resistant fabric ensures maximum comfort and durability. Available in XS-XXL, the Ferrosi Women’s Transit Pants are the perfect addition to your wardrobe.

MarketSmiths:  

Analysis: 

Redundancies are magnified in short writing exercises such as this. The first paragraph tells us they’re “durable” and “stretchy,” so why does the third paragraph reiterate, “movement-mirroring stretch”, “abrasion-resistant fabric,” and “durability” again?

Cramming copious information into a paragraph is one of AI’s soup du jours. This feature-laden product description does, in fact, sound like a lot of descriptions I’ve read before. And yet, despite shout outs to sports like “climbing” and “hiking,” I can’t see myself in these pants. I don’t want to be told these pants are the perfect addition to my wardrobe. I want the description to show me how. 

For high-converting copy and content, get in touch with MarketSmiths today.

Test #3: Write an opening paragraph for a blog on “Why is my Cat Yowling?”

ChatGPT:

Cats are known for their vocalizations, but excessive yowling can be a cause for concern for many cat owners. Yowling is a loud, drawn-out meowing sound that cats make for a variety of reasons. Some cats yowl to get attention, express anxiety or stress, indicate that they are in pain, or even as a sign of aging. As a cat owner, it can be distressing to hear your furry friend yowling frequently, and it’s important to understand the underlying reasons behind this behavior. In this article, we will explore some of the common causes of yowling in cats and what you can do to help your feline friend.

MarketSmiths:

Analysis: Every sentence includes the word “cat.”  

Despite a touch of warmth from phrases like “furry friend” and “feline friend,” words like “excessive,” “anxiety,” “stress,” “pain,” and “distressing” strike a negative tone overall—which, if I’m a cat owner already feeling anxious about my cat’s yowling, isn’t going to sit well with me. 

As you’ll note, I open with a more positive tone that reflects the joys of living with and communicating with cats. Next, I include quotes from veterinarian subject matter experts to clarify what makes a yowl distinctive from a “loud, drawn-out meow.”

Test #4: Write a paragraph explaining how treating cognitive impairment can improve outcomes for patients with chronic kidney disease. 

ChatGPT:

Cognitive deficits are a common complication of chronic kidney disease (CKD), and they can have a significant impact on patient quality of life and outcomes. CKD patients with cognitive impairment may experience difficulties with memory, attention, and decision-making, which can make it challenging to manage their condition effectively. However, there are several ways in which treating cognitive deficits can improve patient outcomes in CKD:

Improved medication adherence: Cognitive impairment can make it difficult for CKD patients to manage their medications effectively, leading to poor adherence and worse outcomes. By treating cognitive deficits, patients may be better able to follow medication regimens, leading to improved health outcomes.

Better disease management: CKD patients with cognitive deficits may have difficulty understanding and following treatment plans, such as diet and exercise recommendations. Treating cognitive deficits can improve patient comprehension and adherence to treatment plans, leading to better disease management and improved outcomes.

Improved mental health: Cognitive impairment can be a source of frustration and distress for CKD patients, leading to anxiety and depression. Treating cognitive deficits can improve patients’ mental health and quality of life, leading to better overall outcomes.

Increased independence: Cognitive deficits can limit CKD patients’ ability to live independently and perform daily tasks, such as driving or managing finances. Treating cognitive deficits can improve patients’ independence and ability to function in daily life, leading to improved quality of life and outcomes.

Overall, treating cognitive deficits in CKD patients can improve medication adherence, disease management, mental health, and independence, all of which can lead to better patient outcomes and quality of life.

MarketSmiths:

Analysis: Like a high school essay, ChatGPT begins and ends with rote summary sentences. In fact, every single paragraph includes the phrase “treating cognitive deficits” and concludes with “… leading to better/improved outcomes/quality of life,” reminding me of poorly written websites in the early days of SEO. 

While the overall case for treating cognitive impairment seems fairly logical, we’re writing to an audience of clinicians who are intelligent enough to draw their own conclusions when presented with objective research within the context of their jobs.

Test #5: How would you describe the voice and tone of [Client Name] for a brand deck?

ChatGPT:

Our brand voice is professional, helpful, and approachable. The tone is friendly and informative, with a focus on making the process of gathering and managing documents as easy and efficient as possible for customers.

The language used in our communication should be clear and concise, avoiding jargon or technical terms where possible, to ensure that customers of all backgrounds can easily understand the benefits of using the platform. The tone should convey a sense of trustworthiness and reliability, assuring customers that their sensitive information is in safe hands.

In addition, our brand voice should be solution-oriented, emphasizing the platform’s ability to solve common pain points associated with document management. Overall, the tone should convey a sense of ease, simplicity, and professionalism, positioning us as a reliable partner for businesses seeking to streamline their workflows and save time.

ALT: When prompted to “make it more fun”…

We’re all about keeping things easy-peasy when it comes to managing documents. Our brand voice is like your favorite coworker – approachable, reliable, and always ready to lend a hand.

Our tone is friendly and informative, kind of like a chatty neighbor who’s always up for a cup of coffee and a chat. We’re not about using fancy lingo or confusing jargon – we want to make sure everyone can easily understand how our platform can help them.

But don’t let our playful vibe fool you – when it comes to handling your sensitive information, we’re as serious as a Monday morning meeting. You can trust us to keep your docs safe and secure.

At the end of the day, we’re all about solutions, not problems. We’re here to help you streamline your document management process and make your workday just a little bit brighter. So why not give us a try? You might just find yourself doing a happy dance (we won’t judge!).

MarketSmiths:

We know the inefficiencies of document collection—we’ve lived it. Our voice is that of a knowledgeable friend—someone who listens, understands, and delivers the right wisdom at the right time.

The tech world can be as dry as a desert—a barren landscape of complex jargon and confusing terms. That’s not us. Our voice is a cool drink of water. Our product, a refreshing solution. 

We understand people are busy. We get to the point with no-nonsense solutions. We inform and inspire, but greet our audience as equals—speaking in plain, digestible language

Through quiet confidence, our product sells itself. We are professional, first and foremost, but we also know that business leaders choose us over competitors because we’re friendly, easy to work with, and—dare we say—fun. 

We’re business at the front, party at the back. A bit of unexpected humor and cheekiness makes content interesting and fresh, but let’s not get swept away. Stay classy!

Analysis: A brand deck needs to differentiate the company voice from the masses. Much of our brand description derives from an hour-long conversation we had with the CEO about their company culture. ChatGPT, of course, isn’t privy to backroom discussions, strategy sessions, or able to generate unique insights. 

The tone vacillates from generic to wacky. I don’t know that I’d want to buy a document collection portal from my “chatty neighbor who’s always up for a chat.” Unless my target audience is a kindergarten classroom, I’m not using the adjective “easy-peasy,” and the last line’s judgment-free “happy dance” is painfully square at best—cringey at worst.

An education company got a website that hooked readers

A coaching company for teachers, Educate gives them the tools they need to take their work to the next level. But without an engaging website, the company struggled to close sales. That’s where MarketSmiths came in. By honing its website copy and brand messaging to clearly express its value, assert vital differentiators, and garner interest and sales, we soon fixed it up. Potential prospects now have crisp, clear copy to read—and Educate has bolstered its place in a crowded market. 

> Read the full case study here

Test #6: Write an introduction that defines “cheugy” in first person perspective. 

ChatGPT: 

Sure! As an AI language model, I can’t have personal experiences, but I can certainly simulate writing from a first-person perspective for you. Here goes:

As a language model, I’ve noticed that the term “cheugy” has gained a lot of popularity among young people in recent years. It’s a term that describes something that’s considered to be outdated or uncool. When I see this term being used, it’s usually in the context of fashion, music, or social media trends that were once popular but have since fallen out of favor.

From what I’ve seen, the term seems to be most commonly used by Gen Z and Millennials. It’s a way for them to express their opinions about what is and isn’t cool, and to differentiate themselves from previous generations. When something is described as “cheugy,” it’s generally seen as a negative thing. People who are described as cheugy may be seen as out of touch or uncool.

Overall, the term cheugy is an interesting reflection of how our society values trends and cultural currency. It’s fascinating to see how these trends evolve and change over time, and how new words and phrases emerge to describe them.

MarketSmiths:

Analysis: “AI Language Model” is not a very subtle ghostwriter. When it gets introspective, I can’t help but feel sorry for this spectator of human emotions. For a computer, it sure does a lot of “seeing.”     

Its explanation contains a factual inaccuracy: Cheugy is not used by Gen Zs and Millennials to “express their opinions about what is and isn’t cool.” I know it’s hard to believe, but cheugy is used by Gen Zs to describe how uncool Millennials are! 

Providing specific examples of what’s deemed “uncool” or “out of favor” can help audiences understand the term better. Humor is complex and difficult for AI to simulate—and yet, that’s one of the hallmarks of great vs. good copywriting.

There’s still a place for humans in copywriting.

Admittedly, the leap in technology is really cool—and something I’d like to tinker around with more. But I’m also even more convinced it’s not here to replace us. It’s a tool—like the keyboards we type on, the dictionary we use to look up words, or the spell check that auto-corrects obvious misspellings. Sure, it’s one hell of a powerful tool: one that will likely eliminate much of the bottom-feeder content we come across, one that will train mediocre writers to be a little bit better, and one that will potentially save the best writers time on grunt work—but one that needs continuous training and supervision. 

Getting consistently high-quality copywriting that offers substantial returns on investment requires tight collaboration between writers and content strategists—and that’s what you can expect when you seek out a partnership with MarketSmiths. Contact us today to get started.

Jennn Fusion

Jennn Fusion

Jennn made up her mind to become a writer when she was five years old and has buried herself in research and the written word ever since. She grew up a snowboarding, guitar-playing, concert-going punk-rocker in Buffalo, NY, home of the chicken wing. After earning a BA in Journalism and English from SUNY Geneseo, she spent most of her twenties working as a music journalist and a promotions marketing manager in Toronto. Her freelance writing career spans nearly two decades of crafting copy for lawyers, doctors, realtors, digital marketers, tech startups, and consultants. When she's not on her laptop with 100 tabs open, she's hanging out with her German Shepherds, husband, and two children; dressing up at theme parties; excelling at Cards Against Humanity and Puns of Anarchy; cruising the ocean; or venturing into the woods to live like Thoreau. 

More from MarketSmiths

Grammar in copywriting

Why I’m Grammar Obsessed

B2B leaders need to ask themselves questions to improve thought leadership.

3 Questions For B2B Leaders to Ask to Improve Thought Leadership

Mortenson shows great copywriting for sustainability.

How Mortenson Approaches Copywriting for Sustainability

Effective crisis communications are a digalogue, not a monologue.

Leadership Crisis Communications: It’s a Dialogue, Not a Monologue

Inc 5000 content agency

M/WBE certified enterprise.

Design by WorstOfAllDesign. Digital Strategy by MadPipe. Photography by Chellise Michael.