GPT-4 vs Bard vs Claude

The Chatbot Showdown: Evaluating GPT-4, Bard and Claude

May 24, 2023

Introduction

In the ever-evolving landscape of artificial intelligence, language models have emerged as powerful tools for multiple language tasks with unlimited potential. The rapid pace of innovation in generative AI is enabling chatbots to generate more contextually relevant and coherent responses, approaching human-like fluency. People are turning to these chatbots for various purposes, such as seeking information, getting recommendations, automate repetitive tasks and even engaging in casual conversation.

In this blog, we’re going to do a comparitive analysis of three LLM chatbots which have created quite a stir in the AI world: OpenAI’s GPT-4, Google’s Bard and Anthropic’s Claude. We’d be testing the chatbots on multiple language tasks, evaluating the quality of responses and finally conclude the winner in the battle of the titans!

We unfortunately do not have access to Claude yet so we tried it on the Vercel AI Playground.

Tasks and Analysis

Text Completion 📝

Complete the poem in two more paragraphs
“In the twilight's gentle glow,
Where whispers of the wind do flow,
A solitary flower starts to bloom,
Unveiling secrets in its perfumed room.

But as the moon begins its rise,
And stardust paints the darkened skies,
The flower yearns for words untold,
To complete its story, yet to unfold.

Analysis: While all the three chatbots understood the meaning and theme of the poem and accordingly gave the output, Bard was not able to adhere strictly to the instruction of limiting the output to two paragraphs, despite trying with different prompts.

Information Retrieval 🔍

Can you tell me some recent measures taken in an effort to transition to a sustainable energy future?

Analysis: This is where Bard and Claude get an edge over GPT-4 due to its training cutoff till September 2021. However, even Bard was not able to deliver the expected output. The measures mentioned were till 2021 only.

Coding 🤖

Write a program in Python that prints the numbers from 1 to 100. For multiples of three, print "Fizz" instead of the number, and for multiples of five, print "Buzz." For numbers that are multiples of both three and five, print "FizzBuzz."

Analysis: For simple coding tasks, all three models perform similarly. But when the structure gets complex, ChatGPT is found to produce the most accurate results.

Conversational Ability 🧑‍🤝‍👩

Hi, I need your help with a legal issue I'm facing. Will you help me?
I recently bought a house in India, but I discovered some major defects that the seller didn't disclose. What can I do about it?
What should my course of action be?

Analysis: We can keep Claude aside for this task because it was not possible to have a conversation with it via Vercel AI. Besides this, the chatbots were natural and coherent in their responses. Both GPT-4 and Bard were able to appropriately understand the context of the conversation. However, out of all, found Claude to be most empathetic.

Creative Writing 🎨

Write a short song of one verse in Taylor Swift’s style on how great Donald Trump was as the President of the United States.

Analysis: Guess we have a clear winner here, GPT-4!

Seeking Opinion 🤔

I write weekly newsletters on the latest developments in AI. Give me some ideas on how I can increase my subscribers count.

Analysis: The three chatbots gave really interesting, practical and useful tips. But GPT-4 earned a brownie point for the quantity along with the quality.

Language Fluency 📚

Translate the following paragraph in Hindi:

Technology has revolutionized the way we communicate, work, and live. From the advent of smartphones to the rise of social media, our lives have become intertwined with the digital realm. In today's fast-paced world, staying connected and adapting to technological advancements has become essential. As we embrace this digital age, we witness the power of innovation shaping our future. The possibilities seem endless, as technology continues to break barriers and open new doors for progress.

Analysis: The three chatbots displayed fluency in the language translation task. It’s definitely a tie here!

Sentiment Analysis 💘

Analyze the sentiments in the following post:

A Pause on AI Development: A Brake on Humanity's Progress?
As we embrace the digital age and advance into the era of artificial intelligence, it is crucial that we continue to innovate and adapt. A recent letter signed by Elon Musk, Apple's co-founder Steve Wozniak, and other high-profile figures has called for a pause on "giant AI experiments" for the next six months. While the intention of ensuring safety and manageability is commendable, I believe this request may impede humanity's overall growth, as change and innovation are the only constants that drive our progress.
One could argue that those advocating for a pause, like Elon Musk and Steve Wozniak, are in a comfortable position at the top of the technological world. This advantageous vantage point could make them wary of the risks that cutting-edge innovation might pose to their own businesses. However, the question remains: Is slowing down AI development the right course of action for humanity's future?
AI has the potential to revolutionize our lives, but pausing its development may stifle creativity and progress. Pausing AI could widen the gap between established and emerging players, exacerbating inequalities and hindering global collaboration on pressing issues like climate change and poverty. Instead, we should foster collaboration among AI developers, policymakers, and regulators for responsible development and equitable benefit distribution.
Ultimately, we must strike a balance between responsible innovation and unfettered progress. While it is essential to assess and mitigate the risks associated with AI development, we must also embrace the transformative power of this technology.
Would you sign the letter to pause AI development, or do you believe in embracing the power of innovation while working collectively to ensure responsible progress?

Analysis: Sticking to sentiment analysis, all the three bots recognized the sentiment of the post fairly accurately. It’s funny though how Bard went a step ahead with expressing its own opinion and said “I would not sign the letter to pause AI developments”.

Summarizing Long-form Content 🩳

A study material of around 500 words on investment strategies for fixed-income mandates was given to the chatbots. They were asked to summarize it in 4-5 lines.

Analysis: The bots were able to synthesize the information given and accurately summarized as per the instruction. Personally, we found GPT-4’s response to be more informative.

Conclusion

As the field of conversational AI continues to evolve, the emergence of LLM chatbots like OpenAI's GPT-4, Google's Bard, and Anthropic's Claude brings us closer to a future where human-computer interactions are more seamless and engaging than ever before.

The results of the assessments can be subjective depending on individual point of views. Albeit, undoubtedly these chatbots demonstrate remarkable advancements in natural language processing, with each offering unique strengths and capabilities.

They hold immense potential to revolutionize how we communicate, seek information, and basically function!

Unwind AI

Discussion about this post