👨💻 TLDR: Following OpenAI’s ChatGPT integration into Bing in February 2023, Bing’s traffic increased by 15.8%, while Google’s declined by 1%, a consequential shift considering each 1% change in search market share equates to $2 billion. A recent study compared user behaviors using Google’s keyword-based method and ChatGPT’s conversational approach, involving 95 participants. Despite ChatGPT’s slower response and misinformation issues, users spent less time on tasks without hindering overall performance. Although it struggled with fact-checking tasks, ChatGPT was great with straightforward questions and leveled user performance across various education levels unlike Google. Users rated ChatGPT’s information quality higher, expressed similar trust levels for both tools, and found ChatGPT more useful and satisfying.
In an intriguing new study, scientists set out to determine how users behave differently when seeking information through traditional search engines versus artificial intelligence-powered chatbots like ChatGPT. The digital experiment showed that the chatbot users managed to complete tasks in less time without hurting the task performance.
Interestingly, ChatGPT seemingly bridged the performance divide between various educational levels. While it effectively answered simple questions and provided basic solutions, it ran into challenges in tasks requiring fact-checking.
Here’s the graphic from the paper describing the performance on two information-gathering tasks comparing five different educational levels from high school to doctorate (highlights by me):
Despite these hiccups, the users deemed ChatGPT’s responses superior in quality to Google Search, and placed equal trust in both platforms. They also reported better overall experience with the chatbot, finding it enjoyable and satisfying. However, researchers warn of potential risks such as overreliance and misinformation propagation through the chatbot.
My Personal View
These insights could be game-changing for search engine development, suggesting the possibility of weaving in chatbot technology for a richer user experience.
Google search has had time to optimize for speed and user experience whereas ChatGPT still has many low-hanging fruits to be deployed from a UX standpoint:
- Improve speed by 10x so ChatGPT will feel much more intuitive and responsive.
- Integrate with many apps (e.g., using Plugins) and browsers so ChatGPT will be 10x more useful.
- Improve the chatbot performance (e.g., integrating real-time data) so ChatGPT output quality will increase by 10x.
Taken together, these and other improvements can easily result in multiple orders of magnitude of performance increase so ChatGPT will not only be faster and produce better quality results but it will also feel much more natural!
✅ Recommended: Free ChatGPT Prompting Cheat Sheet (PDF)
While working as a researcher in distributed systems, Dr. Christian Mayer found his love for teaching computer science students.
To help students reach higher levels of Python success, he founded the programming education website Finxter.com that has taught exponential skills to millions of coders worldwide. He’s the author of the best-selling programming books Python One-Liners (NoStarch 2020), The Art of Clean Code (NoStarch 2022), and The Book of Dash (NoStarch 2022). Chris also coauthored the Coffee Break Python series of self-published books. He’s a computer science enthusiast, freelancer, and owner of one of the top 10 largest Python blogs worldwide.
His passions are writing, reading, and coding. But his greatest passion is to serve aspiring coders through Finxter and help them to boost their skills. You can join his free email academy here.