The smash success of OpenAI’s ChatGPT is a bolt from the blue even for OpenAI and its partner Microsoft. Nobody could have predicted that the chatbot would be the curtain raiser for AI to most of the world. Responding swiftly, Chinese web giant Baidu also sprung into action announcing that it was working on releasing its latest chatbot called ‘Ernie Bot’ in March. The search engine giant’s shares reacted with a 15% jump following the news, reports AIM.
OpenAI’s ChatGPT
ChatGPT was built on top of GPT-3.5, an upgraded version of GPT-3 which OpenAI released quietly. GPT-3.5 was trained on a dataset (text blended with code) published before 2021, which means ChatGPT will not be able to access recent data and events.
While OpenAI did not release an official peer-reviewed paper for the chatbot, the blog mentioned that the model used Reinforcement Learning from Human Feedback or RLHF which was also used to train InstructGPT.
Released in January last year, the company’s range of InstructGPT models were found to be far better than GPT-3 – they were more accurate, produced less toxic output and followed instructions better. Needless to say, this decision to shift to reinforcement learning has worked in favour of models like ChatGPT and InstructGPT because they align better with the user’s intention.
ChatGPT continues to collect data from users to keep training and fine-tuning the model. Plus, users can upvote or downvote ChatGPT’s responses and also fill out a text field given for feedback. ChatGPT is also prone to making factual errors and the phenomenon of ‘hallucination’ that plagues chatbots of its ilk. There are plenty of examples of the chatbot re-evaluating their answer from a correct to an incorrect one, especially if the user expresses their doubt.
Google’s Bard
Against this, Google’s AI chatbot, Bard, will be able to respond to user queries using information drawn from the web. Alphabet CEO Sundar Pichai stated that the platform will be made widely available to the public in the coming weeks. Bard is powered by Google’s Language Model for Dialogue Applications (LaMDA), which first came under attention after engineer Blake Lemoine infamously called the LLM ‘sentient’.
The chatbot is still under testing while being open to a limited number of users. An obvious advantage that Bard has over ChatGPT is that it is expected to have access to more data. But the chatbot has already flubbed its first demo at the tech giant’s event in Paris yesterday proving that Bard too wasn’t infallible.
When Bard was asked about discoveries from the James Webb Telescope, it responded with three bullet points around what was presumed to be the latest developments, one among which stated that the telescope “took the very first pictures of a planet outside of our own solar system”. However, a host of astronomers pointed out that this was in fact incorrect and that the first image of an exoplanet was taken in 2004.
Interestingly when ChatGPT was posed with the same question, it restricted itself to only respond with the potential of the James Webb Telescope to discover exoplanets, stars and galaxies, which now we all know is old news.
Baidu’s Ernie Bot
Famously referred to as China’s Google, Baidu stated that it is internally testing its own ChatGPT-style chatbot called Ernie Bot. Due to be released next month, the bot is trained on its LLM, ERNIE or ‘Enhanced Representation through Knowledge Integration’, which was released in 2019.
ERNIE, a bilingual model expected to understand both English and Chinese performs a range of tasks including language comprehension, language generation and text-to-image generation. Ernie Bot is expected to be built on top of ERNIE 3.0 Titan, a pre-trained language model with 260 billion parameters, which is 50% more parameters than ChatGPT.
ERNIE has a series of advanced LLMs which can perform a variety of functions and while the language generation is from ERNIE 3.0 Titan, its text-to-image generation is from ERNIE-ViLG.
Did you subscribe to our daily Newsletter?
It’s Free! Click here to Subscribe
Source: AIM
