Let us know about free updates
Training a chatbot can be difficult. Last month, Openai rolled back a ChatGPT update because its “default personality” was so empathetic. (Maybe the company’s training data were taken from a transcript of President Donald Trump’s Cabinet meeting.)
While artificial intelligence companies wanted chatbots to be more intuitive, user inquiries were biased towards being overly supportive and dishonest. “A sycopantic interaction can be uncomfortable, anxious and painful. We’re short on it and we’re working to get it right,” the company said in a blog post.
Reprogramming Sycophantic Chatbots may not be the most important dilemma facing Openai, but it chimes with its biggest challenge. Creates a reliable personality throughout the company. This week, Openai was forced to roll back the latest planned company update designed to turn the company into a for-profit enterprise. Instead, they will move to a public benefit corporation under the supervision of a nonprofit committee.
It does not resolve structural tensions at the core of Openai. It also does not satisfy Elon Musk, one of the co-founders of a company that pursues legal action against open eyes because he was lost from his original purpose. Will the company accelerate its AI product deployment to keep financial supporters happy? Or are you pursuing a more deliberative scientific approach to stay true to your humanitarian intentions?
Openai was founded in 2015 as a non-profit research lab dedicated to developing artificial general information for the benefit of humanity. However, the company’s mission, like its definition of AGI, has been blurred ever since.
Openai CEO Sam Altman quickly realized that the company needed a huge amount of capital to pay for the research talent and computing power needed to stay at the forefront of AI research. To that end, Openai created a for-profit subsidiary in 2019. It was a successful breakout for chatbot ChatGpt, where investors were willing to throw money and valued Openai at $260 billion during their latest fundraising. With 500mn users every week, Openai has become a “accidental” consumer internet giant.
Altman, who was fired and re-employed by the nonprofit committee in 2023, says he wants to build a “world brain” that now requires hundreds of billions of dollars more investment. The only problem with his ambitious ambitions is that Openai has yet to develop a viable business model, as tech blogger Ed Zitron yells in increasingly salty words. Last year, the company spent $9 billion and lost $5 billion. Is that financial valuation based on hallucinations? There is growing pressure from investors to quickly and quickly put pressure on Openai and commercialize the technology.
Furthermore, the definition of AGI continues to shift. Traditionally, it has mentioned points that machines outperform humans on a wide range of cognitive tasks. However, in a recent interview with Stratechery’s Ben Thompson, Altman admitted that the term was “almost completely devalued.” However, he embraced the narrower definition of AGI as an autonomous coding agent capable of writing software like humans.
Based on that score, it appears that large AI companies think they are closer to AGI. One giveaway is reflected in their own employment practices. According to Zeki data, the top 15 American AI companies desperately hire software engineers at a rate of up to 3,000 a month, recruiting a total of 500,000 between 2011 and 2024.
A recent research paper from Google Deepmind, which aims to develop AGIs, highlighted four major risks of increasingly autonomous AI models. Misuse by bad actors. Inconsistencies in cases where AI systems do things that are unintended; mistakes that cause unintended harm. There is a multifactorial risk when unpredictable interactions between AI systems have bad results. These are all heart-breaking challenges with potentially catastrophic risks and may require several collaborative solutions. The more powerful AI models, the more careful developers need to deploy them.
So how frontier AI companies are governed is a problem not only for corporate committees and investors, but for all of us. Openai still has a contradictory and contradictory impulse in that respect. The struggle with Sycophancy is the least of the problems as you get closer to AGI, but defines it.
john.thornhill@ft.com
