Spark Capital is leading a $300 million investment in artificial intelligence startup Anthropic. Google invested $400 million in Anthropic. Anthropic is a competitor to OpenAI which makes ChatGPT. Anthropic is now valued at 4.1 billion.
Anthropic was founded in 2021 by former senior members of OpenAI, principally siblings Daniela Amodei and Dario Amodei, the latter whom served as OpenAI’s Vice President of Research. The Amodei siblings were among others who left OpenAI due to directional differences, specifically regarding OpenAI’s ventures with Microsoft in 2019.
Anthropic began development on its own AI chatbot, named Claude. Similar to ChatGPT, Claude uses a messaging interface where users can submit questions or requests and receive highly-detailed and relevant responses.
Arxiv – Constitutional AI: Harmlessness from AI Feedback
As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, and so we refer to the method as ‘Constitutional AI’. The process involves both a supervised learning and a reinforcement learning phase. In the supervised phase we sample from an initial model, then generate self-critiques and revisions, and then finetune the original model on revised responses. In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences. We then train with RL using the preference model as the reward signal, i.e. we use ‘RL from AI Feedback’ (RLAIF). As a result we are able to train a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them. Both the SL and RL methods can leverage chain-of-thought style reasoning to improve the human-judged performance and transparency of AI decision making. These methods make it possible to control AI behavior more precisely and with far fewer human labels.
Initially available in closed beta through a Slack integration, Claude is now accessible to users via the Poe app by Quora along with two other chatbots. The app is currently available for iOS and will soon be available for Android.
Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.
Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.
A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts. He is open to public speaking and advising engagements.
