Chatbots and companion AI need to be regulated to avoid harm to vulnerable users, like kids and the mentally impaired.
The anthropomorphic, addictive, and sycophantic design of chatbots and companion AI creates a lethal cocktail when mixed with the unchecked use of the vulnerable (children and the mentally impaired). Often with unfavorable and negative outcomes for humans and society.
The tragic cases of Sewell and Adam resulting in suicides after interacting with these trending tech calls for urgent and decisive action by policymakers and AI actors. A few US States have already heeded the call and are taking concrete steps through preventive legislation. Worthy of mention are California, Oregon, Colorado, Washington, Utah, New York, Rhode Island, Georgia, and Connecticut.
https://centerforhumanetechnology.substack.com/p/reckless-race-for-ai-market-share