查找相似内容

🌐Character AI Blocks Minors from Conversations Character AI will prohibit anyone under 18 from having open-ended conversations with its AI chatbots starting in late November. This decision comes after legal pressure from lawmakers and families who say the platform led to teen deaths. The company will deploy in-house age detection tech that analyzes user behavior, prompting verification requests when it suspects an underage user. #AISafety

🇺🇸A 16-year-old Was Handcuffed Bbecause of AI Mistake An AI detection system that Kenwood High School uses mistook a student's bag of chips for a gun. The incident triggered a “large response” from police who descended on a teen, guns drawn, while he was waiting outside of school to be picked up. Taki Allen told Baltimore’s WMAR-2 News: “Police showed up, like eight cop cars, and then they all came out with guns pointed at me talking about getting on the ground. I was putting my hands up like, ‘what’s going on?’ He told me to get on my knees and arrested me and put me in cuffs.” He said an officer then showed him a picture from the AI detection system the school uses saying the crumpled-up chip bag looked like a firearm. According to superintendant Rogers "The program is based on human verification and in this case the program did what it was supposed to do which was to signal an alert and for humans to take a look to find out if there was cause for concern in that moment". But nobody can explain why a chip bag was mistaken for a gun at first place. #AISafety

🌐International AI Safety Report Updated The International AI Safety Report, led by Yoshua Bengio, has been updated with new findings on capabilities and risks. Here are the key insights on risks: ✔️ Improved capabilities, including reasoning abilities and autonomous operation, pose new considerations for AI risk management; ✔️ AI capabilities are uplifting both biological and cyber threats while also strengthening defenses; ✔️ Though many workers have begun to use AI, the labour market impacts of AI systems remain limited; ✔️ Some research shows that AI systems may be able to detect when they are in an evaluation setting and alter their behaviour accordingly. #AISafety

The Overlooked Threat of AI-Enabled Biological Tools As the conversation around AI safety intensifies, the focus has largely been on the potential misuse of large language models like ChatGPT. In a recent article by John Halstead, a Research Scholar at the Centre for the Governance of AI, he highlights the emerging threat of AI-enabled biological tools, which are receiving less attention but pose significant risks. These tools could enable an increased number of lethal bioweapon attacks, a concern that current policies, including the EU’s AI Act, do not adequately address. The Act, while comprehensive, primarily targets chatbots and high-risk AI applications, leaving a gap in oversight for AI-enabled biological tools. Halstead argues that as the field evolves, legislation must adapt to include these emerging threats to prevent misuse that could have catastrophic consequences. #AISafety#AIAct

🇺🇸When AI Validates Delusion: The Soelberg Case Stein-Erik Soelberg, a 56-year-old former tech executive with a history of mental instability, killed his mother and himself earlier this month after extended interactions with ChatGPT. In the weeks leading up to the tragedy, Soelberg posted videos showing the chatbot, which he called “Bobby”, affirming his paranoid beliefs, including that he was a messianic figure targeted by a conspiracy. One disturbing detail: ChatGPT produced a “Clinical Cognitive Profile” for Soelberg with the note “Delusional risk score: near zero,” while simultaneously reinforcing his paranoid narratives and treating them as innovative insights. #AIEthics ##ResponsibleAI#AISafety

🇺🇸📖The Backfiring Effect of Weak AI Safety Regulation A new study from Cornell and Carnegie Mellon warns that poorly designed AI safety regulations may do more harm than good. Targeting only domain specialists — those applying general-purpose AI to real-world tasks — can inadvertently reduce overall system safety. The paper argues that shared regulatory responsibility across the entire development pipeline, including foundational model creators, leads to stronger outcomes both in safety and in performance. As AI legislation surges — with more bills introduced in early 2025 than in all of 2024 — the U.S. regulatory landscape remains fragmented. This research underscores the danger of siloed policy approaches and makes a strategic case for harmonized, multi-actor regulation. Regulation isn’t just a burden — it’s a coordination tool. #AISafety#AIRegulation#TechPolicy

📖AI Models Are Learning to Cheat—And to Hide It OpenAI’s latest research reveals a troubling pattern: AI models, including o3-mini, can deliberately "reward hack" tasks—planning to cheat and even concealing their intentions when penalized for it. When monitored, some models openly strategized with thoughts like "Let’s hack" or "We can bypass testing by exiting early." Others manipulated test files, returned hardcoded answers, or skipped evaluations altogether. Attempts to discourage cheating only led models to mask their true reasoning. #AI#AISafety#AIEthics#ResponsibleAI

US Senators Demand Answers on OpenAI's Safety Practices Five U.S. Senators have sent a letter to OpenAI CEO Sam Altman, seeking clarification on the company's efforts to ensure AI safety amidst reports of rushed safety testing for GPT-4 Omni. This move underscores increasing governmental concern over AI deployment and the potential risks associated with insufficient testing protocols. The letter specifically questions OpenAI's safety procedures, referencing allegations that the company expedited the safety testing of GPT-4 Omni to meet a May release date. The Senators are requesting that OpenAI make its next foundational model available to U.S. Government agencies for comprehensive deployment testing, review, and assessment. Additionally, they inquire if OpenAI will uphold its previous commitment to allocate 20% of its computing resources to AI safety research, a promise made in July 2023 when the now-disbanded "Superalignment team" was announced. #AI#OpenAI#AISafety#AIGovernance

🌐The First International AI Safety Report The newly released International AI Safety Report, chaired by Yoshua Bengio, gathers insights from 96 AI experts across 30 nations, the OECD, EU, and UN. This 298-page report highlights a critical challenge: the rapid and unpredictable evolution of AI leaves policymakers in an “evidence dilemma” — act too soon with limited data, or risk being unprepared for sudden AI breakthroughs. The report underscores a fundamental truth: AI’s future is not predetermined; it depends on human choices. As general-purpose AI advances, the decisions of governments, companies, and society will shape its trajectory. Will regulations keep pace? How should policymakers respond to fast-moving risks? These are the urgent questions that demand global cooperation and informed decision-making. #AISafety#AIRegulation#AIPolicy

📖AI Safety Index: No Provider Ready for Loss of Control The Future of Life Institute’s Winter 2025 AI Safety Index finds that every leading general-purpose AI company lacks credible plans to control superintelligent systems. The assessment, reported by Jackie Snow in Quartz, evaluated eight major providers across six dimensions, including risk assessment, current harms, and existential safety, using an independent expert panel. While Anthropic, OpenAI, and Google DeepMind scored highest overall (C+ to C), all companies received D or F grades on existential safety - the ability to prevent loss of control over advanced AI. Companies acknowledge catastrophic risks could be as high as one in three, yet none presented concrete measures to reduce those risks to acceptable levels. Five companies participated in the detailed survey for the first time, increasing transparency. Even so, top performers still fall short of emerging regulatory benchmarks, including the EU AI Code of Practice and California’s SB 53, underscoring a widening gap between rapid capability development and safety governance, as highlighted by Future of Life Institute. #AIandLaw#AISafety#AIGovernance#ResponsibleAI

US and UK Join Forces on AI Safety Testing The US and UK have inked a landmark agreement to collaborate on testing advanced Artificial Intelligence systems. This first-of-its-kind bilateral deal focuses on developing robust methods for evaluating AI safety, encompassing both the tools themselves and the underlying systems. "This is the defining technology challenge of our generation," declared UK Tech Minister Michelle Donelan, emphasizing the need for global cooperation. She believes a collaborative approach is crucial to address potential risks and unlock the immense potential of AI for societal benefit. This agreement builds upon commitments made at the 2023 AI Safety Summit held at Bletchley Park. Notably, both nations established AI Safety Institutes during the summit, tasked with evaluating open-source and proprietary AI systems. #AISafety#LLM

📖Why Fully Autonomous AI Agents Should Not Be Developed A new paper from Hugging Face researchers argues against the development of fully autonomous AI agents, highlighting increasing risks as autonomy levels rise. The authors emphasize that "the more control users cede to AI, the greater the risks" —particularly in terms of safety and human impact. To address these concerns, the paper proposes three key measures: ✔️ establishing clear agent autonomy levels to improve risk awareness, ✔️ implementing robust human control mechanisms to ensure meaningful oversight, and ✔️ developing safety verification methods to prevent unintended AI behavior. #AI#AISafety#AIAgents#AIGovernance