Center for Human-Compatible Artificial Intelligence – News

Learning to Coordinate with Experts

Fri, 07 Mar 2025 14:16:30 GMT

Khanh Nguyen, Benjamin Plaut, Tu Trinh, and Mohamad Danesh introduce a fundamental coordination problem called Learning to Yield and Request Control (YRC), where the objective is to learn a strategy that determines when to act autonomously and when to seek expert assistance. They build an open-source benchmark featuring diverse domains, propose a novel validation approach, and investigate the performance of various learning methods across diverse environments, yielding insights that can guide future research.

Computational Frameworks for Human Care

Thu, 20 Feb 2025 14:08:22 GMT

Brian Christian, CHAI Affiliate, has published an article titled “Computational Frameworks for Human Care” in the most recent issue of Daedalus, the journal of the American Academy of Arts and Sciences. In it, Christian traces how AI alignment has progressed from simple reward mechanisms toward care-like relationships, revealing both the potential and limitations of machine caregiving while deepening our understanding of human care itself. The issue is titled “The Social Science of Caregiving” and was co-edited by CHAI Affiliate Alison Gopnik.

A Practical Definition of Political Neutrality for AI

Tue, 04 Feb 2025 15:00:26 GMT

NEW: Our current research project to build political neutrality evaluations.

RvS: What is Essential for Offline RL via Supervised Learning?

Sat, 18 Jan 2025 09:55:12 GMT

Scott Emmons, PhD student, was an author on “RvS: What is Essential for Offline RL via Supervised Learning?”

Getting By Goal Misgeneralization With a Little Help From a Mentor

Wed, 25 Dec 2024 09:36:25 GMT

“Tu Trinh, Ben Plaut, Khanh Nguyen, and Mohamad Danesh wrote the paper, “Getting By Goal Misgeneralization With a Little Help From a Mentor.” This paper explores whether goal misgeneralization can be mitigated by allowing an agent to ask for help when it is uncertain. The answer is mostly yes, although our current methods have substantial weaknesses and there are lots of interesting avenues for future work.”Tu Trinh, Ben Plaut, Khanh Nguyen, and Mohamad Danesh wrote the paper, This paper explores whether goal misgeneralization can be mitigated by allowing an agent to ask for help when it is uncertain. The answer is mostly yes, although our current methods have substantial weaknesses and there are lots of interesting avenues for future work.

Linear Probe Penalties Reduce LLM Sycophancy

Sat, 14 Dec 2024 09:34:04 GMT

Visiting ETH MsC student Henry Papadatos and supervising CHAI PhD student Rachel Freedman publish an article “Linear Probe Penalties Reduce LLM Sycophancy” at the NeurIPS SoLaR workshop. The paper demonstrates a generalizable methodology for reducing unwanted LLM behaviors that are not sufficiently disincentivized by RLHF fine-tuning

Rachel Freedman selected as inaugural Cooperative AI Fellow

Sat, 30 Nov 2024 09:30:08 GMT

Rachel Freedman, PhD Student, has been selected as one of the fellows for Cooperative AI’s PhD Fellow Program.

Representative Social Choice: From Learning Theory to AI Alignment

Tue, 12 Nov 2024 12:55:26 GMT

Tianyi Qiu, CHAI Intern, wrote this paper which was accepted by NeurIPS 2024 Pluralistic Alignment Workshop. Here is the link to the paper.

Getting By Goal Misgeneralization With a Little Help From a Mentor

Thu, 10 Oct 2024 09:19:23 GMT

Khanh Nguyen, Mohamad Danesh, Ben Plaut, and Alina Trinh wrote this paper which was presented at Towards Safe & Trustworthy Agents Workshop at NeurIPS 2024.

Language-Guided World Models: A Model-Based Approach to AI Control

Wed, 18 Sep 2024 14:32:50 GMT

Khanh Nguyen, CHAI Postdoctoral Fellow, published a paper at the Fourth International Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics (ACL 2024).

“The Alignment Problem” Wins Xingdu Book Award

Thu, 29 Aug 2024 08:50:23 GMT

Brian Christian’s book “The Alignment Problem” was announced as the sole winner in the New Knowledge Category for Imported Editions at the Xingdu Book Award ceremony in China. The Chinese translation was published this past year by Hunan Science & Technology Press.

Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback

Wed, 07 Aug 2024 12:08:47 GMT

Rachel Feedman, CHAI Phd Student, and Wes Holliday, CHAI Affiliate, published a paper at the International Conference on Machine Learning

AI Alignment with Changing and Influenceable Reward Functions

Tue, 23 Jul 2024 12:04:59 GMT

CHAI Researchers, Micah Carroll, Davis Foote, Anand Siththaranjan, Stuart Russell, and Anca Dragan, wrote the paper, “AI Alignment with Changing and Influenceable Reward Functions” which was accepted to ICML.

Forget deepfake videos. Text and voice are this election’s true AI threat.

Mon, 08 Jul 2024 11:59:38 GMT

Jonathan Stray, Senior Scientist at CHAI, and Jessica Alter, tech entrepreneur and co-founder of Tech for Campaigns, wrote an op-ed for The Hill regarding the risks posed by AI in this current election cycle.

Mitigating Partial Observability in Decision Processes via the Lambda Discrepancy

Fri, 28 Jun 2024 10:50:36 GMT

This paper investigates fundamental concepts related to detecting and mitigating partial observability by measuring misalignment between value function estimates. The paper was presented at the “Finding the Frame” workshop at RLC 2024 and the “Foundations of Reinforcement Learning and Control” workshop at ICML 2024.

8th Annual CHAI Workshop

Tue, 18 Jun 2024 10:42:38 GMT

CHAI held its 8th annual workshop at Asilomar Conference Grounds from June 13th to June 16th in Pacific Grove. The workshop had over 200 attendees which was the highest attendance to date. The workshop featured over 60 speakers and panelists and covered a wide array of topics from Societal Effects of AI to Adversarial Robustness.

When Code Isn’t Law: Rethinking Regulation for Artificial Intelligence

Wed, 05 Jun 2024 11:46:09 GMT

Brian Judge, Mark Nitzberg, and Stuart Russell wrote an article that was featured in Oxford Academic’s Policy and Society.

Committing to the wrong artificial delegate in a collective-risk dilemma is better than directly committing mistakes

Mon, 13 May 2024 09:00:00 GMT

New research from computer scientists Inês Terrucha, Elias Fernández Domingos, Pieter Simoens, and Tom Lenaerts at the Vrije Universiteit Brussel, Université Libre de Bruxelles, and UC Berkeley’s Center for Human-Compatible AI

Reinforcement Learning with Human Feedback and Active Teacher Selection (RLHF and ATS)

Tue, 30 Apr 2024 09:00:00 GMT

CHAI PhD graduate student, Rachel Freedman gave a presentation at Stanford University on critical new developments in AI safety, focusing on problems and potential solutions with Reinforcement Learning from Human Feedback (RLHF).

Reinforcement Learning Safety Workshop (RLSW) @ RLC 2024

Mon, 15 Apr 2024 09:00:00 GMT

Important Dates
Paper submission deadline: May 10, 2024 (AoE)
Paper acceptance notification: May 23, 2024