Meta Researchers Want AI Training to Control More Than Just Rewards

Morning, {{first name | folks}}! A lot happened in AI this week. Meta introduced a new way to train models with reinforcement learning, Claude tackled a nine-loop physics calculation, Trump is meeting Anthropic’s CEO, Meta is expanding what Muse can access, and OpenAI fixed a vision bug in GPT-6. Let’s get into it.

Meta has introduced MaD-RL, a reinforcement learning approach that lets developers control the distribution of a model’s outputs instead of simply rewarding individual answers. Meta’s researchers found that common RL methods such as GRPO can concentrate a model’s outputs around one dominant behavior, reducing diversity.

MaD-RL lets developers specify the distribution they want and train the model toward it using different measures such as KL and Jensen-Shannon divergence. Meta tested the approach on mathematical reasoning and programming tasks, where controlling how often different types of outputs appear can matter more than simply maximizing a single reward.

Anthropic says Claude has completed a nine-loop calculation in theoretical physics, a problem researchers had been working on for years. Claude worked through the calculation over several days using existing physics techniques, and physicist Lance Dixon independently checked the result.

What makes this interesting is the amount of work the model was able to handle across a long and complicated calculation. There were plenty of places for errors to creep in, but Claude was able to work through the problem and produce a result that could be checked. It’s another example of AI taking on longer, more technical research tasks rather than simply answering a single question.

President Trump plans to host Anthropic CEO Dario Amodei for a private White House dinner, according to Axios. It would be their first one-on-one meeting, after Amodei was notably absent from last week’s state dinner with other major tech CEOs.

The meeting comes after months of tension between Anthropic and the administration over AI safety and development. Trump has pushed for the U.S. to move quickly on AI, while Amodei has publicly warned about the risks of increasingly capable systems. With frontier AI becoming a bigger part of U.S. technology policy, the relationship between the companies building these systems and the government overseeing them is becoming harder to ignore.

Meta is expanding Muse, its AI agent, with access to more of the tools people already use. Muse will connect to services including Shopify, Stripe, GitHub, Notion, Expedia, and Instacart. It will also be able to use apps on a Mac, work with Meta’s smart glasses and have its own email address. Some tasks can run in the background while Muse works on them.

That gives Muse more ways to handle things without someone sitting there and guiding every step. It can start a task, work across different services, and come back when it is finished. Once an AI can use your apps instead of just talking about them, the question becomes how much access you are comfortable giving it.

OpenAI has fixed a bug affecting image understanding in GPT-6 Sol and GPT-6 Luna across the API and Codex. The issue could hurt performance on visual tasks, including computer use, and OpenAI says developers should rerun evaluations and retry affected workflows.

The update is a useful reminder for teams building on model APIs. Your code can stay exactly the same while the behavior underneath it changes. Production evaluations need to keep checking the model, not just the application around it. That matters even more for systems that rely on vision or computer use.