Learnijoy
ResearchAI ResearchAI Engineering & DevTools

Open-Weight LLMs Show Promise for PDDL Domain Repair.

Nader Karimi Bavandpour, Pascal Bercher· August 19, 2026 View original

Key takeaways

  • Open-weight LLMs can significantly improve PDDL domain repair F1 scores over symbolic baselines.
  • LLMs struggle with consistently satisfying all test constraints for reliable repair.
  • The "LLM-only" approach currently lacks the necessary reliability for automated repair.
  • Hybrid approaches combining LLMs with symbolic methods may be more effective.

Who benefits

RoboticsLogisticsManufacturingAI Development

Summary

This paper evaluates the ability of open-weight large language models to repair errors in Planning Domain Definition Language (PDDL) models, which are crucial for AI planning. While LLMs significantly outperform symbolic baselines in F1 score, they struggle to consistently satisfy all test constraints, indicating limitations in reliable automated repair.

AI planning systems rely on explicit world models, often expressed in the Planning Domain Definition Language (PDDL). Detecting and repairing errors in these PDDL models is an active area of research, typically involving automated methods that modify the model based on positive (successful) and negative (failed) test plans. This study explores whether recent open-weight large language models (LLMs) can perform this PDDL domain repair task autonomously. The researchers adopted an LLM-only approach, evaluating models on their ability to correct PDDL errors. The findings indicate that the best-performing LLM achieved an F1 score of 0.87, a substantial improvement over the symbolic baseline's 0.49. However, despite this strong F1 score, the LLMs struggled with consistently satisfying all test constraints, with test pass rates as low as 0.06 in some domains. This suggests that while LLMs show significant potential for understanding and modifying PDDL, they currently lack the reliability needed for fully automated and trustworthy model repair.

Why it matters

For professionals in AI development and automation, this research highlights both the potential and current limitations of using LLMs for complex symbolic reasoning tasks like PDDL model repair. It suggests that while LLMs can assist, human oversight or more robust verification methods are still necessary for mission-critical applications.

How to implement this in your domain

  1. 1Investigate integrating open-weight LLMs as a preliminary step in PDDL domain repair workflows to generate initial repair suggestions.
  2. 2Develop hybrid systems that combine LLM-generated repairs with symbolic verification or human-in-the-loop validation.
  3. 3Explore fine-tuning open-weight LLMs on specific PDDL error patterns and repair strategies to improve constraint satisfaction.
  4. 4Benchmark LLM performance on domain-specific PDDL repair tasks to assess practical applicability.

Original post by Nader Karimi Bavandpour, Pascal Bercher

"arXiv:2608.17341v1 Announce Type: new Abstract: AI planning is concerned with finding a sequence of actions that achieves a specified goal. It relies on explicit models of the world, commonly represented in the Planning Domain Definition Language (PDDL). An active line of researc…"

View on X

Originally posted by Nader Karimi Bavandpour, Pascal Bercher on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses

More in AI Research

AI Research

New Research Explores Fourth-Moment Geometry of Rademacher Sums

This research determines how higher moments of normalized Rademacher sums depend on their fourth-order mass, establishing Gaussian stability inequalities and sharp Khintchine constants. The findings settle several long-standing conjectures in probability theory.

Peigan Gao, Jian QianAug 19, 2026
AI ResearchAI Engineering & DevTools

Debate Training Curbs Reward Hacking in AI Feedback Systems

This research demonstrates that using a two-player adversarial debate game during reinforcement learning from AI feedback (RLAIF) significantly reduces reward hacking, a common problem where policies exploit judge errors. The method maintains judge performance and achieves higher validation accuracy compared to a single-player RLAIF baseline, even with weaker judges.

Zachary Kenton, Lili Janzer, Rory Greig, Tian Huey Teh, Kirill Tyshchuk, Jonah Brown-Cohen, Harri Edwards, Senthooran Rajamanoharan, Noah Y. Siegel, Natasha Jaques, Rohin ShahAug 19, 2026
AI ResearchAI Engineering & DevTools

MAGPIE-Net Improves Heavy Rainfall Warnings with Satellite Data.

MAGPIE-Net is a new deep-learning model that directly predicts short-duration heavy-rainfall events in station neighborhoods using multitemporal satellite observations. It significantly outperforms gridded-output baselines, achieving higher detection rates and longer lead times for early warnings.

Xiang Lin, Yunying Li, Chengzhi Ye, Zitong Chen, Jing SunAug 19, 2026