CLAP Adapts VLMs to VLAs with Language-Action Grounding.
Summary
CLAP (Causal Language-Action Prediction) is a method that directly converts pretrained Vision-Language Models (VLMs) into Vision-Language-Action models (VLAs) with minimal architectural changes. It achieves this by prepending natural-language action descriptions to numeric action sequences, causally conditioning action prediction on a language-action plan, and significantly improving performance and robustness with single-epoch fine-tuning.
Why it matters
This method offers a more transparent and efficient way to leverage the semantic understanding of large VLMs for robotic control, accelerating the development of capable and robust embodied AI systems.
How to implement this in your domain
- 1Explore CLAP for adapting existing VLMs to robotic control tasks with minimal fine-tuning.
- 2Integrate language-action planning into your robot learning pipelines to improve action prediction.
- 3Utilize the open-weight CLAP models to benchmark and develop new VLA applications.
- 4Investigate how causal language conditioning can enhance other multimodal AI systems.
Who benefits
Key takeaways
- CLAP directly converts VLMs to VLAs with minimal architectural changes.
- It uses language-action grounding to bridge the VLM-VLA output distribution gap.
- Single-epoch fine-tuning yields significant performance and robustness improvements.
- CLAP offers a transparent path to understanding VLM capability transfer to robotics.
Original post by Yuri Ishitoya, Jeremy Siburian, Masashi Hamaya, Kuniaki Saito, Cristian C. Beltran-Hernandez, Mai Nishimura
"arXiv:2607.08974v1 Announce Type: cross Abstract: Vision-language-action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large-scale post-training on robot data and architectural modifications can reshape the backbone so extensively that it becomes difficult…"
View on XOriginally posted by Yuri Ishitoya, Jeremy Siburian, Masashi Hamaya, Kuniaki Saito, Cristian C. Beltran-Hernandez, Mai Nishimura on X · view source
Want to go deeper?
Turn these trends into skills with Learnijoy's hands-on AI & tech courses.
Explore coursesMore in AI Research
Data-Efficient Deep Learning Guidelines for Inertial Sensor Classification
This study provides empirical guidelines for estimating the minimum training set size needed for deep learning models in inertial sensor classification tasks. It reveals that accuracy follows a consistent logarithmic growth pattern, allowing for data-efficient planning of recording campaigns and achieving practical stability with fewer samples than traditionally assumed.
On-Device Adaptive AI Boosts EV Battery Power Prediction
Researchers developed a novel approach for on-device learning in electric vehicles (EVs) that continuously adapts pretrained battery power prediction models to new data. This method significantly improves forecasting performance, reducing mean absolute errors by up to 14.88% with offline adaptation and 7.49% with online adaptation.
New Differentiable Logic Networks Outperform Fixed-Connection Models
Researchers introduce a novel method for optimizing connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs), achieving superior performance with significantly fewer gates. The approach allows for parallel learning of optimal gate types and LUT entries, demonstrating improved accuracy on benchmarks like MNIST.