TLDR Tech / Practical
OpenWAM Framework Enables Combined Video and Action Modeling
A new open framework called OpenWAM has been developed by researchers to serve as a shared base for examining the integration of video prediction and robot control. This framework connects a 5 billion parameter video model with a 2 billion parameter action model through a Mixture-of-Transformers design. It allows for various interaction setups, including video followed by action, action followed by video, simultaneous generation, and separate execution. Tests on robotic manipulation tasks showed OpenWAM achieving high success rates and providing independent local-context inverse and forward dynamics models.
Read the full story