Robot assistance using VLA

HKUST Final Year Project$:$ Robotic Arm System with AI Control Policies

In this final year project at HKUST, we developed a robot assistant that uses Vision-Language-Action (VLA) models to control a robot arm.

Throughout the entire project, we:

  • Developed a web application powered by a LLM to interface with the robot arm.
  • Developed a Gazebo simulator to simulate and verify the robot’s trajectory.
  • Collected the training data and fine-tuned the VLA model.
  • Contribute the collected datasets to open-source community (> 1k downloads)

Project video

Skills Earned: Vision-Language-Action (VLA) Models, VLA Fine-Tuning, Agentic AI