Architecture
┌─────────────────────────────┐
│ LLM Agent │
│ (Groq Llama 3.3-70B, │
│ LangGraph state machine) │
Voice / Text ───────► │ │
│ - Reasons about the task │
│ - Picks tools by phase │
│ - Handles failures │
│ - Sequences waypoints │
└────────────┬─────────────────┘
│
┌────────────┬───────────┼───────────┬──────────────┐
│ │ │ │ │
scan_scene navigate_to check_nav pick_up go_to_checkpoint
│ │ │ │ │
YOLO11 Nav2 (event-driven) arm + named
+ ground- gripper pose
plane actions dict
projection (scripted)
│ │ │
Gazebo Gazebo Gazebo
Camera Base Arm
Tech Stack
| Layer | Tool |
|---|---|
| Robot | TurtleBot3 Waffle Pi + OpenManipulator-X (4-DOF) |
| Simulator | Gazebo Classic 11 |
| ROS Version | ROS 2 Humble |
| Object Detection | YOLO11 (imgsz=1280) |
| 3D Localization | image_geometry ground-plane projection + tf2 |
| Navigation | Nav2 (NavfnPlanner + DWBLocalPlanner) + slam_toolbox |
| Manipulation | ros2_control FollowJointTrajectory + GripperCommand; MoveIt2 wired (IK use TBD) |
| LLM Agent | Groq Llama 3.3-70B via LangGraph |
| Agent Tools | Python functions wrapped as LangGraph tools, calling ROS |