VN-ROS-01v0.9-beta
Fully offline ROS 2 voice-assistant stack for a differential-drive robot. Speak a command — the robot understands, plans, and navigates autonomously. No cloud, no internet required during operation. The entire AI pipeline (wake word, STT, LLM, TTS, Nav2, SLAM, object detection) runs on-device.
Two-computer split over CycloneDDS: the Jetson Orin Nano Super (8GB) runs all AI workloads — LLM, TTS, STT, Nav2, SLAM, YOLO object detection, and the web dashboard. The Raspberry Pi 5 handles real-time motor control, encoder feedback, IMU fusion, LiDAR, and the camera — all in Docker containers.
specifications
languagePython / C++
platformROS 2 Humble · Jetson Orin
statusactive
year2025
operating conditions
in → a spoken command
out → autonomous Nav2 navigation
ROS 2Jetsonfaster-whisperNanoLLMNav2SLAM
hardware
compute (AI)NVIDIA Jetson Orin Nano Super 8GB
compute (robot)Raspberry Pi 5
LiDARSLLiDAR A1M8
cameraIMX708 (object detection)
drive4× DC motors · TB6612FNG driver · encoders
IMUMPU6500 + HMC5883L
power monitorADS1115 battery ADC
voice pipeline
01Mic → openwakeword (always-on, ~10 ms/frame, TFLite)
02faster-whisper STT (CPU int8, base.en model)
03Llama-3.2-1B-Instruct via NanoLLM (MLC q4f16_1) with tool-calling
04FAISS-based RAG for location/object context
05Nav2 (DWB controller · NavFn planner · AMCL localisation)
06Kokoro 82M TTS (TensorRT FP16, Jetson GPU, 24 kHz)
ros2 packages
vector_controlMotor driver, IMU, EKF, system stats
vector_navigationNav2, SLAM Toolbox, sim launches
vector_nav_managerNav/map/mode orchestration
vector_status_managerAggregates subsystem states → /robot_status
vector_llmNanoLLM WebSocket client node
vector_sttWake word + Whisper STT
vector_ttsKokoro TRT TTS engine
vector_detectionYOLO11s object detection node
vector_webFastAPI dashboard + ROS 2 bridge (HTTPS :8080)