We Give Robots Vision
Hardware-agnostic edge robot vision framework with world models for predictive intelligence. Runs on Jetson, Pi, Intel, Hailo & Qualcomm.
Hardware-agnostic edge robot vision framework with world models for predictive intelligence. Runs on Jetson, Pi, Intel, Hailo & Qualcomm.
80+ object classes in real-time
Measure distance to everything
Identify and track faces
Stop, wave, point commands
Detect body positions
Follow objects across frames
Build maps, navigate autonomously
Vision-Language-Action
Point cloud obstacle detection
Camera + Depth + LIDAR
Multiple camera support
10-30 Hz real-time control
Robot manipulation AI
E-STOP, velocity limits
24/7 with auto-recovery
Model updates + rollback
Predictive planning at 200 Hz
Spatiotemporal perception
35.7% better than MiDaS
Warehouse, QA, Agriculture, Retail
Multi-device deployment
TensorRT, OpenVINO, TVM, Hailo, QNN
| Configuration | FPS | Use Case |
|---|---|---|
| Detection only (TensorRT) | 35-40 | YOLO11n FP16 |
| Full pipeline (default) | 4-6 | All models enabled |
| Full pipeline + turbo | 8-12 | Aggressive frame skipping |
| Minimal | 15-20 | Detection + depth + tracking |
| World model (LeWM 15M) | 100-200 Hz | Planning only, <10ms |
40 TOPS, 5-15W
+ AI HAT 26 TOPS
48 TOPS NPU
26 TOPS, 3.5W
15-30 TOPS
CSI, USB, RealSense
git clone https://github.com/mandarwagh9/openeyes.git
cd openeyes
pip install -r requirements.txt
python src/main.py --debug# World model with predictive tracking
python src/main.py --world-model lewm --follow
# With safety evaluation
python src/main.py --world-model lewm --safety-predict
# Industry template (warehouse)
python src/main.py --template warehouse --debug
# Turbo mode for maximum FPS
python src/main.py --turbo --world-model lewm
# Process video files
python src/main.py --video input.mp4 --output output.mp4# Multi-Modal Sensing (v0.7.0)
python src/main.py --lidar --lidar-topic /scan
python src/main.py --realsense
python src/main.py --multi-camera
# VLA & Performance (v0.8.0)
python src/main.py --int8 --dla
python src/main.py --diffusion-policy
python src/main.py --action-chunking --control-freq 20
# Safety & Reliability (v1.0.0)
python src/main.py --safety --max-velocity 1.0 --min-distance 0.3
python src/main.py --health-monitor
python src/main.py --ota-update| Model | Type | Size |
|---|---|---|
| YOLO11n/12n/26n | Detection | 2.6-5.4MB |
| MiDaS + Depth Anything V3 | Depth | 350MB |
| MediaPipe | Face/Gesture/Pose | ~20MB |
| LeWM | World Model | 15M params |
| V-JEPA 2 | Perception | 80-600M params |
| SmolVLA | VLA | 450M params |
| OpenVLA | VLA | 7B params |