valentinfrlch/ha-llmvision

Visual intelligence for your home.

🌟 Features · ⬇️ Quick Start Guide · 📖 Resources · 🪲 How to report Bugs · ☕ Support

Visit Website →

LLM Vision is a Home Assistant integration that uses multimodal large language models to analyze images, videos, live camera feeds, and Frigate events. It can also keep track of analyzed events in a timeline, with an optional Timeline Card for your dashboard.

Features

• Supports OpenRouter, OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure, Groq, Ollama, Open WebUI, LocalAI and any provider with OpenAI compatible endpoints.
• Answers questions and provides descriptions of images, video files, live camera feeds, and Frigate events based on your prompt.
• Remembers people, pets and objects
• Keeps a timeline of camera events, so you can display them on your dashboard or ask Assist about them.
• Seamlessly updates sensors based on data extracted from camera streams, images or videos

See the website for the latest features and examples.

Quick Start Guide

[!TIP] LLM Vision is available in the default HACS repository. You can install it directly through HACS or click the button below to open it there.

1. Install LLM Vision from HACS
2. Restart Home Assistant
3. Search for LLM Vision in Home Assistant Settings/Devices & services
4. Press submit to continue setup with default settings
5. Set up the media folder LLM Vision uses the more secure /media folder for storing snapshots. If you're running Home Assistant Container, you may need to mount a folder to /media in your container settings. See the docs for more details.
6. Return to the LLM Vision Integration Page
7. Press 'Add Entry' to add your first AI Provider

Detailed setup instructions and documentation is available here: LLM Vision Documentation

Blueprint

With the easy to use blueprint, you'll get camera event notifications intelligently summarized by AI. LLM Vision can also store events in a timeline, so you can see what happened on your dashboard.

Learn how to install the blueprint

Resources

Check the docs for detailed instructions on how to set up LLM Vision and each of the supported providers, get inspiration from examples or join the discussion on the Home Assistant Community and Discord.

For technical questions see the discussions tab.

How to report a bug or request a feature

[!IMPORTANT] Bugs: If you encounter any bugs and have followed the instructions carefully, file a bug report. Please check open issues first and include debug logs in your report. Debugging can be enabled on the integration's settings page. Feature Requests: If you have an idea for a feature, create a feature request.

 
Create new Issue 
 

Support

You can support this project by starring this GitHub repository. If you want, you can also buy me a coffee here: