Guides Archives - W Solutions

Video guide: https://youtu.be/puSNKyqAeoU

In this guide we’ll show you how to integrate your IP camera in Home Assistant using Agent DVR via Docker which will connect to your IP camera and will make the camera available in Home Assistant.

Next we’ll use DeepStack Artificial intelligence via Docker for object detection.

Finally we’ll setup A gent DVR MQTT to get a Binary Sensor in Home Assistant that reports on the state of the object detection. Where we then are able to setup all kinds of cool automation’s using Home Assistants build in automation platform or Node-Red.

Artificial intelligence object detection

Camera’s

In our guide we’ve used the Annke C800 and the NC400 (NC400 has awesome color night vision!) which are great camera’s for their prices and are outperforming nearly all competitors in their tiers. Highlights on the Annke camera/recorder line are:

Easy of use
Intuitive recorder UI
Quality
Reliability
Perfect price/quality ratio

Check this video by The Hook Up reviewing the C800 camera in a detailed comparison.

Topics

Prerequisites
What hardware and setup is used in this guide
Links
Setup our Annke C800 Camera
Setup our Annke C800 camera in Home Assistant
1. Install Agent DVR
2. Add camera to Agent DVR
3. Configure camera settings
  1. General config
  2. Actions
  3. Alerts
  4. Detector
  5. Object Recognition
4. Add “Agent DVR” Home Assistant Integration
Setup & integrate DeepStack Artificial intelligence
Setup object detection in Home Assistant with IP cameras
1. MQTT Addon Home Assistant
2. Agent DVR MQTT settings
3. Object detection binary sensors in Home Assistant
Installation one liners

Note: Due to privacy we’ve set a “privacy mask” on some of our screenshots/video’s (ones without are publicly available).

1. Prerequisites

This guide assumes that you have Home Assistant OS installed on a device listed here
This guide assumes that you have Docker setup on a x86 device
The additional software components can be installed:
- on the same device as Home Assistant via Docker x86 (Portainer is also an option, though deprecated, still available here and here).
- on a RaspberryPI (Debian Lite or Ubuntu Server) not that powerful but will still do the job. Agent DVR via Docker is not available on ARM but a one liner to install Agent DVR is available.
- on another Linux or windows device with more resources and possibly a GPU (not required)

2. What hardware and setup is used in this guide?

In our guide we use a single Virtual Machine x86 with a Debian based distro called DietPi to host Home Assistant Supervised, DeepStack Artificial intelligence, and Agent DVR in our Proxmox Virtual Environment.
We have 8 Vcores (Xeon E5-2690) and 8Gb of RAM which is a bit of overkill. You’ll easily get away with half of both for a small multi camera setup.
These steps should work on any Debian or Ubuntu distro (ARM doesn’t run Agent DVR’s Docker but you can install it via the cli with a oneliner)
- Its still possible to host everything on a RaspberryPi 4 but the performance won’t be super.
TIP: It is also possible to use “GPU passthrough” on a VM to get better performance encoding/decoding with an Nvidia GPU

In this guide we’ll reach the following services via:

Agent DVR: 192.168.9.235:8090
DeepStack Artificial intelligence: 192.168.9.235:81
Home Assistant: 192.168.9.235:8132
Annke NC400: 192.168.14.4

3. Links

Docker
AgentDVRv3.9.3.0 TIP: Use a GPU for better performance on encoding/decoding – Docker File nVidia support
DeepStack Artificial intelligence
Home Assistant
- Versions used in guide:
  - HA core 2022.4.7
  - HA supervisor 2022.04.0
  - HA Frontend 20220301.2
- MQTT
  - Broker
  - Home Assistant Add-on: Mosquitto broker
- Annke
  - How to Access the Web interface of ANNKE PoE Cameras via Browser
Network devices discovery tools

4. Setup our Annke C800 Camera

We’ll first start by setting up the camera, in our case we are using the Annke C800.
We’ve powered it up with POE via our Unifi Switch 24 Pro. If your having trouble finding the IP address of your device, please see the “Network discovery tools” links above.

Once you’ve figured out the IP address of the camera, head over to its web UI and login. In our case its http://192.168.14.4:80

We’d like to point out that most if not all IP camera’s have a a Main and a Sub stream. Below we’ll clarify why these are used and in a later section we’ll explain how to use them (in Agent DVR).

Main-stream: stream resolution is higher than sub-stream, for local video streaming, output high definition video to display on the local monitor, also record HD video on your local network network video recorder (NVR). The main-stream resolution are (not limited to) 2560×1920 (5 megapixel), 2048×1536 (3 megapixel) 1920×1080 (1080p) and 1280×720 (720p). Additionally, main-stream delivers real-time (30fps or even high frame rate 60fps) HD video stream, this ensures local recording footage offers detailed information without missing anything important.

Sub-stream: offering low/standard definition video primarily for lower bandwidth situations, the sub-stream resolution are QCIF, CIF, 2CIF, DCIF, 4CIF (D1). Sub-stream may not transmit real time video, but transmit key frames and sub-frames to deliver video. This also can decrease the network bandwidth burden, which in turn achieves smooth video in low bandwidth situations.

You’re able to switch between the two streams to see the difference. At the main page after login, bottom left, click the middel of the 3 icons.

1 = Main-stream
2 = Sub-stream

Presented with a neat looking user interface we’ll start by checking the NTP server functionality for proper time display. Head over to Configuration, System Settings and Time Settings. Set your timezone and NTP server and hit test. If all is in order hit save.

Under Security Service we’ve enabled Enable Illegal Login Lock and set the threshold to 5. So if you’ll enter a wrong password 5 times, you’ll be locked out for a certain period of time.

Under User Management the Annke software also has a cool feature to view the currently logged in “Online Users“. Should you ever want to troubleshoot the connection from a device that pulls in the stream of that cam, you can!

Next we’ll enable HTTPS under Network, Advanced Settings, HTTPS

That concludes the camera configuration, please note your camera IP, model, username and password after setting the camera up for the first time, we’ll need it later!

5. Setup our Annke C800 camera in Home Assistant

5.1 Install Agent DVR

Agent DVR is an advanced video surveillance platform for Windows, macOS, Linux, Raspberry Pi and Docker. Agent has a unified user interface that runs on all modern computers, mobile devices and even Virtual Reality.

It is also the first DVR/NVR Software that supported every camera we threw at it and has a vast pool of features easy to use for everyone.

We’ll install Agent DVR via this link, pick your appropriate platform and install it.

In our case we are using the Docker method on a Debian based x86 OS (Ubuntu should work as well) inside a Virtual Machine (which also has Home Assistant installed via the Linux Installation Method).

Note that the Docker option will not work for ARM devices, you’ll have to use the macOS/Linux/RPi installation option for ARM (RaspberryPI’s etc).

If you need to install Docker on your Debian/Ubuntu server, run:

curl -sSL https://get.docker.com | sudo bash

Use one of the two methods below, if unsure use Docker run

Docker run

docker run --restart always -it -p 8090:8090 -p 3478:3478/udp -p 50000-50010:50000-50010/udp -v /appdata/ispyagentdvr/config/:/agent/Media/XML/ -v /appdata/ispyagentdvr/media/:/agent/Media/WebServerRoot/Media/ -v /appdata/ispyagentdvr/commands:/agent/Commands/ -e TZ=America/Los_Angeles --name agentdvr doitandbedone/ispyagentdvr

Docker compose

version: '3.3'
services:
    ispyagentdvr:
        restart: always
        ports:
            - '8090:8090'
            - '3478:3478/udp'
            - '50000-50010:50000-50010/udp'
        volumes:
            - '/appdata/ispyagentdvr/config/:/agent/Media/XML/'
            - '/appdata/ispyagentdvr/media/:/agent/Media/WebServerRoot/Media/'
            - '/appdata/ispyagentdvr/commands:/agent/Commands/'
        environment:
            - TZ=America/Los_Angeles
        container_name: agentdvr
        image: doitandbedone/ispyagentdvr

It might be the case that you’ll want to store your recordings on a NAS or via another way of remotely storing data. In these cases, file permissions are a quick issue to encounter. If you’re storing elsewhere than the default (/appdata/ispyagentdvr/*) please check the destination folders permissions with

notroot@DietPi:~# ls -la /Recordings/
total 8
ddrwxr-xr-x  2 media media 4096 May  2 19:41 .
drwxr-xr-x 19 root   root   4096 May  2 19:41 ..

Now we know the user and group “media” owns the folder /Recordings and we can get the user id. You can swap out /Recordings with any remotely mounted folder eg /mnt/nfs1.

stillnotroot@DietPi:~# id media
uid=1000(media) gid=1000(media) groups=1000(media)

You can substitute /appdata/ispyagentdvr/media with your preferred directory to store recordings but make sure you set the proper docker PUID and PGID in relation to the actual folders permissions with:


version: '3.3'
services:
    ispyagentdvr:
        restart: always
        environment:
            - PUID=1000 # Change to user ID
            - PGID=1000 # Change to group ID
            - TZ=America/Los_Angeles
        ports:
            - '8090:8090'
            - '3478:3478/udp'
            - '50000-50010:50000-50010/udp'
        volumes:
            - '/appdata/ispyagentdvr/config/:/agent/Media/XML/'
            - '/Recordings/:/agent/Media/WebServerRoot/Media/'
            - '/appdata/ispyagentdvr/commands:/agent/Commands/'
        container_name: agentdvr
        image: doitandbedone/ispyagentdvr

Or for Docker run with

docker run --restart always -it -e PUID=1000 -e PGID=1000 -p 8090:8090 -p 3478:3478/udp -p 50000-50010:50000-50010/udp -v /appdata/ispyagentdvr/config/:/agent/Media/XML/ -v /Recordings/:/agent/Media/WebServerRoot/Media/ -v /appdata/ispyagentdvr/commands:/agent/Commands/ -e TZ=America/Los_Angeles --name agentdvr doitandbedone/ispyagentdvr

5.2 Add camera to Agent DVR

Now that Agent DVR is installed we can add our previously setup Annke C800 to it. Head over to the web UI of Agent DVR, in our case its http://192.168.9.235:8090

Hit the menu icon top left and add a new device. Use the IP Cam Wizard and turn on alerts when prompted.

For make and model we’re selecting the Annke C800, Agent DVR has a huge database of supported cameras so yours should be there as well! If not you can just select “unlisted” and it will search the given IP address for valid protocols and URLs.

I’ve selected the first ONVIF entry shown in the results.

Note that channel 1 is the good quality “Main-stream” and channel 2 is the lower quality “sub-stream” or live stream. You can change the recording channel and live feed channel if you edit the camera settings and click on the 3 dots next to “source type“.

Ideally we’d want to use the “sub” stream as “live” view and only record at the highest quality when motion is detected in order to minimize the network and system resource footprint especially in multi camera environments without a GPU. Where a “sub” stream of lets say 640*480 at 256-768 Kbps with around 5-10 FPS is perfectly usable for object detection. Compare this to using the snapshot images of the “main” stream of 3840*2160 with a max bit-rate of 5120 Kbps at 15. This gives a lot of overhead compared to the “sub” stream which does its job.

5.3 Configure camera settings

5.3.1 General camera config:

Make sure MQTT Events is On to report all events to Home Assistant

For Max Framerate please consider that a higher framerate equals higher bandwidth usage and higher processing times. In a multi camera setup we’ve settled for 10 FPS for live viewing and 30+ for recording. Right now for the sake of the guide we’d like a crispy smooth image so we’ve set it to 30 FPS.

Try toggling the Resize button and see what that does for your image quality. I had to turn ours off to get a better looking image in the Agent DVR web UI.

5.3.2 Actions config:

Next we’ll add an action to run (MQTT publish) whenever DeepStack Artificial intelligence detects a specified object.
We have not yet setup the MQTT broker in Home Assistant but for now we can pre fill these settings.

Go to your camera settings, Actions and hit Add

Then we’ll add the first action that will set the state to ON whenever objects are detected.

If: DeepStack Object Found
Then: MQTT
Topic: ObjectDetection/{NAME}/state {NAME} will automatically be set to your camera name
Post: ON In Caps

Then add the second action which will add details about the detection to the MQTT “json” topic.

If: DeepStack Object Found
Then: MQTT
Topic: ObjectDetection/{NAME}/json
Post: {AIJSON}

The {AIJSON} will report something like this:

 {AIJSON}

5.3.3 Alerts config:

Agent DVR integrates with DeepStack Artificial intelligence to provide intelligent object detection and Facial Recognition services. Agent DVR uses DeepStack object detection to apply an intelligent filter to alerts to reduce the number of false alert events to near zero.

You configure Agent as usual with a motion detector / alert configuration and then add DeepStack alert filtering to ignore alerts that don’t have specific objects in the frame – like car, person, dog etc.

Make sure the settings reflect the image below. Then hit configure and copy over the settings of the next screen. You can tweak your desired object recognition items under the configure menu with the above mentioned in mind, for now we use person and a remote control.

5.3.4 Detector config:

Enable the detector and set the type to Objects

5.3.5 Object Recognition config:

In addition to the intelligent alert filter Agent DVR also offers general object recognition. This will generate events like “DeepStack Object Found” and “DeepStack Object Not Found” that you can perform actions on. General object recognition can be triggered by motion detection, alerts or via API

Edit a camera and go to Object Recognition

5.4 Add “Agent DVR” Home Assistant Integration

Go to Home Assistant’s web UI to Configuration, Devices & Services and hit + Add integration.
Search for “Agent DVR”.

Add Agent DVR and enter the IP address and port (8090 is default) where you host your Agent DVR.

In our case we’ll go to http://192.168.9.235:8123/config/integrations and we’ll point Home Assistant to IP 192.168.9.235:8090 for Agent DVR as in our case they are hosted on the same machine.

Now after adding Agent DVR you’ll have an Alarm Control panel and a new camera device and entity, check it out under devices, search for “camera.” and see if your camera is present.

6. Setup & integrate DeepStack Artificial intelligence

To get started, install DeepStack for your platform (Docker). Use the GPU version if possible (you’ll need an nvidia card for GPU acceleration). Make sure it’s running in Vision Detection mode. To do that via the command line pass in “-e VISION-DETECTION=True”.

For RaspberryPi 4 use deepquestai/deepstack:arm64

Docker Run:

docker run --restart always -v /Recordings/Deepstack/Objectdetection:/datastore/object -v /Recordings/Deepstack/FacialRecognition:/datastore/facial --name DeepStackAI -p 90:5000 -e VISION-DETECTION=True -e VISION-FACE=False -e TZ=Europe/Berlin deepquestai/deepstack

Docker Compose:

version: '3.3'
services:
    deepstackai:
        container_name: DeepStackAI
        restart: always
        environment:
            - PUID=1000 # Change to user ID owning the destination folder
            - PGID=1000 # Change to group ID owning the destination folder
            - VISION-DETECTION=True # Enable for object detection
            - VISION-FACE=False # Enable for facial recognition
            - TZ=Europe/Berlin
        volumes:
            - '/Recordings/Deepstack/Objectdetection:/datastore/object'
            - '/Recordings/Deepstack/FacialRecognition:/datastore/facial'            
        ports:
            - '90:5000'
        image: deepquestai/deepstack

It will take a while to download all the libraries for the container. Afterwards when the Docker run or compose command is finished, check the logs for anything funky docker logs DeepStackAI

Once DeepStack Artificial intelligence is installed and running (docker ps) it will expose itself to the network via port :90 (ss -tunlp should report docker-proxy on port :90) if you’re using the same configuration we’ve provided. Lets enter our IP where DeepStack Artificial intelligence is hosted and port :90. In our case it will be http://192.168.9.235:90

7. Setup object detection in Home Assistant with Annke cameras

7.1 MQTT Addon Home Assistant

To create a sensor in Home Assistant we’ll need data and a supported platform. Agent DVR will provide the data now all we need is a supported platform, here comes MQTT in to play.

We will install The MQTT Add-on to create our own MQTT Broker (server) where our Agent DVR (Write) and Home Assistant (Read) will subscribe to in order to be able to exchange messages about the state of the object detection and update the sensor accordingly.

Install the MQTT Addon in Home Assistant. In our case by going to http://192.168.9.235:8123/hassio/addon/core_mosquitto/info

Configuration
Add-ons, Backup…
Add-on Store
Install Mosquitto broker

7.2 Agent DVR MQTT settings

Now we’ll head over to Agent DVR again to setup the MQTT settings it needs to connect to the Home Assistant MQTT broker Add-on. Do so by going to the menu top left, Configuration, Settings, MQTT

Server: 192.168.9.235
Port: 1883
Client ID: agentdvr
Username: homeassistantuser (change with your actual credentials)
Password: homeassistantpass (change with your actual credentials)
Leave the rest of the settings at default

After adding the MQTT broker to Agent DVR also make sure that “MQTT Events” in the camera’s General config is On

To finalize our Agent DVR settings we need to restart our Docker container with docker restart agentdvr

7.3 Object detection sensors in Home Assistant

We’ll configure our binary sensor for the object detection to point to the configured MQTT topic we’ve set earlier during the camera configuration. But first we’ll connect Home Assistant to the MQTT broker so we can setup our Binary Sensor in Home Assistant.

In Home Assistant go to Settings, Devices & Services, Integrations
There you’ll probably see MQTT as discovered. If so hit configure to set it up. If its not discovered hit + Add Integration and search for MQTT and add it.

We are using IP:PORT 192.168.9.235:1883 as the broker is installed on our VM using port 1883. Use the username and password you’ve setup during the installation of Home Assistant or create a separate user and use that.

We’ll add a Binary Sensor in Home Assistant with the following config, make sure to replace YOURCAMERANAME with the proper value.

binary_sensor:
  - platform: mqtt
    name: "Object detection Tect Cam"
    state_topic: "ObjectDetection/Test Cam/state"
    json_attributes_topic: "ObjectDetection/Test Cam/json"
    json_attributes_template: "{{ value_json.predictions['confidence']['label'] }}"
    device_class: "presence"
    off_delay: 60
    qos: 1

Now in order to edit configuration.yaml we’ll use the Addon VsCode. You can you any means you like to edit it, SSH, SMB etc.

Go to Home Assistant Settings, Add-ons, Backups & Supervisor, Add-on Store and install Studio Code Server

Once installed open VsCode via the sidebar menu in Home Assistant and open up configuration.yaml and add the sensor config.

Once added, check your Home Assistant config for errors in the configuration, settings menu, if all is ok then restart Home Assistant.

Upon restart you should have a binary sensor called “Object detection YOURCAMERANAME”

We can now create cool automations based upon the object detection. Few things we’ve thought about making another guide on, specific to Artificial intelligence and Home Assistant are;

Unlock and lock post box if the postal service (DHL, UPS etc) logo is detected and start recording and output a screenshot to telegram.
Motion / presence detection sensors. Facial recognition flags a family member home or away using door sensors to fire the event.
Upload to specific NAS folders based upon object detection details.
Start stream to Chromecast/Home Assistant Web when object “X” is detected
Notifications for object detections

8. Installation one liners and snippits

Docker auto installation

sudo apt update && sudo apt install curl -y && curl -fsSL get.docker.com | sudo bash

Home Assistant Supervised on Linux (Docker with Addons)

Download the latest Debian package from OS Agent GitHub release page at: https://github.com/home-assistant/os-agent/releases/latest

Note: Replace the deb file with the latest version from the releases page. Also make sure you have Docker installed already!

One thing I noticed that installing network-manager on my DietPi Debian distro cleaned out /etc/network/interfaces thus resulting in no connection to the box. Hence the backup and restore of the interfaces file.

sudo cp /etc/network/interfaces ~/interfaces

sudo apt-get install -y jq wget libglib2.0-bin apparmor curl udisks2 network-manager dbus

sudo mv ~/interfaces /etc/network/interfaces && sudo ifdown -a && sudo ifup -a

sudo wget https://github.com/home-assistant/os-agent/releases/download/1.2.2/os-agent_1.2.2_linux_x86_64.deb 

sudo dpkg -i os-agent_1.2.2_linux_x86_64.deb 

sudo wget https://github.com/home-assistant/supervised-installer/releases/latest/download/homeassistant-supervised.deb  

sudo dpkg -i homeassistant-supervised.deb

Agent DVR

bash <(curl -s "https://raw.githubusercontent.com/ispysoftware/agent-install-scripts/main/install.sh")

Open a web browser at http://hostip:8090 to start configuring Agent DVR.

TLDR;

# Debian/Ubuntu based server
WORKDIRECTORY="~/"
RECORDING="/Recordings"
RECORDINGUID=$(ls -la "$RECORDING" | awk '{print $3}' | head -2 | tail -1 | id | awk '{print $1}' | sed 's/uid=//g' | sed -e 's/([^()]*)//g')
RECORDINGGUID=$(ls -la "$RECORDING"| awk '{print $4}' | head -2 | tail -1 | id | awk '{print $2}' | sed 's/gid=//g' | sed -e 's/([^()]*)//g')
TZ="America/Los_Angeles"

# Change working dir
sudo cd "$WORKDIRECTORY"

# Docker & compose installation
sudo apt update && sudo apt install curl -y && curl -fsSL get.docker.com | sudo bash && sudo apt install docker-compose-plugin -y

# Home Assistant installation
# Backup network settings, needed in our case using DietPi
sudo cp /etc/network/interfaces "$WORKDIRECTORY"/interfaces

# Install dependencies
sudo apt install -y jq wget libglib2.0-bin apparmor curl udisks2 network-manager dbus

# Restore network settings and reload
sudo mv "$WORKDIRECTORY"/interfaces /etc/network/interfaces && sudo ifdown -a && sudo ifup -a

# Download and install os-agent
sudo wget https://github.com/home-assistant/os-agent/releases/download/1.2.2/os-agent_1.2.2_linux_x86_64.deb 
sudo dpkg -i os-agent_1.2.2_linux_x86_64.deb 

# Download and install homeassistant_supervised
sudo wget https://github.com/home-assistant/supervised-installer/releases/latest/download/homeassistant-supervised.deb  
sudo dpkg -i homeassistant-supervised.deb

# Docker Compose AgentDVR & DeepStack Artificial intelligence
sudo cat > "$WORKDIRECTORY"/docker-compose.yaml <<EOF
version: '3.3'
services:
    ispyagentdvr:
        restart: always
        environment:
            - PUID="$RECORDING-UID" # Change to user ID owning the destination folder
            - PGID="$RECORDING-GUID" # Change to group ID owning the destination folder
            - TZ="$TZ"
        ports:
            - '8090:8090'
            - '3478:3478/udp'
            - '50000-50010:50000-50010/udp'
        volumes:
            - '/appdata/ispyagentdvr/config/:/agent/Media/XML/'
            - '$RECORDING/:/agent/Media/WebServerRoot/Media/'
            - '/appdata/ispyagentdvr/commands:/agent/Commands/'
        container_name: agentdvr
        image: doitandbedone/ispyagentdvr
    deepstackai:
        container_name: DeepStackAI
        restart: always
        environment:
            - PUID="$RECORDING-UID" # Change to user ID owning the destination folder
            - PGID="$RECORDING-GUID" # Change to group ID owning the destination folder
            - VISION-DETECTION=True # Enable for object detection
            - VISION-FACE=False # Enable for facial recognition
            - TZ="$TZ"
        volumes:
            - "$RECORDING/Deepstack/Objectdetection:/datastore/object"
            - "$RECORDING/Deepstack/FacialRecognition:/datastore/facial"            
        ports:
            - '90:5000'
        image: deepquestai/deepstack
EOF

# Bring up the stack
docker-compose -f "$WORKDIRECTORY"/docker-compose.yaml up -d

# Change back directory
cd -

# Finished
echo "done"

exit 0

If you have any questions or suggestions please use the comment section. Email/phone support will not be provided for this guide.

Chat on WhatsApp

All Guides

Artificial intelligence in Home Assistant using an IP cam