Using Home Assistant to connect to Open AI

As part of home assistant release 2023.8 you are able to generated AI images with OpenAi’s DALL-E this enables you to ask for images like “London with a bus on a sunny day” and display that on your home assistant dashboard.

To accomplish this you can follow either the video below or the step by step guide here.

Pre requisite

You will need an account with OpenAI (paid) you can use this link to sign up.

You are going to need some billing details as this is a paid API (I would strongly recommend setting some usage limits so you don’t get a surprise bill).

Before moving to home assistant create an API key (remember to keep this secret and don’t share it out), you are going to need that code in the home assistant integration step below.

Home Assistant

We need to add the open AI integration:

• Browse to your Home Assistant instance.
• Go to Settings > Devices & Services.
• Select the integration, then select Configure.
• Add your API Key

Now we need to find out our unique config_entry, the easiest way of doing this is to use the developer tools.

Go to services and find the service called –> openai_conversation.generate_image

Use the UI to input a text, which could be give me an image of London on a sunny day, (the text that you input doesn’t matter towards your config entry). Now click edit in YAML to find your config_entry, something similar to the below code:

data:
  config_entry: f29e6b8696a15e107b4bd843de722249
  prompt: "enter any text"
  size: 1024
response_variable: generated_image

You can actual paste the URL in the response in your browser to test the image (this will be based on your prompt).

Creating an automation

Today’s automation is going to update a dashboard card with a new image based on the weather of your local area. I’m going to use London as a city and fetch an AI image based on the current weather.

This code is extracted from home assistant’s guide:

  - alias: "Update image when weather changes"
    trigger:
      - platform: state
        entity_id: weather.home
    action:
      - alias: "Ask OpenAI to generate an image"
        service: openai_conversation.generate_image
        response_variable: generated_image
        data:
          config_entry: abce6b8696a15e107b4bd843de722249
          size: "512"
          prompt: >-
            New York when the weather is {{ states("weather.home") }}"

      - alias: "Send out a manual event to update the image entity"
        event: new_weather_image
        event_data:
          url: '{{ generated_image.url }}'

Let me explain in a simple way:

You don’t need to use YAML at least for most of it, this is how I did it.

Use the UI, create a trigger (maybe on the weather changing), create an action and now edit in YAML and paste in the action part of the code above. Now change the config_entry with your own and change your prompt with your own. Note how we are using templating to dynamically get the weather state from the entity.

The second action is to update an entity event called new_weather_image, feel free to copy as is.

Now in your configuration.yaml add the following code:

template:
  - trigger:
      alias: "Update image when a new weather image is generated"
      platform: event
      event_type: new_weather_image
    image:
      name: "AI generated image of New York"
      url: "{{ trigger.event.data.url }}"

Couple of things to note: the event_type (new weather image) needs to match with the automation code. The name is going to be the name of the image entity, that you are going to use in the dashboard.