Imran's Blog
Stuff I feel like blogging about.

Using Dhall to manage my docker-compose.yml

Posted on

I saw Dhall mentioned on a job posting for NoRedInk. I had never heard of it before but NoRedInk did create Elm and are fond of functional tooling so it piqued my interest.

For reference, Dhall is a simple language. It's not Turing complete, but it's strongly typed. It's primarily used for managing configuration files. It has some helpers to output JSON/YML/bash, but you can also output to plain text.

The website does a good job of showing what it can do and its use cases so I wont dive into those. Converting my docker-compose.yaml that I use to manage my media server was the first usecase I could think of.

I wont be posting the full source but I hope I convey how much fun I had working with Dhall.

# Experiment

I'll just be focusing on a small part of the docker-compose file, namely the ports.

Let's say this is my compose file

services:
  plex:
    image: linuxserver/plex:latest
    ports:
      - 32400:32400
      - 32400:32400/udp
    environment:
      VERSION: "latest"

A quick translate over would get us

let plex =
      { image = "linuxserver/plex:latest"
      , ports = [ "32400:32400", "32400:32400/udp" ]
      , environment.VERSION = "latest"
      }

in  { services.plex = plex }

When passed into dhall gets us this:

>>> dhall --file test.dhall

{ services.plex
  =
  { environment.VERSION = "latest"
  , image = "linuxserver/plex:latest"
  , ports = [ "32400:32400", "32400:32400/udp" ]
  }
}

dhall-to-yaml produces the following:

>>> dhall-to-yaml --file test.dhall

services:
  plex:
    environment:
      VERSION: latest
    image: linuxserver/plex:latest
    ports:
      - "32400:32400"
      - "32400:32400/udp"

Which is the exact same file (keys are sorted so the output is deterministic).

I always forget what order the ports should go (host:container) so let's take advantage of Dhall to make it easier on me. Before I make any changes to the file I'm going to be grabbing the Dhall SHA so I can compare it to my refactored code's SHA, if they are equal then the output has not changed.

>>> dhall hash --file test.dhall

sha256:db23b00a8e79e084306b72ecc788f0b95579080c883cb72045a3ae6ea43999fd

First lets move ports into it's own variable and verify the hash

let ports
    : List Text
    = [ "32400:32400", "32400:32400/udp" ]

let plex =
      { image = "linuxserver/plex:latest"
      , ports
      , environment.VERSION = "latest"
      }

in  { services.plex = plex }

>>> dhall hash --file test.dhall

sha256:db23b00a8e79e084306b72ecc788f0b95579080c883cb72045a3ae6ea43999fd

OK we can be sure we didn't change our output at all. Don't worry about : List Text that is just a type annotation.

Lets start by making a type that we can store the port mapping in and a function to convert it to text.

let PortMapping
    : Type
    = { host : Natural, container : Natural }

let PortMapping/show
    : PortMapping -> Text
    = \(pm : PortMapping) ->
        "${Natural/show pm.host}:${Natural/show pm.container}"

Now we can supply the ports we want in any order and be confident the output order is correct. Let's update ports to use this new functionality.

let ports
    : List Text
    = [ PortMapping/show { host = 32400, container = 32400 }
      , "${PortMapping/show { host = 32400, container = 32400 }}/udp"
      ]

Our hash is still sha256:db23b00a8e79e084306b72ecc788f0b95579080c883cb72045a3ae6ea43999fd so we haven't broken anything yet.

This is better but we still need to do some string interpolation if we want to create a UDP port mapping. The docker documentation says the notation without /udp or /tcp will default to tcp.

Let's create an enum that will represent these two and update our code to use it.

let Port
    : Type
    = < tcp : PortMapping | udp : PortMapping >

let Port/show
    : Port -> Text
    = \(p : Port) ->
        merge
          { tcp = PortMapping/show
          , udp = \(pm : PortMapping) -> "${PortMapping/show pm}/udp"
          }
          p

let ports
    : List Text
    = [ Port/show (Port.tcp { host = 32400, container = 32400 })
      , Port/show (Port.udp { container = 32400, host = 32400 })
      ]

Much nicer! And our hash is still producing sha256:db23b00a8e79e084306b72ecc788f0b95579080c883cb72045a3ae6ea43999fd.

Now what would be even nicer is we did not have to specify the port twice if they are the same value. Let's do that by expanding our Port enum to supply a tcp_mirror and a udp_mirror.

let Port
    : Type
    = < tcp : PortMapping
      | udp : PortMapping
      | tcp_mirror : Natural
      | udp_mirror : Natural
      >

let Port/show
    : Port -> Text
    = \(p : Port) ->
        let udpPortMapping = \(pm : PortMapping) -> "${PortMapping/show pm}/udp"

        in  merge
              { tcp = PortMapping/show
              , udp = udpPortMapping
              , tcp_mirror =
                  \(port : Natural) ->
                    PortMapping/show { host = port, container = port }
              , udp_mirror =
                  \(port : Natural) ->
                    udpPortMapping { host = port, container = port }
              }
              p

let ports
    : List Text
    = [ Port/show (Port.tcp_mirror 32400), Port/show (Port.udp_mirror 32400) ]

-- sha256:db23b00a8e79e084306b72ecc788f0b95579080c883cb72045a3ae6ea43999fd

We can include the List/map function from Prelude so we do not have manually transform our port list.

# Final test.dhall

let List/map = https://prelude.dhall-lang.org/List/map

let PortMapping
    : Type
    = { host : Natural, container : Natural }

let PortMapping/show
    : PortMapping -> Text
    = \(pm : PortMapping) ->
        "${Natural/show pm.host}:${Natural/show pm.container}"

let Port
    : Type
    = < tcp : PortMapping
      | udp : PortMapping
      | tcp_mirror : Natural
      | udp_mirror : Natural
      >

let Port/show
    : Port -> Text
    = \(p : Port) ->
        let udpPortMapping = \(pm : PortMapping) -> "${PortMapping/show pm}/udp"

        in  merge
              { tcp = PortMapping/show
              , udp = udpPortMapping
              , tcp_mirror =
                  \(port : Natural) ->
                    PortMapping/show { host = port, container = port }
              , udp_mirror =
                  \(port : Natural) ->
                    udpPortMapping { host = port, container = port }
              }
              p

let portsList
    : List Port
    = [ Port.tcp_mirror 32400, Port.udp_mirror 32400 ]

let plex =
      { image = "linuxserver/plex:latest"
      , ports = List/map Port Text Port/show portsList
      , environment.VERSION = "latest"
      }

in  { services.plex = plex }

-- sha256:db23b00a8e79e084306b72ecc788f0b95579080c883cb72045a3ae6ea43999fd

# My Conclusion

Using Dhall is super fun. I can totally see the usefulness in a shared team environment for managing configurations. I think I went a bit overboard as my ~120 line compose file is now ~400 lines of Dhall code 😅. Except now it's statically typed and it will be super easy to add new services to my server (or in context of the experiment, new ports to the Plex service).