what does fd:// mean exactly in dockerd -H fd://

Docker daemon documentation suggests the following hosts option for most setups:

dockerd -H fd://

I guess fd stands for file descriptor. I don’t understand how fd is used for socket communication.

  • docker-compose isn't installing bower components inside the docker container
  • Dockerfile inconsistent caching
  • boot2docker behind proxy - certificate signed by unknown authority
  • Docker RUN Command: When To Group Commands, When Not To?
  • Howto run a Prestashop docker container with persistent data?
  • Docker for Azure - no swarm mode?
  • I understand the following options:

    -H unix:///var/run/docker.sock -H tcp:// -H tcp://

    These are unix domain sockets and tcp sockets. I know how to call docker daemon using these sockets:

    docker -H tcp:// ps

    But if I started docker daemon using -H fd://, the following call gives error:

    $ docker -H fd:// ps
    error during connect: Get http:///v1.26/containers/json: http: no Host in request URL

    So what is the meaning of fd://? Is there any use for it?

  • No space in docker thin pool
  • docker-compose - externalize spring application.properties
  • Network slowness between Docker container and localhost
  • Docker port forwarding stops working after ip of host machine is changed
  • IBM Bluemix Containers : cf ic stats not working
  • How to save config file inside a running container?
  • 2 Solutions collect form web for “what does fd:// mean exactly in dockerd -H fd://”

    When you start Docker daemon, -H fd:// will tell Docker that the service is being started by Systemd and will use socket activation, systemd will create the target socket and pass it to Docker daemon to use. This is the introduction to Systemd and this is the introduction to socket activation. The blogs are pretty long but really worthy reading, here’s a short summary of key points for understanding this question:

    • Systemd is a new init system intending to replace traditional SysV init system, one of its key features is faster init process.
    • Socket activation is one of the technologies used in Systemd to speed up service initialization
    • To receive requests, service needs socket to listen on, take Docker as example it needs unix domain socket like /var/run/docker.sock or TCP socket, of course those sockets needs somebody to create, most of the time is the service itself during start time.
    • With socket activation, SystemD will create those sockets and listen on it for services, and pass those sockets to service with exec when start the service. One benefit is that client requests can be queued in socket buffer once the socket is successfully created, even before the related service is started.
    • The socket info for a certain service used by Systemd is in socket unit file, for Docker it’s [docker.socket][3] with content:

      Description=Docker Socket for the API

    Let’s see how the whole thing works. I have file docker.socket and docker.service under /etc/systemd/system, the ExecStart line for docker.service is:

    ExecStart=/usr/bin/dockerd -H fd://
    1. Stop Docker service: systemctl stop docker

      $> ps aux | grep 'docker' # the `grep` itself in the output is ignored
      $> lsof -Ua | grep 'docker'

      No docker process is running, and no docker.sock

    2. Execute systemctl start docker.socket:

      $> systemctl start docker.socket
      $> ps aux | grep 'docker' 
      $> lsof -Ua | grep 'docker'
      systemd       1    root   27u  unix 0xffff880036da6000      0t0 140748188 /var/run/docker.sock

      After start docker.socket, we can see that there’s still no docker process running, but the socket /var/run/docker.sock has been created, and it’s belongs to process systemd.

      (Off-Topic: Actually the socket is ready to receive requests now, even docker is not running yet, systemd will start docker.service at the moment the first request comes, passing the already created sockets to Docker. This is so-called on-demand auto-spawning)

    3. Start docker.service

      $> systemctl start docker.service
      $> ps aux | grep 'docker'
      root     26302  0.0  1.8 431036 38712 ?        Ssl  14:57   0:00 /usr/bin/dockerd -H fd://

      As you can tell that Docker is running now. Let’s go one step back and try to execute /usr/bin/dockerd -H fd:// manually from terminal:

      $> /usr/bin/dockerd -H fd://
      FATA[0000] no sockets found via socket activation: make sure the service was started by systemd 

      Now you see the differences, when you use -H fd://, docker will expect the socket be passed by its parent process, rather than creating it by itself. When it’s started by Systemd, Systemd will do the job, but when you manually start it on termial, you don’t do the job so the docker daemon process failed and aborted. This is the code of how docker process fd:// when docker daemon starts, you can have a look if you’re interested.

    On the other hand for docker client, docker cli will parse protocol/addr from host specified in -H and make http request to docker daemon, the default host is unix:///var/run/docker.sock. The supported protocols include tcp, unix, npipe and fd. As far as I explore from source code, the transport configuration for fd is the same with tcp, so if you have tcp socket listening, you can just play it with:

    $> docker -H fd://localhost:4322 ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

    which is the same as:

    docker -H tcp://localhost:4322 ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

    The -H fd:// syntax is used when running docker inside of systemd. Systemd itself will create a socket in the docker.socket unit file and listen to it, and this socket is connected to the docker daemon with the fd:// syntax in the docker.service unit file.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.