what does fd:// mean exactly in dockerd -H fd://

Docker daemon documentation suggests the following hosts option for most setups:

dockerd -H fd://

I guess fd stands for file descriptor. I don’t understand how fd is used for socket communication.

  • Keep DNS resolution of a stopped docker container in docker DNS
  • root password inside a docker container
  • docker registry push failed on localhost
  • Connecting Logstash and Rabbitmq docker containers
  • Container watches etcd leader vs container watches etcd cluster
  • What is a Dockerfile.dev and how is it different from Dockerfile
  • I understand the following options:

    -H unix:///var/run/docker.sock -H tcp://192.168.59.106 -H tcp://10.10.10.2
    

    These are unix domain sockets and tcp sockets. I know how to call docker daemon using these sockets:

    docker -H tcp://0.0.0.0:2375 ps
    

    But if I started docker daemon using -H fd://, the following call gives error:

    $ docker -H fd:// ps
    error during connect: Get http:///v1.26/containers/json: http: no Host in request URL
    

    So what is the meaning of fd://? Is there any use for it?

  • Log only real fatal errors from PHP-FPM in Docker container
  • Mac Crashes Every Time I Try to Run Boot2Docker
  • Error while publising to docker on Azure from Visual Studio 2015
  • Docker web-app: bash session
  • Docker-Compose file has yaml.scanner.ScannerError
  • How can i install my package automatically on docker container
  • 2 Solutions collect form web for “what does fd:// mean exactly in dockerd -H fd://”

    When you start Docker daemon, -H fd:// will tell Docker that the service is being started by Systemd and will use socket activation, systemd will create the target socket and pass it to Docker daemon to use. This is the introduction to Systemd and this is the introduction to socket activation. The blogs are pretty long but really worthy reading, here’s a short summary of key points for understanding this question:

    • Systemd is a new init system intending to replace traditional SysV init system, one of its key features is faster init process.
    • Socket activation is one of the technologies used in Systemd to speed up service initialization
    • To receive requests, service needs socket to listen on, take Docker as example it needs unix domain socket like /var/run/docker.sock or TCP socket, of course those sockets needs somebody to create, most of the time is the service itself during start time.
    • With socket activation, SystemD will create those sockets and listen on it for services, and pass those sockets to service with exec when start the service. One benefit is that client requests can be queued in socket buffer once the socket is successfully created, even before the related service is started.
    • The socket info for a certain service used by Systemd is in socket unit file, for Docker it’s [docker.socket][3] with content:

      [Unit]
      Description=Docker Socket for the API
      PartOf=docker.service
      
      [Socket]
      ListenStream=/var/run/docker.sock
      SocketMode=0660
      SocketUser=root
      SocketGroup=docker
      
      [Install]
      WantedBy=sockets.target
      

    Let’s see how the whole thing works. I have file docker.socket and docker.service under /etc/systemd/system, the ExecStart line for docker.service is:

    ExecStart=/usr/bin/dockerd -H fd://
    
    1. Stop Docker service: systemctl stop docker

      $> ps aux | grep 'docker' # the `grep` itself in the output is ignored
      $> lsof -Ua | grep 'docker'
      $> 
      

      No docker process is running, and no docker.sock

    2. Execute systemctl start docker.socket:

      $> systemctl start docker.socket
      $> ps aux | grep 'docker' 
      $> lsof -Ua | grep 'docker'
      systemd       1    root   27u  unix 0xffff880036da6000      0t0 140748188 /var/run/docker.sock
      

      After start docker.socket, we can see that there’s still no docker process running, but the socket /var/run/docker.sock has been created, and it’s belongs to process systemd.

      (Off-Topic: Actually the socket is ready to receive requests now, even docker is not running yet, systemd will start docker.service at the moment the first request comes, passing the already created sockets to Docker. This is so-called on-demand auto-spawning)

    3. Start docker.service

      $> systemctl start docker.service
      $> ps aux | grep 'docker'
      root     26302  0.0  1.8 431036 38712 ?        Ssl  14:57   0:00 /usr/bin/dockerd -H fd://
      <....>
      

      As you can tell that Docker is running now. Let’s go one step back and try to execute /usr/bin/dockerd -H fd:// manually from terminal:

      $> /usr/bin/dockerd -H fd://
      FATA[0000] no sockets found via socket activation: make sure the service was started by systemd 
      

      Now you see the differences, when you use -H fd://, docker will expect the socket be passed by its parent process, rather than creating it by itself. When it’s started by Systemd, Systemd will do the job, but when you manually start it on termial, you don’t do the job so the docker daemon process failed and aborted. This is the code of how docker process fd:// when docker daemon starts, you can have a look if you’re interested.

    On the other hand for docker client, docker cli will parse protocol/addr from host specified in -H and make http request to docker daemon, the default host is unix:///var/run/docker.sock. The supported protocols include tcp, unix, npipe and fd. As far as I explore from source code, the transport configuration for fd is the same with tcp, so if you have tcp socket listening, you can just play it with:

    $> docker -H fd://localhost:4322 ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    

    which is the same as:

    docker -H tcp://localhost:4322 ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    

    The -H fd:// syntax is used when running docker inside of systemd. Systemd itself will create a socket in the docker.socket unit file and listen to it, and this socket is connected to the docker daemon with the fd:// syntax in the docker.service unit file.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.