GNU parallel to keep docker-compose in attached mode

I’m trying to run a bunch of docker-compose commands in parallel using GNU parallel. For some reason though, it looks like starting parallel force docker-compose in detached mode so I can’t access the container output anymore.

I was hoping parallel would just keep docker-compose attached and print the output in order once the process is done.

  • Docker container exits immediately despite Service Running
  • Docker quickstart terminal not opening
  • Jenkins on Docker - Persist data
  • Docker Running the Java -JAR file to create Cassandra KeySpaces. But its not creating
  • ClassNotFoundException when submitting yamr job to remote cluster
  • Invalid repository name while pulling an image in docker
  • Here’s my command

    echo 'tests/test_foo.py' | parallel -X docker-compose --project-name bar run --rm test py.test $*
    

    Is there a way to force docker-compose to stay attached?

  • Where does Docker know where to pull the .sh scripts from when building a Dockerfile?
  • Passing argument to python application running on docker
  • Why does creating a file inside a docker-machine VM result in permission denied?
  • docker build + private NPM (+ private docker hub)
  • Supporting docker based and non docker based deployments
  • Docker Volumes not mounted after host reboot
  • 2 Solutions collect form web for “GNU parallel to keep docker-compose in attached mode”

    There is no detached mode for run, it should always print the log lines to the terminal. This seems to work for me with a single line of input.

    However, calculating the container name is racy, so trying to run multiple instances like this may result in name conflict errors. You might need to use --name as an option to run to set a unique name for each.

    You might also consider using pytest-xdist to run the tests in parallel inside a single test container.

    Further to dnephin’s answer, You can name each container with parallel‘s positional arguments so there is no naming contention

    echo -e "1\ttest1\n2\ttest2\n3\ttest3" | parallel -X --colsep '\t' docker-compose run --name proj{1} web echo {2}
    

    In this demo the tab is used to separate the first field, used for the name and the second string used for the command.

    You either need to use existing fields from tests/test_foo.py for the names or add this additional information if what you have in that file is not unique.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.