GNU parallel to keep docker-compose in attached mode

I’m trying to run a bunch of docker-compose commands in parallel using GNU parallel. For some reason though, it looks like starting parallel force docker-compose in detached mode so I can’t access the container output anymore.

I was hoping parallel would just keep docker-compose attached and print the output in order once the process is done.

  • Having trouble setting up a persistent data volume for a Docker image
  • Where pulled images of docker are stored in Windows?
  • Docker localhost IP
  • Connecting to mongo container using node getting MongoError
  • docker mysql persistent storage
  • invalid argument creating a ruby dev env with docker & fig
  • Here’s my command

    echo 'tests/test_foo.py' | parallel -X docker-compose --project-name bar run --rm test py.test $*
    

    Is there a way to force docker-compose to stay attached?

  • Deploying Docker images from Gitlab's private Docker registry to Openshift
  • How to understand docker container disk space usage?
  • Is it reasonable to containerize development environment with Docker?
  • Arch Linux, Docker “No space left on device.”
  • spring rabbitmq client closes channel with error “Channel shutdown: connection error”
  • Port forwarding in Jelastic with Docker
  • 2 Solutions collect form web for “GNU parallel to keep docker-compose in attached mode”

    There is no detached mode for run, it should always print the log lines to the terminal. This seems to work for me with a single line of input.

    However, calculating the container name is racy, so trying to run multiple instances like this may result in name conflict errors. You might need to use --name as an option to run to set a unique name for each.

    You might also consider using pytest-xdist to run the tests in parallel inside a single test container.

    Further to dnephin’s answer, You can name each container with parallel‘s positional arguments so there is no naming contention

    echo -e "1\ttest1\n2\ttest2\n3\ttest3" | parallel -X --colsep '\t' docker-compose run --name proj{1} web echo {2}
    

    In this demo the tab is used to separate the first field, used for the name and the second string used for the command.

    You either need to use existing fields from tests/test_foo.py for the names or add this additional information if what you have in that file is not unique.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.