Load data from host machine to docker hive container
I have a csv file in my host machine and I have a docker VM. I want to upload the data from the csv file in my host to the VM. I am using python‘s hive_service library to connect to and make queries. However, I am stuck as to how to put data into the VM.
For example the script below connects and is able to query hive but fails at the second query. I need to upload smpl.txt from my host machine to the docker vm
import sys from hive_service import ThriftHive from hive_service.ttypes import HiveServerException from thrift import Thrift from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol try: print "1111" transport = TSocket.TSocket("192.168.99.100", 10000) transport = TTransport.TBufferedTransport(transport) protocol = TBinaryProtocol.TBinaryProtocol(transport) print "2222" client = ThriftHive.Client(protocol) transport.open() print "3333" client.execute("CREATE TABLE names (name string, value int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ','") client.execute("LOAD DATA LOCAL INPATH './smpl.txt' OVERWRITE INTO TABLE names") # client.execute("SELECT * FROM orders") # while (1): # row = client.fetchOne() # if (row == None): # break # print row #client.execute("SELECT * FROM r") # print client.fetchAll() print "4444" transport.close() except Thrift.TException, tx: print '%s' % (tx.message)
One Solution collect form web for “Load data from host machine to docker hive container”
If that file must already be in the docker VM, you can consider using
docker cp, which allows for copying a local file to a running container.
Using a library like
pypi/docker-py/, that would translate into:
c = docker.Client(base_url='unix://var/run/docker.sock', version='1.12', timeout=10) c.copy(container, resource)