Started a ubuntu container that just runs "ping 8.8.8.8" and tried
the sample code in https://gist.github.com/dims/c3327f633c526847c8e5
to recreate the problem mentioned in:
https://github.com/docker/docker-py/issues/300
To debug the problem i printed the byte array read in recvall
when reading STREAM_HEADER_SIZE_BYTES and realized that the data
being read was far ahead of the actual start of the header documented
in the vnd.docker.raw-stream of the docker remote api. This is
possibly because the requests/urllib3 is reading ahead a bit more
and we shouldn't be trying to hack the internals of those projects.
So just using the documented file-like response.raw is good enough
for us to get the functionality we need which is being able to
read for exactly where the stream header starts. With this change
i can reliably stream the logs just like "docker logs --follow".
Note that we still need to access the underlying socket to set
the timeout to prevent read time outs. The original fix was for
client.logs() only but on further review it made sense to replace
all occurances of _multiplexed_socket_stream_helper with the
new method.
Currently TestImportFromStream fails, because something is going wrong
with the HTTP chunked transfer-encoding. This didn't work before,
either (for a different reason).
Rather than have one function that can do 5 things and attempts to guess
what the user wanted, it's good to have a function for each possible
method of importing an image. Error cases are handled a lot better this
way.
The unit test test_import_image_from_file() was changed to be
test_import_image_from_bytes(), because if we try to import a temporary
file in the test, the 'data' parameter of the 'fake_request' object is
an instance of a funny internal type like <_io.BufferedReader
name='/tmp/tmpc9chux'> and it's very hard to match such a thing with
`mock.assert_called_with()`.
This allows streaming a system to the 'import' command, rather than
having to read the whole thing into memory before sending it.
Previously both the UnixAdapter and the docker.Client objects would
track the 'base URL' of the Docker daemon (socket path in the case of
local Unix-domain socket connections). The Client object would construct
URLs which contained the path to the socket with the path of the Docker
API call appended. The UnixHTTPConnection instance would then remove the
known socket path from the URL.
This relied on all calls going through the HTTPConnection.request()
function, where the URL could be rewritten. In the case of 'chunked'
HTTP POST requests this doesn't happen, so such calls would request
a path still including the socket path and would receive a 404 error.
The client now constructs URLs containing just the path of the desired API
endpoint, and expects the Unix socket transport to know the path to the
Docker daemon's socket.