Commit Graph

108 Commits

Author SHA1 Message Date
Solomon Hykes b9c316df29 inmem: complete Copy implementation
Copy can now handle arbitrary nesting of streams, without knowing the
underlying type of either the source or destination.

It does this by attempting `SendTo` and `ReceiveFrom`, then if that is
not supported, falling back to nested copies in separate goroutines.

Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:48:13 -07:00
Solomon Hykes 1508c1d46f inmem: PipeSender and PipeReceiver return clean nil values
This is necessary because `(*PipeSender)nil` is not equal to
`(interface{})nil`.

Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:48:13 -07:00
Solomon Hykes 048bed0cd1 inmem: a message is simply 1 opcode + an array of binary strings.
Remove redundant `Data` field. Every argument can be arbitrary data.

Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:48:13 -07:00
Solomon Hykes 5b89b2fd83 inmem.StackSender.Add returns a reference to the previous chain
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:47:34 -07:00
Solomon Hykes 7d0ba6e26b inmem: Sender implements Close
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:46:24 -07:00
Solomon Hykes 655a0af862 inmem: NopReceiver and NopSender
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:46:23 -07:00
Solomon Hykes 12b5459a1c inmem.StackSender: a dynamic beam forwarded with health checks
StackSender forwards beam messages to a dynamic list of backend receivers.
New backends are stacked on top. When a message is sent, each backend is
tried until one succeeds. Any failing backends encountered along the way
are removed from the queue.

Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:46:23 -07:00
Solomon Hykes 6616f08fe0 inmem: gofmt
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:46:23 -07:00
Solomon Hykes ffa47f5445 inmem: fix the tests
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:46:23 -07:00
Solomon Hykes f22b8f52f3 inmem: Receiver and Sender don't need to implement Close
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:46:23 -07:00
Solomon Hykes 91133d63be inmem: Fix PipeSender to implement ReceiverFrom
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:46:23 -07:00
Solomon Hykes 1eb832dcbc inmem: Copy requires either SenderTo or ReceiverFrom
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:46:23 -07:00
Solomon Hykes 3d03104af4 data: expose EncodeList and DecodeList
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:46:11 -07:00
Solomon Hykes 0feebd2320 inmem: Implement `Copy`, `ReceiveFrom` and `SendTo`
* Copy is the beam equivalent of `io.Copy`
* `ReceiveFrom` is the beam equivalent of `io.ReadFrom`
* `SendTo` is the beam equivalent of `io.WriteTo`

Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:44:23 -07:00
Solomon Hykes a5cb570bfb inmem: more testing
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:44:20 -07:00
Solomon Hykes 338a7d50a8 inmem: fix nested in/out channels in each message
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:41:38 -07:00
Solomon Hykes caa6004064 inmem: use pkg/testutils
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:41:21 -07:00
Solomon Hykes a1ed1ec1fa inmem: switch to a Sender/Receiver/Pipe interface similar to io.Pipe
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-05-16 18:40:31 -07:00
Solomon Hykes c0364720db beam: no target field yet 2014-05-09 21:00:46 -07:00
Solomon Hykes 3725628467 Beam in-mem: an in-memory implementation of Beam 2014-05-09 21:00:46 -07:00
Solomon Hykes 2c9e18009f Fix stdin handling in engine.Sender and engine.Receiver
This introduces a superficial change to the Beam API:

* `beam.SendPipe` is renamed to the more accurate `beam.SendRPipe`
* `beam.SendWPipe` is introduced as a mirror to `SendRPipe`

There is no other change in the beam API.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-05-09 15:39:55 -07:00
Alexander Larsson e9651077fc beam: Add simple framing system for UnixConn
This is needed for Send/Recieve to correctly handle borders between
the messages.

The framing uses a single 32bit uint32 length for each frame, of which
the high bit is used to indicate whether the message contains a file
descriptor or not. This is enough to separate out each message sent
and to decide to which message each file descriptors belongs, even
though multiple Sends may be coalesced into a single read, and/or one
Send can be split into multiple writes.

Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-05-01 11:06:01 -07:00
Tianon Gravi d0b1155e89 Fix various MAINTAINERS format inconsistencies
Docker-DCO-1.1-Signed-off-by: Andrew Page <admwiggin@gmail.com> (github: tianon)
2014-04-30 11:22:11 -06:00
Alexander Larsson 00480dcc8c beam: Add more tests to unix_test.go
These are failing, and indicate things that need to be fixed.  The
primarily problem is the lack of framing between beam messages.

Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
[solomon@docker.com: rebased on master]
Signed-off-by: Solomon Hykes <solomon@docker.com>
2014-04-30 02:10:09 -07:00
Solomon Hykes e1ace86c5a Fix Go formatting in beam and dockerscript
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 16:16:29 -07:00
Solomon Hykes 2a5908261b beam/examples/beamsh: 'chdir' changes the current directory
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:24 -07:00
Solomon Hykes 47751cc50e beam/examples/beamsh: 'exec' can communicate with its child via beam.
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:24 -07:00
Solomon Hykes d599ab2f65 beam/examples/beamsh: use beam.Router to simplify 'multiprint' and fix job passthrough
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:23 -07:00
Solomon Hykes d04102e5dd beam: new routing functions Route.KeyEquals, Route.KeyIncludes, Route.NoKey
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:23 -07:00
Solomon Hykes 8bc15104e2 beam/examples/beamsh: use beam.Router to simplify 'trace'
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:23 -07:00
Solomon Hykes f84e7624aa beam/examples/beamsh: move builtins to a separate file for readability
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:23 -07:00
Solomon Hykes 6a727eb78d beam/examples/beamsh: simplify commands by always creating and sending stdout and stderr for them
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:23 -07:00
Solomon Hykes cfee34a9a7 beam/examples/beamsh: use beam.Router to simplify 'logger'
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:23 -07:00
Solomon Hykes e94309f056 beam/examples/beamsh: use beam.Router to simplify the 'stdio' command
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:23 -07:00
Solomon Hykes 0e85dc3423 beam: Router can route beam messages with a convenient set of rules and handlers
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:23 -07:00
Solomon Hykes b645a78485 beam/examples/beamsh: move example scripts to scripts/
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Alexander Larsson faa83a0fc0 beam: Fix double close of fds in SendUnix
Instead of calling syscall.Close() on the fds in sendUnix() we call
Close() on the *os.File in Send(). Otherwise the fd will be closed, but
the *os.File will continue to live, and when it is finalized the
fd will be closed (which by then may be reused and can be anything).

This also adds a note to Send() the the file is closed.

Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
2014-04-22 15:50:22 -07:00
Solomon Hykes 80df1aaaaa beam: more unit tests
This adds testing to SendConn.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes 7436d1a9ca beam/examples/beamsh: add scripts to reproduce various bugs
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes 1cfa87002c beam/examples/beamsh: fix 'print' to be pass-through
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes 09c76a202f beam/examples/beamsh: rename 'log' to 'logger' to avoid conflict with stdout/stderr
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes 107326fa98 beam/examples/beamsh: 'prompt' asks the user for a value and sends it.
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes 2bc94d4bdc beam/examples/beamsh: commands are messages.
Commands in the pipeline should either implement or pass-through command messages.

This amounts to a proof-of-concept implementation of the "pipeline"
design of Docker plugins.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes 3cf484f376 beam/examples/beamsh: cosmetic fix in Fatalf
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes 65a0f0c27d beam: replace SendPair() with the simpler SendConn()
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Solomon Hykes 5f8c6df2a0 beam/examples/beamsh: miniserver.ds demo
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Solomon Hykes 626ef1644c beam: improve the API with Sender/Receiver interfaces and utilities: Copy/SendPipe/SendPair
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Alexander Larsson 979277c7a0 beam: Make extracted Fds CloseOnExec
Grab forklock to make sure no forks accidentally inherit the new fds
before they are made CLOEXEC There is a slight race condition between
ReadMsgUnix returns and when we grap the lock, so this is not
perfect. Unfortunately There is no way to pass MSG_CMSG_CLOEXEC to
recvmsg() nor any way to implement non-blocking i/o in go, so this is
hard to fix.

Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
2014-04-22 15:50:21 -07:00
Solomon Hykes e64b46fb78 beam/examples/beamsh: add a few example dockerscripts
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Solomon Hykes bc22e5589e beam/examples/beamsh: simplify code by using sendWPipe utility
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00