Updates for moby 28.0 networking (#21612)

## Description

Updates for moby 28.0 networking.

## Related issues or tickets

Series of commits ...
- Fix description of 'inhibit_ipv4'
- Not changed in moby 28.0, updated to clarify difference from (new)
IPv6-only networks.
- Updates to default bridge address config
  - https://github.com/moby/moby/pull/48319
- Describe IPv6-only network config
  - https://github.com/moby/moby/pull/48271
  - https://github.com/docker/cli/pull/5599
- Update description of gateway modes
  - https://github.com/moby/moby/pull/48594
  - https://github.com/moby/moby/pull/48596
  - https://github.com/moby/moby/pull/48597
- Describe gateway selection in the networking overview.
  - https://github.com/docker/cli/pull/5664
- Describe gateway mode `isolated`
  - https://github.com/moby/moby/pull/49262

## Reviews

<!-- Notes for reviewers here -->
<!-- List applicable reviews (optionally @tag reviewers) -->

- [ ] Technical review
- [ ] Editorial review
- [ ] Product review

---------

Signed-off-by: Rob Murray <rob.murray@docker.com>

content/manuals/engine/network/_index.md

@@ -64,6 +64,41 @@ networking functionality:
 For more information about the different drivers, see [Network drivers
 overview](./drivers/_index.md).
 
+### Connecting to multiple networks
+
+A container can be connected to multiple networks.
+
+For example, a frontend container may be connected to a bridge network
+with external access, and a
+[`--internal`](/reference/cli/docker/network/create/#internal) network
+to communicate with containers running backend services that do not need
+external network access.
+
+A container may also be connected to different types of network. For example,
+an `ipvlan` network to provide internet access, and a `bridge` network for
+access to local services.
+
+When sending packets, if the destination is an address in a directly connected
+network, packets are sent to that network. Otherwise, packets are sent to
+a default gateway for routing to their destination. In the example above,
+the `ipvlan` network's gateway must be the default gateway.
+
+The default gateway is selected by Docker, and may change whenever a
+container's network connections change.
+To make Docker choose a specific default gateway when creating the container
+or connecting a new network, set a gateway priority. See option `gw-priority`
+for the [`docker run`](/reference/cli/docker/container/run.md) and
+[`docker network connect`](/reference/cli/docker/network/connect.md) commands.
+
+The default `gw-priority` is `0` and the gateway in the network with the
+highest priority is the default gateway. So, when a network should always
+be the default gateway, it is enough to set its `gw-priority` to `1`.
+
+```console
+$ docker run --network name=gwnet,gw-priority=1 --network anet1 --name myctr myimage
+$ docker network connect anet2 myctr
+```
+
 ## Container networks
 
 In addition to user-defined networks, you can attach a container to another
@@ -145,6 +180,14 @@ direct routing to containers, see
 
 ## IP address and hostname
 
+When creating a network, IPv4 address allocation is enabled by default, it
+can be disabled using `--ipv4=false`. IPv6 address allocation can be enabled
+using `--ipv6`.
+
+```console
+$ docker network create --ipv6 --ipv4=false v6net
+```
+
 By default, the container gets an IP address for every Docker network it attaches to.
 A container receives an IP address out of the IP subnet of the network.
 The Docker daemon performs dynamic subnetting and IP address allocation for containers.

content/manuals/engine/network/drivers/bridge.md

@@ -106,15 +106,15 @@ The following table describes the driver-specific options that you can pass to
 `--opt` when creating a custom network using the `bridge` driver.
 
 | Option                                                                                          | Default                     | Description                                                                                         |
-|-------------------------------------------------------------------------------------------------|-----------------------------|------------------------------------------------------------------------------------------------|
+|-------------------------------------------------------------------------------------------------|-----------------------------|-----------------------------------------------------------------------------------------------------|
 | `com.docker.network.bridge.name`                                                                |                             | Interface name to use when creating the Linux bridge.                                               |
 | `com.docker.network.bridge.enable_ip_masquerade`                                                | `true`                      | Enable IP masquerading.                                                                             |
-| `com.docker.network.bridge.gateway_mode_ipv4`<br/>`com.docker.network.bridge.gateway_mode_ipv6` | `nat`                       | Enable NAT and masquerading (`nat`), or only allow direct routing to the container (`routed`). |
+| `com.docker.network.bridge.gateway_mode_ipv4`<br/>`com.docker.network.bridge.gateway_mode_ipv6` | `nat`                       | Control external connectivity. See [Packet filtering and firewalls](packet-filtering-firewalls.md). |
 | `com.docker.network.bridge.enable_icc`                                                          | `true`                      | Enable or Disable inter-container connectivity.                                                     |
 | `com.docker.network.bridge.host_binding_ipv4`                                                   | all IPv4 and IPv6 addresses | Default IP when binding container ports.                                                            |
 | `com.docker.network.driver.mtu`                                                                 | `0` (no limit)              | Set the containers network Maximum Transmission Unit (MTU).                                         |
 | `com.docker.network.container_iface_prefix`                                                     | `eth`                       | Set a custom prefix for container interfaces.                                                       |
-| `com.docker.network.bridge.inhibit_ipv4`                                                        | `false`                     | Prevent Docker from [assigning an IP address](#skip-ip-address-configuration) to the network.  |
+| `com.docker.network.bridge.inhibit_ipv4`                                                        | `false`                     | Prevent Docker from [assigning an IP address](#skip-bridge-ip-address-configuration) to the bridge. |
 
 Some of these options are also available as flags to the `dockerd` CLI, and you
 can use them to configure the default `docker0` bridge when starting the Docker
@@ -229,6 +229,20 @@ When you create your network, you can specify the `--ipv6` flag to enable IPv6.
 $ docker network create --ipv6 --subnet 2001:db8:1234::/64 my-net
 ```
 
+If you do not provide a `--subnet` option, a Unique Local Address (ULA) prefix
+will be chosen automatically.
+
+## IPv6-only bridge networks
+
+To skip IPv4 address configuration on the bridge and in its containers, create
+the network with option `--ipv4=false`, and enable IPv6 using `--ipv6`.
+
+```console
+$ docker network create --ipv6 --ipv4=false v6net
+```
+
+IPv4 address configuration cannot be disabled in the default bridge network.
+
 ## Use the default bridge network
 
 The default `bridge` network is considered a legacy detail of Docker and is not
@@ -259,7 +273,12 @@ the settings you need to customize.
 }
 ```
 
-Restart Docker for the changes to take effect.
+In this example:
+
+- The bridge's address is "192.168.1.1/24" (from `bip`).
+- The bridge network's subnet is "192.168.1.0/24" (from `bip`).
+- Container addresses will be allocated from "192.168.1.0/25" (from `fixed-cidr`).
+
 
 ### Use IPv6 with the default bridge network
 
@@ -270,22 +289,34 @@ These three options only affect the default bridge, they are not used by
 user-defined networks. The addresses in below are examples from the
 IPv6 documentation range.
 
-- Option `ipv6` is required
+- Option `ipv6` is required.
-- Option `fixed-cidr-v6` is required, it specifies the network prefix to be used.
+- Option `bip6` is optional, it specifies the address of the default bridge, which
+  will be used as the default gateway by containers. It also specifies the subnet
+  for the bridge network.
+- Option `fixed-cidr-v6` is optional, it specifies the address range Docker may
+  automatically allocate to containers.
   - The prefix should normally be `/64` or shorter.
   - For experimentation on a local network, it is better to use a Unique Local
-    prefix (matching `fd00::/8`) than a Link Local prefix (matching `fe80::/10`).
+    Address (ULA) prefix (matching `fd00::/8`) than a Link Local prefix (matching
+    `fe80::/10`).
 - Option `default-gateway-v6` is optional. If unspecified, the default is the first
   address in the `fixed-cidr-v6` subnet.
 
 ```json
 {
   "ipv6": true,
+  "bip6": "2001:db8::1111/64",
   "fixed-cidr-v6": "2001:db8::/64",
   "default-gateway-v6": "2001:db8:abcd::89"
 }
 ```
 
+If no `bip6` is specified, `fixed-cidr-v6` defines the subnet for the bridge
+network. If no `bip6` or `fixed-cidr-v6` is specified, a ULA prefix will be
+chosen.
+
+Restart Docker for changes to take effect.
+
 ## Connection limit for bridge networks
 
 Due to limitations set by the Linux kernel, bridge networks become unstable and
@@ -295,20 +326,22 @@ to a single network.
 For more information about this limitation, see
 [moby/moby#44973](https://github.com/moby/moby/issues/44973#issuecomment-1543747718).
 
-## Skip IP address configuration
+## Skip Bridge IP address configuration
+
+The bridge is normally assigned the network's `--gateway` address, which is
+used as the default route from the bridge network to other networks.
 
 The `com.docker.network.bridge.inhibit_ipv4` option lets you create a network
-that uses an existing bridge and have Docker skip configuring the IPv4 address
+without the IPv4 gateway address being assigned to the bridge. This is useful
-on the bridge. This is useful if you want to configure the IP address for the
+if you want to configure the gateway IP address for the bridge manually. For
-bridge manually. For instance if you add a physical interface to your bridge,
+instance if you add a physical interface to your bridge, and need it to have
-and need to move its IP address to the bridge interface.
+the gateway address.
 
-To use this option, you should first configure the Docker daemon to use a
+With this configuration, north-south traffic (to and from the bridge network)
-self-managed bridge, using the `bridge` option in the `daemon.json` or the
+won't work unless you've manually configured the gateway address on the bridge,
-`dockerd --bridge` flag.
+or a device attached to it.
 
-With this configuration, north-south traffic won't work unless you've manually
+This option can only be used with user-defined bridge networks.
-configured the IP address for the bridge.
 
 ## Next steps
 

content/manuals/engine/network/packet-filtering-firewalls.md

@@ -129,31 +129,78 @@ clients. No routes are normally set up in the host's network for container
 addresses that exist within a host.
 
 But, particularly with IPv6 you may prefer to avoid using NAT and instead
-arrange for external routing to container addresses.
+arrange for external routing to container addresses ("direct routing").
 
 To access containers on a bridge network from outside the Docker host,
 you must set up routing to the bridge network via an address on the Docker
 host. This can be achieved using static routes, Border Gateway Protocol
 (BGP), or any other means appropriate for your network.
 
-The bridge network driver has options
+Within a local layer 2 network, remote hosts can set up static routes
-`com.docker.network.bridge.gateway_mode_ipv6=<nat|routed>` and
+to a container network using the Docker daemon host's address on the local
-`com.docker.network.bridge.gateway_mode_ipv4=<nat|routed>`.
+network. Those hosts can access containers directly. For remote hosts
+outside the local network, direct access to containers requires router
+configuration to enable the necessary routing.
+
+#### Gateway modes
+
+The bridge network driver has the following options:
+- `com.docker.network.bridge.gateway_mode_ipv6`
+- `com.docker.network.bridge.gateway_mode_ipv4`
+
+Each of these can be set to one of the gateway modes:
+- `nat`
+- `nat-unprotected`
+- `routed`
+- `isolated`
 
 The default is `nat`, NAT and masquerading rules are set up for each
-published container port. With mode `routed`, no NAT or masquerading rules
+published container port. Packets leaving the host will use a host address.
-are set up, but `iptables` are still set up so that only published container
+
-ports are accessible.
+With mode `routed`, no NAT or masquerading rules are set up, but `iptables`
+are still set up so that only published container ports are accessible.
+Outgoing packets from the container will use the container's address,
+not a host address.
+
+In `nat` mode, when a port is published to a specific host address, that
+port is only accessible via the host interface with that address. So,
+for example, publishing a port to an address on the loopback interface
+means remote hosts cannot access it.
+
+However, using direct routing, published container ports are always
+accessible from remote hosts, unless the Docker host's firewall has
+additional restrictions. Hosts on the local layer-2 network can set up
+direct routing without needing any additional network configuration.
+Hosts outside the local network can only use direct routing to the
+container if the network's routers are configured to enable it.
+
+In `nat-unprotected` mode, unpublished container ports are also
+accessible using direct routing, no port filtering rules are set up.
+This mode is included for compatibility with legacy default behaviour.
+
+The gateway mode also affects communication between containers that
+are connected to different Docker networks on the same host.
+- In `nat` and `nat-unprotected` modes, containers in other bridge
+  networks can only access published ports via the host addresses they
+  are published to. Direct routing from other networks is not allowed.
+- In `routed` mode containers in other networks can use direct
+  routing to access ports, without going via a host address.
 
 In `routed` mode, a host port in a `-p` or `--publish` port mapping is
 not used, and the host address is only used to decide whether to apply
 the mapping to IPv4 or IPv6. So, when a mapping only applies to `routed`
-mode, only addresses `0.0.0.0` or `::1` are allowed, and a host port
+mode, only addresses `0.0.0.0` or `::` should be used, and a host port
-must not be given.
+should not be given. If a specific address or port is given, it will
+have no effect on the published port and a warning message will be
+logged.
 
-Mapped container ports, in `nat` or `routed` mode, are accessible from
+Mode `isolated` can only be used when the network is also created with
-any remote address, if routing is set up in the network, unless the
+CLI flag `--internal`, or equivalent. An address is normally assigned to the
-Docker host's firewall has additional restrictions.
+bridge device in an `internal` network. So, processes on the docker host can
+access the network, and containers in the network can access host services
+listening on that bridge address (including services listening on "any" host
+address, `0.0.0.0` or `::`). No address is assigned to the bridge when the
+network is created with gateway mode `isolated`.
 
 #### Example