Review Interlock upgrade

This commit is contained in:
Joao Fernandes 2018-03-26 15:52:45 -07:00 committed by Jim Galasyn
parent 50c2475dce
commit 72e6b3d49a
3 changed files with 113 additions and 67 deletions

View File

@ -1733,6 +1733,8 @@ manuals:
path: /ee/ucp/interlock/usage/host-mode-networking/
- title: Service labels reference
path: /ee/ucp/interlock/usage/labels-reference/
- title: Layer 7 routing upgrade
path: /ee/ucp/interlock/upgrade/
- sectiontitle: Deploy apps with Kubernetes
section:
- title: Deploy a workload

View File

@ -1,67 +0,0 @@
# Migration
In Docker EE UCP 2.X, the layer 7 routing of the platform was implemented using HRM. Starting with Docker EE UCP 3.X, the layer 7 routing is now implemented using Interlock. This article describes the process in which the migration from HRM to Interlock happens.
For context, let me recap on the HRM to interlock migration process:
0. HRM service is already enabled and configured by the customer.
1. Migration from 2.2.4 to 3.0 starts
2. Reconcilation process for interlock starts and HRM service is inspected.
3. An interlock config is created from the HRM service spec and deployed as a docker config
4. The HRM service is removed.
5. The interlock service is deployed.
6. The interlock extension and proxy service are deployed by the interlock service.
Currently, this process is not atomic, so any failure along the way will not result in the reversal of previous steps. In any case, if a failure occurs today the migration would need manual fixing to operate normally again.
In the following section, I cover different failure points, how to diagnostic, and to solve (when possible) from a customer/support standpoint.
### Before starting migration
**TODO**: Information [on interlock manual deploymenbt](https://beta.docs.docker.com/ee/ucp/interlock/install/manual-deployment/#deployment) is incorrect. Please update with procedure in this PR: https://github.com/docker/orca/issues/12227.
#### Migration check procedure
Record which of your HRM-enabled applications are routable through HRM by doing a curl request:
- For HTTP apps: ```curl -vs http://${CLUSTER_ID}:${HRM_HTTP_PORT}/ -H "Host: ${HOSTNAME}"```
- For HTTPS apps: ```curl -vs http://${HOSTNAME}:${HRM_HTTPS_PORT}"```
When the migration, check that all applications that where routable before the migration still are.
#### Failure happens during step 1 before step 2
Not related to HRM interlock migration
#### Failure happens during step 2 before step 3
Codepath: https://github.com/docker/orca/blob/master/agent/agent/components/interlockservice/interlockservice.go
- **Diagnostic**: Only happens if the HRM service gets deleted somehow after being detected as present but before the migration to interlock starts.
- **Resolution**: Follow the *manual hrm migration* procedure.
#### Failure happens during step 3 before step 4
- **Diagnostic**: `ucp-hrm` is still present, but the config `com.docker.ucp.interlock.conf-1` could not be created. Could happen due to name conflicts.
- **Resolution**: Follow the *manual hrm migration* procedure.
#### Failure happens during step 4 before step 5
- **Diagnostic**: `ucp-hrm` failed to remove. The ucp interlock config is already created. Only happens if the service delete operation for the `ucp-hrm` service fails for some reason (unlikely).
- **Resolution**: Follow the *manual hrm migration* procedure, but replace the interlock swarm config name from `com.docker.ucp.interlock.conf-1` to `com.docker.ucp.interlock.conf-2`.
#### Failure happens during step 5 before step 6
- **Diagnostic**: The `ucp-interlock` service failed to deploy. Only happens if the service create operation for the `ucp-interlock` service fails for some reason (unlikely).
- **Resolution**: Enable interlock through the UI, making sure that the `PublishedPort` and `PublishedSSLPort` match the http and https ports you were using with HRM.
#### Failure happens during step 6
- **Diagnostic**: The `ucp-interlock-extension` or `ucp-interlock-proxy` services failed to deploy. May happen if there is a port conflict.
- **Resolution**: Disable and then re-enable interlock through the UI, making sure that the `PublishedPort` and `PublishedSSLPort` match the http and https ports you were using with HRM.
#### Manual Migration procedure
1. delete the `ucp-hrm` service.
2. Enable interlock manually (https://beta.docs.docker.com/ee/ucp/interlock/install/manual-deployment/), but instead of creating a new overlay network called `ucp-interlock`, reuse the existing `ucp-hrm`.
3. Wait until the service named `ucp-interlock-proxy` is deployed with all replicas and then try the *migration check procedure* again.

111
ee/ucp/interlock/upgrade.md Normal file
View File

@ -0,0 +1,111 @@
---
title: Layer 7 routing upgrade
description: Learn how to route layer 7 traffic to your swarm services
keywords: routing, proxy, hrm
---
The [HTTP routing mesh](/datacenter/ucp/2.2/guides/admin/configure/use-domain-names-to-access-services.md)
functionality was redesigned in UCP 3.0 for greater security and flexibility.
The functionality was also renamed to "layer 7 routing", to make it easier for
new users to get started.
[Learn about the new layer 7 routing functionality](index.md).
To route traffic to your service you apply specific labels to your swarm
services, describing the hostname for the service and other configurations.
Things work in the same way as they did with the HTTP routing mesh, with the
only difference being that you use different labels.
You don't have to manually update your services. During the upgrade process to
3.0, UCP updates the services to start using new labels.
This article describes the upgrade process for the routing component, so that
you can troubleshoot UCP and your services, in case something goes wrong with
the upgrade.
# UCP upgrade process
If you are using the HTTP routing mesh, and start an upgrade to UCP 3.0:
1. UCP starts a reconciliation process to ensure all internal components are
deployed. As part of this, services using HRM labels are inspected.
2. UCP creates the `com.docker.ucp.interlock.conf-<id>` based on HRM configurations.
3. The HRM service is removed.
4. The `ucp-interlock` service is deployed with the configuration created.
5. The `ucp-interlock` service deploys the `ucp-interlock-extension` and
`ucp-interlock-proxy-services`.
The only way to rollback from an upgrade is by restoring from a backup taken
before the upgrade. If something goes wrong during the upgrade process, you
need to troubleshoot the interlock services and your services, since the HRM
service won't be running after the upgrade.
[Learn more about the interlock services and architecture](architecture.md).
## Check that routing works
After upgrading to UCP 3.0, you should check if all swarm services are still
routable.
For services using HTTP:
```bash
curl -vs http://<ucp-url>:<hrm-http-port>/ -H "Host: <service-hostname>"
```
For services using HTTPS:
```bash
curl -vs http://<ucp-url>:<hrm-https-port>
```
After the upgrade, check that you can still use the same hostnames to access
the swarm services.
## The ucp-interlock services are not running
After the upgrade to UCP 3.0, the following services should be running:
* `ucp-interlock`: monitors swarm workloads configured to use layer 7 routing.
* `ucp-interlock-extension`: Helper service that generates the configuration for
the `ucp-interlock-proxy` service.
* `ucp-interlock-proxy`: A service that provides load balancing and proxying for
swarm workloads.
To check if these services are running, use a client bundle with administrator
permissions and run:
```bash
docker ps -a | grep ucp-interlock
```
* If the `ucp-interlock` service doesn't exist or is not running, something went
wrong with the reconciliation step.
* If this still doesn't work, it's possible that UCP is having problems creating
the `com.docker.ucp.interlock.conf-1`, due to name conflicts. Make sure you
don't have any configuration with the same name by running:
* If either the `ucp-interlock-extension` or `ucp-interlock-proxy` services are
not running, it's possible that there are port conflicts.
As a workaround re-enable the layer 7 routing configuration from the
[UCP settings page](deploy/index.md). Make sure the ports you choose are not
being used by other services.
## Workarounds and clean-up
If you have any of the problems above, disable and enable the layer 7 routing
setting on the [UCP settings page](deploy/index.md). This redeploys the
services with their default configuration.
When doing that make sure you specify the same ports you were using for HRM,
and that no other services are listening on those ports.
You should also check if the `ucp-hrm` service is running. If it is, you should
stop it since it can conflict with the `ucp-interlock-proxy` service.
## Optionally remove labels
As part of the upgrade process UCP adds the
[labels specific to the new layer 7 routing solution](usage/labels-reference.md).
You can update your services to remove the old HRM labels, since they won't be
used anymore.