From 92759efddaa3c9ff8c8834d7af34d1b996d8e5db Mon Sep 17 00:00:00 2001 From: Olly P Date: Fri, 11 Oct 2019 18:42:20 +0100 Subject: [PATCH] Added commentary around Azure NSGs for UCP (#9612) * Added commentary around Azure NSGs * Added reference to Pod CIDR * Adjusted for Peer Review --- .../cloudproviders/install-on-azure.md | 23 +++++++++++++++---- ee/ucp/admin/install/system-requirements.md | 15 +++++++++--- 2 files changed, 30 insertions(+), 8 deletions(-) diff --git a/ee/ucp/admin/install/cloudproviders/install-on-azure.md b/ee/ucp/admin/install/cloudproviders/install-on-azure.md index 011fe07def..1b3cdca1c7 100644 --- a/ee/ucp/admin/install/cloudproviders/install-on-azure.md +++ b/ee/ucp/admin/install/cloudproviders/install-on-azure.md @@ -56,6 +56,19 @@ You must meet the following infrastructure prerequisites to successfully deploy needed as part of the UCP prerequisites. If you are using a separate Resource Group for the networking components, the same Service Principal will need `Network Contributor` access to this Resource Group. +- Kubernetes pods integrate into the underlying Azure networking stack, from + an IPAM and routing perspective with the Azure CNI IPAM module. Therefore + Azure Network Security Groups (NSG) impact pod to pod communication. End users + may expose containerized services on a range of underlying ports, resulting in + a manual process to open an NSG port every time a new containerized service is + deployed on to the platform. This would only affect workloads deployed on to + the Kubernetes orchestrator. It is advisable to have an "open" NSG between + all IPs on the Azure Subnet passed into UCP at [install time](#install-ucp). + To limit exposure, this Azure subnet should be locked down to only be used + for Container Host VMs and Kubernetes Pods. Additionally, end users can + leverage [Kubernetes Network + Policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) + to provide micro segmentation for containerized applications and services. UCP requires the following information for the installation: @@ -207,7 +220,7 @@ addresses, from the same Azure Subnet as the hosts, for each Virtual Machine in the cluster. However if you have manually attached additional IP addresses to the Virtual Machines (via an ARM Template, Azure CLI or Azure Portal) or you are deploying in to small Azure subnet (less than /16), an `--azure-ip-count` -flag can be used at install time. +flag can be used at install time. > Note: Do not set the `--azure-ip-count` variable to a value of less than 6 if > you have not manually provisioned additional IP addresses for each Virtual @@ -216,7 +229,7 @@ flag can be used at install time. > to the Virtual Machine's private IP address. Below are some example scenarios which require the `--azure-ip-count` variable -to be defined. +to be defined. **Scenario 1 - Manually Provisioned Addresses** @@ -232,16 +245,16 @@ addresses to a custom value due to: - Primarily using the Swarm Orchestrator - Deploying UCP on a small Azure subnet (for example /24) -- Plan to run a small number of Kubernetes pods on each node. +- Plan to run a small number of Kubernetes pods on each node. For example if you wanted to provision 16 addresses per virtual machine, then -you would pass `--azure-ip-count 16` into the UCP installation command. +you would pass `--azure-ip-count 16` into the UCP installation command. If you need to adjust this value post-installation, see [instructions](https://docs.docker.com/ee/ucp/admin/configure/ucp-configuration-file/) on how to download the UCP configuration file, change the value, and update the configuration via the API. If you reduce the value post-installation, existing virtual machines will not -be reconciled, and you will have to manually edit the IP count in Azure. +be reconciled, and you will have to manually edit the IP count in Azure. ### Install UCP diff --git a/ee/ucp/admin/install/system-requirements.md b/ee/ucp/admin/install/system-requirements.md index ef7a3f1ae9..a1bf441586 100644 --- a/ee/ucp/admin/install/system-requirements.md +++ b/ee/ucp/admin/install/system-requirements.md @@ -16,7 +16,7 @@ You can install UCP on-premises or on a cloud provider. Common requirements: * [Docker Engine - Enterprise](/ee/supported-platforms.md) version {{ site.docker_ee_version }} * Linux kernel version 3.10 or higher * [A static IP address for each node in the cluster](/ee/ucp/admin/install/plan-installation/#static-ip-addresses) - + ### Minimum requirements * 8GB of RAM for manager nodes @@ -59,6 +59,15 @@ indicated as the "Scope" of that port. The three scopes are: - Internal: Traffic arrives from other hosts in the same cluster. - Self: Traffic arrives to that port only from processes on the same host. +> Note +> +> When installing UCP on Microsoft Azure, an overlay network is not used for +> Kubernetes; therefore, any containerized service deployed onto Kubernetes and +> exposed as a Kubernetes Service may need its corresponding port to be opened +> on the underlying Azure Network Security Group. For more information see +> [Installing on +> Azure](/ee/ucp/admin/install/cloudproviders/install-on-azure/#azure-prerequisites). + Make sure the following ports are open for incoming traffic on the respective host types: @@ -87,10 +96,10 @@ host types: | managers | TCP 12388 | Internal | Internal Port for the Kubernetes API Server | ## Disable `CLOUD_NETCONFIG_MANAGE` for SLES 15 -For SUSE Linux Enterprise Server 15 (SLES 15) installations, you must disable `CLOUD_NETCONFIG_MANAGE` +For SUSE Linux Enterprise Server 15 (SLES 15) installations, you must disable `CLOUD_NETCONFIG_MANAGE` prior to installing UCP. - 1. In the network interface configuration file, `/etc/sysconfig/network/ifcfg-eth0`, set + 1. In the network interface configuration file, `/etc/sysconfig/network/ifcfg-eth0`, set ``` CLOUD_NETCONFIG_MANAGE="no" ```