Merge pull request #4670 from gjtempleton/CA-AWS-IAM-Permissions-Rework

CA - AWS - Rework IAM Docs/Snippets

cluster-autoscaler/cloudprovider/aws/CA_with_AWS_IAM_OIDC.md

@@ -53,9 +53,11 @@ D) Set up [Cluster Autoscaler Auto-Discovery] using the [tutorial] .
 
 Note: The keys for the tags that you entered don't have values. Cluster Autoscaler ignores any value set for the keys.
 
-- Create an IAM Policy for cluster autoscaler and to enable AutoDiscovery. 
+__NOTE:__ Please see [the README](README.md#IAM-Policy) for more information on best practices with this IAM role.
 
-```sh
+- Create an IAM Policy for cluster autoscaler and to enable AutoDiscovery as well as discovery of instance types.
+
+```json
 {
     "Version": "2012-10-17",
     "Statement": [
@@ -66,17 +68,23 @@ Note: The keys for the tags that you entered don't have values. Cluster Autoscal
                 "autoscaling:DescribeAutoScalingInstances",
                 "autoscaling:DescribeLaunchConfigurations",
                 "autoscaling:DescribeTags",
+                "ec2:DescribeInstanceTypes",
+                "ec2:DescribeLaunchTemplateVersions"
+            ],
+            "Resource": ["*"]
+        },
+        {
+            "Effect": "Allow",
+            "Action": [
                 "autoscaling:SetDesiredCapacity",
                 "autoscaling:TerminateInstanceInAutoScalingGroup"
             ],
-            "Resource": "*"
+            "Resource": ["*"]
         }
     ]
 }
 ```
 
-NOTE: ``` autoscaling:DescribeTags ``` is very important if you are making use of the AutoDiscovery feature of the Cluster AutoScaler. 
-
 - Attach the above created policy to the *instance role* that's attached to your Amazon EKS worker nodes.
 - Download a deployment example file provided by the Cluster Autoscaler project on GitHub, run the following command:
 

cluster-autoscaler/cloudprovider/aws/README.md

@@ -19,7 +19,69 @@ the EC2 instance on which the Cluster Autoscaler pod runs.
 
 ### IAM Policy
 
-The following policy provides the minimum privileges necessary for Cluster Autoscaler to run:
+There are a number of ways to run the autoscaler in AWS, which can significantly
+impact the range of IAM permissions required for the Cluster Autoscaler to function
+properly. Two options are provided below, one which will allow use of all of the
+features of the Cluster Autoscaler, the second with a more limited range of IAM
+actions enabled, which enforces using certain configuration options in the
+Cluster Autoscaler binary.
+
+It is strongly recommended to restrict the target resources for the autoscaling actions
+by either [specifying Auto Scaling Group ARNs](https://docs.aws.amazon.com/autoscaling/latest/userguide/control-access-using-iam.html#policy-auto-scaling-resources) in the `Resource` list of the policy or
+[using tag based conditionals](https://docs.aws.amazon.com/autoscaling/ec2/userguide/control-access-using-iam.html#security_iam_service-with-iam-tags). The [minimal policy](#minimal-iam-permissions-policy)
+includes an example of restricting by ASG ARN.
+
+#### Full Cluster Autoscaler Features Policy (Recommended)
+
+Permissions required when using [ASG Autodiscovery](#Auto-discovery-setup) and
+Dynamic EC2 List Generation (the default behaviour). In this example, only the second block of actions
+should be updated to restrict the resources/add conditionals:
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": [
+        "autoscaling:DescribeAutoScalingGroups",
+        "autoscaling:DescribeAutoScalingInstances",
+        "autoscaling:DescribeLaunchConfigurations",
+        "autoscaling:DescribeTags",
+        "ec2:DescribeInstanceTypes",
+        "ec2:DescribeLaunchTemplateVersions"
+      ],
+      "Resource": ["*"]
+    },
+    {
+      "Effect": "Allow",
+      "Action": [
+        "autoscaling:SetDesiredCapacity",
+        "autoscaling:TerminateInstanceInAutoScalingGroup"
+      ],
+      "Resource": ["*"]
+    }
+  ]
+}
+```
+
+#### Minimal IAM Permissions Policy
+
+*NOTE:* The below policies/arguments to the Cluster Autoscaler need to be modified as appropriate
+for the names of your ASGs, as well as account ID and AWS region before being used.
+
+The following policy provides the minimum privileges necessary for Cluster Autoscaler to run.
+When using this policy, you cannot use autodiscovery of ASGs. In addition, it restricts the
+IAM permissions to the node groups the Cluster Autoscaler is configured to scale.
+
+This in turn means that you must pass the following arguments to the Cluster Autoscaler
+binary, replacing min and max node counts and the ASG:
+
+```bash
+--aws-use-static-instance-list=false
+--nodes=1:100:exampleASG1
+--nodes=1:100:exampleASG2
+```
 
 ```json
 {
@@ -32,27 +94,14 @@ The following policy provides the minimum privileges necessary for Cluster Autos
         "autoscaling:DescribeAutoScalingInstances",
         "autoscaling:DescribeLaunchConfigurations",
         "autoscaling:SetDesiredCapacity",
-        "autoscaling:TerminateInstanceInAutoScalingGroup",
-        "ec2:DescribeInstanceTypes"
+        "autoscaling:TerminateInstanceInAutoScalingGroup"
       ],
-      "Resource": ["*"]
+      "Resource": ["arn:aws:autoscaling:${YOUR_CLUSTER_AWS_REGION}:${YOUR_AWS_ACCOUNT_ID}:autoScalingGroup:*:autoScalingGroupName/${YOUR_ASG_NAME}"]
     }
   ]
 }
 ```
 
-If you'd like Cluster Autoscaler to [automatically
-discover](#auto-discovery-setup) EC2 Auto Scaling Groups **(recommended)**, add
-`autoscaling:DescribeTags` to the `Action` list. Also add
-`autoscaling:DescribeLaunchConfigurations` (if you created your ASG using a
-Launch Configuration) and/or `ec2:DescribeLaunchTemplateVersions` (if you
-created your ASG using a Launch Template) to the `Action` list.
-
-If you prefer, you can restrict the target resources for the autoscaling actions
-by specifying Auto Scaling Group ARNs in the `Resource` list of the policy. More
-information can be found
-[here](https://docs.aws.amazon.com/autoscaling/latest/userguide/control-access-using-iam.html#policy-auto-scaling-resources).
-
 ### Using OIDC Federated Authentication
 
 OIDC federated authentication allows your service to assume an IAM role and interact with AWS services without having to store credentials as environment variables. For an example of how to use AWS IAM OIDC with the Cluster Autoscaler please see [here](CA_with_AWS_IAM_OIDC.md).