From 02f26a19133bc3ca66baebd4d277b806d0032dee Mon Sep 17 00:00:00 2001 From: David Deyo Date: Fri, 31 Aug 2018 09:30:45 -0700 Subject: [PATCH 01/14] Create Audit Log topic drafted --- ee/ucp/admin/configure/create-audit-logs.md | 45 +++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 ee/ucp/admin/configure/create-audit-logs.md diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md new file mode 100644 index 0000000000..6f2f305dde --- /dev/null +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -0,0 +1,45 @@ +Audit logs can be considered a type of event that are more focused on external user/agent actions and security than understanding state or events of the system itself. They are a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators or other components of the system. + +Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP API, Swarm API and Kubernetes API endpoints that are invoked (except for the ignored list) and sent to Docker Engine via stdout. zCreating audit logs is mainly CLI driven and is an UCP component that integrates with Swarm, K8s, and UCP APIs. + +## Procedure + +1. Download the UCP Client bundle. +2. +2. Retrieve JSON for current audit log configuration. +``` +export DOCKER_CERT_PATH=~/ucp-bundle-dir/ +curl --cert ${DOCKER_CERT_PATH}/cert.pem --key ${DOCKER_CERT_PATH}/key.pem --cacert ${DOCKER_CERT_PATH}/ca.pem -k -X GET https://ucp-domain/api/ucp/config/logging > auditlog.json +``` +3. Modify the auditLevel field to metadata or request. +``` +vi auditlog.json + + {"logLevel":"INFO","auditLevel":"metadata","supportDumpIncludeAuditLogs":false} + ``` +4. Send the JSON request for the auditlog config with the same API path but with the `PUT` method +``` +curl --cert ${DOCKER_CERT_PATH}/cert.pem --key ${DOCKER_CERT_PATH}/key.pem --cacert ${DOCKER_CERT_PATH}/ca.pem -k -H "Content-Type: application/json" -X PUT --data $(cat auditlog.json) https://ucp-domain/api/ucp/config/logging +``` + +5. Create any workload or RBAC grants in Kube and generate a support dump to check the contents of ucp-controller.log file for audit log entries. +6. +6. Optionally, configure the Docker Engine driver to logstash and collect and query audit logs within ELK stack after deploying ELK: https://success.docker.com/article/elasticsearch-logstash-kibana-logging + +### API Endpoints ignored + +The following endpoints are ignored since they are API endpoints that are not considered security events and may cause a large amount of entries to be triggered. + +- /_ping +- /ca +- /auth +- /trustedregistryca +- /kubeauth +- /metrics +- /info +- /version* +- /debug +- /openid_keys +- /apidocs +- /kubernetesdocs +- /manage From c6ad01898e56445d27461ebce15d035b0ff5dc36 Mon Sep 17 00:00:00 2001 From: David Deyo Date: Fri, 31 Aug 2018 09:35:30 -0700 Subject: [PATCH 02/14] edits --- ee/ucp/admin/configure/create-audit-logs.md | 1 - 1 file changed, 1 deletion(-) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index 6f2f305dde..d9c13cb703 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -5,7 +5,6 @@ Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP A ## Procedure 1. Download the UCP Client bundle. -2. 2. Retrieve JSON for current audit log configuration. ``` export DOCKER_CERT_PATH=~/ucp-bundle-dir/ From 803308c52fd0de44617f9e4a427e862c540f84f0 Mon Sep 17 00:00:00 2001 From: ddeyo Date: Fri, 31 Aug 2018 10:16:35 -0700 Subject: [PATCH 03/14] added links --- ee/ucp/admin/configure/create-audit-logs.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index d9c13cb703..df8f00ddd8 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -4,7 +4,8 @@ Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP A ## Procedure -1. Download the UCP Client bundle. +1. Download the UCP Client bundle [topic] (https://success.docker.com/article/download-client-bundle-from-the-cli). + 2. Retrieve JSON for current audit log configuration. ``` export DOCKER_CERT_PATH=~/ucp-bundle-dir/ @@ -22,7 +23,7 @@ curl --cert ${DOCKER_CERT_PATH}/cert.pem --key ${DOCKER_CERT_PATH}/key.pem --cac ``` 5. Create any workload or RBAC grants in Kube and generate a support dump to check the contents of ucp-controller.log file for audit log entries. -6. + 6. Optionally, configure the Docker Engine driver to logstash and collect and query audit logs within ELK stack after deploying ELK: https://success.docker.com/article/elasticsearch-logstash-kibana-logging ### API Endpoints ignored From 7adbdb04942110424298a6a74643aee507cbc7d7 Mon Sep 17 00:00:00 2001 From: ddeyo Date: Fri, 31 Aug 2018 10:28:36 -0700 Subject: [PATCH 04/14] last changes --- ee/ucp/admin/configure/create-audit-logs.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index df8f00ddd8..15e4826338 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -4,7 +4,7 @@ Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP A ## Procedure -1. Download the UCP Client bundle [topic] (https://success.docker.com/article/download-client-bundle-from-the-cli). +1. Download the UCP Client bundle [Download client bundle from the command line] (https://success.docker.com/article/download-client-bundle-from-the-cli). 2. Retrieve JSON for current audit log configuration. ``` @@ -26,9 +26,9 @@ curl --cert ${DOCKER_CERT_PATH}/cert.pem --key ${DOCKER_CERT_PATH}/key.pem --cac 6. Optionally, configure the Docker Engine driver to logstash and collect and query audit logs within ELK stack after deploying ELK: https://success.docker.com/article/elasticsearch-logstash-kibana-logging -### API Endpoints ignored +### API endpoints ignored -The following endpoints are ignored since they are API endpoints that are not considered security events and may cause a large amount of entries to be triggered. +The following API endpoints are ignored since they are not considered security events and may create a large amount of log entries. - /_ping - /ca From 508a76844f0a3288822a1b1874a71bb826e5ddf1 Mon Sep 17 00:00:00 2001 From: ddeyo Date: Fri, 31 Aug 2018 11:31:37 -0700 Subject: [PATCH 05/14] disclaimer added --- ee/ucp/admin/configure/create-audit-logs.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index 15e4826338..73aa00acd3 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -1,4 +1,16 @@ -Audit logs can be considered a type of event that are more focused on external user/agent actions and security than understanding state or events of the system itself. They are a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators or other components of the system. +--- +title: Create UCP audit logs +description: Learn how to create audit logs of all activity in UCP +keywords: logs, ucp, swarm, kubernetes +--- + +> BETA DISCLAIMER +> +> This is beta content. It is not yet complete and should be considered a work in progress. This content is subject to change without notice. +{: .important} + + +Audit logs are a type of event that are more focused on external user/agent actions and security than understanding state or events of the system itself. They are a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators or other components of the system. Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP API, Swarm API and Kubernetes API endpoints that are invoked (except for the ignored list) and sent to Docker Engine via stdout. zCreating audit logs is mainly CLI driven and is an UCP component that integrates with Swarm, K8s, and UCP APIs. @@ -24,7 +36,7 @@ curl --cert ${DOCKER_CERT_PATH}/cert.pem --key ${DOCKER_CERT_PATH}/key.pem --cac 5. Create any workload or RBAC grants in Kube and generate a support dump to check the contents of ucp-controller.log file for audit log entries. -6. Optionally, configure the Docker Engine driver to logstash and collect and query audit logs within ELK stack after deploying ELK: https://success.docker.com/article/elasticsearch-logstash-kibana-logging +6. Optionally, configure the Docker Engine driver to logstash and collect and query audit logs within ELK stack after deploying ELK. https://success.docker.com/article/elasticsearch-logstash-kibana-logging ### API endpoints ignored From 892acca3ad8439818928740aa25f8fea1b851d54 Mon Sep 17 00:00:00 2001 From: ddeyo Date: Fri, 31 Aug 2018 11:33:15 -0700 Subject: [PATCH 06/14] header added --- ee/ucp/admin/configure/create-audit-logs.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index 73aa00acd3..fe4c6aa3b1 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -7,10 +7,8 @@ keywords: logs, ucp, swarm, kubernetes > BETA DISCLAIMER > > This is beta content. It is not yet complete and should be considered a work in progress. This content is subject to change without notice. -{: .important} - -Audit logs are a type of event that are more focused on external user/agent actions and security than understanding state or events of the system itself. They are a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators or other components of the system. +Audit logs are focused on external user/agent actions and security than understanding state or events of the system itself. They are a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators or other components of the system. Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP API, Swarm API and Kubernetes API endpoints that are invoked (except for the ignored list) and sent to Docker Engine via stdout. zCreating audit logs is mainly CLI driven and is an UCP component that integrates with Swarm, K8s, and UCP APIs. From aa2e52ba9e5697344708f39ded1628a5c6dc4869 Mon Sep 17 00:00:00 2001 From: ddeyo Date: Sun, 2 Sep 2018 17:49:54 -0700 Subject: [PATCH 07/14] added use cases from the PRD --- ee/ucp/admin/configure/create-audit-logs.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index fe4c6aa3b1..bbd4e56424 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -12,6 +12,15 @@ Audit logs are focused on external user/agent actions and security than understa Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP API, Swarm API and Kubernetes API endpoints that are invoked (except for the ignored list) and sent to Docker Engine via stdout. zCreating audit logs is mainly CLI driven and is an UCP component that integrates with Swarm, K8s, and UCP APIs. +## Benefits + +You can use audit logs to help with the following use cases: + +- **Historical Troubleshooting** - Audit logs are helpful in determining a sequence of past events that explain why an issue occured. +- **Security Analysis and Auditing** - Security is one of the primary uses for audit logs. A full record of all user interactions with the container infrastructure gives your security team full visibility to questionable or unauthorized accesses. +- **Chargeback** - You can use audit logs and information about the resources to generate chargeback information. +- **Alerting** - If there is a watch on an event stream or a notification created by the event, alerting features can be built on top of event tools that generate alerts for ops teams (PagerDuty, OpsGenie, Slack, or custom solutions). + ## Procedure 1. Download the UCP Client bundle [Download client bundle from the command line] (https://success.docker.com/article/download-client-bundle-from-the-cli). From f77a2ab23f57467a0a7b13e466ffb2099b60165a Mon Sep 17 00:00:00 2001 From: ddeyo Date: Sun, 2 Sep 2018 17:51:26 -0700 Subject: [PATCH 08/14] fixed headings --- ee/ucp/admin/configure/create-audit-logs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index bbd4e56424..fed1af61f0 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -45,7 +45,7 @@ curl --cert ${DOCKER_CERT_PATH}/cert.pem --key ${DOCKER_CERT_PATH}/key.pem --cac 6. Optionally, configure the Docker Engine driver to logstash and collect and query audit logs within ELK stack after deploying ELK. https://success.docker.com/article/elasticsearch-logstash-kibana-logging -### API endpoints ignored +## API endpoints ignored The following API endpoints are ignored since they are not considered security events and may create a large amount of log entries. From ea0ac2201a23401835844abc639402fae3c36a01 Mon Sep 17 00:00:00 2001 From: ddeyo Date: Fri, 7 Sep 2018 08:42:33 -0700 Subject: [PATCH 09/14] review update --- ee/ucp/admin/configure/create-audit-logs.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index fed1af61f0..93db2456c2 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -12,6 +12,19 @@ Audit logs are focused on external user/agent actions and security than understa Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP API, Swarm API and Kubernetes API endpoints that are invoked (except for the ignored list) and sent to Docker Engine via stdout. zCreating audit logs is mainly CLI driven and is an UCP component that integrates with Swarm, K8s, and UCP APIs. +To allow more control to administrators over the audit Logging, three audit logging levels are provided: + +- None: audit logging is disabled +- Metadata: includes the following: + - Method and API endpoint for the request + - UCP user which made the request + - Response Status (success/failure) + - Timestamp of the call + - Object ID of any created or updated resource (for create/update calls). We are not including names of created/updated resources unless more customers are requesting this feature. + - License Key + - Remote Address +- Request: includes all fields from the Metadata level as well as the request payload. + ## Benefits You can use audit logs to help with the following use cases: From 96ac8bef9b5bc256d0aec2d6f8eac0cec8535094 Mon Sep 17 00:00:00 2001 From: ddeyo Date: Fri, 7 Sep 2018 08:45:16 -0700 Subject: [PATCH 10/14] review edits --- ee/ucp/admin/configure/create-audit-logs.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index 93db2456c2..feec1694bf 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -12,6 +12,8 @@ Audit logs are focused on external user/agent actions and security than understa Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP API, Swarm API and Kubernetes API endpoints that are invoked (except for the ignored list) and sent to Docker Engine via stdout. zCreating audit logs is mainly CLI driven and is an UCP component that integrates with Swarm, K8s, and UCP APIs. +## Logging levels + To allow more control to administrators over the audit Logging, three audit logging levels are provided: - None: audit logging is disabled From df479a6f4ad20db59914cf3543b0c81b805533b2 Mon Sep 17 00:00:00 2001 From: ddeyo Date: Sat, 8 Sep 2018 09:45:12 -0700 Subject: [PATCH 11/14] Review changes --- ee/ucp/admin/configure/create-audit-logs.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index feec1694bf..ded5a6ee1c 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -10,7 +10,7 @@ keywords: logs, ucp, swarm, kubernetes Audit logs are focused on external user/agent actions and security than understanding state or events of the system itself. They are a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators or other components of the system. -Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP API, Swarm API and Kubernetes API endpoints that are invoked (except for the ignored list) and sent to Docker Engine via stdout. zCreating audit logs is mainly CLI driven and is an UCP component that integrates with Swarm, K8s, and UCP APIs. +Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP API, Swarm API and Kubernetes API endpoints that are invoked (except for the ignored list) and sent to Docker Engine via stdout. Creating audit logs is CLI driven and is an UCP component that integrates with Swarm, K8s, and UCP APIs. ## Logging levels @@ -49,14 +49,14 @@ curl --cert ${DOCKER_CERT_PATH}/cert.pem --key ${DOCKER_CERT_PATH}/key.pem --cac ``` vi auditlog.json - {"logLevel":"INFO","auditLevel":"metadata","supportDumpIncludeAuditLogs":false} - ``` -4. Send the JSON request for the auditlog config with the same API path but with the `PUT` method +{"logLevel":"INFO","auditLevel":"metadata","supportDumpIncludeAuditLogs":false} +``` +4. Send the JSON request for the auditlog config with the same API path but with the `PUT` method. ``` curl --cert ${DOCKER_CERT_PATH}/cert.pem --key ${DOCKER_CERT_PATH}/key.pem --cacert ${DOCKER_CERT_PATH}/ca.pem -k -H "Content-Type: application/json" -X PUT --data $(cat auditlog.json) https://ucp-domain/api/ucp/config/logging ``` -5. Create any workload or RBAC grants in Kube and generate a support dump to check the contents of ucp-controller.log file for audit log entries. +5. Create any workload or RBAC grants in Kubernetes and generate a support dump to check the contents of ucp-controller.log file for audit log entries. 6. Optionally, configure the Docker Engine driver to logstash and collect and query audit logs within ELK stack after deploying ELK. https://success.docker.com/article/elasticsearch-logstash-kibana-logging From d036a3b4cfb12b9025a459298d71d28eb72317c6 Mon Sep 17 00:00:00 2001 From: ddeyo Date: Sat, 8 Sep 2018 10:06:00 -0700 Subject: [PATCH 12/14] copy edits --- ee/ucp/admin/configure/create-audit-logs.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index ded5a6ee1c..42d26f759f 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -19,10 +19,10 @@ To allow more control to administrators over the audit Logging, three audit logg - None: audit logging is disabled - Metadata: includes the following: - Method and API endpoint for the request - - UCP user which made the request - - Response Status (success/failure) + - UCP user who made the request + - Response Status (success or failure) - Timestamp of the call - - Object ID of any created or updated resource (for create/update calls). We are not including names of created/updated resources unless more customers are requesting this feature. + - Object ID of any created or updated resource (for create or update API calls). We do not include names of created or updated resources - License Key - Remote Address - Request: includes all fields from the Metadata level as well as the request payload. From 17aed965ade06dd3a3f1ec4dcfe6583ac4408cce Mon Sep 17 00:00:00 2001 From: ddeyo Date: Sat, 8 Sep 2018 10:30:41 -0700 Subject: [PATCH 13/14] beta disclaimer --- ee/ucp/admin/configure/create-audit-logs.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index 42d26f759f..92cf55a946 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -1,10 +1,10 @@ --- title: Create UCP audit logs description: Learn how to create audit logs of all activity in UCP -keywords: logs, ucp, swarm, kubernetes +keywords: logs, ucp, swarm, kubernetes, audits --- -> BETA DISCLAIMER +> Beta disclaimer > > This is beta content. It is not yet complete and should be considered a work in progress. This content is subject to change without notice. From c5a1d0b7a84483022ffd3af72121dbc0bee7dac9 Mon Sep 17 00:00:00 2001 From: ddeyo Date: Mon, 10 Sep 2018 14:41:55 -0700 Subject: [PATCH 14/14] @jin edits incorped --- ee/ucp/admin/configure/create-audit-logs.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ee/ucp/admin/configure/create-audit-logs.md b/ee/ucp/admin/configure/create-audit-logs.md index 92cf55a946..f3cf53cda9 100644 --- a/ee/ucp/admin/configure/create-audit-logs.md +++ b/ee/ucp/admin/configure/create-audit-logs.md @@ -8,7 +8,7 @@ keywords: logs, ucp, swarm, kubernetes, audits > > This is beta content. It is not yet complete and should be considered a work in progress. This content is subject to change without notice. -Audit logs are focused on external user/agent actions and security than understanding state or events of the system itself. They are a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators or other components of the system. +Audit logs are focused on external user/agent actions and security rather than understanding state or events of the system itself. They are a chronological record of security-relevant activities by individual users, administrators or software components that have affected the system. Audit Logs capture all HTTP actions (GET, PUT, POST, PATCH, DELETE) to all UCP API, Swarm API and Kubernetes API endpoints that are invoked (except for the ignored list) and sent to Docker Engine via stdout. Creating audit logs is CLI driven and is an UCP component that integrates with Swarm, K8s, and UCP APIs. @@ -32,7 +32,7 @@ To allow more control to administrators over the audit Logging, three audit logg You can use audit logs to help with the following use cases: - **Historical Troubleshooting** - Audit logs are helpful in determining a sequence of past events that explain why an issue occured. -- **Security Analysis and Auditing** - Security is one of the primary uses for audit logs. A full record of all user interactions with the container infrastructure gives your security team full visibility to questionable or unauthorized accesses. +- **Security Analysis and Auditing** - Security is one of the primary uses for audit logs. A full record of all user interactions with the container infrastructure gives your security team full visibility to questionable or attempted unauthorized accesses. - **Chargeback** - You can use audit logs and information about the resources to generate chargeback information. - **Alerting** - If there is a watch on an event stream or a notification created by the event, alerting features can be built on top of event tools that generate alerts for ops teams (PagerDuty, OpsGenie, Slack, or custom solutions).