Flesh out sig-governance diff, sketch change process

sig-testing/charter.md

@@ -1,6 +1,6 @@
 # SIG Testing Charter
 
-This charter adheres to the conventions described in the 
+This charter adheres to the conventions described in the
 [Kubernetes Charter README] and uses the Roles and Organization Management
 outlined in [sig-governance].
 
@@ -9,10 +9,10 @@ outlined in [sig-governance].
 SIG Testing is interested in effective testing of Kubernetes. We focus on
 creating and running tools and infrastructure that make it easier for the
 community to write and run tests, and to contribute, analyze and act upon
-test results. 
+test results.
 
-We are not responsible for writing, fixing, nor actively troubleshooting the 
+We are not responsible for writing, fixing, nor actively troubleshooting the
-project's tests, as this is the responsiblity of the respective test, feature,
+project's tests, as this is the responsibility of the respective test, feature,
 and subproject owners. We will however act as an escalation point of last
 resort for remediation if it is clear that misbehaving tests are harming the
 immediate health of the project.
@@ -24,18 +24,19 @@ immediate health of the project.
 - Project CI and merge automation via tools such as [prow] and [tide]
 - Infrastructure to support running project CI at scale, including tools
   such as [boskos], [ghproxy] and [greenhouse]
-- Extraction of test results from GCS and populating a public accessible
+- Providing a place and schema in which to upload test results for
-  BigQuery dataset via [kettle]
+  contributors who wish to provide additional test results not generated
-- Display and analysis of test artifacts via tools like [gubernator], 
+  by the project's CI
-  [testgrid], [triage] and [velodrome]
+- Extraction, display and analysis of test artifacts via tools like
+  [gubernator], [kettle], [testgrid], [triage] and [velodrome]
 - Configuration management of jobs and ensuring they use a consistent
   process via tools such as [job configs], [kubetest]
-- Tools that facilitate local testing of kubernetes such as [greenhouse] 
+- Tools that facilitate local testing of kubernetes such as [greenhouse]
   and [kind]
 - Jobs that automate away project toil via [@fejta-bot]
 - Ensuring all of the above is kept running on a best effort basis
 - Tools, frameworks and libraries that make it possible to write tests against
-  kubernetes such as e2e\* or integration test frameworks. 
+  kubernetes such as e2e\* or integration test frameworks.
 
   \* Note that while we are the current de facto owners of the kubernetes e2e
   test framework, we are not staffed to actively maintain or rewrite it and
@@ -43,42 +44,73 @@ immediate health of the project.
 
 #### Cross-cutting and Externally Facing Processes
 
+**Ongoing Support**
+
 - The [Release Team test-infra role] is staffed by a member of SIG Testing, as
   such their responsibilities are within the scope of this SIG, including
   the maintenance of release jobs
-- When rolling out changes that may potentially impact the project as a whole
-  we consult with SIG Contributor Experience, and follow [lazy consensus] by 
-  notifying kubernetes-dev, providing a deadline, and a rationale for the 
-  deadline if necessary.
 - We reserve the right to halt automation and infrastructure that we own,
-  or disable tests that we don't own if the project as a whole is being 
+  or disable tests that we don't own if the project as a whole is being
   impacted
 - We are actively assisting with the transition of project infrastructure to
   the CNCF and enabling non-Googlers to support this
 
+**Deploying Changes**
+
+We aspire to remain agile and deploy quickly, while ensuring a disruption-free
+experience for project contributors. As such, the amount of notice we provide
+and the amount of consensus we seek is driven by our estimation of risk. We
+don't currently define risk in terms of objective metrics, so here is a rough
+description of the guidelines we follow. We anticipate refining these over
+time.
+
+- **Low risk** changes do not break existing contributor workflows, are easy
+  to roll back, and impact at most a few project repos or SIGs. These should
+  be reviewed by another member of SIG Testing or the affected SIG(s),
+  preferably an approver.
+
+- **Medium risk** changes may impact existing contributor workflows, should be
+  easy to roll back, and may impact all of the project's repos. These should
+  be shared with SIG Contributor Experience, may require a lazy consensus
+  issue with [kubernetes-dev@] notice.
+
+- **High risk changes** likely break existing contributor workflows, may be
+  difficult to roll back, and likely impact all of the project's repos. These
+  require a consultation with SIG Contributor Experience, and a lazy consensus
+  issue with [kubernetes-dev@] notice.
+
 ### Out of scope
 
-- We are not resonpsible for troubleshooting or writing tests or jobs for 
+- We are not resonpsible for troubleshooting or writing tests or jobs for
-  features or subprojects owned by other SIGs.
+  features or subprojects owned by other SIGs
 
 ## Roles and Organization Management
 
-This sig adheres to the Roles and Organization Management outlined in 
+This sig adheres to the Roles and Organization Management outlined in
 [sig-governance] and opts-in to updates and modifications to [sig-governance].
 
 ### Deviations from [sig-governance]
 
 - Chairs also fulfill the role of Tech Lead
+- Proposing and making decisions _MAY_ be done without the use of KEPS so long
+  as the decision is documented in a linkable medium. We prefer to use issues
+  on [kubernetes/test-infra] to document technical decisions, and mailing list
+  threads on [kubernetes-sig-testing@] to document administrative decisions on
+  leadership, meetings and subprojects.
+- We do not consistently review sig-testing testgrid dashboards as part of our
+  meetings
 
 ### Subproject Creation
 
-Subprojects are created by Tech Leads as defined in the [sig-governance]
+Subprojects are created by Tech Leads following the process defined in [sig-governance]
 
 
 [sig-governance]: https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance.md
 [Kubernetes Charter README]: https://github.com/kubernetes/community/blob/master/committee-steering/governance/README.md
+[lazy consensus]: http://en.osswiki.info/concepts/lazy_consensus
 
 [@fejta-bot]: https://github.com/fejta-bot
+[boskos]: https://git.k8s.io/test-infra/boskos
 [ghproxy]: https://git.k8s.io/test-infra/ghproxy
 [greenhouse]: https://git.k8s.io/test-infra/greenhouse
 [gubernator]: http://k8s-gubernator.appspot.com
@@ -93,5 +125,7 @@ Subprojects are created by Tech Leads as defined in the [sig-governance]
 [triage]: https://go.k8s.io/triage
 [velodrome]: https://velodrome.k8s.io
 
-[lazy consensus]: https://rave.apache.org/docs/governance/lazyConsensus.html
 [Release Team test-infra role]: https://git.k8s.io/sig-release/release-team/role-handbooks/test-infra
+[kubernetes-dev@]: https://groups.google.com/forum/#!forum/kubernetes-dev
+[kubernetes-sig-testing@]: https://groups.google.com/forum/#!forum/kubernetes-sig-testing
+