cncf
/
techdocs
mirror of https://github.com/cncf/techdocs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #229 from dwelsch-esi/analysis-howto 

Revised TechDoc Analysis Tools
This commit is contained in:
Nate W 2024-05-31 09:16:25 -07:00 committed by GitHub
parent 917642768e edebb31f76
commit 81199a22ff
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
14 changed files with 844 additions and 183 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.prettierignore
View File

@ -6,3 +6,5 @@
assessments
docs
/README.md
assistance.md

37
README.md
View File

@ -1,20 +1,29 @@
# CNCF TechDocs office hours
# CNCF TechDocs
This repository holds team information and meeting notes for tech docs office hours.
This repository holds resources provided by the CNCF Technical Documentation team. The repo contains the following directories:
- `analysis` contains instructions, templates, and criteria for requesting and performing an analysis of an OSS project's website and technical documentation. Completed analyses are stored here as well.
- `resources` contains information that OSS teams can use to set up a documentation project as suggested by the TechDocs team.
## CNCF TechDocs team
GitHub ID | Role
---|---
[@chalin](https://github.com/chalin) | Senior technical writer
[@nate-double-u](https://github.com/nate-double-u) | Developer advocate & technical writer
[@thisisobate](https://github.com/thisisobate) | Developer advocate
The full-time staff of the CNCF Tech Docs team is:
| GitHub ID | Role |
|----------------------------------------------------|---------------------------------------|
| [@chalin](https://github.com/chalin) | Senior technical writer |
| [@nate-double-u](https://github.com/nate-double-u) | Developer advocate & technical writer |
| [@thisisobate](https://github.com/thisisobate) | Developer advocate |
<!-- cSpell:ignore chalin nate thisisobate -->
Various consultants and volunteers also contribute to CNCF Tech Docs projects.
## Office hours
The CNCF tech docs team generally holds office hours on the [fourth Wednesday of every month at 8am Pacific time](https://tockify.com/cncf.public.events/monthly?search=CNCF%20Tech%20Writers%20Office%20Hours).
The CNCF tech docs team holds office hours on the [fourth Wednesday of every month at 8am Pacific time](https://tockify.com/cncf.public.events/monthly?search=CNCF%20Tech%20Writers%20Office%20Hours).
Office hours started on 30 September, 2020.
Office hours started on 30 September 2020.
### Meeting link
@ -24,4 +33,12 @@ Zoom link: https://zoom-lfx.platform.linuxfoundation.org/meeting/95471930872?pas
We store ongoing meeting notes in a [Google doc](https://docs.google.com/document/d/1roexHTLCrErYjNT2NEoRsVnn_YNbQzZ1gyXNK8hXR4Q/).
<!-- cSpell:ignore chalin nate thisisobate -->
## Assistance program for technical documentation
The TechDocs team can assist CNCF projects analyze and improve their documentation. For details, see the TechDocs [assistance-program][].
[assistance-program]: ./assistance.md
### Resources
The `docs/` directory contains collected resources for building websites and developing documentation, including recommended tools and practices, how-tos, and evaluation checklists.

34
assessments/README.md Normal file
View File

@ -0,0 +1,34 @@
# CNCF TechDocs Analysis for OSS Projects
## Purpose
The goals of a CNCF technical documentation analysis are to:
- Examine the current project technical documentation and website against the CNCF's analysis framework, as described in the doc analysis [criteria](../docs/analysis/criteria.md).
- Compare the documentation against the current or proposed maturity level for the overall project.
- Recommends a program of key improvements with the largest return on investment. These improvements are documented as *recommendations* in the analysis document and expanded in a companion [implementation plan](../docs/analysis/resources/implementation-template.md) and [issues backlog](../docs/analysis/resources/umbrella-issue-template.md).
## Audience
Analyses are written for the purpose of improving a project's documentation and are available for anyone to read. Among the intended benefits to project stakeholders are these:
- **Project maintainers** can gain a critical overview of the technical documentation to plan work on the project's documentation. This work can increase the effectiveness of the project software, speed adoption, and improve user satisfaction.
- **Project contributors** can take on the recommended backlog issues to improve the documentation.
The analyses also provide information of value to organizations with an interest in promoting open source software:
- **CNCF Foundation members** can see what benefits can (and cannot) be expected of a documentation improvement effort.
- **Members of other open-source software foundations** can use these analyses as a model for their own documentation improvement processes. (Please contact the Cloud Native Computing Foundation to discuss licensing and permission of analysis templates and tools.)
## Contents
This directory contains completed analyses of the technical documentation for selected CNCF incubating and graduated software projects.
The analyses are in one of two formats depending on when they were written. Earlier analyses (**0001** - **0007**) are Markdown files, each of which is the sole artifact of the analysis.
Subsequent analyses (**0008-** forward) each has its own directory containing three analysis artifacts:
- `projectname-analysis.md` evaluates the project documentation and provides comments and recommendations in a manner very similar to the Round 1 tech doc assessments. This document is based on the analysis template and accompanying criteria developed for the Round 1.
- `projectname-implementation.md` provides concrete actions that can be implemented by project contributors. Its focus is on specific, achievable work that will have a strong positive impact on document effectiveness.
- `projectname-issues.md` is a backlog of improvement actions, meant to be entered as GitHub Issues, derived from `projectname-implementation.md`.
Each directory might also contain other documents, such as CSV-formatted surveys of documentation pages.

51
assessments/howto.md
View File

@ -1,51 +0,0 @@
# CNCF Project documentation assessments
**Who this document is for:** Members of the CNCF Techdocs team. This document provides guidance and a template on executing documentation assessments for CNCF projects.
## Purpose
The aim of a documentation assessment is to:
- Measure against the CNCF’s standards for documentation
- Recommend areas to improve
- Provide examples of great documentation as reference
- Identify key areas which will net the largest improvement if addressed
**What an assessment is**
- An overview with specific recommendations
- As short as possible – err on the side of bulleted lists
- Geared at providing actionable feedback (litmus test: could you turn a piece of feedback into a backlog issue with little extra work?)
**What it isn’t**
- Full lists of exhaustive solutions for each and every issue with a website
- Fluffy and unspecific
- Judgmental
## Assessment criteria and examples
See [Assessment definitions and reference examples](criteria.md).
**Doing an assessment**
When doing an assessment:
1. Locate a project’s main documentation repository and any community/contributor/org/governance repositories
Review all documentation available & the website (if present).
2. Note:
- Any quick wins
- Larger, systemic issues at play
- Main issues to fix
- What the project does exceptionally well!
3. Draft the assessment using the template provided
4. Send it to the CNCF techdocs team for a review
5. Send it to the project maintainers and schedule a zoom meeting to discuss with them in person.
6. PR to the techdocs repository in this directory for archiving

112
assessments/template.md
View File

@ -1,112 +0,0 @@
# Assessment template
Prepared by: \<name> ([@add-link-to-your-github-id](https://github.com/cncf/techdocs))<br>
Date: 2021-mm-dd
## Introduction
This document assesses the quality and completeness of a project's documentation and website (if present).
This document:
- Measures existing documentation quality against the CNCF’s standards
- Recommends specific and general improvements
- Provides examples of great documentation as reference
- Identifies key improvements with the largest return on investment
## How this document works
The assessment is divided into three sections:
- **Project documentation:** for end users of the project; aimed at people who intend to use it
- **Contributor documentation:** for new and existing contributors to the project
- **Website:** branding, website structure, and maintainability
Each section rates content based on different [criteria](criteria.md).
## Project documentation
| Criteria | 1 | 2 | 3 | 4 | 5 |
| --- | --- | --- | --- | --- | --- |
| Information architecture | | | | | |
| New user content | | | | | |
| Content maintainability | | | | | |
| Content creation processes | | | | | |
Scale:
- 1 = (Is not present or requires significant work)
- 3 = (Is present, but needs work)
- 5 = (Is executed extremely well or no improvement required)
**Comments**
_Provide comments for each rating above, 1-2 sentences max, bullet point list_
**Recommendations**
_Provide a list of recommendations to improve in this area_
## Contributor documentation
| Criteria | 1 | 2 | 3 | 4 | 5 |
| --- | --- | --- | --- | --- | --- |
| Communication methods documented | | | | | |
| Beginner friendly issue backlog | | | | | |
| “New contributor” getting started content | | | | | |
| Project governance documentation | | | | | |
Scale:
- 1 = (Is not present or requires significant work)
- 3 = (Is present, but needs work)
- 5 = (Is executed extremely well or no improvement required)
**Comments**
_Provide comments for each rating above, 1-2 sentences max, bullet point list_
**Recommendations**
_Provide a list of recommendations to improve in this area_
## Website
| Criteria | 1 | 2 | 3 | 4 | 5 |
| --- | --- | --- | --- | --- | --- |
| Single-source for all files | | | | | |
| Meets min website req. (for maturity level) | | | | | |
| Branding and design | | | | | |
| Case studies/social proof | | | | | |
| Maintenance planning | | | | | |
| A11y plan & implementation | | | | | |
| Mobile-first plan & impl. | | | | | |
| HTTPS access & HTTP redirect | | | | | |
| Google Analytics 4 for production only | | | | | |
| Indexing allowed for production server only | | | | | |
| Intra-site / local search | | | | | |
| Account custodians are documented | | | | | |
Scale:
- 1 = (Is not present or requires significant work)
- 3 = (Is present, but needs work)
- 5 = (Is executed extremely well or no improvement required)
**Comments**
_Provide comments for each rating above, 1-2 sentences max, bullet point list_
_Include a list of the top 404s, as reported through analytics or a search console._
**Recommendations**
_Provide a list of recommendations to improve in this area_
## Recommendations
_From the recommendations above, lis the top 1-3 concerns for this particular project and expand on them in enough detail that you could either:_
- _Pass the work off to a contractor or other member of the CNCF techdocs team_
- _Write a GitHub issue for the project team and place it in the backlog and someone not involved in the docs assessment process could execute on it_

53
assistance.md Normal file
View File

@ -0,0 +1,53 @@
# Assistance program for technical documentation
This document outlines the Cloud Native Computing Foundation (CNCF) Technical Documentation Assistance Program (the Program), a service offered by CNCF Tech Docs for evaluating and improving an OSS project's technical documentation. The process is designed to:
1. Provide a baseline analysis of the project's documentation quality measured against the project's [maturity level](docs/analysis/criteria.md). Often, projects request an analysis in support of promotion to a new maturity level.
1. Recommend changes that will reduce the gap between the documentation and the maturity of the overall project.
1. Expand on the recommended changes in an implementation plan.
1. Break down the implementation into a documentation project backlog comprising a GitHub Issues list.
1. Support the project team's contributors in taking up and completing the issue backlog items.
1. Leave the project team with an improved understanding and skill base for improving and maintaining the project documentation.
## Phase 0: Training
Some level of familiarity with the technical documentation process is required to:
- Work effectively with technical writers
- Draft technical documentation (for non-writers)
- Get the best results out of the Assistance Program
For this reason, CNCF offers free training on documentation essentials for project contributors and maintainers. To get the most from the Assistance Program, project contributors are encouraged to do the training *before* engaging a documentation specialist to complete the documentation analysis.
The training program consists of the following online courses. Anyone can sign up for and complete the courses at their own pace.
1. [Open Source Technical Documentation Essentials (LFC111)](https://training.linuxfoundation.org/training/open-source-technical-documentation-essentials-lfc111/)
1. [Creating Effective Documentation for Developers (LFC112)](https://training.linuxfoundation.org/training/creating-effective-documentation-for-developers-lfc112/)
## Phase 1: Documentation analysis
A technical writer (on CNCF staff or on contract) analyzes the documentation. Based on the standards developed as part of the CNCF TechDocs program, the writer:
1. Estimates the maturity level of the documentation compared to the current or desired maturity level of the software project using a rubric developed by CNCF. The rubric is divided into three categories:
1. Project documentation: The end-user documentation for the project's work product, typically (but not always) an application, API, protocol, or some other software product.
1. Contributor documentation: Documentation about the project, aimed at project contributors and describing procedures, infrastructure, and customs for doing project work. This includes artifacts that define procedures and governance; recruit, orient, and train project contributors; and name responsible parties (project leaders, often generically called *maintainers*).
1. Website: The technical infrastructure behind the documentation and the project's web presence, including website generation, versioning, SEO, analytics, and security.
1. Collaborates with project leadership to identify user roles and objectives for software users.
1. Proposes changes, if necessary, to the organization and content of the documentation to close gaps with the target maturity level.
1. Writes an implementation plan describing improvements that address the gaps identified by the analysis.
## Phase 2: Backlog creation
Once a high-level improvement plan has been written and approved, the tech writer analyzes the proposed changes to create a backlog of actionable, individual writing assignments and other tasks to improve the documentation. These tasks should have two primary characteristics:
- As much as possible, they should be independent of each other so they can be completed in any order by any combination of contributors; and
- They should be time-constrained. If possible, large tasks should be broken down into smaller, independent tasks that require hours or days rather than weeks to complete.
## Phase 3: Documentation improvement
Community members work on the issues created in the previous phase. Ideally, tech writers are available to advise contributors and edit work, especially if the contributing community members are not trained technical writers. Remember that the training courses in Phase 0 are available to prepare contributors with general knowledge of the process.
We know that recruiting contributors to write documentation can be difficult. While this is largely the responsibility of the project leadership, CNCF is actively working on ways to encourage doc contributions. For example, creating a backlog of time-bounded issues is an attempt to lower barriers, both psychological and logistical, to documentation creation and maintenance.
## Phase 4: Impact analysis
Projects are encouraged to collect metrics (using Google analytics and page feedback data) on documentation usage as a means of assessing the effectiveness of documentation improvements.

10
docs/analysis/README.md Normal file
View File

@ -0,0 +1,10 @@
# TechDoc Analysis
## Contents
This directory contains instructions and criteria for completing a documentation analysis, including a [how-to][] guide and analysis [criteria][].
Templates for the analyses are in the resources directory.
[how-to]: ./howto.md
[criteria]: ./criteria.md

20
assessments/criteria.md → docs/analysis/criteria.md
View File

@ -137,10 +137,12 @@ Examples:
### Single-source requirement
Source files for _all website pages_ should reside in a _single_ repo.
Otherwise, having source files in two places will confuse contributors (who
won't know which file(s) to update) and you'll run the risk of losing updates
&mdash; [as has happened already][otel-changes-lost].
Source files for _all website pages_ should reside in a single repo.
Among other problems, keeping source files in two places:
- confuses contributors
- requires you to keep two sources in sync
- increases the likelihood of errors
- makes it more complicated to generate the documentation from source files
Ideally, all website files should be in the **website repo** itself.
Alternatively, files should be brought into the website repo via [git
@ -149,9 +151,6 @@ submodules][].
If a project chooses to keep source files in multiple repos, they need a clearly
documented strategy for managing mirrored files and new contributions.
[otel-changes-lost]: https://github.com/open-telemetry/opentelemetry.io/issues/673
[git submodules]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
### Minimal website requirements
Listed here are the _minimal_ website requirements for projects based on their
@ -162,7 +161,7 @@ maturity levels so, for example, incubating projects must satisfy the
requirements for sandbox projects.
- **Sandbox**
- [Website guidelines](../docs/website-guidelines-checklist.md): majority of the guidelines are satisfied
- [Website guidelines](../../docs/website-guidelines-checklist.md): majority of the guidelines are satisfied
- [Docs assessment][]: consider [submitting a request][service desk] for an
assessment as early as possible to avoid documentation and website rework.
- **Project documentation** may or may not be present -- it is acceptable at
@ -191,11 +190,11 @@ requirements for sandbox projects.
documentation.
[archived state]: https://docs.github.com/en/repositories/archiving-a-github-repository/archiving-repositories
[docs assessment]: /assessments/howto.md
[docs assessment]: ./howto.md
[maturity level]: https://github.com/cncf/toc/tree/main/process#ii-stages---definitions--expectations
[service desk]: https://servicedesk.cncf.io
[single-source requirement]: #single-source-requirement
[website guidelines]: /howto/website-guidelines-checklist.md
[website guidelines]: ../website-guidelines-checklist.md
### Usability, accessibility and devices
@ -220,6 +219,7 @@ Plan for suitable [accessibility][] measures for your website. For example:
It is up to each project to set their own guidelines.
[accessibility]: https://developer.mozilla.org/en-US/docs/Web/Accessibility
### Branding
CNCF seeks to support enterprise-ready open source software. A key aspect of

111
docs/analysis/howto.md Normal file
View File

@ -0,0 +1,111 @@
# CNCF TechDocs Analysis How-To
## Audience
This document is for members of the CNCF TechDocs team, including contractors or consultants, who need to conduct or assist with an analysis of a CNCF open-source project's technical documentation.
## Purpose
The goals of a CNCF technical documentation analysis are to:
- Examine the current project technical documentation and website against the CNCF's analysis framework, as described in the doc analysis [criteria](./criteria.md).
- Compare the documentation against the current or proposed maturity level for the overall project.
- Recommends a program of key improvements with the largest return on investment. These improvements are documented as *recommendations* in the analysis document and expanded in a companion [implementation plan](./resources/implementation-template.md) and [issues backlog](./resources/umbrella-issue-template.md).
## Doing a Tech Docs Analysis
The tech docs analysis consists of some repository bookkeeping (Prerequisites), then of three overall tasks:
1. Write the analysis document: Evaluate the existing project documentation with respect to the project maturity level (or proposed maturity level, if the analysis is associated with an upgrade request). Identify gaps with CNCF criteria. Write general recommendations to close the largest and most important gaps.
2. Write the implementation plan: Decompose the recommendations to specific improvement suggestions. These can be additions or revisions to the docs; reorganization; website infrastructure changes; or any other work that will close the gaps. Make suggestions specific (if you propose reorganizing a section, for example, provide an outline) but provide enough information that a contributor could solve the problem differently if they have a different idea (make it clear that your proposed outline is only one possible reorganization, e.g.).
3. Write the issue backlog.
Finally, there are follow-up steps including creating GitHub issues and a pull request, and getting approval from project maintainers.
### Prerequisites
This process assumes you have some familiarity with GitHub repositories and pull requests (PRs).
1. Clone the [CNCF tech docs repository](https://github.com/cncf/techdocs).
1. Create a branch for the analysis.
1. In the new branch, create a directory for the analysis in the CNCF tech docs /analysis directory. Name the directory `00NN-_PROJECT_`, where *NN* is the next index available in the directory (check for PRs as well, if someone else is working on tech doc analyses), and where _PROJECT_ is a short but not abbreviated project name. For example, for Kubernetes _PROJECT_ would be *kubernetes*, not *k8s*.
1. Copy and rename the analysis doc templates from the `/analysis/analysis-tools` directory as follows: `analysis-template.md` > `_PROJECT_-analysis.md`; `implementation-template.md` > `_PROJECT_-implementation.md`; and `umbrella-issue-template.md` > `_PROJECT_-issues.md`.
### Writing the Analysis document
Edit `_PROJECT_-analysis.md` and follow these steps to complete the first step, the analysis:
1. Define the scope of the analysis. Edit "Scope of analysis" to reflect URLs and repositories included and excluded from the analysis.
1. Review the in-scope URLs and repositories for compliance with the rubric criteria. Note any gaps, as well as any areas that exceed criteria or are exceptionally well executed. I find it easiest to do this separately for each of the three areas of concern (project doc, contributor doc, website), making a pass through the documentation once for each section (Information architecture, New user content, Content maintainability, etc.). Don't worry about a numerical score during this step; instead, note how the documentation complies, or not, with each criterion with respect to the project maturity level (or proposed maturity level, if the analysis is part of a petition for upgrade). Write comments to note the most important gaps and best-executed features of the documentation.
1. Assign ratings to each criterion based on your comments and compliance with the maturity level expectations in the rubric. The ratings are self-explanatory. Keep in mind that "needs improvement" or "meets standards" is with respect to the current (or proposed) maturity level.
1. Write recommendations. The template implies that you'll do this for every criterion; the "Recommendations" headings mirror the "Comments" headings. However, if some alternative framework makes more sense, use that. For example, it might be that two or three of the product documentation criteria are improved by reorganizing the documentation. In this case, rather than repeat the recommendation to reorganize in each section, write a single recommendation and explain how it improves all the areas. This is the first step in moving from the analysis to specific, actionable, time-bound backlog items.
#### General tips
Things to keep in mind while doing the analysis:
- Look for:
- Quick wins – low-effort changes that would have a major impact.
- Large, systemic issues that you can organize a number of issues around.
- The two or three most important issues that impede documentation effectiveness.
- Anything the project does exceptionally well. We can call these out as examples in other evaluations!
- Don't get bogged down in detail when writing comments. Include enough detail that you can describe how to implement a suggested solution. A sentence or two is sufficient for most issues.
- Keep in mind the overall goal of the technical documentation: to make its users more effective. Focus on issues that get in the way of that goal.
- It is not necessary to come up with a recommendation in every case, especially for elements that are satisfactory or if a recommendation would result in minimal improvement.
- Don't worry about grammar and style if they don't affect documentation effectiveness. If writing style impedes understanding, make a note; otherwise move on. An exception: Insist that tasks and procedures be written as clear, step-by-step instructions.
- Some of the criteria, especially around contributor documentation, project governance, and website infrastructure, are essentially check-boxes. These you can quickly investigate, note, and move on. Spend more time analyzing the effectiveness of the project documentation.
#### Common issues
Many common issues seem to come about because open-source software documentation is written by software developers while they are writing the software. This results in documentation that:
1. Is organized around features, not users and use cases.
2. Explains technical concepts well, including architecture and design decisions.
3. Contains complete reference information for APIs, CLIs, and configuration parameters.
4. Has missing or incomplete user-oriented "how-to" explanations and operational procedures.
You may or may not find the following issues in your analysis, but it's worth keeping them in mind:
- Ambiguity around user roles.
- Missing or unclear task-based documentation.
- Assumptions about the reader's level of knowledge.
- Organization that buries or scatters important information, especially tasks and procedures.
- Missing or unclear new-user workflows.
### Writing the implementation plan
Write the implementation plan. Edit the `_PROJECT_-implementation.md` file.
The gist of the implementation plan is to break down the recommendations in the analysis document. This is an intermediate stage between general recommendations and the issues backlog. For small projects and where the recommendations are independent and time-bound, an implementation plan might not be necessary and you can move straight to writing the backlog.
If you do write an implementation plan, start with recommendations in the analysis document. Rewrite the recommendations, making them more specific, with a suggested (but not mandatory) implementation. For example, if you recommend reorganizing the documentation, provide a suggested outline, along with an explanation of the reason for the reorg.
### Writing an issues backlog
Write an issues backlog. Edit `_PROJECT_-issues.md`.
The goal of writing an issues backlog is to offer project contributors the opportunity to make the recommended changes.
Rewrite each action in the implementation plan. If possible, break large actions into smaller issues. Each issue should be:
- Independent. As much as possible, no issue should have another issue as a prerequisite. A contributor should be able to choose an issue, resolve it, and close it without reference to any other issue.
- Time-bound. A contributor should be able to complete an issue in a reasonably short time, say a few hours or a couple of days at most.
Make the suggested solution even more specific. At this point, the issue should almost be a recipe for making the doc improvement, with the caveat that a contributor is not required to implement the solution as suggested in the issue.
## Next Steps
### Including supporting documentation
If you have supporting material that might be helpful to a contributor working on the documentation issues, include them in the directory with the other documents. For example, you might inventory the existing tech doc pages in a spreadsheet; in this case, include a CSV file of the inventory.
### Creating a pull request
If you have not created a pull request with the analysis documents, do so now. Tag project maintainers and CNCF documentation staff, and ask for comments.
### Getting contributor feedback
If you haven't met with the project's maintainers yet, do so before you create the issues in GitHub. Ideally you'd like to have a Zoom meeting with any interested parties to get feedback on the analysis and implementation plan.
### Creating GitHub issues
Enter the backlog issues from the issues document into the project documentation GitHub repository using the format in the umbrella-issues-template.md and issues-template.md files. Create one GitHub issue per backlog issue, and create an umbrella issue that contains a checklist item for each issue.

15
docs/analysis/resources/README.md Normal file
View File

@ -0,0 +1,15 @@
# TechDoc Analysis Templates
This directory contains templates for analyzing a CNCF project's documentation.
## Contents
Use the templates in this directory to perform a documentation analysis for CNCF. These materials provide:
- A relatively objective set of criteria (a "rubric") for evaluating existing documentation and website content, infrastructure, and support.
- An attempt to make the documentation analysis appropriate to the current (or proposed) maturity level for the overall project.
- Emphasis on effective documentation that serves all users associated with the project.
Resources for completing a documentation analysis, including a [how-to][] guide and analysis [criteria][], are in the `docs` directory.
[how-to]: ../howto.md
[criteria]: ../criteria.md

476
docs/analysis/resources/analysis-template.md Normal file
View File

@ -0,0 +1,476 @@
---
title: _PROJECT_ Documentation Analysis
tags: _PROJECT_
created: YYYY-MM-DD
modified: YYYY-MM-DD
author: _NAME_ (@_HANDLE_)
---
<!-- TO USE THIS TEMPLATE: Search and replace:
_PROJECT_ with the project name
YYYY-MM-DD with the creation and modification dates of the analysis document
_NAME_ with the name of the analysis author
@_HANDLE_ with the GitHub handle of the analysis author
_PROJECT-WEBSITE_ with the landing page of the project's information website
_PROJECT-DOC-URL_ with the main page of the technical documentation for the current project revision; this might be on the main website server, for example as _PROJECT-WEBSITE_/doc
_PROJECT-DOC-REPO_ with the repository where the project technical documentation is stored; this might be its own repo or a directory in the project main repo
See howto.md for a discussion of the analysis procedure.
-->
# Introduction
This document analyzes the effectiveness and completeness of the [_PROJECT_][project-website] open source software (OSS) project's documentation and website. It is funded by the CNCF Foundation as part of its overall effort to incubate, grow, and graduate open source cloud native software projects.
According to CNCF best practices guidelines, effective documentation is a prerequisite for program graduation. The documentation analysis is the first step of a CNCF process aimed at assisting projects with their documentation efforts.
## Purpose
This document was written to analyze the current state of _PROJECT_ documentation. It aims to provide project leaders with an informed understanding of potential problems in current project documentation. A second document, `_PROJECT_-impementation.md`, outlines an actionable plan for improvement. A third document, `_PROJECT_-issues.md`, enumerates a backlog of issues to be added to the project documentation repository. These issues can be taken up by contributors to improve the documentation.
This document:
- Analyzes the current _PROJECT_ technical documentation and website
- Compares existing documentation against the CNCF’s standards
- Recommends a program of key improvements with the largest return on investment
## Scope of analysis
The documentation discussed here includes the entire contents of the website, the technical documentation, and documentation for contributors and users on the _PROJECT_ GitHub repository.
The _PROJECT_ website and documentation are written in [Markdown, ReStructured Text, other] and are compiled using the [Hugo, Docusaurus, Sphynx, other] static site generator with the [Docsy, other] theme and served from [the Netlify platform, other]. The site's code is stored on the _PROJECT_ GitHub repo.
**In scope:**
- Website: _PROJECT-WEBSITE_
- Documentation: _PROJECT-DOC-URL_
- Website repo: _PROJECT-DOC-REPO_
- _[Other; might include a demo server, governance site, or other relevant repositories]_
**Out of scope:**
- Other _PROJECT_ repos: _[In general, do not include sub-projects or related "ecosystem" projects]_
## How this document is organized
This document is divided into three sections that represent three major areas of concern:
- **Project documentation:** concerns documentation for users of the _PROJECT_ software, aimed at people who intend to use the project software
- **Contributor documentation:** concerns documentation for new and existing contributors to the _PROJECT_ OSS project
- **Website:** concerns the mechanics of publishing the documentation, and includes branding, website structure, and maintainability
Each section begins with summary ratings based on a rubric with appropriate [criteria][criteria-doc] for the section, then proceeds to:
- **Comments**: observations about the existing documentation, with a focus on how it does or does not help _PROJECT_ users achieve their goals.
- **Recommendations**: suggested changes that would improve the effectiveness of the documentation.
An accompanying document, [`_PROJECT_-implementation.md`][implementation-doc], breaks the recommendations down into concrete actions that can be implemented by project contributors. Its focus is on drilling down to specific, achievable work that can be completed in constrained blocks of time. Ultimately, the implementation items are decomposed into a series of [issues][issues-doc] and entered as GitHub [issues][project-doc-website]/issues.
## How to use this document
Readers interested only in actionable improvements should skip this document and read the [implementation plan][implementation-doc] and [issues list][issues-doc].
Readers interested in the current state of the documentation and the reasoning behind the recommendations should read the section of this document pertaining to their area of concern:
- [Project documentation][project-heading]
- [Contributor documentation][contributor-heading]
- [Website and documentation infrastructure][website-heading]
Examples of CNCF documentation that demonstrate the analysis criteria are linked from the [criteria][criteria-doc] specification.
### Recommendations, requirements, and best practices
This analysis measures documentation against CNCF project maturity standards, and suggests possible improvements. In most cases there is more than one way to do things. Few recommendations here are meant to be prescriptive. Rather, the recommended implementations represent the reviewers' experience with how to apply documentation best practices. In other words, borrowing terminology from the lexicon of [RFCs][rfc-spec], the changes described here should be understood as "recommended" or "should" at the strongest, and "optional" or "may" in many cases. Any "must" or "required" actions are clearly denoted as such, and pertain to legal requirements such as copyright and licensing issues.
# Project documentation
<!-- Pick the CNCF maturity level of the project: -->
_PROJECT_ is a **graduated** project of CNCF. This means that the project should have [*very high*][criteria-doc] standards for documentation.
<!-- or -->
_PROJECT_ is an **incubating** project of CNCF. This means that the project should be [*developing*][criteria-doc] professional-quality documentation alongside the project code.
| Criterion | Rating (1-5) |
|----------------------------|----------------|
| Information architecture | (rating value) |
| New user content | (rating value) |
| Content maintainability | (rating value) |
| Content creation processes | (rating value) |
| Inclusive language | (rating value) |
<!-- Rating values:
1 - not present
2 - needs improvement
3 - meets standards
4 - meets or exceeds standards
5 - exemplary
-->
## Comments
<!--
Make any overall comments about the Project Documentation here.
-->
The following sections contain brief assessments of each element of the Project Documentation rubric.
<!-- For each heading below, discuss how well the in-scope items, and especially the technical documentation, meet these criteria. (Criteria are copied from criteria.md) -->
### Information architecture
The overall structure (pages/subpages/sections/subsections) of your project documentation. We evaluate on the following:
* Is there high level conceptual/“About” content?
Is the documentation feature complete? (i.e., each product feature is documented)
* Are there step-by-step instructions (tasks, tutorials) documented for features?
* Are there any key features which are documented but missing task documentation?
* Is the “happy path”/most common use case documented?
Does task and tutorial content demonstrate atomicity and isolation of concerns? (Are tasks clearly named according to user goals?)
* If the documentation does not suffice, is there a clear escalation path for users needing more help? (FAQ, Troubleshooting)
* If the product exposes an API, is there a complete reference?
* Is content up to date and accurate?
### New user content
New users are the most avid users of documentation, and need content specifically for them. We evaluate on the following:
* Is “getting started” clearly labeled? (“Getting started”, “Installation”, “First steps”, etc.)
* Is installation documented step-by-step?
* If needed, are multiple OSes documented?
* Do users know where to go after reading the getting started guide?
* Is your new user content clearly signposted on your site’s homepage or at the top of your information architecture?
* Is there easily copy-pastable sample code or other example content?
### Content maintainability & site mechanics
As a project scales, concerns like localized (translated) content and versioning become large maintenance burdens, particularly if you don’t plan for them.
We evaluate on the following:
* Is your documentation searchable?
* Are you planning for localization/internationalization with regards to site directory structure? Is a localization framework present?
* Do you have a clearly documented method for versioning your content?
### Content creation processes
Documentation is only as useful as it is accurate and well-maintained, and requires the same kind of review and approval processes as code.
We evaluate on the following:
* Is there a clearly documented (ongoing) contribution process for documentation?
* Does your code release process account for documentation creation & updates?
* Who reviews and approves documentation pull requests?
* Does the website have a clear owner/maintainer?
### Inclusive language
Creating inclusive project communities is a key goal for all CNCF projects.
We evaluate on the following:
* Are there any customer-facing utilities, endpoints, class names, or feature names that use non-recommended words as documented by the [Inclusive Naming Initiative](https://inclusivenaming.org) website?
* Does the project use language like "simple", "easy", etc.?
## Recommendations
<!-- Write general recommendations based on the comments from the previous section. -->
### Information architecture
### New user content
### Content maintainability & site mechanics
### Content creation processes
### Inclusive language
# Contributor documentation
<!-- Pick the CNCF maturity level of the project: -->
_PROJECT_ is a **graduated** project of CNCF. This means that the project should have [*very high*][criteria-doc] standards for documentation.
<!-- or -->
_PROJECT_ is an **incubating** project of CNCF. This means that the project should be [*developing*][criteria-doc] professional-quality documentation alongside the project code.
| Criterion | Rating (1-5) |
|-------------------------------------------|----------------|
| Communication methods documented | (rating value) |
| Beginner friendly issue backlog | (rating value) |
| “New contributor” getting started content | (rating value) |
| Project governance documentation | (rating value) |
<!-- Rating values:
1 - not present
2 - needs improvement
3 - meets standards
4 - meets or exceeds standards
5 - exemplary
-->
## Comments
<!--
Make any overall comments about the Contributor Documentation here.
-->
The following sections contain brief assessments of each element of the Contributor Documentation rubric.
<!-- For each heading below, discuss how well the in-scope items meet these criteria. Keep in mind that much of the contributor documentation might be contained in the documentation repository. (Criteria are copied from criteria.md) -->
### Communication methods documented
One of the easiest ways to attract new contributors is making sure they know how to reach you.
We evaluate on the following:
* Is there a Slack/Discord/Discourse/etc. community and is it prominently linked from your website?
* Is there a direct link to your GitHub organization/repository?
* Are weekly/monthly project meetings documented? Is it clear how someone can join those meetings?
* Are mailing lists documented?
### Beginner friendly issue backlog
We evaluate on the following:
* Are docs issues well-triaged?
* Is there a clearly marked way for new contributors to make code or documentation contributions (i.e. a “good first issue” label)?
* Are issues well-documented (i.e., more than just a title)?
* Are issues maintained for staleness?
### New contributor getting started content
Open source is complex and projects have many processes to manage that. Are processes easy to understand and written down so that new contributors can jump in easily?
We evaluate on the following:
* Do you have a community repository or section on your website?
* Is there a document specifically for new contributors/your first contribution?
* Do new users know where to get help?
### Project governance documentation
One of the CNCF’s core project values is open governance.
We evaluate on the following:
* Is project governance clearly documented?
## Recommendations
<!-- Write general recommendations based on the comments from the previous section. -->
### Communication methods documented
### Beginner friendly issue backlog
### New contributor getting started content
### Project governance documentation
# Website and infrastructure
<!-- Pick the CNCF maturity level of the project: -->
_PROJECT_ is a **graduated** project of CNCF. This means that the project should have [*very high*][criteria-doc] standards for documentation.
<!-- or -->
_PROJECT_ is an **incubating** project of CNCF. This means that the project should be [*developing*][criteria-doc] professional-quality documentation alongside the project code.
| Criterion | Rating (1-5) |
|---------------------------------------------|----------------|
| Single-source for all files | (rating value) |
| Meets min website req. (for maturity level) | (rating value) |
| Usability, accessibility, and design | (rating value) |
| Branding and design | (rating value) |
| Case studies/social proof | (rating value) |
| SEO, Analytics, and site-local search | (rating value) |
| Maintenance planning | (rating value) |
| A11y plan & implementation | (rating value) |
| Mobile-first plan & impl. | (rating value) |
| HTTPS access & HTTP redirect | (rating value) |
| Google Analytics 4 for production only | (rating value) |
| Indexing allowed for production server only | (rating value) |
| Intra-site / local search | (rating value) |
| Account custodians are documented | (rating value) |
<!-- Rating values:
1 - not present
2 - needs improvement
3 - meets standards
4 - meets or exceeds standards
5 - exemplary
-->
## Comments
<!--
Make any overall comments about the Website and documentation infrastructure here.
-->
The following sections contain brief assessments of each element of the Website and documentation infrastructure rubric.
<!-- For each heading below, discuss how well the in-scope items meet these criteria. Keep in mind that much of the website infrastructure criteria depend on the tools (static site generator, website framework and hosting, analytics tools, etc.) and processes (project CI, release procedures, goverance, etc.) used to produce the documentation. (Criteria are copied from criteria.md) -->
### Single-source requirement
Source files for _all website pages_ should reside in a single repo.
Among other problems, keeping source files in two places:
- confuses contributors
- requires you to keep two sources in sync
- increases the likelihood of errors
- makes it more complicated to generate the documentation from source files
Ideally, all website files should be in the **website repo** itself.
Alternatively, files should be brought into the website repo via [git
submodules][git-submodules].
If a project chooses to keep source files in multiple repos, they need a clearly
documented strategy for managing mirrored files and new contributions.
### Minimal website requirements
Listed here are the minimal website requirements for projects based on their [maturity level][maturity-level], either incubating or graduated. (These are the only two levels for which a tech docs analysis can be requested.)
| Criterion | Incubating Requirement | Graduated Requirement |
|------------------------------------------|---------------------------------------------------------|-------------------------------------------|
| [Website guidelines][website-guidelines] | All guidelines satisfied | All guidelines satisfied |
| [Docs analysis][analysis-doc] (this) | Requested through CNCF [service desk][cncf-servicedesk] | All follow-up actions addressed |
| **Project doc**: stakeholders | Roles identified and doc needs documented | All stakeholder need identified |
| **Project doc**: hosting | Hosted directly | Hosted directly |
| **Project doc**: user docs | Comprehensive, addressing most stakeholder needs | Fully addresses needs of key stakeholders |
[git-submodules]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
[website-guidelines]: ../../website-guidelines-checklist.md
[maturity-level]: https://github.com/cncf/toc/tree/main/process#ii-stages---definitions--expectations
[cncf-servicedesk]: https://servicedesk.cncf.io
### Usability, accessibility and devices
Most CNCF websites are accessed from mobile and other non-desktop devices at least 10-20% of the time. Planning for this early in your website's design will be much less effort than retrofitting a desktop-first design.
* Is the website usable from mobile?
* Are doc pages readable?
* Are all / most website features accessible from mobile -- such as the top-nav,
site search and in-page table of contents?
* Might a [mobile-first] design make sense for your project?
[mobile-first]: https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Responsive/Mobile_first
Plan for suitable [accessibility][] measures for your website. For example:
* Are color contrasts significant enough for color-impaired readers?
* Are most website features usable using a keyboard only?
* Does text-to-speech offer listeners a good experience?
It is up to each project to set their own guidelines.
[accessibility]: https://developer.mozilla.org/en-US/docs/Web/Accessibility
### Branding and design
CNCF seeks to support enterprise-ready open source software. A key aspect of
this is branding and marketing.
We evaluate on the following:
* Is there an easily recognizable brand for the project (logo + color scheme)
clearly identifiable?
* Is the brand used across the website consistently?
* Is the website’s typography clean and well-suited for reading?
### Case studies/social proof
One of the best ways to advertise an open source project is to show other organizations using it.
We evaluate on the following:
* Are there case studies available for the project and are they documented on the website?
* Are there user testimonials available?
* Is there an active project blog?
* Are there community talks for the project and are they present on the website?
* Is there a logo wall of users/participating organizations?
### SEO, Analytics and site-local search
SEO helps users find your project and it's documentation, and analytics helps
you monitor site traffic and diagnose issues like page 404s. Intra-site search,
while optional, can offer your readers a site-focused search results.
We evaluate on the following:
* Analytics:
- Is analytics enabled for the production server?
- Is analytics disabled for all other deploys?
- If your project used Google Analytics, have you migrated to GA4?
- Can Page-not-found (404) reports easily be generated from you site
analytics? Provide a sample of the site's current top-10 404s.
* Is site indexing supported for the production server, while disabled for
website previews and builds for non-default branches?
* Is local intra-site search available from the website?
* Are the current custodian(s) of the following accounts clearly documented:
analytics, Google Search Console, site-search (such as Google CSE or Algolia)
### Maintenance planning
Website maintenance is an important part of project success, especially when project maintainers aren’t web developers.
We evaluate on the following:
* Is your website tooling well supported by the community (i.e., Hugo with the
Docsy theme) or commonly used by CNCF projects (our recommended tech stack?)
* Are you actively cultivating website maintainers from within the community?
* Are site build times reasonable?
* Do site maintainers have adequate permissions?
### Other
* Is your website accessible via HTTPS?
* Does HTTP access, if any, redirect to HTTPS?
## Recommendations
<!-- Write general recommendations based on the comments from the previous section. -->
### Single-source requirement
### Minimal website requirements
### Usability, accessibility and devices
### Branding and design
### Case studies/social proof
### SEO, Analytics and site-local search
### Maintenance planning
### Other
<!--- References --->
[project-website]: _PROJECT-WEBSITE_
[project-doc-website]: _PROJECT-DOC-URL_
[criteria-doc]: ../criteria.md
[implementation-template]: ./implementation-template.md
[issues-template]: ./issue-template.md
[umbrella-template]: ./umbrella-issue-template.md
[implementation-doc]: ./_PROJECT_-implementation.md
[issues-doc]: ./_PROJECT_-issues.md
[project-heading]: #project-documentation
[contributor-heading]: #contributor-documentation
[website-heading]: #website
[rfc-spec]: https://www.rfc-editor.org/rfc/rfc2119
[website-guidelines]: ../../website-guidelines-checklist.md

35
docs/analysis/resources/implementation-template.md Normal file
View File

@ -0,0 +1,35 @@
---
title: Implementing _PROJECT_ Doc Improvements
tags: _PROJECT_
---
# Introduction
This document provides actionable suggestions for improving the _PROJECT_ technical documentation.
For an analysis and general discussion of recommendations on _PROJECT_ technical documentation, see [_PROJECT_-analysis.md][].
## Recommendations, requirements, and best practices
This analysis measures documentation against CNCF project maturity standards and suggests possible improvements. In most cases there is more than one way to do things. Few recommendations here are meant to be prescriptive. Rather, recommendations are based on documentation best practices as understood by the reviewers. The recommended implementations represent the reviewers' experience with how to apply those best practices. In other words, borrowing terminology from the lexicon of [RFCs][rfc-keywords], the changes described here should be understood as "recommended" or "should" at the strongest, and "optional" or "may" in many cases. Any "must" or "required" actions are clearly denoted as such, and pertain to legal requirements such as copyright and licensing issues.
The top-level documentation recommendations for this project are:
<!-- Provide a summary or outline of the recommendations. Depending on the analysis findings, recommended actions might be organized into two or three high-level items that contain multiple actions, or might just be a list of independent changes. For examples, see a completed implementation plan such as 0008-Backstage or 0010-etcd. -->
# High-level action 1
## Issue 1
## Issue 2
# High-level action 2
## Issue 1
## Issue 2
<!--- References --->
[rfc-keywords]: https://www.rfc-editor.org/rfc/rfc2119

40
docs/analysis/resources/issue-template.md Normal file
View File

@ -0,0 +1,40 @@
---
title: _PROJECT_ Issue
tags: _PROJECT_
---
<!-- This template provides one possible format for the individual issues filed in the Issues of a project repository. Within the CNCF tech docs repo, include all issues in one document, `_PROJECT_-issues.md`. See any completed analysis for an example. -->
# Overview
<!-- Summarize the documentation changes prescribed by this issue. -->
Audience: <!-- Provide the user role to which the issue is most applicable. -->
Type: <!-- What type of documentation topic the change affects. One of Task, Reference, or Conceptual. -->
# Context
<!-- This is boilerplate text linking back to the doc analysis. -->
This issue tracks recommended changes resulting from an analysis of the etcd documentation commissioned by CNCF. The analysis and supporting documents are here: https://github.com/cncf/techdocs/tree/main/assessments/00NN-project/
# Possible Implementation
<!-- Include a bullet list of links to pages that are affected by the change: -->
Related material in the current doc:
- https://github.com/_PROJECT_/website/tree/main/content/en/docs/v3.5/tutorials
<!-- Describe in detail a suggested way to achieve the goals of the issue. This should be specific enough to provide a contributor with a recipe for making the change; however, the contributor should feel free to solve the problem differently if they have an idea how it should be done. -->
Suggested changes: <!-- (example) -->
Use the following outline to write a procedure for each task:
- Prerequisites (bullet list of prerequisite conditions, if any)
- Procedure
1. Step 1 (keep steps discrete and atomic. Put command-line input and expected output in a code block.)
2. Step 2 ...
- Result (optional; describe output or side effects if they're notable or unexpected.)

31
docs/analysis/resources/umbrella-issue-template.md Normal file
View File

@ -0,0 +1,31 @@
---
title: _PROJECT_ Umbrella Issue
tags: _PROJECT_
---
# Overview
<!-- Provide an outline or high-level description of the recommended changes. Note any general patterns that occur throughout the documentation, such as a lack of step-by-step procedures. -->
<!-- Items might be disjoint and unrelated; that's OK. If there are high-level items that must be broken down into smaller issues, use the high-level items to organize the issues in this plan. Otherwise, list issues in order from the analysis document. This is an improvement plan, not a legal brief. -->
<!-- The following is boilerplate language to include in the umbrella issue in the repo: -->
This issue tracks recommended changes resulting from an analysis of the _PROJECT_ documentation commissioned by CNCF. The analysis and supporting documents are here: https://github.com/cncf/techdocs/tree/main/assessments/00NN-project/
The CNCF etcd documentation effort is tracked in the CNCF Tech Docs repo:
https://github.com/cncf/techdocs/issues/NNN
# Issues
This is a list of issues representing the recommended work on the _PROJECT_ website and technical documentation.
## Issue: Item 1
<!-- Summarize the documentation changes prescribed by this issue. Use enough detail to estimate the scope of the issue. Fine-grained detail can go in the issue itself. In the GitHub umbrella issue, link to the sub-issue using a Markdown checkbox as shown below. -->
- [ ] https://github.com/project/repo/issues/NNN
## Issue: Item 2
<!-- ... and so on. -->