diff --git a/assessments/0008-backstage/README.md b/assessments/0008-backstage/README.md index 9066bb1..89df73f 100644 --- a/assessments/0008-backstage/README.md +++ b/assessments/0008-backstage/README.md @@ -3,8 +3,8 @@ title: Backstage TechDocs Analysis tags: backstage --- -- [Backstage Analysis](backstage-analysis.md) -- [Backstage Implementation](backstage-implementation.md) -- [User Roles](user-roles.md) -- [Backstage Insights Summary](backstage-insights-summary.md) -- [Backstage Docs Survey](backstage-doc-survey.csv) +- [Backstage Doc Analysis](backstage-analysis.md) - Analyzes the exisitng the Backstage documentation and provides recommendations. +- [Backstage Implementation](backstage-implementation.md) - Provides detailed and actionable recommendations, intended to be worked on as GitHub issues. +- [User Roles](user-roles.md) - A discussion of Backstage stakeholders with respect to how they use documentation. Folded into the Implementation doc. Reference. +- [Backstage Insights Summary](backstage-insights-summary.md) - A summary of conclusions from Spotify's survey of Backstage adopters. Background. +- [Backstage Docs Survey](backstage-doc-survey.csv) - A spreadsheet of every page of the Backstage technical documentation at the time of the analysis. About 200 lines. diff --git a/assessments/0008-backstage/backstage-analysis.md b/assessments/0008-backstage/backstage-analysis.md index 530d537..eb53ee0 100644 --- a/assessments/0008-backstage/backstage-analysis.md +++ b/assessments/0008-backstage/backstage-analysis.md @@ -1,13 +1,11 @@ --- title: Backstage Documentation Analysis tags: backstage +created: 2023-09-01 +modified: 2023-11-28 +author: Dave Welsch (@dwelsch-esi) --- - -Prepared by: Dave Welsch ([@dwelsch-esi][dwelsch-esi-github])
-Date: 2023-09-01 -Most recent update: 11/02/2023 - # Introduction This document analyzes the effectiveness and completeness of the [Backstage][backstage-io] open source software (OSS) project's documentation and website. It is funded by the CNCF Foundation as part of its overall effort to incubate, grow, and graduate open source cloud native software projects. @@ -26,7 +24,7 @@ This document: ## Scope of analysis -The documentation discussed here includes the entire contents of the website (which contains the technical docs, as well as documentation for contributors and users on the Backstage GitHub repository. The analysis does not include Spotify's Backstage website at https://backstage.spotify.com/. +The documentation discussed here includes the entire contents of the website (which contains the technical docs), as well as documentation for contributors and users on the Backstage GitHub repository. The analysis does not include Spotify's Backstage website at https://backstage.spotify.com/. The Backstage website and documentation are written in Markdown and are compiled using the Docusaurus static site generator. The site's code is stored on the Backstage GitHub repo. @@ -49,13 +47,13 @@ This document is divided into three sections that represent three major areas of - **Project documentation:** concerns documentation for users of the Backstage software, aimed at people who intend to use it - **Contributor documentation:** concerns documentation for new and existing contributors to the Backstage OSS project -- **Website:** concerns the mechanics of publishing the documentation; includes branding, website structure, and maintainability +- **Website:** concerns the mechanics of publishing the documentation, and includes branding, website structure, and maintainability Each section begins with summary ratings based on a rubric with appropriate [criteria][cncf-doc-criteria] for the section, then proceeds to: - **Comments**: observations about the existing documentation, with a focus on how it does or does not help Backstage users achieve their goals. - **Recommendations**: suggested changes that would improve the effectiveness of the documentation. -An accompanying document, [backstage-implementation.md][implementation-doc], attempts to break the recommendations down into concrete actions that can be implemented by project contributors. Its focus is on drilling down to specific, achievable work that can be completed in constrained blocks of time. Ultimately, the implementation items should be tracked as a series of Github [issues][backstage-issues]. +An accompanying document, [backstage-implementation.md][implementation-doc], breaks the recommendations down into concrete actions that can be implemented by project contributors. Its focus is on drilling down to specific, achievable work that can be completed in constrained blocks of time. Ultimately, the implementation items should be tracked as a series of Github [issues][backstage-issues]. ## How to use this document @@ -68,6 +66,8 @@ Readers interested in the current state of the documentation and the reasoning b - [Contributor documentation][contributor-doc] - [Website and documentation infrrastructure][website] +Examples of CNCF documentation that demonstrates the analysis criteria are linked from the [Criteria][cncf-doc-criteria] specification. + ### Recommendations, requirements, and best practices @@ -139,11 +139,6 @@ The documentation seems **feature complete**. All software features are document It's not clear that the documentation is completely **up to date**, although [release notes][backstage-doc-rn] exist and are easily findable. -Examples: - -* https://prometheus.io/docs/introduction/overview/ -* https://kubernetes.io/docs/home/ - ### New user content @@ -151,11 +146,6 @@ Examples: There is **example content** available, including tutorials for a variety of tasks and a very nice [demo server][backstage-demo]. -Examples: - -* https://falco.org/docs/getting-started/ -* https://prometheus.io/docs/introduction/first_steps/ - ### Content maintainability & site mechanics @@ -163,10 +153,6 @@ The documentation is **searchable**. There does not seem to be a **localization* There does not seem to be any mechanism for **versioning** documentation content. -Examples: - -* https://github.com/kubernetes/website - ### Content creation processes @@ -176,11 +162,6 @@ These procedures are presumably included in any updates to (and subsequent **rel The website does not have a clear individual **owner or maintainer**. -Examples: - -* https://github.com/nats-io/nats-site/blob/master/MAINTAINERS.md (clearly documented maintainers) -* https://thanos.io/tip/contributing/how-to-contribute-to-docs.md/ - ### Inclusive language @@ -191,7 +172,7 @@ The website makes occasional use of words like "simple" and "easy", but avoids e ## Recommendations: project documentation -The following sections contain recommendations for improvements to the Backstage project documentation. These recommendations are for broad improvements that would greatly increase documentation effectiveness. For details, see [Implementation][implementation] +The following sections contain recommendations for improvements to the Backstage project documentation. These recommendations are for broad improvements that would greatly increase documentation effectiveness. For details, see [Implementation][implementation]. ### Clarify user roles and objectives @@ -209,13 +190,22 @@ Most of the Backstage documentation seems to have evolved from architecture and For every user role: -**Define use cases**: Define the common use cases for each role. Some typical examples: +**Define use cases** + +Define the common use cases for each role. Some typical examples: *Administrator*: Installation; deployment; configuration; maintenance; upgrades; extension (plugins); disaster recovery. *User (developer)*: Development environment setup; Adding an existing element; creating a new element; creating a template; viewing projects; searching; creating documentation; CI/CD; deploying; reverting a deployment. -**Write procedures**: For each use case, develop instructional content, including: "happy path" procedures that can be followed by anyone familiar with the product generally; examples; troubleshooting guides; and for new users, tutorials. +**Write procedures** + +For each use case, develop instructional content, including: "happy path" procedures that can be followed by anyone familiar with the product generally; examples; troubleshooting guides; and for new users, tutorials. + +**Support procedures with conceptual and reference topics** + +Much of the existing documentation can and should be re-used when possible. Existing documentation pages should be analyzed by type of content (this analysis has already been done; see the [documentation survey][doc-survey]). Pages should be rewritten so that each page serves one purpose: conceptual, task, or reference. In some cases this will mean that two pages are extracted from a single existing page. Conversely, some pages may prove to be redundant. The new, more focused pages can then be reorganized as described below. + ### Reorganize the documentation site @@ -255,9 +245,6 @@ Communication channels are clearly documented on the [Community][backstage-commu The **[GitHub][backstage-backstage] repository** is directly linked from the main menu. There are 22 related repositories listed on the [Backstage project page][backstage-github-project]. A little digging is required to suss out the purpose of some of these projects. -Examples: -https://kubernetes.io/community/ - ### Beginner friendly issue backlog @@ -275,19 +262,11 @@ The Backstage OSS **community** is visible and accessible on a [community page][ Backstage is listed in the [Clotributor][clotributor] tool. -Examples: - -* https://github.com/helm/community -* https://github.com/backstage/community ### Project governance documentation Governance is clearly documented in [GOVERNANCE.md][backstage-governance]. -Examples: - -* https://github.com/kubernetes/steering - ## Recommendations: contributor documentation @@ -353,10 +332,6 @@ The Backstage **brand** is easily recognizable through the logo and color scheme The website **typography** is easy to read. -Examples: - -* https://helm.sh/ - ### Case studies/social proof @@ -364,12 +339,6 @@ Examples: The project has an **active blog**. **Community discussions** are available on the website. -Examples: - -* https://www.fluentd.org/testimonials (user testimonials) -* https://goharbor.io/ (logo wall) -* https://blog.rook.io/ (blog) - ### SEO, Analytics and site-local search @@ -400,9 +369,6 @@ The **website tooling** (Docusaurus static site build) is well supported. I tested the instructions for using `yarn` to build the website. The **site build time** was under 30 seconds for a local build on a Mac M1. Maintainers have sufficient **permissions** to download and build the doc. Checking in the doc requires a PR and approval from a project maintainer. -Examples: - -* http://kubernetes.io ### Usability, accessibility and devices @@ -470,7 +436,7 @@ Improve compliance in these areas: [backstage-io]: https://backstage.io [backstage-microsite]: https://github.com/backstage/backstage/tree/master/microsite [clotributor]: https://clotributor.dev/ -[cncf-doc-criteria]: criteria.md +[cncf-doc-criteria]: ../criteria.md [cncf-doc-criteria]: https://github.com/cncf/techdocs/blob/main/assessments/criteria.md [cncf-docs-howto]: https://github.com/cncf/techdocs/blob/main/assessments/howto.md [cncf-maturity-stages]: https://github.com/cncf/toc/tree/main/process#ii-stages---definitions--expectations @@ -495,4 +461,4 @@ Improve compliance in these areas: [proj-doc]: #project-documentation [user-roles]: #user-roles [website-rec]: #recommendations-website -[website]: #website \ No newline at end of file +[website]: #website diff --git a/assessments/0008-backstage/backstage-implementation.md b/assessments/0008-backstage/backstage-implementation.md index 7582073..0a7ba1d 100644 --- a/assessments/0008-backstage/backstage-implementation.md +++ b/assessments/0008-backstage/backstage-implementation.md @@ -13,8 +13,6 @@ For an analysis and general discussion of recommendations on Backstage technical Notwithstanding the fact that this analysis measures documentation against CNCF project maturity standards, in most cases there is more than one way to do things. Few recommendations here are meant to be prescriptive. Rather, recommendations are based on documentation best practices as understood by the reviewers. The recommended implementations represent the reviewers' experience with how apply those best practices. In other words, borrowing terminology from the lexicon of [RFCs][rfc-keywords], the changes described here should be understood as "recommended" or "should" at the strongest, and "optional" or "may" in many cases. Any "must" or "required" actions are clearly denoted as such, and pertain to legal requirements such as copyright and licensing issues. - - The top-level documentation recommendations for this project are: - Fill gaps in instructional documentation for all stakeholder roles, including project contributors. @@ -78,15 +76,9 @@ There is already some instructional content in the Backstage documentation. In m The sections below give recommendations for the most important instructional documentation improvements to Backstage for each user role. -## Evaluator - -The website guidelines contain elements that are intended to help evaluators decide whether the product is suitable to their needs, such as a logo wall and case studies. - -Technical documentation contains little that is actually geared toward evaluators, but a good technical overview – valuable to all stakeholders – is essential in evaluating the product. - ## Administrator -The following artifacts need to be written and made findable for administrators. +The following artifacts should be written and made findable for administrators. 1. A server installation and setup guide for administrators. Provide clear, step-by-step instructions for downloading and deploying Backstage to an organization. @@ -103,7 +95,7 @@ The following artifacts need to be written and made findable for administrators. ## Developer -The following artifacts need to be written and made findable for developers. +The following artifacts should be written and made findable for developers. 1. A getting started guide for developers. Provide a clear work path that describes how to: 1. Downloead and install any necessary software components @@ -122,7 +114,7 @@ The following artifacts need to be written and made findable for developers. There are a dizzying array of issues with writing, modifying, and maintaining plugins in Backstage. This is not a detailed recipe for documenting those issues. For integrators, at a high level, a program should be undertaken to: -1. Organize integrator tasks from most basic and common (write a simple plugin; decide between backend and frontend plugin) to more complex (Integrate with external systems; use a proxy; implement authentication). +1. Organize integrator tasks from most basic and common (write a simple plugin; decide between backend and frontend plugin) to more complex (integrate with external systems; use a proxy; implement authentication). 2. Where possible, using the exisitng documentation as a starting point, write step-by-step procedures for discrete integration tasks (starting with how to write a basic plugin). 3. Organize existing reference and conceptual information (such as API references and architecture discussions) into supporting documentation, referenced from the integration tasks. @@ -148,7 +140,7 @@ Some documents are used by more than one user role. These docs are listed first | Document | Description | | --- | --- | -| Technical overview | A discussion of what the product is and what problems it solves. Ideally, the discussion starts with a summary and provides explanations of increasing depth to address to satisfy different audiences (evaluator -> developer -> contributor, e.g.). | +| Technical overview | A discussion of what the product is and what problems it solves. Ideally, the discussion starts with a summary and provides explanations of increasing depth to address different audiences (evaluator -> developer -> contributor, e.g.). | | Release notes | Release-specific information, including: new features; performance improvements; bugs and known issues; deprecated features; software dependency changes; and experimental or beta features. | | Glossary | A dictionary of product-specific terms. Also commonly includes domain- and industry-specific terms that are necessary to understanding the product. | | Knowledge base | An encyclopedic collection of related background, conceptual, and reference information that doesn't fit elsewhere in the documentation. Similar to a FAQ, but more structured, more searchable and, therefore, more useful. | @@ -175,7 +167,7 @@ Some documents are used by more than one user role. These docs are listed first | Document | Description | | --- | --- | | Getting started guide | A document that usually walks a developer through setting up a development environment (for a language or API). In the case of Backstage, this is more of an integration with their existing environment. Nonetheless, this should explain how to configure all the tools the developer needs to begin using Backstage. | -| Developer guide | Contains all tasks that the developer needs to use the Backstage app under normal circumstances: adding, modifying, and searching for products; writing documentation; using templates. | +| Developer guide | Contains all tasks that the developer needs to use the Backstage app under normal circumstances: adding products ("catalog population"), modifying, and searching for products; writing documentation; using templates. | | CLI reference | An indexed reference to the command-line interface. The Backstage CLI does a wide variety of tasks and is used both the administrator and by developers. | | Tutorials | Tasks that are good candidates for tutorials are difficult, often-used tasks that must be mastered to use the product effectively. Many of these are probably in daily use by developers. | | Cookbooks | There might be specialized tasks required of developers by an organization that should be documented, especially if they are performed infrequently. |