diff --git a/analyses/0015-knative/analysis.md b/analyses/0015-knative/analysis.md new file mode 100644 index 0000000..e985156 --- /dev/null +++ b/analyses/0015-knative/analysis.md @@ -0,0 +1,864 @@ +--- +title: Knative Documentation Analysis +tags: Knative +created: 2025-07-25 +modified: 2025-07-25 +author: Dave Welsch (@dwelsch-esi) +--- + + + +## Introduction + +This document analyzes the effectiveness and completeness of the +[Knative][project-website] open source software (OSS) project's documentation +and website. It is funded by the CNCF Foundation as part of its overall effort +to incubate, grow, and graduate open source cloud native software projects. + +According to CNCF best practices guidelines, effective documentation is a +prerequisite for program graduation. The documentation analysis is the first +step of a CNCF process aimed at assisting projects with their documentation +efforts. + +### Purpose + +This document was written to analyze the current state of Knative +documentation. It aims to provide project leaders with an informed understanding +of potential problems in current project documentation. A second +document is a list of [issues] to be added to the project +documentation repository. These issues can be taken up by contributors to +improve the documentation. + +This document: + +- Analyzes the current Knative technical documentation and website +- Compares existing documentation against the CNCF’s standards +- Recommends a program of key improvements with the largest return on investment + +### Scope of analysis + +The documentation discussed here includes the entire contents of the website, +the technical documentation, and documentation for contributors and users on the +Knative GitHub repository. + +The Knative website and documentation are written in Markdown and are compiled +using the mkdocs static site generator with the Material theme. +The site's code is stored on the knative/docs GitHub repo. + +#### In scope + +- Website: https://knative.dev/docs/ (not an error; identical to the + Documentation site) +- Documentation: https://knative.dev/docs/ +- Website repo: https://github.com/knative/docs +- https://github.com/knative/eventing (The Knative Eventing API is built from + this repo) +- https://github.com/knative/community (Community and governance for the + project) +- https://github.com/knative/client (`kn`, a Knative CLI) +- https://github.com/knative/func (`func`, another CLI) +- https://github.com/knative/serving (a major component of Knative) + +#### Out of scope + +- Other Knative repos: https://github.com/knative/\, for any \ not + listed in "In scope". + +### How this document is organized + +This document is divided into three sections that represent three major areas of +concern: + +- **Project documentation:** concerns documentation for users of the Knative + software, aimed at people who intend to use the project software +- **Contributor documentation:** concerns documentation for new and existing + contributors to the Knative OSS project +- **Website:** concerns the mechanics of publishing the documentation, and + includes branding, website structure, and maintainability + +Each section begins with summary ratings based on a rubric with appropriate +[criteria] for the section, then proceeds to: + +- **Comments**: observations about the existing documentation, with a focus on + how it does or does not help Knative users achieve their goals. +- **Recommendations**: suggested changes that would improve the effectiveness of + the documentation. + +The accompanying [issues] document contains actionable improvements +for inclusion in [project-doc-website]/issues. + +### How to use this document + +Readers interested only in actionable improvements should skip this document and +read the **[issues] list**. + +Readers interested in the current state of the documentation and the reasoning +behind the recommendations should read the section of this document pertaining +to their area of concern: + +- [Project documentation](#project-documentation) +- [Contributor documentation](#contributor-documentation) +- [Website and documentation infrastructure](#website-and-infrastructure) + +Examples of CNCF documentation that demonstrate the analysis criteria are linked +from the [criteria] specification. + +#### Recommendations, requirements, and best practices + +This analysis measures documentation against CNCF project maturity standards, +and suggests possible improvements. In most cases there is more than one way to +do things. Few recommendations here are meant to be prescriptive. Rather, the +recommended implementations represent the reviewers' experience with how to +apply documentation best practices. In other words, borrowing terminology from +the lexicon of [RFCs][rfc-spec], the changes described here should be understood +as "recommended" or "should" at the strongest, and "optional" or "may" in many +cases. Any "must" or "required" actions are clearly denoted as such, and pertain +to legal requirements such as copyright and licensing issues. + +## Project documentation + +Knative is a **graduated** project of CNCF. This means that the project should +have [_very high_][criteria] standards for documentation. + +| Criterion | [Rating (1-5)] | +| -------------------------- | -------------- | +| Information architecture | 2 - Needs improvement | +| New user content | 2 - Needs improvement | +| Content maintainability | 3 - Meets standards | +| Content creation processes | 3 - Meets standards | +| Inclusive language | 5 - Exemplary | + +### Comments + +Project documentation overall is very good. There's room for improvement mostly +around organization and the new user experience. + +Following are general comments that don't address specific CNCF criteria. + +The project main landing page also serves as the documentation landing page, +mixing marketing and technical information. It's helpful to users if there's +a clear separation between technical documentation and other information. +The technical documentation is goal-driven and should be factual, specific, +and purposeful. + +The technical documentation outline in the banner headings is good from a +organizational perspective: Concepts, Tutorial, and Installing, then +instructional information, then reference information. See notes below on the +Left-hand sidebar (LHS) table of contents. Some other notes: + +- The "Home" item is unnecessary for the project landing page. By convention +the upper-left logo links to the site landing page. +- The "Concepts" section is a brief introduction. Instead, it should contain +a complete technical overview that's accessible to a new user. +- "Installing" does a good job of providing complete installation instructions +for all user roles. Some additional "road signs" would help to guide users to +the correct installation for their role, OS, and so on. +- "Serving" and "Eventing" organize the instructional information by functional + component, and by user role within the component. The alternative is to flip + this and organize by user role first: + - Admin Guide + - Eventing + - Serving + - Developer Guide + - Eventing + - Serviing +- Knative CLI - The title implies a dedicated Knative CLI. There are two Knative + CLIs documented here, `kn` and `func`. As well, `kubectl` is the first CLI + documented on the page. + +The `kn` CLI is documented in the knative/client repo, with a link from here. +The reference to the repo-based documentation is OK, but why is the not +documented here? Its installation is documented on the site; this seems +inconsistent. + +***Table of contents*** + +The table of contents (TOC) in the LHS is hard to navigate. Some issues: + +1. The LHS contains only one technical documentation section at a time (see + previous comment about the banner menu). It's more usual for the LHS to + contain the entire TOC and to not change between pages (except for expanding + and collapsing headings). +2. The expand/collapse items in the TOC are navigational only; they do not + bring up information pages. This isn't uncommon, but it's tedious to navigate + multi-level topics. It's also less efficient than displaying the overview or + introduction with the top-level entry. +3. There are singleton sub-headings in the TOC. For example, see [Concepts > + Resources > Serving Resources > Revisions] + (https://knative.dev/development/concepts/serving-resources/revisions/). + Especially with the navigation-only expand/collapse items, these represent + extra mouse clicks for the user with no added benefit. +4. There are multiple entry points to the same documentation in the TOC. *And* + there are duplicate information sections. For example: + Installing > Install the Knative CLI sends you to the `Knative CLI` page. This + completely changes the left-hand TOC but the banner menu is not set up to + highlight `Knative CLI`, so are no cues as to where you were just teleported + (violating the [principle of least astonishment] + (https://en.wikipedia.org/wiki/Principle_of_least_astonishment)). + +***New user experience*** + +The blurb under "Need to know more?" on the landing page sounds like it's going +to an information resource; the button says "Explore Knative". Instead it +goes to the Quickstart tutorials. It sounds more like it should go to "Serving +Architecture". + +***Graphics*** + +In many places, large graphics are placed before any explanatory text. Also, +many graphics, especially in the E2E tutorial (incidentally, "end-to-end" should +be spelled out), provide no information but serve to distract, clutter, and take +up valuable screen space. + +***Headings and titles*** + +Tasks are for the most part well named using appropriate verbs. + +TOC titles that don't agree with page titles confuse the reader. + +The following subsections address individual items in the CNCF criteria for +Project Documentation. + +#### Information architecture + +***Is there high level conceptual content?*** + +The "Concepts" document at the top of the websiteis a brief introduction to two +of the components of Knative. It does not introduce the overall architecture or +explain how it works. + +There is other high-level conceptual information about Knative scattered +throughout the website. The existing conceptual information seems to assume that +you're already familiar with Kubernetes and with the problems that Knative is +intended to solve. + +***Is every product feature documented?*** + +The documentation seems a little behind being feature-complete, based on open +Github issues requesting feature documentation. See for example issues +[6216](https://github.com/knative/docs/issues/6216) and +[5331](https://github.com/knative/docs/issues/5331). + +Every component section (Eventing, Serving) is organized differently. This +requires extra cognitive work from the readers. + +***Does the documentation define all user roles (personas) for the product?*** + +The "Audiences information" in Community is a good description of user roles. +This information is reflected in subsections of the "Eventing" and "Serving" +sections in the division between Developer and Admin topics. + +***Are there instructions (tasks, tutorials) documented for features?*** + +Tasks are documented for most features. + +***Are there instructions for all major use cases for each user role?*** + +Tasks are covered for different user roles; it's unknown whether this coverage +is complete. + +***Are tasks organized by user role and use case?*** + +The documentation is organized by the key components. Tasks for each component +seem to be present, but could be more clearly written and better labeleled. + +***Does instructional content demonstrate atomicity*** +Are individual tasks clearly named according to their goals? + +For the most part tasks seem separate and well named. + +***Are tasks written as numbered step-by-step instructions?*** + +Tasks are broken down into steps. In most cases steps are not numbered. Some +procedures have embedded subtasks. Numbering of tasks and subtasks is +inconsistent. + +***Are task descriptions in headings and the TOC described with a verb phrase?*** + +Most task headings accurately describe the task. + +***Is the documentation free of features that lack task documentation?*** + +Task documentation seems to exist for key features. It is sometimes difficult to +locate. + +***Is the “happy path” — the most common use case — documented?*** + +The "Happy Path" seems to be building and running a simple service using all +three of the Knative components. This is documented in the "E2E" tutorial. + +***Is there a clear escalation path for users needing more help?*** + +In general, Knative help and community support are strong throughout the +documentation and repositories. Troubleshooting could provide clearer +instructions and be more closely linked to the tasks it supports. Readers will +probably eventually find their way to help through the community via a Slack +channel. + +There is a FAQ for Eventing. It contains only one question. + +There are troubleshooting instructions for various components within the +documentation. The troubleshooting steps are little more than command examples +showing the non-exception case output. In some cases the output is cryptic and +it's not clear what to do. + +***If the product exposes an API, is there a complete reference?*** + +There is a reference for the Eventing API. It contains no explanatory or +introductory text, and many descriptions of functions and fields are missing, +rudimentary, or tautological. + +***If the product has CLIs, are there complete references?*** + +There are references for the two CLIs, `kn` and `func`. The references are +contained in their respective repos, not in the technical documentation. + +***Is content up to date and accurate?*** + +The content seems up to date based on the versioning and release mechanisms. + +***Does the doc separate conceptual, instructional, and reference info?*** + +Individual topics are separated by information type. + + +#### New user content + +***Is “getting started” clearly labeled?** +(“Getting started”, “Installation”, “First steps”, or the equivalent.) + +There is no clearly labeleld "Getting Started" page. There is a "Quickstart" +tutorial, with an accompanying plugin, that serves (primarily) as a beginner +getting-started procedure. + +***Is there a “getting started” path for all user roles?*** + +The "Quckstart" tutorial does not discuss roles. It acknowledges that the +exercise does a simplified local installation and directs the reader to YAML or +Operator-based installs for production server installation. + +***Is installation documented step-by-step?*** + +Installation is thoroughly documented. Instructions are step-by-step and well +organized. That said, I could not install using the Quickstart tutorial because +I ran into an error that was not documented. the Install pages link to the +Release Notes pages for binary downloads. This was confusing. Downloading the +binary from the release page is not straightforward. The download links are +under "Assets", far down the page. + +***Are different types of installation documented if necessary?*** +(development, test, production) + +The Quickstart tutorial documents a local install. Other (server) installs are +documented in the Install section. + +***If needed, are multiple OSes documented?*** + +Installs for multiple OSes are documented implicitly (for example, the install +binaries are characteristic of their respective OSes). A discussion of OSes up +front in the prereqs section would be appropriate. + +***Do users know where to go after reading the getting started guide?*** + +Next steps are documented at the end of the quickstart and some other sections. + +***Is new user content clearly signposted on the site’s homepage?*** + +There is no clearly labeled new user content available from the landing page. +The closest is the Quickstart tutorial. + +***Is there easily copy-pastable sample code or other example content?*** + +There are code samples available in "Code samples", organized by Knative +component. + +#### Content maintainability and site mechanics + + +***Is your documentation searchable?*** + +The documentation is searchable. The Search function does not truncate results. +For example, the "Puppet" entry when searching "getting started" displays the +page's entire 900 words. + +***Are you planning for localization/internationalization?*** + +***Is a localization framework present?*** + +Apparently not. There is no evident localization or mechanism for localization. + +***Do you have a clearly documented method for versioning your content?*** + +Content is versioned up to the most recent release using branches in the +knative/docs Github repo. The process is documented in the contribute-to-docs +directory in the repo. The version drop-down contains the latest three releases +plus pre-release. No older archived versions are offered on the site. + +***Is release-specific information documented in release notes?*** + +The repo contains appropriate release notes. + +***Is the documentation free of duplicate sections of information?*** + +Duplicated information is present in the doc. It is deliberately included from a +collection of reusable doc snippets. + +***Do informational graphics add value by providing information?*** + +Graphics are large and contain little new information, especially in the E2E +tutorial. Some of the conceptual diagrams (component diagrams, flow charts) are +helpful. + +***Will graphics require modifications?*** +For example, due to software changes, GUI updates, or translation? + +Graphics are unlikely to require frequent updates; for example, there are no +screen shots. + + +#### Content creation processes + +***Is there a clearly documented contribution process for documentation?*** + +The documentation contribution process is well documented in the repo. + +***Does the code release process include documentation creation & updates?*** + +Documentation releases are controlled by the release process. The contribution +process documents how to update previous versions of the documentation, if +necessary. + +***Who reviews and approves documentation pull requests?*** + +***Does the website have a clear owner/maintainer?*** + +Review and approval of documentation releases is role-based and is controlled by +the OWNERS and OWNER_ALIASES files in the repo. A steering committee is among +the leadership identified in these files. + + +#### Inclusive language + +***Are there customer-facing utilities, endpoints, class names, or features*** +that use non-recommended words as documented by the Inclusive Naming +Initiative website? + +The documentation contains few non-recommended words as documented by the +[Inclusive Naming Initiative](https://inclusivenaming.org) website. + +***Does the project use language like "simple", "easy", etc.?*** + +The Knative documentation avoids ableist language. The Knative style guide +explicitly discourages language that assumes a level of understanding ("just", +"easily", "simply"). + +### Recommendations + +#### Organization + +The simplest solution to untangling the documentation from the landing page is +probably to: + +- Add a "Documentation" tab on the right alongside "Blog," "About," and +"Community" +- Move the tech docs items from the banner to the left-hand sidebar (LHS) + +Other suggestions: + +- Remove the "Home" item. +- Write a comprehensive "Concepts" section using conceptual material from the +rest of the documentation. +- Rename "Tutorial" to "Tutorials". +- Add navigational cues to the "Installation" section to better guide users to +the right instructions and downloads for their user role, OS, and use case. +- Rename "Developer Topics" and "Admin Topics" to "Developer Tasks" and "Admin + Tasks" to cue readers. These are effectively user guides and need to be + recognizable as such by users. +- Rename Eventing - "Knative Eventing - The Event-driven application platform + for Kubernetes". This is technical documentation. The marketing stuff goes + elsewhere. +- In the "Eventing" section, move "Event Mesh" under "Concepts". +- Rename "Knative CLI" to "CLI tools". Include the `kn` CLI documentation on the + documentation site. +- In "Reference": Rewrite at the top of the page: "This page describes Knative + security and disclosure information." + to + "This page describes how to validate code and report security vulnerabilites + in Knative. For a complete description of the Knative threat model, see the + [Knative security working group repo] + (https://github.com/knative/community/blob/main/working-groups/security/threat-model.md)." + Then change headings: "Verifying code signature" and "Reporting a + vulnerability." +- Move Release Notes out of Reference to be a top-level TOC item. +- Make the "Explore Knative" button the landing page link to the conceptual + overview. Or, change the button label to "Quick Start". +- Remove the Eventing FAQ. Its (single) response is documented elsewhere. +- Add a glossary in the Reference section. +- Reevaluate each graphic on the site. Is it adding value? If not, eliminate it. + Does it take up so much page space that it forces unnecessary scrolling? + Reduce the size. Especially in the E2E tutorial, get rid of the large graphics. + Their only purpose here is to cue the user to the type of task. Make them 32x32 + icons, or get rid of them. In the text, tell the user what the graphic is + before they see it, and make sure the picture is adding value, or else delete + it. +- Make TOC titles agree with page titles. Rename "About" in the banner to + something more descriptive: "Why Knative?", "Testimonials and Case Studies", + or "Endorsements". +- Revise to conform to the style guide: + - Rewrite "please" requests to straightforward instructions. For example: + "Please see XYZ" to "See XYZ". + - Remove "About" from headings. For example: "About the Prometheus Stack" to + "The Prometheus Stack". + - Spell out words to avoid abbreviations. For example: Replace "E2E" with "end + to end". + + +#### Information architecture + +***Conceptual overview*** + +Update the "Concepts" section to be a complete overview of the Knative system. +Start with an explanation of Knative's purpose that's understandable to new +users and to readers who are only passingly familiar with containerized software. + +Include a complete explanation of Knative's components and how they're related. +This is a good short explanation of the Knative components (though a little +marketing-ish) that's on the CNCF website: + + Knative is a developer-focused serverless application layer which is a great + complement to the existing Kubernetes application constructs. Knative consists + of three components: an HTTP-triggered autoscaling container runtime called + “Knative Serving”, a CloudEvents-over-HTTP asynchronous routing layer called + “Knative Eventing”, and a developer-focused function framework which leverages + the Serving and Eventing components, called "Knative Functions". + +None of the instructional material is going to make sense without an +understanding of the Knative architecture. + +Some specifics: + +Rewrite: "The documentation in this section explains commonly referenced Knative +concepts and abstractions, and helps to provide you with a better understanding +of how Knative works." +as +"This documentation explains what Knative is for and how it works." + +***Escalation path*** + +Expand or eliminate the FAQ. My preference is to eliminate it. + +Organize the Troubleshooting procedures by symptom (within component). Or at +least expand the explanations in the Troubleshooting sections to describe what +each command is intended to diagnose. + +Expand the Eventing API reference to include meaningful explanations of every +field and function. Add a short introduction to the API reference. + +***New user content*** + +Add a "Getting Started" page at the top level of the documentation. This page +should map a reader's user role and goals to content that helps them install, +evaluate, or learn about the product, depending on their goal. For exmaple, +direct new users to the Quickstart tutorial, evaluators or potential buyers to +the conceptual overview, and so on. + +Currently a user who clicks in the install section to get an install file +download has to scroll through a lot of release notes before they reach the +downloads list. Move the install download links to the top of the Release Notes +or clearly link to them from the top of the Release Notes. Or, put them on their +own Downloads page. + +***Content maintainability & site mechanics*** + +Configure the Search to truncate results after 100 characters or so. Displaying +the entire page text of each result hinders users' ability to find their result. + +Address localization. If you have no intention to localize the documentation, +say so in the introduction. + +## Contributor documentation + +Knative is a **graduated** project of CNCF. This means that the project should +have [_very high_][criteria] standards for documentation. + +| Criterion | [Rating (1-5)] | +| ----------------------------------------- | -------------- | +| Communication methods documented | 5 - Exemplary | +| Beginner friendly issue backlog | 3 - Meets standards | +| “New contributor” getting started content | 3 - Meets standards | +| Project governance documentation | 4 - Meets or exceeds standards | + +### Comments + + +#### Communication methods documented + +***Is there a Slack/Discord or equivalent community linked from your website?*** + +***Is there a direct link to your GitHub project or repository?*** + +***Can users find and join periodic project meetings, if applicable?*** + +***Are mailing lists documented?*** + +The site lists community resources that include: + +- A number of general and subtopic-specific Slack channels +- A link to the Github doc repository +- An up-to-date published meeting schedule +- A Stack Overflow topic +- A Google Groups mailing list + + +#### Beginner friendly issue backlog + +***Are docs issues well-triaged?*** + +Doc issues are well triaged; there are tags classifying changes according to +size and status. + +***Is there a clearly marked way for new contributors to make contributions?*** +(A “good first issue” label?) + +There is a "good first issue" label in the issues list. + +***Are issues well-documented (i.e., more than just a title)?*** + +Issue descriptions are uneven. Many issues are well documented, but some lack +basic information. + +***Are issues maintained for staleness?*** + +There is a archiving bot for stale doc issues, but there are issues as old as +four years in the repo. + + +#### New contributor getting started content + +***Do you have a community repository or section on your website?*** + +There is a separate [community repo](https://github.com/knative/community) in +the Knative project. + +***Is there a document welcoming new contributors?*** +And documenting a first contribution process? + +There is ample documentation for contributors but no documentation specifically +for new contributors. + +***Can new users find where to get help?*** + +There are ample resources for new users to get support from the community. + + +#### Project governance documentation + +***Is project governance clearly documented?*** + +Project +[governance](https://github.com/knative/community/blob/main/GOVERNANCE.md) is +documented in the [community repo](https://github.com/knative/community) in the +Knative project. + + +### Recommendations + +No recommendations. + +#### Communication methods documented + +#### Beginner friendly issue backlog + +#### New contributor getting started content + +#### Project governance documentation + +## Website and infrastructure + +**TBD** + +Knative is a **graduated** project of CNCF. This means that the project should +have [_very high_][criteria] standards for documentation. + +| Criterion | [Rating (1-5)] | +| ------------------------------------------- | -------------- | +| Single-source for all files | [rating (1-5)] | +| Meets min website req. (for maturity level) | [rating (1-5)] | +| Usability, accessibility, and design | [rating (1-5)] | +| Branding and design | [rating (1-5)] | +| Case studies/social proof | [rating (1-5)] | +| SEO, Analytics, and site-local search | [rating (1-5)] | +| Maintenance planning | [rating (1-5)] | +| A11y plan & implementation | [rating (1-5)] | +| Mobile-first plan & impl. | [rating (1-5)] | +| HTTPS access & HTTP redirect | [rating (1-5)] | +| Google Analytics 4 for production only | [rating (1-5)] | +| Indexing allowed for production server only | [rating (1-5)] | +| Intra-site / local search | [rating (1-5)] | +| Account custodians are documented | [rating (1-5)] | + +### Comments + + +#### Single-source requirement + +The project website and documentaiton are contained in a single repo. + +Every code repo in the project has its own `docs` folder. + +#### Minimal website requirements + +Listed here are the minimal website requirements for projects based on their +[maturity level][maturity-level], either incubating or graduated. (These are the +only two levels for which a tech docs analysis can be requested.) + + + +| Criterion | Incubating Requirement | Graduated Requirement | +| ----------------------------- | ------------------------------------------------------- | ----------------------------------------- | +| [Website guidelines] | All guidelines satisfied | All guidelines satisfied | +| **Docs analysis** (this) | Requested through CNCF [service desk][cncf-servicedesk] | All follow-up actions addressed | +| **Project doc**: stakeholders | Roles identified and doc needs documented | All stakeholder need identified | +| **Project doc**: hosting | Hosted directly | Hosted directly | +| **Project doc**: user docs | Comprehensive, addressing most stakeholder needs | Fully addresses needs of key stakeholders | + + + +[git-submodules]: https://git-scm.com/book/en/v2/Git-Tools-Submodules +[maturity-level]: + https://github.com/cncf/toc/tree/main/process#ii-stages---definitions--expectations +[cncf-servicedesk]: https://servicedesk.cncf.io + +#### Usability, accessibility and devices + +Most CNCF websites are accessed from mobile and other non-desktop devices at +least 10-20% of the time. Planning for this early in your website's design will +be much less effort than retrofitting a desktop-first design. + +- Is the website usable from mobile? +- Are doc pages readable? +- Are all / most website features accessible from mobile -- such as the top-nav, + site search and in-page table of contents? +- Might a [mobile-first] design make sense for your project? + +[mobile-first]: + https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Responsive/Mobile_first + +Plan for suitable [accessibility][] measures for your website. For example: + +- Are color contrasts significant enough for color-impaired readers? +- Are most website features usable using a keyboard only? +- Does text-to-speech offer listeners a good experience? + +It is up to each project to set their own guidelines. + +[accessibility]: https://developer.mozilla.org/en-US/docs/Web/Accessibility + +#### Branding and design + +CNCF seeks to support enterprise-ready open source software. A key aspect of +this is branding and marketing. + +We evaluate on the following: + +- Is there an easily recognizable brand for the project (logo + color scheme) + clearly identifiable? +- Is the brand used across the website consistently? +- Is the website’s typography clean and well-suited for reading? + +Footer: "Knative is a Cloud Native Computing Foundation incubation project". Should be "incubating". + + +#### Case studies/social proof + +One of the best ways to advertise an open source project is to show other +organizations using it. + +We evaluate on the following: + +- Are there case studies available for the project and are they documented on + the website? +- Are there user testimonials available? +- Is there an active project blog? +- Are there community talks for the project and are they present on the website? +- Is there a logo wall of users/participating organizations? + +#### SEO, Analytics and site-local search + +SEO helps users find your project and it's documentation, and analytics helps +you monitor site traffic and diagnose issues like page 404s. Intra-site search, +while optional, can offer your readers a site-focused search results. + +We evaluate on the following: + +- Analytics: + - Is analytics enabled for the production server? + - Is analytics disabled for all other deploys? + - If your project used Google Analytics, have you migrated to GA4? + - Can Page-not-found (404) reports easily be generated from you site + analytics? Provide a sample of the site's current top-10 404s. +- Is site indexing supported for the production server, while disabled for + website previews and builds for non-default branches? +- Is local intra-site search available from the website? +- Are the current custodian(s) of the following accounts clearly documented: + analytics, Google Search Console, site-search (such as Google CSE or Algolia) + +#### Maintenance planning + +Website maintenance is an important part of project success, especially when +project maintainers aren’t web developers. + +We evaluate on the following: + +- Is your website tooling well supported by the community (i.e., Hugo with the + Docsy theme) or commonly used by CNCF projects (our recommended tech stack?) +- Are you actively cultivating website maintainers from within the community? +- Are site build times reasonable? +- Do site maintainers have adequate permissions? + +#### Other + +- Is your website accessible via HTTPS? +- Does HTTP access, if any, redirect to HTTPS? + +### Recommendations + +> AUTHOR NOTE: Write general recommendations based on the comments from the +> previous section. + +#### Single-source requirement + +#### Minimal website requirements + +#### Usability, accessibility and devices + +#### Branding and design + +#### Case studies/social proof + +#### SEO, Analytics and site-local search + +#### Maintenance planning + +#### Other + +#### References and notes + +##### Rating values + +The numeric rating values used in this document are as follows + +1. Not present +2. Needs improvement +3. Meets standards +4. Meets or exceeds standards +5. Exemplary + +[criteria]: ../criteria.md +[implementation]: ./implementation.md +[issues]: ./issues.md +[project-doc-website]: https://knative.dev/docs/ +[project-website]: https://knative.dev/docs/ +[Rating (1-5)]: #rating-values +[rfc-spec]: https://www.rfc-editor.org/rfc/rfc2119 +[website guidelines]: ../../website-guidelines-checklist.md diff --git a/analyses/0015-knative/issues.md b/analyses/0015-knative/issues.md new file mode 100644 index 0000000..f7d5a71 --- /dev/null +++ b/analyses/0015-knative/issues.md @@ -0,0 +1,721 @@ +--- +title: Knative Issues +tags: Knative +created: 2025-07-28 +modified: 2025-07-28 +author: Dave Welsch (@dwelsch-esi) +--- + +# Separate technical documentation from the project page + +## Overview + +The Knative project [landing page] redirects to the doc page: +`https://knative.dev/docs/`. The project landing page contains all the +elements you'd expect on an open-source software page, including case studies, +user endorsements, and links to the community. + +It also includes the technical documentation in the banner menu. This +arrangement conflates the project and the technical documentation, mixing +marketing and technical information. + +It's helpful to users if there's a clear separation between technical +documentation and other information. The technical documentation is goal-driven +and should be factual, specific, and purposeful. + +The request: Separate the technical documentation from the project landing page. + +Audience: All + +Type: All + + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://github.com/knative/ +- https://github.com/knative/docs/ + + +Suggested changes: + +The simplest solution to untangling the documentation from the landing page is +probably to: + +- Reconfigure the site to use the base URL `https://knative.dev` as the project + landing page. +- Add a "Documentation" tab on the right alongside "Blog," "About," and + "Community" on the project page that links to the docs page, + `https://knative.dev/docs`. +- Move the tech docs items from the banner to the left-hand sidebar (LHS) on the + tech docs page. +- Remove the version drop-down from the project landing page (and the other + non-documentation project pages). + + +# Rename pages + +## Overview + +Well named pages and headings help users quickly find the right information. +Consistent naming aids in scanning the table of contents, and accurate names +make search more effective. + +Audit page and heading titles for consistency and accuracy. + +Audience: All + +Type: All + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + +## Possible Implementation + +Related material in the current doc: + +- https://github.com/knative/docs/ +- https://knative.dev/docs/client/ +- https://knative.dev/docs/getting-started/tutorial/ +- https://knative.dev/docs/serving/ +- https://knative.dev/docs/eventing/ + + +Suggested changes: + +- Remove "About" from titles: + - "About Installing Knative" > "Installing Knative" + - "About Eventing Features" > "Eventing Features" + - and so on. +- Rename "Tutorial" to "Tutorials" (there is more than one tutorial here). +- Make verb tense consistent in procedure titles. Use "-ing" verbs to title tasks + and processes (thiis is already done in the majority of cases!): + - [Configure Broker defaults] > "Configuring Broker defaults" + - [Install Knative using quickstart] > "Installing Knative using quickstart" + - and so on. +- In [Serving] and [Eventing], rename "Developer Topics" and + "Admin Topics" to "Developer Tasks" and "Admin Tasks" to cue readers. +- Retitle this page: + [Knative Eventing - The Event-driven application platform for Kubernetes] + to "Eventing". +- Rename "Knative CLI" to "CLI tools". +- Make TOC titles agree with page titles. Some examples (not a complete list): + - Change "Install the Knative CLI" in the TOC to match the page title: + [Installing the Knative CLI]. + - Change the page title: [Metrics] to match "Configuring metrics" in the TOC. + - Change "About Apache Kafka Broker" in the TOC to match the page title: + [Knative Broker for Apache Kafka]. +- Rename "About" in the landing page banner to something less misleading: "Why + Knative?", "Testimonials and Case Studies", or "Endorsements". + + +# Update technical overview + +## Overview + +Write a conceptual overview of Knative. This should go where the "Concepts" +section is in the current documentation, but be much more comprehensive. + +This conceptual overview serves two purposes: + +1. It helps potential users evaluate whether Knative is the right solution to +their problem; +2. It provides all users with enough background information to understand the +instructional and reference material later in the documentation. User don't have +to read it all before proceeding, but they will refer to it when they don't +understand how the product's pieces fit together. + +Audience: All + +Type: Conceptual + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + +## Possible Implementation + +Related material in the current doc: + +- https://knative.dev/docs/concepts/ +- https://knative.dev/docs/serving/ +- https://knative.dev/docs/serving/architecture/ +- https://knative.dev/docs/serving/request-flow/ +- Almost any page currently titled "About (something)" + + +Suggested changes: + +Update the "Concepts" page to be a comprehensive conceptual overview of the +entire system. Start with an introduction that explains what the product is +and what problems it is designed to solve. This information should be accessible +to someone who is not an expert container developer or admin. If some background +knowledge is required to understand the concepts, state what that knowledge is. + +Follow with a description of the product architecture that explains its +components and how they interoperate. From the CNCF website: + + Knative is a developer-focused serverless application layer which is a great + complement to the existing Kubernetes application constructs. Knative consists + of three components: an HTTP-triggered autoscaling container runtime called + “Knative Serving”, a CloudEvents-over-HTTP asynchronous routing layer called + “Knative Eventing”, and a developer-focused function framework which leverages + the Serving and Eventing components, called "Knative Functions". + +Finally, explain essential but tangential topics like authenication and +authorization, still at a conceptual level. + +# Update Installation to better orient users + +## Overview + +The installation instructions are generally very good, but there are a few speed +bumps to finding and performing the right installation procedure. + +Add "roadmap" information to orient readers in the Installation section of the +documentation. + +Audience: Admins and developers + +Type: Instructional + + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://github.com/knative/install + + +Suggested changes: + +On the [Installing Knative] page, provide a roadmap at the top to direct users +to the correct installation. Aside from installing the different components +(Eventing and Serving) using different modalities (YAML, Kubernetes Operator, +and upgrading), disclose what operating systems are supported, and differentiate +between local and server installations. + +# Reorganization issues + +## Overview + +This issue suggests several changes to the documentation's organization to +improve usability and consistency. For example, several pages appear as +"orphans" (single subordinate headings) in the table of contents (TOC). This is +usually avoidable with better organization. + +Audience: All + +Type: All + + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://github.com/knative/eventing +- https://knative.dev/docs/eventing/event-mesh/ +- https://knative.dev/docs/reference/relnotes/ + +Suggested changes: + +- In the [Eventing] section, move "Event Mesh" under "Concepts". +- Move the [Release Notes] to the top level of the TOC + (and change the URL to https://knative.dev/docs/relnotes/). +- Make the "Explore Knative" button the landing page link to the conceptual + overview. Or, change the button label to "Quick Start". Its current label + is misleading. +- Remove the Eventing FAQ. It has only one entry, and that topic is covered + elsewhere in the documentation. +- Are [Revisions] the only Knative Resource? Roll this menu up: + `Serving > Resources > Revisions > About Revisions` + to `Serving > Revisions > About Revisions`. + +# Document the kn CLI on the site + +## Overview + +The `kn` CLI documentation is maintained in the +[/knative/kn] repo, except for the install instrutions, which are on the doc +site. + +Consolidate the `kn` CLI documentation so that it's maintained in one place. + +Audience: Admin and developer + +Type: Reference + + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://github.com/knative/ + + +Suggested changes: + +Document the `kn` CLI on the documentation website rather than linking to the +[/knative/kn] repo documentation. + + +# Revise the Security and threat disclosure page + +## Overview + +The +[Knative Security and Disclosure Information] page has some misleading and +confusing elements. Rewrite the introduction to this page so that it's clear +what procedures and information are available. + +Audience: Admin + +Type: Instructional + + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://knative.dev/docs/reference/security/ + +Suggested changes: + +In [Reference]: + +Rewrite the introduction at the top of the page, including the bullet point link +to to the threat model, to: + +"This page describes how to validate code and report security vulnerabilites in +Knative. For a complete description of the Knative threat model, see the +[Knative security working group repo]." +Then change headings: "Verifying code signature" and "Reporting a vulnerability." + +Rename the heading "Code Signature Verification" to +"Verifying a code signature". + +Rename the heading "Report a vulnerability" to "Reporting a vulnerability". + +# Add a glossary + +## Overview + +A software product like Knative typically uses many terms specific to the +product and to its knowledge domain. Users, especially those new to the +software, need help making sense of these terms while reading the documentation. + +Add a glossary of terms to the technical documentation. + +Audience: All + +Type: Reference + + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://github.com/knative/ + + +Suggested changes: + +Add a glossary to the reference section of the documentation. Include terms that +are essential concepts in related software domains (containerization, server and +event architecture, etc.) as well as terms that are specific to Knative. A few +such terms: Custom resource, Eventing, func (CLI), kn (CLI), Operator, Serving, +Sink, Source, Webhook. + +A glossary entry should provide readers with a definition of the term, but not +try to explain it in detail. A sentence or two is usually enough. It's OK to +refer to other glossary entries in the definition. + +Order the entries alphabetically. + + +# Review graphics + +## Overview + +Graphics should enhance understanding by illustrating concepts and processes. +Many graphics on the Knative site detract from, rather than enhance, the +usability of documentation. + +Review the graphics on the site to make sure they meet these criteria: + +- Does the graphic enhance understanding? Does it add information that can't be + conveyed in a few words? +- Is the graphic placed in context by the surrounding text? +- Does the graphic's size and aspect ratio enable proper placement and text flow + on a variety of screens sizes? +- Is the graphic maintainable: Will it require frequent updating for + localization, software updates, or interface changes? + +Review the graphics that are on the site. Remove graphics that don't justify the +space they take up. Scale the graphics to a more appropriate size if necessary. + +Audience: All + +Type: Conceptual + + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://knative.dev/docs/concepts/ +- https://knative.dev/docs/concepts/eventing-resources/brokers/ +- https://knative.dev/docs/getting-started/tutorial/ +- https://knative.dev/docs/getting-started/next-steps/ +- https://knative.dev/docs/bookstore/page-0/welcome-knative-bookstore-tutorial/ +- https://knative.dev/docs/bookstore/disclaimer/ +- https://knative.dev/docs/bookstore/* (all bookstore pages) +- https://knative.dev/docs/serving/architecture/ +- https://knative.dev/docs/serving/request-flow/ +- https://knative.dev/docs/serving/encryption/encryption-overview/ + + +Suggested changes: + +- Briefly introduce the system diagram in the [Concepts overview]. This can be a + single phrase added to the sentence below the graphic: "The primary Knative + Serving resources are Services, Routes, Configurations, and Revisions, as + shown in the previous figure:". +- Similarly for other conceptual graphics on the site: Refer to them from the + text. + + +# Edit for conformance to style guide + +## Overview + +Edit the website for conformance to the [Knative style guide]. + +Audience: All + +Type: All + + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://github.com/knative/ + + +Suggested changes: + +Specific Knative style rules that will have the biggest impact: +- *Use imperatives for headings of procedures.* Technical writing best practice + has traditionally been to use gerunds ("-ing" verbs), but this is okay as well. + More imporant is to be consistent. Use the same form for all procedure headings. +- *Use simple and direct language.* To the extent possible, follow the "Voice + and language" guidelines in the style guide. For example, there are 50 instances + of the word "please" in the Knative documentation. Remove these and use simple + imperative sentences for instructions. +- *Avoid statements that will soon be out of date.* This addresses + maintainability. See the [example] in the style guide. There are many instances + of the word "currently" in the Knative documentation. +- Spell out words to avoid abbreviations. For example: Replace "E2E" with "end + to end". +- *Use simple and direct language.* For example, from [Concepts]: "The + documentation in this section explains commonly referenced Knative concepts + and abstractions, and helps to provide you with a better understanding of how + Knative works" could be: "This documentation explains what Knative is for and + how it works." + + +# Improve troubleshooting + +## Overview + +Troubleshooting instructions are too terse in Knative. They provide commands for +diagnosing issues, but do not explain a procedure. + +Improve troubleshooting by adding explanations. + +Audience: All + +Type: Instructional + + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://knative.dev/docs/install/troubleshooting/ +- https://knative.dev/docs/bookstore/page-0.5/environment-setup/#troubleshooting +- https://knative.dev/docs/install/installing-backstage-plugins/#troubleshooting +- https://knative.dev/docs/serving/troubleshooting/debugging-application-issues/ +- https://knative.dev/docs/bookstore/page-1/send-review-comment-to-broker/#step-2-create-broker +- https://knative.dev/docs/bookstore/page-2/sentiment-analysis-service-for-bookstore-reviews/#prerequisite-1-install-knative-func-cli + + +Suggested changes: + +Two possible approaches: + +1. Expand each troubleshooting step to explain what the step is trying to + diagnose. (Many of the current troubleshooting steps are no more than "use this + command to diagnose ..."). Make sure that there are links to each step from the + procedure where the issue can be encountered. +2. Consolidate all the troubleshooting procedures to a single troubleshooting + guide, organized by component. Again, provide links to the relevant points in + the guide from each procedure. + Exception: The end-to-end tutorial example has a lot of troubleshooting + information. As is the case currently, it should have its own troubleshooting + guide or guides. + +# Fill out the Eventing API + +## Overview + +The Eventing API contains no explanatory or introductory text, and many +descriptions of functions and fields are missing, rudimentary, or tautological. + +Improve the explanations in the Eventing API. + +Audience: Developers + +Type: Reference + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://knative.dev/docs/eventing/reference/eventing-api/ + + +Suggested changes: + +Add a brief introduction to the Eventing API, explaining what the API is for, +what kind of API it is (REST, presumably), where to find it, and how to include +it. + +Provide semantic information in descriptions about endpoints and fields. Do not +only restate information available from the element name. For example, the +`BackoffPolicyType` option has no explanation. It would be helpful to know what +the backoff is for: "Backoff is the delay before trying to reconnect after +failing to deliver an event" (or whatever). The description of its first value, +"exponential", reads: "Exponential backoff policy". More helpful would provide +more information: "Retry after an exponentially increasing number of seconds (1, +2, 4, 8, and so on)". + +# Write a Getting Started page + +The Quickstart tutorial is useful, but it doesn't fully orient new users. A +getting started page would help, with meta information about what to expect +and where to go on the documentation site. + +## Overview + +Audience: New users + +Type: Instructional + + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://github.com/knative/ +- https://knative.dev/docs/getting-started/ + +Suggested changes: + +Add a "Getting Started" page at the top level of the documentation. This page +should map a reader's user role and goals to content that helps them install, +evaluate, or learn about the product, depending on their goal. For exmaple, +direct new users to the Quickstart tutorial, evaluators or potential buyers to +the conceptual overview, and so on. + +See, for example, getting started pages for: + +- [Kubernetes] +- [The Update Framework] + (a little unusual because it's a specification, not a software product) +- [Python] (a kitchen-sink approach, to be sure) + +# Make install downloads findable + +Currently the binary install download links in the Installing section redirect +to the release notes, where the list of download files is far down the page. +Move and relabel the download file sectionm so that the files are easy to find. + +## Overview + +Audience: All + +Type: Instructional + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://knative.dev/docs/install/quickstart-install/#__tabbed_1_2 +- https://knative.dev/docs/client/install-kn/#__tabbed_1_2 +- https://knative.dev/docs/install/operator/knative-with-operator-cli/#__tabbed_1_1 +- https://knative.dev/docs/install/operator/knative-with-operator-cli/#__tabbed_1_2 +- https://github.com/knative/client/releases + +Suggested changes: + +Move the install download links to the top of the Release Notes or clearly link +to them from the top of the Release Notes. Or, put them on their own Downloads +page. + +Rename the heading for the download files from "Assets" to "Downloads" or +"Install files". + +# Truncate search results + +## Overview + +Audience: All + +Type: All + +The Search function does not truncate results. For example, the "Puppet" entry +when searching "getting started" displays the page's entire 900 words. + +Configure the Search to truncate results after 100 characters or so. Displaying +the entire page text of each result hinders users' ability to find their result. + +## Context + +This issue tracks recommended changes resulting from an analysis of the Knative +documentation commissioned by CNCF. The analysis and supporting documents are +here: https://github.com/cncf/techdocs/tree/main/analyses under `0015-knative`. + + +## Possible Implementation + +Related material in the current doc: + +- https://github.com/knative/ +- https://github.com/knative/docs/blob/main/mkdocs.yml + +Suggested changes: + +Search result entries are too long. The most straightforward solution is +probably to cut off each search result after a number of characters, say 100 or +so. + +This change probably has to be implemented in the `mkdocs` configuration. I'd +start by looking at the [mkdocs config file]. + + +[/knative/kn]: https://github.com/knative/client +[Concepts]: https://knative.dev/docs/concepts/ +[Concepts overview]: https://knative.dev/docs/concepts/ +[Configure Broker defaults]: https://knative.dev/docs/eventing/configuration/broker-configuration/ +[Eventing]: https://knative.dev/docs/eventing/ +[example]: https://github.com/knative/docs/blob/main/contribute-to-docs/style-guide/voice-and-language.md#avoid-statements-that-will-soon-be-out-of-date +[Install Knative using quickstart]: https://knative.dev/docs/install/quickstart-install/ +[Installing Knative]: https://github.com/knative/install +[Installing the Knative CLI]: https://knative.dev/docs/client/install-kn/ +[Knative Broker for Apache Kafka]: https://knative.dev/docs/eventing/brokers/broker-types/kafka-broker/ +[Knative Eventing - The Event-driven application platform for Kubernetes]: https://knative.dev/docs/eventing/ +[Knative Security and Disclosure Information]: https://knative.dev/docs/reference/security/ +[Knative security working group repo]: https://github.com/knative/community/blob/main/working-groups/security/threat-model.md +[Knative style guide]: https://github.com/knative/docs/tree/main/contribute-to-docs/style-guide +[Kubernetes]: https://kubernetes.io/docs/setup/ +[landing page]: https://knative.dev/ +[Metrics]: https://knative.dev/docs/serving/autoscaling/autoscaling-metrics/ +[mkdocs config file]: https://github.com/knative/docs/blob/main/mkdocs.yml +[Python]: https://wiki.python.org/moin/BeginnersGuide +[Reference]: https://knative.dev/docs/reference/ +[Release Notes]: https://knative.dev/docs/reference/relnotes/ +[Revisions]: https://knative.dev/docs/serving/revisions/ +[Serving]: https://knative.dev/docs/serving/ +[The Update Framework]: https://theupdateframework.io/docs/getting-started/