techdocs/analyses/0004-tremor.md

---
cSpell:ignore: Horgan onramps offramps LLDB Wayfair
---

# Assessment: Project Tremor

Prepared by: Celeste Horgan Date: July 2021

## Introduction

This document assesses the quality and completeness of a project's documentation
and website (if present).

This document:

- Measures existing documentation quality against the CNCF’s standards
- Recommends specific and general improvements
- Provides examples of great documentation as reference
- Identifies key improvements with the largest return on investment

## How this document works

The assessment is divided into three sections:

- **Project documentation:** for end users of the project; aimed at people who
  intend to use it
- **Contributor documentation:** for new and existing contributors to the
  project
- **Website:** branding, website structure, and maintainability

Each section rates content based on different
[criteria](../docs/analysis/criteria.md).

## Project documentation

| Criteria                   | 1   | 2   | 3   | 4   | 5   |
| -------------------------- | --- | --- | --- | --- | --- |
| Information architecture   |     | ✅  |     |     |     |
| New user content           |     | ✅  |     |     |     |
| Content maintainability    |     | ✅  |     |     |     |
| Content creation processes | ✅  |     |     |     |     |

Scale:

- 1 = (Is not present or requires significant work)
- 3 = (Is present, but needs work)
- 5 = (Is executed extremely well or no improvement required)

**Comments**

- **Information Architecture:** Project Tremor's documentation makes a number of
  individually small IA choices that culminate in a confusing experience for the
  user.

  - The [Getting Started](https://www.tremor.rs/getting-started/) section covers
    a number of seemingly random topics, with some appearing to be focused on
    selling the project to new users (i.e.
    [Good User Experience](https://www.tremor.rs/getting-started/tooling/)),
    more traditional "getting started" content
    ([Quick Install](https://www.tremor.rs/getting-started/install/)), and
    high-level conceptual documentation
    ([Understanding Data](https://www.tremor.rs/getting-started/codecs/)).
  - Tremor has ample high-level conceptual overview information, but it's
    somewhat scattered.
    - The
      [Architecture Overview - Goodness of fit](https://docs.tremor.rs/overview/)
      section is probably the best overview for brand new users available,
      however the rest of the page gets very detailed very quickly, and feels
      like a collection of semi-random topics that someone felt the need to
      write down, with no real direction. This page probably needs to be broken
      up into many pages within a section, with some of the more esoteric topics
      currently under "Getting Started" included.
    - Low-level task content often gets buried in
      [Recipes](https://docs.tremor.rs/tremor-query/recipes/), generically
      titled
      [Walkthrough](https://docs.tremor.rs/operations/configuration-walkthrough/)
      pages, and the extremely helpful but poorly named
      [Workshops](https://docs.tremor.rs/workshop/examples/00_passthrough/)
      section. This makes it hard for a user with a specific query ("How do I
      do....") to know, at a glance, if the documentation contains the answer to
      their question or not.
    - [The Configuration walkthrough](https://docs.tremor.rs/operations/configuration-walkthrough/)
      was the single most useful page in terms of understanding the overall
      'picture' of what Tremor is and how a developer needs to move through the
      documentation. But it's buried in the Configuration section, and I
      wouldn't think to look there until I was much further in the process.
    - Given how valuable the
      [Workshops](https://docs.tremor.rs/workshop/examples/00_passthrough/)
      section is, it needs a better name. "Recipes", "Tutorials", "How to" would
      all be clearer.
  - Some specific comments about the Tremor Query and Script sections:
    - The [Tremor Query Overview](https://docs.tremor.rs/tremor-query/) and
      [Tremor Script Overview](https://docs.tremor.rs/tremor-script/) would
      benefit from splitting the "Language" topic and subtopics into their own
      page in the section.
    - As someone unfamiliar with the project, I'm not sure how
      [Tremor Query](https://docs.tremor.rs/tremor-query/) and
      [Tremor Script](https://docs.tremor.rs/tremor-script/) are different based
      purely on their overviews. I'm also not sure if they work together or not,
      or if I as a user need to use one language versus another.
    - Given the complexity of Tremor's various query methods, I think focusing
      on producing more [Recipes](https://docs.tremor.rs/tremor-script/recipes/)
      would be a helpful next step for the team.
  - [History](https://docs.tremor.rs/history/) probably doesn't belong as a
    top-level section.
  - [Development](https://docs.tremor.rs/development/) and
    [Governance](https://docs.tremor.rs/CodeOfConduct/) are targeted at
    contributor users rather than end users, and should be in a different
    section of the site if possible.

- **New user content:** Project Tremor has a Getting Started section but it's a
  little disorganized as mentioned above. This makes it difficult for an actual
  new user to understand.

  - [Starting Tremor for the first time](https://www.tremor.rs/getting-started/starting/)
    is the best part of the getting started guide. However, it mentions that
    there are "many ways" to install Tremor - it would be good to point users to
    documentation on those other ways of installing.
    - Is the
      [Quick developer install](https://www.tremor.rs/getting-started/install/)
      the "other methods"? There's no introduction to this page, so I can't
      tell.
  - [Talking to other systems](https://www.tremor.rs/getting-started/connectivity/)
    is definitely a foundational concept for Tremor (as an events processing
    system) but the style in which it's written isn't appropriate for a getting
    started guide. This goes for
    [Understanding Data](https://www.tremor.rs/getting-started/codecs/),
    [Scripting](https://www.tremor.rs/getting-started/scripting/) and
    [Specialize Tremor Pipelines](https://www.tremor.rs/getting-started/specialize/).
    - These topics should be under Docs in their own section, perhaps titled
      "Overview".
  - Ensuring that all pages, like
    [The Tremor CLI tool](https://docs.tremor.rs/operations/cli/), have
    overviews goes a long way in ensuring the docs are user friendly for new
    users.
  - I like the [FAQ](https://www.tremor.rs/faq/) as new user content, but FAQs
    often become a catch-all and get messy quickly. Proceed with caution.

- **Content maintainability:**
  - Docs are mostly separated into their own repository,
    https://github.com/tremor-rs/tremor-www-docs, though as the maintainers note
    there are many other repositories with docs available in them. These need to
    be consolidated for easier maintenance.
  - The
    [scripts](https://github.com/tremor-rs/tremor-www-docs/tree/main/python_scripts)
    which generate certain pieces of documentation introduce fragility into the
    system: if the location of files, names of repos, etc. change then the
    documentation pipeline breaks. This also makes it hard for non-"technical"
    users to make changes to these pages, and to reorganize these pages if
    needed.
    - The automation benefits might be worth it however.
  - Your
    [release process](https://github.com/tremor-rs/tremor-www-docs/blob/main/RELEASE_PROCESS.md)
    is documented, which is great to see!
  - Your site is only partially searchable, with sections not within /docs
    inaccessible by search.
- **Content creation processes:** There are no content creation processes
  documented and no CONTRIBUTING.md file for the main docs repository. Some
  things to think about:
  - If I write documentation, who verifies that it's technically accurate?
  - If I'm not a native English speaker, who can I ask for help with grammar and
    language review?
  - If I'm a trained technical writer and want to rearrange, create new, or
    split existing topics, who do I tag in issues/on Slack for comment?
  - As a developer user, how can I ensure that content is accurate and up to
    date? Are alpha and beta features flagged? What happens when content (+
    functionality) gets deprecated or changed in a major way?
  - Who reviews documentation PRs?

**Recommendations**

- **Get specific about your Getting Started:** Getting Started Guides contain a
  very predictable set of content:

  - In one paragraph or less, what is this piece of software
  - What prerequisite software (i.e. Kubernetes) and knowledge (i.e., Go) do I
    need on my system to launch this?
    - What special features does this piece of software have that other similar
      software might not have?
  - What are the installation instructions?
  - Is there a sample application I can set up and execute basic commands on?

- **Reorganize the Operations section and retitle it "Using Tremor"**:
  Additionally, move some topics from the existing Getting Started section:

  - [Configuring Tremor](https://docs.tremor.rs/operations/configuration-walkthrough/)
    - [Walkthrough: Configure a Tremor Deployment](https://docs.tremor.rs/operations/configuration-walkthrough/)
  - [Connectivity: Onramps and Offramps](https://www.tremor.rs/getting-started/connectivity/)
    - Link to the [Onramps](https://docs.tremor.rs/artefacts/onramps/) and
      [Offramps](https://docs.tremor.rs/artefacts/offramps/) sections to point
      users to their next steps
    - [Guaranteed Delivery](https://docs.tremor.rs/operations/gdcb/) (Note: this
      topic needs revision as it is not user friendly)
    - [Linked Transports](https://docs.tremor.rs/operations/linked-transports/)
  - Pipelines (Overview: what are pipelines? You can probably borrow content
    from the [Pipeline Model](https://docs.tremor.rs/overview/#pipeline-model)
    section of the architecture overview)
    - Create a Tremor Pipeline
    - [Specialize Tremor Pipelines](https://www.tremor.rs/getting-started/specialize/)
    - [Processing Data](https://www.tremor.rs/getting-started/scripting/)
    - Recipes/Examples
  - [Monitoring](https://docs.tremor.rs/operations/monitoring/)
  - [Tremor CLI Tool](https://docs.tremor.rs/operations/cli/)
  - [IDE Add-ons](https://www.tremor.rs/getting-started/tooling/)

- **Reorganize [Overview](https://docs.tremor.rs/overview/) with
  user-centeredness in mind**: I suggest creating a new section with the
  following content:

  - The information on [The docs index](https://docs.tremor.rs/) as a page
    titled "Overview", with subpages:
    - [Architecture Overview](https://docs.tremor.rs/overview/) (note: you could
      break out each of the "Model" sections into their own subpages)
    - [History](https://docs.tremor.rs/history/), and I would consider titling
      this "Release log" or something similar
    - [Understanding Data](https://www.tremor.rs/getting-started/codecs/)

- **Audit for colloquial/casual language, abbreviations, and other "basic"
  English:**

  - This recommendation has less to do with casual language or abbreviations
    being _bad_, and more to do with the perception of professionalism for the
    project.
  - Review _all_ documentation for the following:
    - Unnecessary abbreviations (for example
      [This page](https://docs.tremor.rs/operations/gdcb/)).
    - Casual language or colloquialisms, for example:
      [A short "canned" synopsis](https://docs.tremor.rs/operations/gdcb/),
      referring to processes either as
      [painless](https://www.tremor.rs/getting-started/starting/#requirements)
      (on the same page: "let's be real"), etc.
    - Spelling, consistent capitalization of titles, consistent verbiage in
      titles:
      - In general when it comes to page titles, prefer either proper nouns
        (Good examples of this:
        [Lexical Preprocessor](https://docs.tremor.rs/tremor-script/pp/),
        [Linked Transports](https://docs.tremor.rs/operations/linked-transports/)),
        or verb-based phrases for tasks (For example,
        [Debugging with LLDB](https://docs.tremor.rs/development/debugging/)).

- **Rename Workshops section to "Recipes" or "Examples"**

- **Create a CONTRIBUTING.md file for the docs repository**

## Contributor documentation

| Criteria                                  | 1   | 2   | 3   | 4   | 5   |
| ----------------------------------------- | --- | --- | --- | --- | --- |
| Communication methods documented          |     |     |     |     | ✅  |
| Beginner friendly issue backlog           |     |     |     |     | ✅  |
| “New contributor” getting started content |     |     |     | ✅  |     |
| Project governance documentation          |     |     |     |     |     |

Scale:

- 1 = (Is not present or requires significant work)
- 3 = (Is present, but needs work)
- 5 = (Is executed extremely well or no improvement required)

**Comments**

- **Communication methods documented:**
  [Contributing](https://docs.tremor.rs/Contributing/) documents these, but it
  would be useful to pull these up to the Community page

- **Beginner friendly issue backlog:** You document your
  [GitHub best practices](https://docs.tremor.rs/Contributing/#pull-requests)
  and
  [issues for new users are tagged](https://github.com/tremor-rs/tremor-www-docs/issues?q=is%3Aissue+is%3Aclosed).
  Great work!

  - One thing to note is when tagging an issue
    [as a good first issue](https://github.com/tremor-rs/tremor-www-docs/issues/101),
    assume the reader knows nothing and must be hand-held through the entire
    process. [This issue](https://github.com/kubernetes/website/issues/28621) is
    a great example of the granularity needed by a first time contributor.

- **New contributor getting started content:** The
  [Community](https://www.tremor.rs/community/) page exists but doesn't provide
  much guidance for new users. How do I get involved? When do community meetings
  happen? Where do I find the team on Discord or Slack?

  - As mentioned above, [Development](https://docs.tremor.rs/development/) and
    [Governance](https://docs.tremor.rs/CodeOfConduct/) are targeted at
    contributor users and should be under the community section.

- **Governance documentation:** Your
  [Project Governance](https://www.tremor.rs/governance/) is linked front and
  center, which I love! The only critique I have is that as a new user I am
  unlikely to click through to all of the links available. It would be lovely to
  distill these documents into something Tremor maintains at some point in the
  future.

**Recommendations**

- **Turn the Community page into a subsection and house Governance and
  Development documentation under it:**

  - Specifically, move [Development](https://docs.tremor.rs/development/) to
    /community/development
  - Move
    [The entire governance section](https://docs.tremor.rs/Contributing/#pull-requests)
    as well
  - Consider moving [RFCs](https://rfcs.tremor.rs/) under community as well

- **Pull meeting and contact information out of contributing and onto the
  community page:** people cannot join a meeting or chat that they can't find a
  link to.

## Website

| Criteria                  | 1   | 2   | 3   | 4   | 5   |
| ------------------------- | --- | --- | --- | --- | --- |
| Branding and design       |     |     |     | ✅  |     |
| Case studies/social proof | ✅  |     |     |     |     |
| Maintenance planning      |     | ✅  |     |     |     |

Scale:

- 1 = (Is not present or requires significant work)
- 3 = (Is present, but needs work)
- 5 = (Is executed extremely well or no improvement required)

**Comments**

- **Branding and design:**

  - On the whole, the Tremor docs site looks professional, if a little plain.
  - Because of the way the site deploys (from different repositories via GitHub
    pages), navigation is disjointed: if you're in the RFCs, you don't have the
    full site's navigation available to you and clicking the logo takes you back
    to https://rfcs.tremor.rs, with similar behavior occurring in the docs. This
    is a bad user experience.
  - Tremor could do a bit more selling on its homepage:
    - What are the project's key features that make it different from other
      event processing systems?
    - Are there any known use cases you can talk about or use the logos of?
    - Have you given any talks on Tremor, or is there other (particularly video)
      content you can showcase about the project?

- **Case studies/social proof**: None available.

  - Case studies/talks/generally showing that other projects use Tremor is one
    of the most powerful ways to gain new users. People love real-world examples
    of a codebase in action.
  - Because Tremor was developed by Wayfair, we suggest documenting Wayfair's
    use as a "use case" or case study, and then adding a section on the website
    which allows people to submit their own use cases!

- **Maintenance planning:** In general, we recommend that projects keep their
  documentation all in one repository. Tremor's documentation currently deploys
  from https://github.com/tremor-rs/tremor-www-main,
  https://github.com/tremor-rs/tremor-rfcs, and
  https://github.com/tremor-rs/tremor-www-docs.
  - It's confusing for contributors who don't know where to file issues, open
    pull requests, or whom to tag for reviews (and where).
  - The easiest fix for this pain point is to deploy everything using and Hugo.
  - In general, we prefer sites to deploy via Netlify.

**Recommendations**

- Develop a case study or other content around Wayfair's use of Tremor, to give
  users a tangible example of what they can do with the project.

- Fix navigation and improve site maintainability by putting all your
  documentation in one repository.

## Recommendations

**Reorganize your documentation's information architecture:** In addition to the
suggestions above, I recommend an overall reorganization of the documentation,
along the lines of:

- Docs
  - Overview (renamed from Architecture Overview)
  - Using Tremor
    - subpages as described previously
  - Artifacts
  - Tremor Query
  - Tremor Script
  - Examples (renamed from Workshops)
  - API Reference
  - Glossary
  - FAQ (moved from top level navigation)

**Reorganize the top-level navigation of the site:** once again, as mentioned
previously:

- Getting Started
- Docs
  - subsections as described above
- Community
  - RFCs
  - Contributing (moved from Docs section)
  - Governance (moved from Docs section)
- Community Chat
- Blog

**Consolidate documentation into one repository, and document contributing
guidelines for users:** I recommend collapsing everything into
https://github.com/tremor-rs/tremor-www-main, barring any technical difficulties
doing so, and switching deploys over to Netlify to access Deploy previews and
other useful features.