Problem:
Currently the web UI's resource autocomplete also lists authorities.
However you can't tap authorities in this way, you have to use --authority
in addition to whatever resource you're trying to tap.
The web UI is confusing as it presents authorities in that list.
Those authorities should instead be moved to the Authority box in the advanced filter form.
Solution:
* Don't present authorities as options in the Resource dropdowns
* Add authority autocomplete to authority form input
Follow up to @kl in #1391 there is an error when we try to tap an authority
Add client side filtering to the tap table, so that we can narrow down
queries while still tapping a whole resource.
There are two general kinds of filters here:
- filters where the number of possible values is bounded/small and
we know them (e.g. inbound/outbound, grpc status). here, I've tried to
hardcode the list of possible options with explanations (see the GRPC status filters)
- filters where the number of possible values can be very large (e.g. paths)
here, I've generated the list of options as we process the incoming data.
I also periodically delete the oldest filter option so the list of filters
doesn't grow unbounded
Filters added:
- GRPC status code filters
- http status filters
- path filters
- scheme filters
- tls, destination and source filters
* Make use of the Web UI to render tap events in a table
- Return JSON tap events instead of the command line output
- Experiment with a different way of rendering the EventList
- changed the default width back to 100% of the screen because this
table does not look great squished
* Update ant to 3.7.2
* Add autocomplete of namespaces/resources to Tap in web ui
* Add form fields for authority/path/method/rps/scheme
* Add the ability to clear error messages to the error banner
* Add error listener to ws object
Speed up incremental rebuilds by avoiding relinking the controller
and/or web executables when changes are made to unrelated files.
Before this change, any time the git tag changed, the executables
would have to be rebuilt (relinked at least), even if no Go files
changed.
Validated by running the integration tests.
Signed-off-by: Brian Smith <brian@briansmith.org>
* Allow docker-build-proxy to override the proxy version
* Update based on review feedback
* fetch-proxy should return full path to executable
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
Adds a tap endpoint in the web api that communicates with the dashboard
via websockets.
I've moved a bunch of code from the cli tap.go into utils so that the code
can be shared between web and CLI. I think we should consider making the
display more suited to web, but in the short term, reusing the CLI's
rendering of tap events works.
Adds a Tap page in the Web UI that you can use to make tap requests.
The form currently only allows you to enter a resource and namespace,
other filters coming in a follow-up branch.
- Remove a conduit image from our img folder
- Add a linkerd favicon, should no longer get the favicon not found console error
- Configure webpack to not hash image names
* This commit adds an application topology graph within the namespace tab. As a developer / operator one would like to see an overview of the services running to identify dependencies. Adding this graph gives Linkerd2 users a good overview of service dependencies.
* networkgraphtest added
Fixes: #924
Signed-off-by: Franziska von der Goltz <franziska@vdgoltz.eu>
* Stop using `installsuffix` when building Go code.
See https://plus.google.com/117192131596509381660/posts/eNnNePihYnK.
`-installsuffix cgo` isn't necessary as of Go 1.10 (where build caching
changed substantially) and it probably wasn't necessary earlier.
Signed-off-by: Brian Smith <brian@briansmith.org>
* update grafana dashboards to remove conduit reference and replace with linkerd instances
* update test install fixtures to reflect changes
Fixes: #1315
Signed-off-by: Franziska von der Goltz <franziska@vdgoltz.eu>
This PR adjusts the colour of a popup in the sidebar, as well as removes
references to conduit in the frontend test fixtures.
All that's left in the Web UI code now is a few references to the conduit sites / githubs,
as well as the CLI name.
* Remove a touch of conduit blue from the sidebar popup
* Remove minor references to conduit throughout the web code
* Fully colour the sidebar in new bg colour
This PR begins to migrate Conduit to Linkerd2:
* The proxy has been completely removed from this repo, and is now located at
github.com/linkerd/linkerd2-proxy.
* A `Dockerfile-proxy` has been added to fetch the most-recently published proxy
binary from build.l5d.io.
* Proxy-specific protobuf bindings have been moved to
github.com/linkerd/linkerd2-proxy-api.
* All docker images now use the gcr.io/linkerd-io registry.
* `inject` now uses `LINKERD2_PROXY_` environment variables
* Go paths have been updated to reflect the new (future) repo location.
This PR starts removing all references to the word "Conduit" in the web UI.
In the interest of not making huge changes all at once, I'll gradually start moving away
from the usage of "conduit" in the Web UI. For example, there are a lot of components that
have conduit in their names but they don't need to.
This branch is mostly component / variable names. There should be no visible changes except
the spinner is no longer a Conduit spinner.
See #1262 for visible branding changes.
- Rename ConduitLink to PrefixedLink
- Remove ConduitSpinner in favour of antd.Spin
- Remove css classnames that are conduit- centered
- Parameterize the current Product Name so that it's easier to change in the future
Tracking ticket: linkerd/linkerd#2018
- Add Reason to the error data passed from the api
- Rewrite error logic in the UI to try to make it clearer
- Show 0/0 pods meshed instead of 0/0 pods meshed (N/A) if 0 pods are meshed
Create a ephemeral, in-memory TLS certificate authority and integrate it into the certificate distributor.
Remove the re-creation of deleted ConfigMaps; this will be added back later in #1248.
Signed-off-by: Brian Smith brian@briansmith.org
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
- Return pod uptimes from the GetPods endpoint
- Adds filtering by namespace to api.GetPods
- Adds a --namespace filter to conduit get pods
- Adds pod uptimes to the controller component toolitps on the ServiceMesh page
- Moves the ServiceMesh page back to using /api/pods
Adds the ability to query by a new non-kubernetes resource type, "authorities",
in the StatSummary api.
This includes an extensive refactor of stat_summary.go to deal with non-kubernetes
resource types.
- Add documentation to Resource in the public api so we can use it for authority
- Handle non-k8s resource requests in the StatSummary endpoint
- Rewrite stat summary fetching and parsing to handle non-k8s resources
- keys stat summary metric handling by Resource instead of a generated string
- Adds authority to the CLI
- Adds /authorities to the Web UI
- Adds some more stat integration and unit tests
Add Sidebar links to Pods, Deployments, and Replication Controllers
In #1016 we removed the sidebar links to individual resource pages in favour of a namespace
page that lists all resources. These resource pages require no additional code so they're still
in our UI (accessible under /pods, /deployments etc), just not easily findable. I find them
useful to check when in development mode, or when debugging something, so I'd like to
re-add links.
If we don't want them in permanently, we can gate them behind `NODE_ENV=development`
* Add CA certificate bundle distributor to conduit install
* Update ca-distributor to use shared informers
* Only install CA distributor when --enable-tls flag is set
* Only copy CA bundle into namespaces where inject pods have the same controller
* Update API config to only watch pods and configmaps
* Address review feedback
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Add controller admin servers and readiness probes
* Tweak readiness probes to be more sane
* Refactor based on review feedback
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
Don't allow the CLI or Web UI to request named resources if --all-namespaces is used.
This follows kubectl, which also does not allow requesting named resources
over all namespaces.
This PR also updates the Web API's behaviour to be in line with the CLI's.
Both will now default to the default namespace if no namespace is specified.
- If error messages are very long, truncate them and display a toggle to show the full message
- Tweak the headings - remove Pod, Container and Image - instead show them as titles
- Also move over from using Ant's Modal.method to the plain Modal component, which is a
little simpler to hook into our other renders.
* Display proxy container errors in the Web UI
Add an error modal to display pod errors
Add icon to data tables to indicate errors are present
Display errors on the Service Mesh Overview Page and all the resource pages
Add an emitWarning to the webpack config so that webpack will compile despite lint
errors when running in development mode. This is necessary to enable development
on the frontend using webpack-dev-server's automatic reloading.
Also sets a NODE_ENV in travis.yml so that the build will fail if linting fails.
* Update destination service ot use shared informer instead of custom endpoints informer
* Add additional tests for dst svc endpoints watcher
* Remove service ports when all listeners unsubscribed
* Update go deps
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
Common blacklists have `/api/stat` in them. This causes the dashboard to not load.
`/api/tps-reports` is not in any blacklists, suggests what this route does and is slightly tongue in cheek. Fixes#970
* Display font-awesome icons no matter what URL is originally loaded
The URLs in the dashboard need to be relative. Unfortunately, this means that if
you load something that isn't the base route ... font-awesome icons look broken.
There's no real way to solve this from within webpack (or the web server without
some work). Instead, just load font-awesome from a CDN as there's no real
benefit we get from including it in the bundle. Fixes#1019.
* Moving font-awesome to styles
* Web: remove ns column from tables on individual ns page
* Add prop types and tests for MetricsTable component
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Add propType validation
When refactoring components, it is hard to know what is required and isn't.
Adds propTypes to the existing components and enables eslint errors for anything
moving forward. This should keep us documenting the API for components.
* Remove extra newline
On the individual namespace pages, the filter should not be shown, as all results that appear on the that page will be for on namespace.
Added a boolean property, showNamespaceFilter, to MetricsTable that allows you to define if the filter should be shown.
Tested that the filter is not shown on namespace pages.
Fixes#972
Signed-off-by: Kim Christensen <kimworking@gmail.com>
As part of the HOC + Context merges, ResourceList missed out on the api injection and errors out on the Namespaces tab.
Wrap the returned HOC in `withContext` to make sure it is there, no matter where it is in the tree. (Fixes#1034)
* Add a HOC for the REST API tooling
We're copying and duplicating logic all over the place with components that need to talk to the API.
Moves most of the REST API tooling into a HOC that can be used by other components. Now, a component can use `withREST`, pass in the promises that it would like resolved and receive the responses as props.
* Show PageHeader whether there's an error or not
* Hiding page header during loading
* Test updates to work with namespace restructuring
In an effort to highlight the namespace overview pages, remove the Deployments,
Replication Controllers and Pods items from the sidebar and replace them with direct
links to individual Namespace pages. If the user has more than 8 namespaces, only
list the first 8 (the rest can be accessed by the namespace list page).
The Deployments/RCs/Pods endpoints are still available if you go directly to
/deployments, /pods, etc. but they're not highlighted to the user.
Previously, we would filter out stats coming from Conduit itself and from the kube-*
namespaces on some views in the Web UI. Remove this filtering, so that we display
all the resource information we get back from the Stat API. (Fixes#997)
On the Resource pages, the call to action would show up when there were no
metrics present, but that's actually not actionable by the user. Instead, I'm
going to show a blank table with a "no s detected" message.
* Remove special-case filtering out of kube-* namespaces, and conduit namespaces
* Remove the call to action for no metrics
* Linkify the namespace column for the resource pages
* Add an app-wide context for global props.
We've been passing the `api` object down from the top of the react tree. With
16.x, there's now the ability to have context that can inject anywhere in the
tree. This creates a top level context provider that contains most of the global
variables we've been using (api, appData, ...). It subsequently cleans up some
of the routes and nested components.
- Bumps `react-dom` to 16.3.2 (to match `react`).
- Adds `enzyme-context-patch` for now. This is fixed in enzyme master, but there
has not been a release yet. Needs to be removed when that is fixed.
* Use a default inside appData for controllerNamespace
* Update syntax of if to use curly brackets
- Update the `response_total` prometheus query of the StatSummary endpoint to also
break queries out by a `meshed` label.
- Add a 'Secured' column to the web UI/CLI stat displays, which indicate the percentage of traffic
starting and ending in the mesh
This meshed label is used in the CLI/Web UI to display a column of the percentage of traffic that
starts/ends in the mesh. (Which is a proxy indicator for whether that traffic is 'secured' when we
add TLS by default for intra mesh requests).
The `meshed` label is not yet added anywhere, so until it is supplied by the proxy, all traffic will
show up as 0% secured in the web/CLI.
- Switched from `es2015` to `env` for the default preset. This is the recommended preset and allows us to track the latest and greatest moving forward.
- Added `react-app` as a preset. We get class properties (and thus => for context) as well as the current recommended settings for react apps.
- Created a `web` script that provides functions for common tasks. `react-app` requires that BABEL_ENV/NODE_ENV is set and this guarantees it.
- Updated the web dockerfile to set NODE_ENV correctly and use `bin/web`.
- Moved the babel related modules over to devDependencies.
Debugging issues in the dashboard is a little frustrating without source maps and the full source map takes awhile to build.
Just enables one of the cheaper source maps by default. It is good enough (tm) for what is there now.
This PR modifies the Namespace page in the web UI to replace the 3 existing api calls
with a single call.
* Consolidate calls to /metrics to use the new resource type all
* Simplify urlsForResource, add comment with assumptions
Problem
If you navigate directly to (or do a hard refresh on) a path with more than one segment,
e.g. http://localhost:8084/namespaces/conduit, the dashboard js is not served.
Pages with two paths have to be accessed by loading the dashboard on a different
path and then clicking through.
When accessing the dashboard via conduit dashboard we append a path prefix so that
we can connect using the k8s proxy. This means that moving the dashboard to serve
images off relative paths won't work, because we need to serve images whether the
dashboard is loaded from http://localhost:8084/namespaces/conduit or
from http://localhost:8084/namespaces.
Solution
Check whether we're serving the dashboard with the proxy url, and if we are, adjust
the url at which we serve the index bundle from.
I've also added a very manual override if the conduit logo can't be found at the usual url.
This enables the removal of the inline-block display for links and
fixes menu items not showing up when sidebar is expanded on firefox
Problem
Previously we were linking the icon and expand text of the menu bar separately.
This caused the clickable areas of the menus to be inconsistent, which we were
fixing via css. This wasn't consistently displayed across browsers.
Fix
Linkify the whole Menu Item rather than linking the icon and text separately.
This enables the removal of the inline-block display for links and
fixes menu items not showing up when sidebar is expanded on firefox.
Additionally it makes the clicking of menu links way more consistent.
The frontend assets was not optimized, resulting in suboptimal page load times.
Enabled webpack production mode in the Dockerfile, this still allows good development
and debugging experience when running the web interface locally during development.
Also added minification of the CSS handled by css-loader.
The web interface still works as expected.
The size of the JS file has been reduced from 3.6 MB to 1.2 MB.
And the CSS minification has resulted in sidebar.css from 5.71 kB to 4.33 kb,
and styles.css from 4.18 kB to 3.1 kB.
Fixes#378
Signed-off-by: Kim Christensen <kimworking@gmail.com>
* Fix issue where we were waiting for the next polling interval when switching tabs
Fix issue where we were waiting for the next polling interval when switching tabs.
When we switch tabs, we update the Props of the ResourceList component, but we weren't
resetting how we poll the server. This meant we'd wait until the end of the current polling interval
(2s) to get the data for the tab we just switched to.
I've added stopServerPolling and startServerPolling methods so that we can cancel the resource
requests of the page we're leaving and immediately start polling for new data if the resource type
changes.
The way that git-related version information is linked into go binaries
busts Docker's cache such that every commit causes all binaries to
rebuilt.
In order to ameliorate this, we can build each binary once without
version information first so that its artifacts are cached. When Go
sources are not changed and only the version information changes, builds
are 4.3x faster than before (from 5+ minutes to <90s).
On `master`
Branch off of master and build (mostly cached):
```
:; time DOCKER_TRACE=1 bin/docker-build
...
DOCKER_TRACE=1 bin/docker-build 9.10s user 6.30s system 5% cpu 4:26.47 total
```
Rebuild without changing anything (highly cached):
```
:; time DOCKER_TRACE=1 bin/docker-build
...
DOCKER_TRACE=1 bin/docker-build 9.23s user 6.04s system 47% cpu 32.017 total
```
Update only the git sha and rebuild:
```
:; git ci -am 'bump it' --allow-empty
[ver/eg 2749eb3] bump it
:; time DOCKER_TRACE=1 bin/docker-build
...
DOCKER_TRACE=1 bin/docker-build 8.55s user 6.08s system 4% cpu 5:22.25 total
```
On this branch:
Rebuild without changing anything (highly cached):
```
:; time DOCKER_TRACE=1 bin/docker-build
...
DOCKER_TRACE=1 bin/docker-build 8.94s user 5.97s system 46% cpu 32.257 total
```
Update only the git sha and rebuild:
```
:; git ci -am 'bump it' --allow-empty
[ver/go-docker-cache-versionless 77a80b5] bump it
:; time DOCKER_TRACE=1 bin/docker-build
...
DOCKER_TRACE=1 bin/docker-build-cli-bin 2.02s user 1.34s system 9% cpu 34.144 total
```
* Turn the status bars red if there exist failed pods in the namespace
* Also use failed pods in conduit component table
Now that the API returns the number of failed pods, use this info to indicate failed pods in
the ServiceMesh page.
The bars will turn red if there are any failed pods present in the namespace.
They'll be green if they have non-zero pods meshed, and grey otherwise.
Add namespaces as a top level resource in the Web UI
This PR does the following:
- Replace the deployments table in the service mesh page with namespaces
- Add a Namespaces index page that lists all namespaces and their stats
- Add an individual namespace page showing all resources for that namespace
- Make the incomplete mesh message more generic to any resource type
- Revamp rest of service mesh page to move off ListPods
Make the sidebar icon based and collapsed by default
I had to move the call to version check into the sidebar component, indicator
when the sidebar was minimized if there was a conduit update.
Currently I just have letters representing the icons for Deployments, RCs and Pods,
but we can change this in the future.
* Modify the Stat endpoint to also return the count of failed pods
* Add comments explaining pod count stats
* Rename total pod count to running pod count
This is to support the service mesh overview page, as I'd like to include an indicator of
failed pods there.
Enables filtering by one or more namespaces. Table updates are prevented
when the filter menu is open, as table updates will rerender the menu,
unselecting anything the user has selected but not confirmed.
* Add a namespace column to the metrics tables, support long resource names
* Add a test for GrafanaLink
* Change the PodList.jsx component to not use the ListPods api
We removed individual Deployment pages a while ago, but left the autocomplete search bar in. Clicking on searches goes to a 404 because we don't have /deployment any more.
This will be revisited in the future with direct links to grafana dashboards to all the
resources we support.
* Add a Replication Controllers page in the Web UI
@siggy pointed out that we don't need to use the PodsList api any more, since the new stats endpoint (#671) includes meshedPodCount and totalPodCount, which is all we need to determine whether the deployment/rc has been added to the mesh (which is what we were using ListPods to determine).
This PR modifies deployments to not use the pods api any more, and adds a Replication Controllers page. This page is quite similar to the Deployments page in logic, so I've made a PodOwnersList component to share the code.
I haven't added Replication Controllers to the Service Mesh page yet, because that page does require a list of component pods. Also, we don't need the calls to Prometheus for the Service Mesh page, so I don't want to use the existing stat apis for it. I figure that is a large enough change for a separate PR.
After this was implemented we found that ExternalName services are
represented in DNS as CNAMEs, which means that the proxy's DNS
fallback logic can be used instead of doing DNS in the control
plane. Besides simplifying the controller, this will also increase
fidelity with the proxied pods' DNS configuration (improve
transparency).
Signed-off-by: Brian Smith <brian@briansmith.org>
The `conduit tap` command is now deprecated.
Replace `conduit tap` with `connduit tapByResource`. Rename tapByResource
to tap. The underlying protobuf for tap remains, the tap gRPC endpoint now
returns Unimplemented.
Fixes#804
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
public-api and and tap were both using their own implementations of
the Kubernetes Informer/Lister APIs.
This change factors out all Informer/Lister usage into the Lister
module. This also introduces a new `Lister.GetObjects` method.
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
The TapByResource endpoint was previously a stub.
Implement end-to-end tapByResource functionality, with support for
specifying any kubernetes resource(s) as target and destination.
Fixes#803, #49
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
The Destination service does not provide ReplicaSet information to the
proxy.
The `pod-template-hash` label approximates selecting over all pods in a
ReplicaSet or ReplicationController. Modify the Destination service to
provide this label to the proxy.
Relates to #508 and #741
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* Expose pod stats in CLI, web UI, and Grafana
* Fix js api helpers test
* Add outbound traffic stats to pod dashboard
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
The public-api previously only permitted 4 hard-coded time windows:
10s, 1m, 10m, 1h. This was primarily a relic of the recently removed
telemetry system.
Modify the public-api to validate the time string, but allow for any
window size, which is then passed through to Prometheus.
Fixes#686
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* Add namespace as a resource type in public-api
The cli and public-api only supported deployments as a resource type.
This change adds support for namespace as a resource type in the cli and
public-api. This also change includes:
- cli statsummary now prints `-`'s when objects are not in the mesh
- cli statsummary prints `No resources found.` when applicable
- removed `out-` from cli statsummary flags, and analagous proto changes
- switched public-api to use native prometheus label types
- misc error handling and logging fixes
Part of #627
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* Refactor filter and groupby label formulation
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Rename stat_summary.go to stat.go in cli
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Update rbac privileges for namespace stats
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
Conduit was relying on apps/v1 to Deployment and ReplicaSet APIs.
apps/v1 is not available on Kubernetes 1.8. This prevented the
public-api from starting.
Switch Conduit to use apps/v1beta2. Also increase the Kubernetes API
cache sync timeout from 10 to 60 seconds, as it was taking 11 seconds on
a test cluster.
Fixes#761
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* Remove the telemetry service
The telemetry service is no longer needed, now that prometheus scrapes
metrics directly from proxies, and the public-api talks directly to
prometheus. In this branch I'm removing the service itself as well as
all of the telemetry protobuf, and updating the conduit install command
to no longer install the service. I'm also removing the old version of
the stat command, which required the telemetry service, and renaming the
statsummary command to stat.
* Fix time window tests
* Remove deprecated controller scrape config
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
The new StatSummary endpoint was only providing request volume and
successs rate information.
Add support for retrieving latency stats via StatSummary. Also make
all prometheus calls in parallel, and implement kubernetes test
fixtures.
Fixes#681
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* Switch public API to use cached k8s resources
* Move shared informer code to separate goroutine
* Fix spelling issue
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
The Grafana dashboards key off of deployment, but had no awareness of
namespaces, causing incorrect metrics aggregation and display.
This change makes the Grafana dashboards key off of namespaces, and also
modifies the Grafana links in the Conduit dashboard to link to
namespace+deployment.
Fixes#704
Part of #420
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
The new statsummary command accepted friendly k8s names, which worked
for k8s queries, but Prometheus requires a specific key.
Modify the statsummary query to map friendly k8s names to canonical k8s
names when constructing the query. Then during the query, map the
canonical k8s name to a specific Prometheus label.
Fixes#695
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* Link to Grafana from Conduit Dashboard
Previously the only way to access the Grafana dashboards was via direct
link, provided by the `conduit dashboard` command.
Add Grafana links throughout the Conduit Dashboard, next to all
Deployment objects. This change also modifies the behavior of the
ConduitLink helper, to enable linking to other deployments proxied by
the `conduit dashboard` command.
Part of #420
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* review feedback
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* review feedback, fix console, remove absolute
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
Start implementing new conduit stat summary endpoint.
Changes the public-api to call prometheus directly instead of the
telemetry service. Wired through to `api/stat` on the web server,
as well as `conduit statsummary` on the CLI. Works for deployments only.
Current implementation just retrieves requests and mesh/total pod count
(so latency stats are always 0).
Uses API defined in #663
Example queries the stat endpoint will eventually satisfy in #627
This branch includes commits from @klingerf
* run ./bin/dep ensure
* run ./bin/update-go-deps-shas
* fix pod status and count display in control plane dashboard section:
- the control plane would show terminated and stale deployments in the UI, this is confusing and might indicate errors
- this filters out temrinated and failed component deploys from the UI
- it is to note that pending deploys will still be counted and represented with a greyed out status dot
- Fixes: #606
Signed-off-by: Franziska von der Goltz <franziska@vdgoltz.eu>
Signed-off-by: Franziska von der Goltz <franziska@vdgoltz.eu>
remove toggle sorting functionality from TableComponent:
- tables displaying metrics allowed to toggle between being sorted and unsorted when clicking the same button. This was confusing behavior for the user.
- this PR removes the toggle functionality and introduces a BaseTable Component that extends antd's component without the capability to toggle
- Fixes: #566
Signed-off-by: Franziska von der Goltz <franziska@vdgoltz.eu>
* Add tests/utils/scripts for running integration tests
Add a suite of integration tests in the `test/` directory, as well as
utilities for testing in the `testutil/` directory.
You can use the `bin/test-run` script to run the full suite of tests,
and the `bin/test-cleanup` script to cleanup after the tests.
The test/README.md file has more information about running tests.
@pcalcado, @franziskagoltz, and @rmars also contributed to this change.
* Create TEST.md file at the root of the repo
* Update based on review feedback
* Relax external service IP timeout for GKE
* Update TEST.md with more info about different types of test runs
* More updates to TEST.md based on review feedback
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
Shortly after conduit is installed in k8s environment. The control plane component that establishes a watch endpoint with k8s run in to networking issues during proxy initialization. During failure, each watcher fails to retry its connection to k8s watch endpoint which leads to timeouts and eventually, multiple controller pod restarts.
This PR adds retry logic to each "watch" enabled package.
fixes#478
Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
* Use Go 1.10.0 to build Go components.
Take advantage of the new build cache in Go 1.10. Future work on improving
build performance will utilize the build cache further.
Signed-off-by: Brian Smith <brian@briansmith.org>
Previously Dockerfile-go-deps was converted from a multi-stage Dockefile
to a single-stage Dockerfile in anticipation of enabling efficient use
of `--cache-from` in CI. However, that resulted in the image ballooning
in size because it contained the Git repo for every package downloaded
by `dep ensure`.
Bring the image back down to the proper size by removing the temporary
files created.
Signed-off-by: Brian Smith <brian@briansmith.org>
* Display more decimal points for truncated numbers, add hover info
* Filter completed pods out of web UI
* Decrease the polling interval from 10s to 2s
* Add more detailed pod categorization based on status
* Tweak filtering of pods, tweak explanations in status table
- reduce row spacing on tables to make them more compact
- Rename TabbedMetricsTable to MetricsTable since it's not tabbed any more
- Format latencies greater than 1000ms as seconds
- Make sidebar collapsible
- poll the /pods endpoint from the sidebar in order to refresh the list of deployments in the autocomplete
- display the conduit namespace in the service mesh details table
- Use floats rather than Col for more responsive layout (fixes#224)
On my system (i9-7960x running Docker natively in Linux) this regularly saves
over 11 seconds of build time when a file under pkg/ changes and over 1.5
seconds of build time when a file under controller/ changes. Since most
contributors are running Docker in a VM on less powerful computers, the
savings for most contributors should be significantly greater.
I imagine the savings for web/ and cli/ and proxy-init/ are similar, but I
did not measure them.
Signed-off-by: Brian Smith <brian@briansmith.org>
Previously Dockerfile-go-deps would run `dep ensure` whenever anything in the
source tree changed. Also, because it was a multi-stage Dockerfile it did not
work well with Docker's `--cache-from` feature.
Change Dockerfile-go-deps to only re-run `dep ensure` when Gopkg.{toml,lock}
and/or bin/dep change. Simplify it to a single stage so that it works better
with Docker's `--cache-from` feature.
Signed-off-by: Brian Smith <brian@briansmith.org>
bin/dep verifies the digest of the `dep` downloaded `dep` executable,
whereas previously Dockerfile-go-deps wasn't.
Signed-off-by: Brian Smith <brian@briansmith.org>
UI cleanups. Remove repetitive labels in the UI, remove unused elements,
remove graphs until we improve their utility.
- remove “Deployment” from the headers of the Deployment Detail Page
- remove Routes in sidebar
- kill leftmost 100px of sidebear
- remove word controller from service mesh page first table
- add twitter and GitHub and slack links
- kill the graphs, replace with one large header (request rate, success rate, latency top bar)
put upstream/downstream diagram before upstream downstream tables
* Clean up DeploymentList page (#321)
- remove "Most active deployments" graphs from the Deployments List page
- remove the scatterplot sections of the page as I don't think we'll be using them for a while
Follow-up from #315.
Now that the UIs don't report per-path metrics, we can remove the path label from Prometheus, the path aggregation and filtering options from the telemetry API, and the path field from the proxy report API.
I've modified the tests to no longer expect the removed fields, and manually verified that Conduit still works after making these changes.
Closes#265
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
I've removed per-path metrics from the web dashboard and from the `conduit stat` command.
Manually validated that these metrics are no longer displayed.
Closes #263
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Prometheus queries from the Telemetry service were taking seconds or 10s
of seconds.
Optimize these queries:
- Move all summary queries requiring a single point data off of Prometheus'
QueryRange() endpoint, onto Query()
- Set `defaultVectorRange` to 30s, and also use it regardless of time
window
Also add tests for grpc_server and telemetry server
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
Fixes#260
This PR updates the web UI to remove the pod detail page, and to remove the links to that page from pod names in metrics tables. It also removes the `pods` option from `conduit stat`, and the `sourcePod` and `targetPod` fields from the controller API proto's `MetricMetadata` message.
I've updated the `conduit stat` tests to reflect these changes, and manually verified the web UI changes.
Closes#261
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
* Consolidate latency, success, request metrics into one tab
on the SortableMetricsTable
- removes sparklines from the table
- makes tables sortable by default
- move pod table in DeploymentDetail to its own row
* remove request distribution column, reorder columns
* Control metricsWindow from root of app
- Add buttons [currently hidden] on metrics pages to control window of metrics requests
- Consolidate metricsWindow usage (stop passing it around)
- Add a ConduitLink component so we can stop passing around pathPrefix
- Add tests for ApiHelpers
* Hide the time window buttons; fix bug in absolute links
* Add a note explaining why metricWindow buttons are disabled
* Convert ConduitLink in to a component that wraps another
The conduit dashboard command asychronously shells out and runs "kubectl
proxy".
This change replaces the shelling out with calls to kubernetes proxy
APIs. It also allows us to enable race detection in our go tests, as the
shell out code tests did not pass race detection.
Fixes#173
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* Add `bin/dep` which fetches a fixed version of `dep` to be used.
* Upgrade from dep 0.3.1 to 0.4.1
* Fix inconsistent Gopkg.lock by checking in the result of `bin/dep ensure`
Signed-off-by: Alex Leong <alex@buoyant.io>
The conduit.io/* k8s labels and annotations we're redundant in some
cases, and not flexible enough in others.
This change modifies the labels in the following ways:
`conduit.io/plane: control` => `conduit.io/controller-component: web`
`conduit.io/controller: conduit` => `conduit.io/controller-ns: conduit`
`conduit.io/plane: data` => (remove, redundant with `conduit.io/controller-ns`)
It also centralizes all k8s labels and annotations into
pkg/k8s/labels.go, and adds tests for the install command.
Part of #201
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
We added basic prometheus instrumentation, but this only encapsulated basic go metrics and
request counts. This adds latency and response size metrics exporting as well, to the
public-api server, theweb server and the telemetry server.
Since the util function in grpc.go was basically used to wrap the server creation in a prometheus handler, I added the other prometheus constants in there and renamed the file to prometheus.go.
- Add request duration and response size instrumentation to web and public api
- Also add latency monitoring to telemetry service requests
- Rename util/grpc.go to util/prometheus.go
* Various small UI naming tweaks
- align top two tables in the service mesh page
- "All Deployments" -> "Deployments"
- reorder latency p50, p95, p99
- "Current success" -> "Success rate"
* Add margin to incomplete mesh message, reorder latency in TabbedMetricsTable
* Right align numbers in service mesh page
* Set conduit version to match conduit docker tags
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Remove --skip-inbound-ports for emojivoto
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Rename git_sha => git_sha_head
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Switch to using the go linker for setting the version
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Log conduit version when go servers start
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Cleanup conduit script
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Add --short flag to head sha command
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Set CONDUIT_VERSION in docker-compose env
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Switch to ant sider/content Layout modules, to help style sidebar
This fixes the problem of the sidebar not extending all the way on long pages.
* Fix a bug where the autocomplete options weren't being reset when an item was selected
* Rename components to clarify component relationships
* Rename Deployment to DeploymentDetail to match PodDetail
* Rename Deployments to DeploymentsList to clarify which page this is
* Rename StatPane to ResourceMetricsOverview to be a less generic name
* Rename HealthPane -> ResourceHealthOverview
* Rename StatPaneStat -> ResourceOverviewMetric
Signed-off-by: Risha Mars <mars@buoyant.io>
Previously, proxy-deps and go-deps included the source tree for local
projects. This can cause build conflicts when files are renamed.
By adopting a multi-stage build for the proxy-deps image, we can be sure
that we only preserve essential dependencies & manifests in the
proxy-deps and go-deps images.
Furthermore, `bin/update-go-deps-shas` and `bin/update-proxy-deps-shas` have
been added to ease maintenance when files are changed.
Fixes#159
Signed-off-by: Oliver Gould <ver@buoyant.io>
* Use more descriptive CSS variable names (#135)
Signed-off-by: Nathan Murthy <drip.dr0p.dr0p@gmail.com>
* Decouple the names of the colors from their use
Signed-off-by: Nathan Murthy <drip.dr0p.dr0p@gmail.com>
* Rename a few more root CSS color var names
Signed-off-by: Nathan Murthy <drip.dr0p.dr0p@gmail.com>
* Update more CSS color vars referenced elsewhere
Signed-off-by: Nathan Murthy <drip.dr0p.dr0p@gmail.com>
* Add autocomplete search bar back into sidebar
Signed-off-by: Risha Mars <mars@buoyant.io>
* Use getPodsByDeployment
Signed-off-by: Risha Mars <mars@buoyant.io>
* Revive scatterplot: re-add scatterplot to Deployments page
Tried to make some UI improvements to address previous problems:
* added a hover bar and tooltip that displays all of the nodes under the bar,
in descending order of successRate (to correspond with their order in the chart)
* the tooltip looked weird in the empty state so I also added the max/min latencies
observed there
Also cleans up the Deployments page a little when there are not any "least healthy deployments".
* Previously, the sidebar tooltip would still render the last
highlighted nodes' information when the dots updated. Fix that
by selecting a datapoint to highlight when the dots update.
* Add overlay tooltip with names of highlighted nodes
* Align the node labels with the node, except in cases of label overlap
Signed-off-by: Risha Mars <mars@buoyant.io>
* Use stdout as writer for tap command
fixes#136
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Add --log-level to command line
Signed-off-by: Phil Calcado <phil@buoyant.io>
Our build instructions were scattered across a few README's.
This consolidates all instructions relevant to Conduit development into
a single BUILD.md.
Fixes#134
Signed-off-by: Andrew Seigner <andrew@sig.gy>
* Abstract Conduit API client from protobuf interface to add new features
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Consolidate mock api clients
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Add simple implementation of healthcheck for conduit api
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Change NextSteps to FriendlyMessageToUser
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Add grpc check for status on the client
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Add simple server-side check for Conduit API
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Fix feedback from PR
Signed-off-by: Phil Calcado <phil@buoyant.io>
* add ability to add flash to latest datapoint on line graph based on passed down prop.
- added flash to all graphs except the sparklines
Signed-off-by: Franziska von der Goltz <franziska@vdgoltz.eu>
* Add /paths page that shows rollup metrics by path
* Clean up ApiHelpers a bit
Adds ability to sort by column in the tabbed metrics table (to make a TabbedMetricsTable sortable, set sortable={true})
Adds a page, accessible via /paths that shows a table of all paths, with their request/success/latency metrics. I haven't exposed it in the sidebar as it doesn't have design treatment.
* Sort imports
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Upgrade k8s.io/client-go to v6.0.0
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Make k8s store initialization blocking with timeout
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
* Upgrade ant to 3.1.0
* Adjust styles for updated ant
- Locale is enUS by default now, so removing our config
- Adjust table styles
* Upgrade react and react-dom to 16.2.0
* Upgrade enzyme to 3.3.0, fix tests accordingly
The existing startup/shutdown log info messages had spacing issues and
used fmt.
Update the log messages to use logrus for consistency, and fix spacing
issues.
Signed-off-by: Andrew Seigner <andrew@sig.gy>
The image tags for gcr.io/runconduit/go-deps and
gcr.io/runconduit/proxy-deps were not updating to account for all
changes in those images.
Modify SHA generation to include all files that affect the base
dependency images. Also add instructions to README.md for updating
hard-coded SHAs in Dockerfile's.
Fixes#115
Signed-off-by: Andrew Seigner <andrew@sig.gy>
The web README.md mentions running locally, but it's not explicit about
how to connect to public-api.
Add notes on various ways to connect a local web to a public-api
process.
Signed-off-by: Andrew Seigner <andrew@sig.gy>
* Rename constructor functions from MakeXyz to NewXyz
As it is more commonly used in the codebase
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Make Conduit client depend on KubernetesAPI
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Move Conduit client and k8s logic to standard go package dir for internal libs
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Move dependencies to /pkg
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Make conduit client more testable
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Remove unused config object
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Add more test cases for marhsalling
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Move client back to controller
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Sort imports
Signed-off-by: Phil Calcado <phil@buoyant.io>
* Make TabbedMetricsTable in charge of fetching timeseries
All of the parent components in charge of fetching timeseries data
don't actually use them, but pass them to this table. It would simplify
a lot of the parents if this component handled the ts fetching too.
This introduces a urlsForResource helper in ApiHelpers to allow
us to consolidate url generating in the app to one place. It also
associates the appropriate groupBy for various fetches.
Signed-off-by: Risha Mars <mars@buoyant.io>
Previously Conduit would render an iframe, received from
versioncheck.conduit.io.
Modify the client to retrieve the latest released version, via CORS.
Signed-off-by: Andrew Seigner <andrew@sig.gy>
The README.md in the Web project describes building and running the Web
service locally, against a public API server in Docker.
Add instructions for running the Web Service locally against a public
API Server in Kubernetes.
Signed-off-by: Andrew Seigner <andrew@sig.gy>
* Have better empty states in Deployment Detail and Pod Detail
* Fix call to action on deployment detail page
Adds various checks to hide sections of pages that don't have data:
* Removes the timeseries graphs and latency overview from Deployment/Pod
detail if there are no metrics for it
* Removes the Upstreams/Downstreams if num upstreams / downstreams is 0
* Removes the pod barcharts from the Deployment detail if there are no pods
* If there are no pod metrics, populate the list of pods from /pods
These changes strip the pod detail view down a lot if there's no pod data, but that might be fine
* fix inconsistent deployment count on servicemesh page.
* tests added for deploy count messaging on servicemesh page
* refactored code for Call To Action component to use 'instructions' in util
* refactored correlating css
* Remove integration test clause from travis file
* Use correct channel for asyn process error reporting
* Fix test for sorting stat keys
* Make integration tests only run if CONDUIT_INTEGRATION_TESTS_ENABLED is set
* Fix timeouts in tests
* Fix handler test check for version
* Removes smoke test that required kubectl as not present on travis
* Replace tail with sleep to avoid leaking subprocesses during tests
* Fix typo & extract constant
* fix sidebar highlighting when dashboard is opened via cli:
- took path prefix into account
* addressed review feedback: using pathprefix
* addressed review feedback:
- revert to using this.props.location
* updates sidebar, initial progress indicator
* Update web README
* Revamp metrics processing in MetricUtils
- Try to improve documentation of functions
- Remove nested data from processed metrics - they were often duplicated data
- Split out timeseries processing from rollup processing
- Remove the 'rollup' level of nesting
- Limit the number of timeseries we fetch on the deployments page
Fix various loose ends in the web app:
* Add Tooltip to status dots explaining color meaning
* Adjust formatting of RPS numbers
* Pass in current success rate to the HealthPane
* Fix page header styling caused by content being in wrong div
* Align the metric value for inbound SR in Health Pane
* Status Dot tweaks
* Improve rendering of status dots when there are a lot of them
* Add support for path stats in cli and web api
The cli stat command supports grouping by pod and deployment. With this
change, it will also support grouping by path, in order to facilitate a
summary stats per individual endpoint.
* Right-align numeric columns in stat output
We’ve built Conduit from the ground up to be the fastest, lightest,
simplest, and most secure service mesh in the world. It features an
incredibly fast and safe data plane written in Rust, a simple yet
powerful control plane written in Go, and a design that’s focused on
performance, security, and usability. Most importantly, Conduit
incorporates the many lessons we’ve learned from over 18 months of
production service mesh experience with Linkerd.
This repository contains a few tightly-related components:
- `proxy` -- an HTTP/2 proxy written in Rust;
- `controller` -- a control plane written in Go with gRPC;
- `web` -- a UI written in React, served by Go.
Risha Mars
Kevin Lingerfelt
Sean McArthur
Brian Smith
Franziska von der Goltz
Oliver Gould
Thomas Rampelberg
Kim Christensen
Andrew Seigner
Dennis Adjei-Baah
Eliza Weisman
Alex Leong
Nathan Murthy
Phil Calçado
Sterling White