Adding local and cloud integration testing for dashboard (#2586)

Adds local and cloud integration testing for the dashboard using WebdriverIO and
SauceLabs. Includes documentation on how to set up and run the Sauce Connect
proxy locally. Adds a `bin/web integration` script that takes `local` or `cloud`
arguments to run the tests. 

Note: for web development, the web server launched by `bin/web run` and `bin/web
dev` is now 7777, not 8084, because the Sauce Connect proxy can only tunnel to
certain ports.
This commit is contained in:
Carol A. Scott 2019-03-29 15:48:00 -07:00 committed by GitHub
parent 23454d248d
commit 0251f50fa4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
8 changed files with 1020 additions and 35 deletions

View File

@ -262,7 +262,7 @@ These commands assume working [Go](https://golang.org) and
bin/web run
```
The web server will be running on `localhost:8084`.
The web server will be running on `localhost:7777`.
#### Webpack dev server
@ -276,7 +276,7 @@ To develop with a webpack dev server:
Note: this will start up:
- `web` on :8084. This is the golang process that serves the dashboard.
- `web` on :7777. This is the golang process that serves the dashboard.
- `webpack-dev-server` on :8080 to manage rebuilding/reloading of the
javascript.
- `controller` is port-forwarded from the Kubernetes cluster via `kubectl`
@ -284,7 +284,7 @@ To develop with a webpack dev server:
- `grafana` is port-forwarded from the Kubernetes cluster via `kubectl`
on :3000
1. Go to [http://localhost:8084](http://localhost:8084) to see everything
2. Go to [http://localhost:7777](http://localhost:7777) to see everything
running.
#### Dependencies

87
TEST.md
View File

@ -194,6 +194,93 @@ To cleanup the namespaces after the test has finished, run:
$ bin/test-cleanup
```
### Testing the dashboard
We use [WebdriverIO](https://webdriver.io/) to test how the web dashboard looks
and operates locally in Chrome. For cross-browser testing, we use
[SauceLabs](https://saucelabs.com/), which runs simulataneous tests on different
browsers in the cloud.
If you're new to the repo, make sure you've installed web dependencies via
[Yarn](https://yarnpkg.com):
```bash
brew install yarn # if you don't already have yarn
bin/web setup
```
Then start up the dashboard at `localhost:7777`. You can do that in one of two
ways:
```bash
# standalone
bin/web run
```
OR
```bash
# with webpack-dev-server
bin/web dev
```
#### Local
To run a local WebdriverIO instance that will run the tests on a local instance
of Chrome, run:
```bash
bin/web integration local
```
#### Cloud
To run cross-browser tests via SauceLabs, you need to do a few things first:
1. Sign up for a (free) SauceLabs sub-account for the account 'buoyant'. If you
are not a Buoyant staffer, the best way to get an account invite is to ask in
the [Linkerd Slack channel](https://slack.linkerd.io).
2. Once you have your username and key, set them as permanent environment
variables. This keeps your credentials private, and means that everyone on
the team can run the tests via their unique login without modifying the test
files. Open your `~/.bash_profile` file and add:
```bash
export SAUCE_USERNAME="your Sauce username"
export SAUCE_ACCESS_KEY="your Sauce access key"
```
3. Now you'll [download Sauce
Connect](https://wiki.saucelabs.com/display/DOCS/Sauce+Connect+Proxy), the
proxy server that will open a secure tunnel between a SauceLabs VM and the
Linkerd dashboard instance you're running on `localhost:7777`. You'll want to
save it in a separate directory from the rest of your development files.
After downloading it, navigate to that directory and start it up:
```bash
SC=sc-4.5.3-osx # OSX example
wget -O - https://saucelabs.com/downloads/$SC.zip | tar xfz - -C ~/
cd ~/$SC
bin/sc -u $SAUCE_USERNAME -k $SAUCE_ACCESS_KEY
```
Wait until you see `Sauce Connect is up, you may start your tests` in your
terminal. Open a separate terminal window and run:
```bash
bin/web integration cloud
```
SauceLabs will start running the tests in the cloud. If any tests fail,
you'll immediately get the URL in your terminal window with a video of the
test and information about what happened. The test(s) will also appear in
[your SauceLabs archives](https://app.saucelabs.com/archives) a minute or so
after they end. (Depending on time of day and server load, it may take longer
for the tests to appear in the archives.)
4. When you're finished, close the tunnel by pressing `CTRL-C` in the Sauce
Connect window. If you forget to do this, it will close on its own after a
few minutes.
## Writing tests
To add a new test, create a new subdirectory inside the `test/` directory.

16
bin/web
View File

@ -91,7 +91,7 @@ function run {
)
cd $ROOT/web && \
../bin/go-run . $*
../bin/go-run . --addr=:7777 $*
}
function setup {
@ -104,6 +104,20 @@ function test {
yarn jest "$*"
}
function integration {
if [[ "$#" -ne 1 || ($1 != "cloud" && $1 != "local") ]]; then
echo "usage: bin/web integration (cloud|local)" >&2
exit 1
fi
if [ $1 = "cloud" ]; then
cd $ROOT/web/app && \
./node_modules/wdio/node_modules/.bin/wdio ./integration/wdio-sauce.conf.js
else
cd $ROOT/web/app && \
./node_modules/wdio/node_modules/.bin/wdio ./integration/wdio.conf.js
fi
}
function main {
setup
build

View File

@ -0,0 +1,11 @@
const assert = require('assert');
describe('logo link test', function() {
it('should redirect to the home view if logo is clicked', () => {
browser.url('http://localhost:7777/tap');
const link = $('.linkerd-word-logo');
link.click();
const breadcrumbHeader = $('.breadcrumb-link');
const pageTitle = breadcrumbHeader.getText();
assert.equal(pageTitle, 'Overview');
});
});

View File

@ -0,0 +1,28 @@
exports.config = {
runner: 'local',
user: process.env.SAUCE_USERNAME,
key: process.env.SAUCE_ACCESS_KEY,
sauceConnect: true,
specs: [
'./integration/specs/*.js'
],
// Patterns to exclude.
exclude: [
// 'path/to/excluded/files'
],
maxInstances: 10,
capabilities: [
{browserName: 'firefox', platform: 'Windows 10', version: '60.0'},
{browserName: 'chrome', platform: 'OS X 10.13', version: '69.0'}
],
bail: 0,
baseUrl: 'http://localhost',
waitforTimeout: 10000,
connectionRetryTimeout: 90000,
connectionRetryCount: 3,
framework: 'mocha',
mochaOpts: {
ui: 'bdd',
timeout: 60000
}
}

View File

@ -0,0 +1,24 @@
exports.config = {
port: 9515, // default for ChromeDriver
path: '/',
services: ['chromedriver'],
runner: 'local',
specs: [
'./integration/specs/*.js'
],
exclude: [
// 'path/to/excluded/files'
],
maxInstances: 10,
capabilities: [{browserName: 'chrome', platform: 'OS X 10.13', version: '69.0'}],
bail: 0,
baseUrl: 'http://localhost',
waitforTimeout: 10000,
connectionRetryTimeout: 90000,
connectionRetryCount: 3,
framework: 'mocha',
mochaOpts: {
ui: 'bdd',
timeout: 60000
}
}

View File

@ -25,6 +25,7 @@
"whatwg-fetch": "2.0.3"
},
"devDependencies": {
"@wdio/cli": "^5.7.6",
"babel-core": "6.26.0",
"babel-eslint": "8.0.3",
"babel-jest": "23.6.0",
@ -34,6 +35,7 @@
"babel-preset-env": "1.7.0",
"babel-preset-react-app": "3.1.1",
"babel-runtime": "^6.26.0",
"chromedriver": "^2.46.0",
"clean-webpack-plugin": "1.0.0",
"css-loader": "0.28.7",
"enzyme": "3.7.0",
@ -58,6 +60,10 @@
"sinon-stub-promise": "4.0.0",
"style-loader": "0.21.0",
"url-loader": "1.0.1",
"wdio": "^3.0.3",
"wdio-chromedriver-service": "^5.0.1",
"wdio-mocha-framework": "^0.6.4",
"webdriverio": "^5.7.6",
"webpack": "4.20.2",
"webpack-bundle-analyzer": "^3.0.3",
"webpack-cli": "3.0.7",

File diff suppressed because it is too large Load Diff