From 4c9b1f1131ebd15443ad42731cc17090331bcc6d Mon Sep 17 00:00:00 2001 From: Daniel Hiltgen Date: Tue, 18 Aug 2015 14:47:06 -0700 Subject: [PATCH] Add support for pprof This likely still needs some tuning/tweaking, but may be helpful as we start to do more scale/performance testing of the system. --- profiling.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 profiling.md diff --git a/profiling.md b/profiling.md new file mode 100644 index 0000000000..81cc6c3415 --- /dev/null +++ b/profiling.md @@ -0,0 +1,49 @@ +# Profiling Orca + +If you run the Orca server with the debug flag set, not only will you get more logging output, but we enable +remote pprof access. + +Links: +* http://blog.golang.org/profiling-go-programs +* https://golang.org/pkg/net/http/pprof/ + + +# Examples + +* First deploy orca with debug. +* If you're using TLS (e.g., with bootstrap install) you'll need to add the certs to the local system's trusted certs (unfortunately pprof doesn't have an "--insecure" flag) + ```bash +sudo bash -c "docker run --rm -it \ + --name orca-bootstrap \ + -v /var/run/docker.sock:/var/run/docker.sock \ + dockerorca/orca-bootstrap \ + dump-certs > /usr/local/share/ca-certificates/orca.crt" +sudo update-ca-certificates +``` + +* Now you can point pprof at the server + +```bash +ORCA=https://192.168.104.73:443 +go tool pprof ${ORCA}/debug/pprof/block + +web +``` + +That will pop up a nice SVG image on your default browser showing the accumulated blocking calls + +* Other URL endpoints of interest: ("web" should produce a nice summary in each of these) + * go tool pprof ${ORCA}/debug/pprof/profile - The CPU usage + * go tool pprof ${ORCA}/debug/pprof/heap - The memory usage + * go tool pprof ${ORCA}/debug/pprof/goroutine - The goroutine usage + * curl --insecure ${ORCA}/debug/pprof/ - Display the entry points (or use your browser) + +# Dropped nodes + +* It looks like nodefraction can be used to include nodes that would otherwise be dropped, but the web visualization seems to ignore it + ```bash +go tool pprof -nodefraction=0.00000001 ${ORCA}/debug/pprof/block + +peek swarm +``` +