Commit Graph

33 Commits

Author SHA1 Message Date
justinsb 3ea73f47f8 Better dumping via private IP when bastion is not set
Previously this would always fail in a confusing way,
regardless of whether we had connectivity,
because we tried to connect to an empty-string host.

Now we are more explicit about the error,
and will at least try to connect directly.
2025-02-19 08:25:33 -05:00
Antonio Ojea 5b08cef72c improve log dumper
get more networking information usefult to troubleshoot network issues.
2025-01-07 15:52:26 +00:00
Peter Rifel 825f908d1d
Dump ip routes as job artifacts
This will help troubleshoot networking issues we're seeing with Ubuntu 24.04
2024-12-22 19:34:45 -06:00
Peter Rifel 3b5fa6fc35
Dont fail resource dump if not all APIs can be discovered 2024-10-05 07:54:14 -05:00
Peter Rifel 1eb8d3ce57
Prefix resource dump filenames with group
Certain api services have conflicting Resource types, causing resource dumps to overwrite eachother.
This ensures filenames remain unique by prefixing them with the api group
2024-10-03 20:51:17 -05:00
upodroid 4717450b1c fix dumpling logs for GCE scale tests 2024-01-20 13:33:04 +00:00
Peter Rifel a97ac7d8ff
Dump and redact secrets
This will help with troubleshooting some cert-manager issues where secrets aren't being created
2024-01-03 19:53:04 -06:00
Ciprian Hacman 5c2c0c0671 Dump previous and current container logs separately 2024-01-02 09:01:47 +02:00
Peter Rifel f731080867
Update podlogs.go
Co-authored-by: Ciprian Hacman <ciprian@hakman.dev>
2024-01-01 21:51:26 -06:00
Peter Rifel 8ad0661975
Dump pod logs in parallel 2024-01-01 13:11:25 -06:00
Peter Rifel db32f982d1
Remove unused cluster name param
Signed-off-by: Peter Rifel <pgrifel@gmail.com>
2024-01-01 13:10:37 -06:00
Peter Rifel 8524175f86
Dump k8s resources with kops toolbox dump
Signed-off-by: Peter Rifel <pgrifel@gmail.com>
2023-12-30 19:22:53 -08:00
upodroid 6828213063 dump nodes even if controlplane is broken 2023-10-10 10:32:44 +01:00
Ciprian Hacman 5c785609b2 test: Dump special nodes first and limit to 500 nodes 2023-09-20 19:45:46 +03:00
Ciprian Hacman 772b1d3d37 test: Reduce connection timeout when dumping nodes 2023-09-17 16:08:21 +03:00
Justin SB 309a2c0b3c tests: Capture iptables chains
This is helpful debugging information when we see network issues.
2023-01-22 11:30:29 -05:00
John Gardiner Myers 2947c75836 Include kops-controller logs in toolbox dump 2022-12-25 17:58:44 -08:00
John Gardiner Myers 6bd629e9c7 Can only request agent forwarding once per connection 2022-11-10 22:42:23 -08:00
John Gardiner Myers 71017f0307 Use bastion to dump private instances 2022-11-09 21:34:04 -08:00
John Gardiner Myers 8675336725 Also dump logs from IPv6 nodes 2022-11-05 16:10:53 -07:00
Peter Rifel f4e06eafa1
Include sysctls in toolbox dump 2022-04-28 20:22:24 -05:00
Ole Markus With ce2e877aeb Remove bazel files from vendor 2022-04-12 13:29:03 +02:00
Ciprian Hacman ea7df00719 Run hack/update-gofmt.sh 2021-12-01 22:39:50 +02:00
John Gardiner Myers 77fe133869 Include the amazonvpc logs in toolbox dump 2021-11-08 14:31:28 -08:00
Peter Rifel 29e9276080
Have toolbox dump include contents of /etc/hosts
This should help troubleshoot DNS issues in gossip clusters
2021-09-26 19:04:13 -05:00
Peter Rifel a6d47b6002
Decrease connection timeout when dumping logs from instances
This is currently over 2 minutes. This change should speed up the end of tests when SSH isn't available.
2021-09-22 21:09:13 -05:00
Peter Rifel a467aa17a5
Add .log suffix to dumped pod logs 2021-09-05 11:31:04 -05:00
Peter Rifel 43906b993d
Dump external-dns and dns-controller pod logs
This helps troubleshooting when the API DNS record never resolves in a prow job
2021-09-05 09:05:09 -05:00
Ciprian Hacman d378128895 Capture logs from the etcd-cilium pod 2021-07-07 08:32:19 +03:00
Ciprian Hacman acae316957 Capture logs from the containerd service 2021-07-03 08:03:12 +03:00
Peter Rifel 4d9f0128a3
Upgrade to klog2
This splits up the kubernetes 1.19 PR to make it easier to keep up to date until we get it sorted out.
2020-08-16 20:56:48 -05:00
Peter Rifel 1faeb36d37
Address feedback and test failures 2020-07-22 22:19:00 -05:00
Justin Santa Barbara d8b69ab2e3
Capture logs from a kops cluster
This is derived from the dumping code in kubetest.  If we want to run
tests outside of kubetest (e.g. upgrade tests), we're going to need
that functionality.
2020-07-17 10:00:06 -05:00