summaryrefslogtreecommitdiffhomepage
path: root/cmd/testwrapper
AgeCommit message (Collapse)AuthorFilesLines
2025-02-18cmd/testwrapper: wip ideaicio/testwrapper2Paul Scott4-3/+278
Signed-off-by: Paul Scott <paul@tailscale.com>
2025-02-17cmd/testwrapper: temporarily remove test coverage supportBrad Fitzpatrick1-148/+1
testwrapper doesn't work with Go 1.24 and the coverage support is making it harder to debug. Updates #15015 Updates tailscale/corp#26659 Change-Id: I0125e881d08c92f1ecef88b57344f6bbb571b569 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2025-01-03util/slicesx: add MapKeys and MapValues from golang.org/x/exp/mapsBrad Fitzpatrick1-2/+2
Importing the ~deprecated golang.org/x/exp/maps as "xmaps" to not shadow the std "maps" was getting ugly. And using slices.Collect on an iterator is verbose & allocates more. So copy (x)maps.Keys+Values into our slicesx package instead. Updates #cleanup Updates #12912 Updates #14514 (pulled out of that change) Change-Id: I5e68d12729934de93cf4a9cd87c367645f86123a Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-10-24cmd/testwrapper: add pkg runtime to output (#13894)Paul Scott2-10/+21
Fixes #13893 Signed-off-by: Paul Scott <paul@tailscale.com>
2024-02-21cmd/testwrapper: apply results of all unit tests to coverage for all packagesPercy Wegmann1-0/+1
This allows coverage from tests that hit multiple packages at once to be reflected in all those packages' coverage. Updates #cleanup Signed-off-by: Percy Wegmann <percy@tailscale.com>
2023-12-21all: cleanup unused code, part 2 (#10670)Andrew Lytvynov1-16/+0
And enable U1000 check in staticcheck. Updates #cleanup Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2023-11-21ci: report test coverage to coveralls.ioPercy Wegmann1-2/+150
This records test coverage for the amd64 no race tests and uploads the results to coveralls.io. Updates #cleanup Signed-off-by: Ox Cart <ox.to.a.cart@gmail.com>
2023-10-16tstest, tstest/integration, github/workflows: shard integration testsBrad Fitzpatrick1-0/+3
Over four jobs for now. Updates #cleanup Change-Id: Ic2b1a739a454916893945a3f9efc480d6fcbd70b Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-10-16cmd/testwrapper: parse args betterMaisem Ali3-54/+249
Previously we were just smushing together args and not trying to parse the values at all. This resulted in the args to testwrapper being limited and confusing. This makes it so that testwrapper parses flags in the exact format as `go test` command and passes them down in the provided order. It uses tesing.Init to register flags that `go test` understands, however those are not the only flags understood by `go test` (such as `-exec`) so we register these separately. Updates tailscale/corp#14975 Signed-off-by: Maisem Ali <maisem@tailscale.com>
2023-10-14tstest/integration: add tests for tun mode (requiring root)Brad Fitzpatrick1-4/+12
Updates #7894 Change-Id: Iff0b07b21ae28c712dd665b12918fa28d6f601d0 Co-authored-by: Maisem Ali <maisem@tailscale.com> Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-10-03tstest/deptest: add check that x/exp/{maps,slices} imported as xfooBrad Fitzpatrick1-1/+1
Updates #cleanup Change-Id: I4cbb5e477c739deddf7a46b66f286c9fdb106279 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-09-28cmd/testwrapper: include flake URL in JSON metadataBrad Fitzpatrick2-10/+25
Updates tailscale/corp#14975 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-09-28cmd/testwrapper: output machine-readable JSON on test flakesBrad Fitzpatrick1-12/+14
For parsing by other tools. Updates tailscale/corp#14975 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-09-11cmd/testwrapper: fix exit deflake (#9342)Paul Scott1-7/+7
Sometimes `go test` would exit and close its stdout before we started reading it, and we would return that "file closed" error then forget to os.Exit(1). Fixed to prefer the go test subprocess error and exit regardless of the type of error. Fixes #9334 Signed-off-by: Paul Scott <paul@tailscale.com>
2023-09-11cmd/testwrapper/flakytest: don't spam stderr in Mark when not under wrapperBrad Fitzpatrick1-3/+8
If the user's running "go test" by hand, no need to spam stderr with the sentinel marker. It already calls t.Logf (which only gets output on actual failure, or verbose mode) which is enough to tell users it's known flaky. Stderr OTOH always prints out and is distracting to manual "go test" users. Updates #cleanup Change-Id: Ie5e6881bae291787c30f75924fa132f4a28abbb2 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-09-07cmd/testwrapper: exit code 1 when go build fails (#9276)Paul Scott2-16/+262
Fixes #9275 Fixes #8586 Fixes tailscale/corp#13115 Signed-off-by: Paul Scott <paul@tailscale.com>
2023-09-04cmd/testwrapper: emit logs of failed tests on timeoutMaisem Ali1-24/+28
It would just fail the entire pkg, but would not print any logs. It was already tracking all the logs, so have it emit them when the pkg fails/times out. Updates #9231 Signed-off-by: Maisem Ali <maisem@tailscale.com>
2023-08-29cmd/testwrapper: handle timeouts as test failuresMaisem Ali1-0/+6
While investigating the fix in 7538f386710b80c6b4c1997797be28a661210d4a, I was curious why the testwrapper didn't fail. Turns out if the test times out and there was no explicit failure, the only message we get is that the overall pkg failed and no failure information about the individual test. This resulted in a 0 exit code. This fixes that by failing the explicit case of the pkg failing when there is nothing to retry for that pkg. Updates #8493 Signed-off-by: Maisem Ali <maisem@tailscale.com>
2023-08-24cmd/testwrapper: fix off-by-one error in maxAttempts checkMaisem Ali1-1/+1
It was checking if `>= maxAttempts` which meant that the third attempt would never run. Updates #8493 Signed-off-by: Maisem Ali <maisem@tailscale.com>
2023-08-17all: import x/exp/maps as xmaps to distinguish from Go 1.21 "maps"Brad Fitzpatrick1-2/+2
Updates #8419 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-07-11cmd/testwrapper: handle build failuresMaisem Ali1-0/+10
`go test -json` outputs invalid JSON when a build fails. Handle that case by reseting the json.Decode and continuing to read. Updates #8493 Signed-off-by: Maisem Ali <maisem@tailscale.com>
2023-06-30cmd/testwrapper: stream output resultsMaisem Ali1-32/+40
Previously it would wait for all tests to run before printing anything, instead stream the results over a channel so that they can be emitted immediately. Updates #8493 Signed-off-by: Maisem Ali <maisem@tailscale.com>
2023-06-30cmd/testwrapper: output packages testedMaisem Ali1-32/+71
Previously it would only print the failures without providing more information on which package the failures from. This commit makes it so that it prints out the package information as well as the attempt numbers. ``` ➜ tailscale.com git:(main) ✗ go run ./cmd/testwrapper ./cmd/... ok tailscale.com/cmd/derper ok tailscale.com/cmd/k8s-operator ok tailscale.com/cmd/tailscale/cli ok tailscale.com/cmd/tailscaled === RUN TestFlakeRun flakytest.go:38: flakytest: issue tracking this flaky test: https://github.com/tailscale/tailscale/issues/0 flakytest_test.go:41: First run in testwrapper, failing so that test is retried. This is expected. --- FAIL: TestFlakeRun (0.00s) FAIL tailscale.com/cmd/testwrapper/flakytest Attempt #2: Retrying flaky tests: ok tailscale.com/cmd/testwrapper/flakytest ``` Updates #8493 Signed-off-by: Maisem Ali <maisem@tailscale.com>
2023-06-28cmd/testwrapper/flakytest: clearly describe why TestFlakeRun failsMaisem Ali1-1/+4
Fixes #8474 Signed-off-by: Maisem Ali <maisem@tailscale.com>
2023-06-26cmd/testwrapper: only retry flaky failed testsMaisem Ali3-53/+230
Redo the testwrapper to track and only retry flaky tests instead of retrying the entire pkg. It also fails early if a non-flaky test fails. This also makes it so that the go test caches are used. Fixes #7975 Signed-off-by: Maisem Ali <maisem@tailscale.com>
2023-01-27all: update copyright and license headersWill Norris3-9/+6
This updates all source files to use a new standard header for copyright and license declaration. Notably, copyright no longer includes a date, and we now use the standard SPDX-License-Identifier header. This commit was done almost entirely mechanically with perl, and then some minimal manual fixes. Updates #6865 Signed-off-by: Will Norris <will@tailscale.com>
2023-01-23tstest/integration: mark all integration tests as flakyAndrew Dunham1-1/+1
Updates #7036 Change-Id: I3aec5ad680078199ba984bf8afc20b2f2eb37257 Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
2023-01-18cmd/testwrapper: move from corp; mark magicsock test as flakyAndrew Dunham3-0/+136
Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: Ibab5860f5797b3db151d3c27855333e43a9088a4
Copyright © 2025 - 2026 Wayne Cole. WTFPL. Attribution appreciated. No warranty. Not liable for bodily damages.