Remove detected goroutine leaks #2833

codebien · 2022-12-22T17:10:52Z

No description provided.

codecov-commenter · 2022-12-22T17:21:27Z

Codecov Report

Merging #2833 (8cc0c9e) into master (367163b) will increase coverage by 0.07%.
The diff coverage is 83.48%.

❗ Current head 8cc0c9e differs from pull request most recent head 6ce8ddf. Consider uploading reports for the commit 6ce8ddf to get more accurate results

@@            Coverage Diff             @@
##           master    #2833      +/-   ##
==========================================
+ Coverage   76.84%   76.92%   +0.07%     
==========================================
  Files         229      229              
  Lines       16924    16946      +22     
==========================================
+ Hits        13005    13035      +30     
+ Misses       3077     3074       -3     
+ Partials      842      837       -5

Flag	Coverage Δ
ubuntu	`76.84% <83.48%> (+0.08%)`	⬆️
windows	`76.75% <83.48%> (+0.10%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
cmd/tests/tests.go	`46.15% <0.00%> (+1.70%)`	⬆️
log/loki.go	`30.70% <0.00%> (-6.36%)`	⬇️
log/file.go	`79.66% <50.00%> (-1.59%)`	⬇️
lib/testutils/httpmultibin/httpmultibin.go	`92.64% <89.18%> (+2.69%)`	⬆️
cmd/root.go	`95.33% <94.54%> (+13.51%)`	⬆️
js/initcontext.go	`80.00% <0.00%> (-4.00%)`	⬇️
lib/executor/vu_handle.go	`93.45% <0.00%> (-0.94%)`	⬇️
api/v1/metric.go	`77.27% <0.00%> (+9.09%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

na--

❤️

lib/testutils/httpmultibin/httpmultibin.go

The base branch was changed.

cmd/stdlog_integration_test.go

cmd/root.go

codebien · 2023-01-18T11:43:56Z

Rebased after the e2e tests refactor

cmd/root.go

imiric

It mostly LGTM, just a few non-blocking comments and questions.

@codebien How did you see the goleak report? I couldn't see anything from go test -v -race -count=1 ./cmd/tests on master, and I can't figure it out from their README 😕 I would like to see leaks detected on master and then fixed on this branch.

lib/testutils/httpmultibin/httpmultibin.go

cmd/root.go

codebien · 2023-01-21T11:59:33Z

Hi @imiric, thanks for your review.

On master, the leak is not reported because we are skipping the check. You should see the report removing this option.

k6/cmd/tests/tests.go

Lines 55 to 57 in 0b563d4

 // TODO: figure out why logrus' `Entry.WriterLevel` goroutine sticks 

 // around and remove this exception. 

 opt := goleak.IgnoreTopFunction("io.(*pipe).read")

codebien · 2023-01-25T11:59:59Z

Hey @imiric,
I tried to apply some fixes for your request changes, the current log handling code is not as clean as I would but I think to do something better we need to start creating a dedicated component for logging (#1939, #2287) then it should be moved to a dedicated PR.

Regarding Httpmultibin I've moved the ws-close as a dedicated handler with the expectation that the server closes the connection. Instead, the echo handler waits for the client to start the closing ops.
Some ws tests weren't aligned with this strict principle so I had to adjust how we are using them.

The current commit history is not optimal so if we have a consensus on the current design I will re-arrange for having atomic commits.

.github/workflows/test.yml

imiric

Almost LGTM 😄

I'd only like to move asyncHook to the log package.

There are many subtle changes to the logging mechanism here, and while it visually seems safe, I hope we're not breaking some obscure behavior. The testing for this isn't great, so hopefully we can catch any issues in manual test runs. 🤞

lib/testutils/httpmultibin/httpmultibin.go

log/file_test.go

cmd/root.go

cmd/stdlog_integration_test.go

cmd/tests/cmd_run_test.go

na-- · 2023-03-06T16:24:20Z

Generally LGTM, with a few nitpicks I commented on or 👍 already existing comments

Refactor for better handling and asserting closing operations.

na--

🤞

After #2833 log outputs were stopped but loki didn't wait until it push was finished to signal it was done. Which leads to the actual k6 process exiting before loki could flush its messages.

codebien self-assigned this Dec 22, 2022

na-- previously approved these changes Dec 23, 2022

View reviewed changes

codebien force-pushed the no-go-leaks branch from 8d033d1 to 2189dc4 Compare December 23, 2022 11:23

na-- reviewed Dec 23, 2022

View reviewed changes

lib/testutils/httpmultibin/httpmultibin.go Outdated Show resolved Hide resolved

codebien force-pushed the no-go-leaks branch 5 times, most recently from 6c51141 to 5b191f4 Compare December 23, 2022 15:42

na-- added this to the v0.43.0 milestone Jan 3, 2023

na-- mentioned this pull request Jan 10, 2023

Ensure REST API server is shut down after test ends and check for goroutine leaks #2803

Merged

codebien changed the base branch from remove-the-engine to master January 13, 2023 14:16

codebien force-pushed the no-go-leaks branch from 5b191f4 to 546bba9 Compare January 13, 2023 14:21

na-- reviewed Jan 13, 2023

View reviewed changes

cmd/stdlog_integration_test.go Show resolved Hide resolved

na-- reviewed Jan 13, 2023

View reviewed changes

cmd/root.go Outdated Show resolved Hide resolved

codebien force-pushed the no-go-leaks branch 2 times, most recently from c0d84e0 to be16867 Compare January 17, 2023 15:17

codebien marked this pull request as ready for review January 17, 2023 16:59

github-actions bot requested review from imiric and na-- January 17, 2023 16:59

codebien force-pushed the no-go-leaks branch 4 times, most recently from e7b6db8 to 1e60b4f Compare January 18, 2023 11:43

codebien commented Jan 18, 2023

View reviewed changes

cmd/root.go Outdated Show resolved Hide resolved

imiric reviewed Jan 20, 2023

View reviewed changes

lib/testutils/httpmultibin/httpmultibin.go Show resolved Hide resolved

cmd/root.go Outdated Show resolved Hide resolved

cmd/root.go Outdated Show resolved Hide resolved

na-- modified the milestones: v0.43.0, v0.44.0 Feb 8, 2023

codebien force-pushed the no-go-leaks branch 2 times, most recently from c2e08fb to 64c9809 Compare February 21, 2023 18:59

mstoykov reviewed Feb 22, 2023

View reviewed changes

.github/workflows/test.yml Outdated Show resolved Hide resolved

imiric suggested changes Feb 24, 2023

View reviewed changes

lib/testutils/httpmultibin/httpmultibin.go Show resolved Hide resolved

log/file_test.go Outdated Show resolved Hide resolved

cmd/root.go Outdated Show resolved Hide resolved

na-- reviewed Mar 6, 2023

View reviewed changes

cmd/stdlog_integration_test.go Show resolved Hide resolved

na-- reviewed Mar 6, 2023

View reviewed changes

cmd/tests/cmd_run_test.go Outdated Show resolved Hide resolved

codebien force-pushed the no-go-leaks branch 4 times, most recently from ba09a69 to 6df4ba0 Compare March 9, 2023 09:26

codebien added 4 commits March 9, 2023 10:29

log: Refactor for using better async management

65680b5

cmd: Refactor init loggers ops

0cb4e00

Refactor of the HTTPMultibin WebSocket handlers

d6079e2

Refactor for better handling and asserting closing operations.

ws: Drop a not valid test

6ce8ddf

codebien force-pushed the no-go-leaks branch from 6df4ba0 to 6ce8ddf Compare March 9, 2023 09:29

codebien requested a review from imiric March 9, 2023 09:42

imiric approved these changes Mar 9, 2023

View reviewed changes

codebien requested a review from na-- March 9, 2023 15:42

na-- approved these changes Mar 9, 2023

View reviewed changes

codebien merged commit 333a3d1 into master Mar 9, 2023

codebien deleted the no-go-leaks branch March 9, 2023 16:27

na-- mentioned this pull request Mar 31, 2023

Fix panic handling and add a new exit code and a test for panics #2997

Merged

mstoykov mentioned this pull request Jul 13, 2023

Fix loki log output exiting before push is finished #3194

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove detected goroutine leaks #2833

Remove detected goroutine leaks #2833

codebien commented Dec 22, 2022

codecov-commenter commented Dec 22, 2022 •

edited

Loading

na-- left a comment

codebien commented Jan 18, 2023

imiric left a comment •

edited

Loading

codebien commented Jan 21, 2023

codebien commented Jan 25, 2023 •

edited

Loading

imiric left a comment

na-- commented Mar 6, 2023

na-- left a comment

Remove detected goroutine leaks #2833

Remove detected goroutine leaks #2833

Conversation

codebien commented Dec 22, 2022

codecov-commenter commented Dec 22, 2022 • edited Loading

Codecov Report

na-- left a comment

Choose a reason for hiding this comment

codebien commented Jan 18, 2023

imiric left a comment • edited Loading

Choose a reason for hiding this comment

codebien commented Jan 21, 2023

codebien commented Jan 25, 2023 • edited Loading

imiric left a comment

Choose a reason for hiding this comment

na-- commented Mar 6, 2023

na-- left a comment

Choose a reason for hiding this comment

codecov-commenter commented Dec 22, 2022 •

edited

Loading

imiric left a comment •

edited

Loading

codebien commented Jan 25, 2023 •

edited

Loading