Skip to content

test: fix flaky test-http2-session-timeout#20692

Closed
Trott wants to merge 2 commits intonodejs:masterfrom
Trott:one-less-timer
Closed

test: fix flaky test-http2-session-timeout#20692
Trott wants to merge 2 commits intonodejs:masterfrom
Trott:one-less-timer

Conversation

@Trott
Copy link
Member

@Trott Trott commented May 13, 2018 •
edited
Loading

Check actual expired time rather than relying on a number of calls to setTimeout() in test-http2-session-timeout more robust.

Fixes: #20628

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added the test Issues and PRs related to the tests. label May 13, 2018
@Trott Trott added the wip Issues and PRs that are still a work in progress. label May 13, 2018
@Trott Trott changed the title test: fix flaky test-http2-session-timeout [WIP] test: fix flaky test-http2-session-timeout May 13, 2018
@Trott Trott force-pushed the one-less-timer branch from d852b27 to 7a2833e Compare May 13, 2018 01:47
@Trott Trott changed the title [WIP] test: fix flaky test-http2-session-timeout test: fix flaky test-http2-session-timeout May 13, 2018
@Trott Trott removed the wip Issues and PRs that are still a work in progress. label May 13, 2018
@Trott
Copy link
Member Author

Trott commented May 13, 2018

This version of the test is demonstrably more reliable than the current version when tested this way:

  • Copy the current version of the test to test/parallel.
  • Run the test with tools/test.py -j 16 --repeat 192 tools/test/parallal/test-<WHATEVER_YOU_COPIED_IT_TO>
  • Repeat with the new version of the test (copied to test/parallel).

You may need to adjust the value of the -j option to be not too large that both versions fail so often that it's hard to tell the difference and to not be so small that both versions fail so infrequently as to not make a different. 16 was a good value on my laptop. Here are the last three runs of each version of the test with that value (successes/failures):

Current version of test: 142/50, 159/33, 153/39
This PR's version of the test: 184/8, 191/1, 186/6

/cc @apapirovski

@Trott Trott mentioned this pull request May 13, 2018
Closed
@Trott
Copy link
Member Author

Trott commented May 13, 2018

CI: https://ci.nodejs.org/job/node-test-pull-request/14828/

apapirovski
apapirovski reviewed May 13, 2018
View reviewed changes
test/sequential/test-http2-session-timeout.js Outdated
Copy link
Contributor

@apapirovski apapirovski May 13, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm -1 on this change. process.hrtime isn't cheap so it's more — not less — likely to create situations where this test fails on slow platforms.

Actually, scratch that, I see what's making this more reliable.

lpinca
lpinca reviewed May 13, 2018
View reviewed changes
test/sequential/test-http2-session-timeout.js Outdated
Copy link
Member

@lpinca lpinca May 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: if I'm not wrong we can pass makeReq directly without wrapping it in another function.

Copy link
Member Author

@Trott Trott May 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Changed. Thanks!

@lpinca lpinca added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 13, 2018
@Trott Trott force-pushed the one-less-timer branch from 7a2833e to f836321 Compare May 13, 2018 17:11
@richardlau
Copy link
Member

richardlau commented May 14, 2018

#20692 (comment) suggests this doesn't fix the test but instead makes it less likely to fail, i.e., it's still flaky? If so can we reword the commit to something other than fix (maybe improve or make less flaky)?

Trott added 2 commits May 14, 2018 16:09
@Trott
test: improve reliability of http2-session-timeout
06deaf5 
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

Fixes: nodejs#20628
@Trott
squash! nit fix
34160c7
@Trott Trott force-pushed the one-less-timer branch from f836321 to 34160c7 Compare May 14, 2018 23:10
@Trott
Copy link
Member Author

Trott commented May 14, 2018

@richardlau I went with "improve reliability" although I had to abbreviate the test name to keep it under 50 chars. (I removed test- which is implied by the test: for the subsystem. And the full name of the test appears in the main text.)

@Trott
Copy link
Member Author

Trott commented May 15, 2018

CI: https://ci.nodejs.org/job/node-test-pull-request/14891/

@apapirovski
Copy link
Contributor

apapirovski commented May 16, 2018

Landed in 9e4ae56

apapirovski pushed a commit that referenced this pull request May 16, 2018
@Trott @apapirovski
test: improve reliability of http2-session-timeout
9e4ae56 
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

PR-URL: #20692
Fixes: #20628
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
MylesBorins pushed a commit that referenced this pull request May 22, 2018
@Trott @MylesBorins
test: improve reliability of http2-session-timeout
a4cbe30 
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

PR-URL: #20692
Fixes: #20628
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
@addaleax addaleax mentioned this pull request May 22, 2018
Merged
kjin pushed a commit to kjin/node that referenced this pull request Aug 23, 2018
@Trott @kjin
test: improve reliability of http2-session-timeout
c34872f 
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

PR-URL: nodejs#20692
Fixes: nodejs#20628
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
kjin pushed a commit to kjin/node that referenced this pull request Sep 19, 2018
@Trott @kjin
test: improve reliability of http2-session-timeout
769bbb8 
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

PR-URL: nodejs#20692
Fixes: nodejs#20628
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
kjin pushed a commit to kjin/node that referenced this pull request Oct 16, 2018
@Trott @kjin
test: improve reliability of http2-session-timeout
9cf40c5 
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

PR-URL: nodejs#20692
Fixes: nodejs#20628
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
BethGriggs pushed a commit that referenced this pull request Oct 17, 2018
@Trott @BethGriggs
test: improve reliability of http2-session-timeout
e5f8b08 
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

Backport-PR-URL: #22850
PR-URL: #20692
Fixes: #20628
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
@BethGriggs BethGriggs mentioned this pull request Oct 30, 2018
Merged
@beevelop beevelop mentioned this pull request Nov 20, 2018
Closed
@Trott Trott deleted the one-less-timer branch January 13, 2022 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lpinca lpinca lpinca left review comments

@jasnell jasnell jasnell approved these changes

@BridgeAR BridgeAR BridgeAR approved these changes

@trivikr trivikr trivikr approved these changes

+1 more reviewer

@apapirovski apapirovski apapirovski approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

author ready PRs that have at least one approval, no pending requests for changes, and a CI started. test Issues and PRs related to the tests.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Investigate flaky test-http2-session-timeout

8 participants

@Trott @richardlau @apapirovski @jasnell @lpinca @BridgeAR @trivikr @nodejs-github-bot