Skip to content

test: fix flaky test-timers-blocking-callback#9198

Closed
Trott wants to merge 1 commit intonodejs:masterfrom
Trott:timers-blocking-callback
Closed

test: fix flaky test-timers-blocking-callback#9198
Trott wants to merge 1 commit intonodejs:masterfrom
Trott:timers-blocking-callback

Conversation

@Trott
Copy link
Member

@Trott Trott commented Oct 20, 2016

Checklist
  • make -j8 test (UNIX), or vcbuild test nosign (Windows) passes
  • commit message follows commit guidelines
Affected core subsystem(s)

test timers

Description of change

This test was failing on FreeBSD from time to time in the project CI.
The bug the test was written for would guarantee that the timer would
fire at least 100ms late, but the assertion was firing if it was more
than 50ms late.

This changes the assertion to fire when the timer is more than 100ms
late.

I ran a modified version of this test using 0.10.38 (which has the bug)
and 0.10.39 (which has the fix) to confirm that it still fails in the
buggy one and passes in the fixed one.

/cc @misterdjules @Fishrock123

@Trott
test: fix flaky test-timers-blocking-callback
76ad5b2 
This test was failing on FreeBSD from time to time in the project CI.
The bug the test was written for would guarantee that the timer would
fire at least 100ms late, but the assertion was firing if it was more
than 50ms late.

This changes the assertion to fire when the timer is more than 100ms
late.

I ran a modified version of this test using 0.10.38 (which has the bug)
and 0.10.39 (which has the fix) to confirm that it still fails in the
buggy one and passes in the fixed one.
@Trott Trott added timers Issues and PRs related to the timers subsystem / setImmediate, setInterval, setTimeout. test Issues and PRs related to the tests. labels Oct 20, 2016
@nodejs-github-bot nodejs-github-bot added the test Issues and PRs related to the tests. label Oct 20, 2016
Fishrock123
Fishrock123 approved these changes Oct 20, 2016
View reviewed changes
Copy link
Contributor

@Fishrock123 Fishrock123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

¯_(ツ)_/¯

@misterdjules
Copy link

misterdjules commented Oct 20, 2016 •
edited
Loading

This is definitely an improvement, so the changes look good to me. However the test would still be inherently flaky, because there's no guarantee of when the timers will fire regardless of the bug that was fixed by the change that came with this test.

For instance, one recent test failure showed that it took around 1.2 seconds for the test to fail, when in total it should take ~600ms for the test to complete if the process running that program is always on CPU.

In other words, I'd expect these changes to result in this test being less flaky, but still flaky.

@Trott
Copy link
Member Author

Trott commented Oct 20, 2016

CI: https://ci.nodejs.org/job/node-test-pull-request/4598/

@Trott
Copy link
Member Author

Trott commented Oct 20, 2016

OMG, CI is yellow, I'll take it!

Trott added a commit to Trott/io.js that referenced this pull request Oct 21, 2016
@Trott
test: fix flaky test-timers-blocking-callback
5d818cc 
This test was failing on FreeBSD from time to time in the project CI.
The bug the test was written for would guarantee that the timer would
fire at least 100ms late, but the assertion was firing if it was more
than 50ms late.

This changes the assertion to fire when the timer is more than 100ms
late.

I ran a modified version of this test using 0.10.38 (which has the bug)
and 0.10.39 (which has the fix) to confirm that it still fails in the
buggy one and passes in the fixed one.

PR-URL: nodejs#9198
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Julien Gilli <jgilli@nodejs.org>
@Trott
Copy link
Member Author

Trott commented Oct 21, 2016

Landed in 5d818cc

@Trott Trott closed this Oct 21, 2016
jasnell pushed a commit that referenced this pull request Oct 24, 2016
@Trott @jasnell
test: fix flaky test-timers-blocking-callback
a717be8 
This test was failing on FreeBSD from time to time in the project CI.
The bug the test was written for would guarantee that the timer would
fire at least 100ms late, but the assertion was firing if it was more
than 50ms late.

This changes the assertion to fire when the timer is more than 100ms
late.

I ran a modified version of this test using 0.10.38 (which has the bug)
and 0.10.39 (which has the fix) to confirm that it still fails in the
buggy one and passes in the fixed one.

PR-URL: #9198
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Julien Gilli <jgilli@nodejs.org>
@misterdjules misterdjules mentioned this pull request Oct 26, 2016
Closed
MylesBorins pushed a commit that referenced this pull request Nov 17, 2016
@Trott
test: fix flaky test-timers-blocking-callback
7ef2f7a 
This test was failing on FreeBSD from time to time in the project CI.
The bug the test was written for would guarantee that the timer would
fire at least 100ms late, but the assertion was firing if it was more
than 50ms late.

This changes the assertion to fire when the timer is more than 100ms
late.

I ran a modified version of this test using 0.10.38 (which has the bug)
and 0.10.39 (which has the fix) to confirm that it still fails in the
buggy one and passes in the fixed one.

PR-URL: #9198
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Julien Gilli <jgilli@nodejs.org>
MylesBorins pushed a commit that referenced this pull request Nov 19, 2016
@Trott @MylesBorins
test: fix flaky test-timers-blocking-callback
60704fb 
This test was failing on FreeBSD from time to time in the project CI.
The bug the test was written for would guarantee that the timer would
fire at least 100ms late, but the assertion was firing if it was more
than 50ms late.

This changes the assertion to fire when the timer is more than 100ms
late.

I ran a modified version of this test using 0.10.38 (which has the bug)
and 0.10.39 (which has the fix) to confirm that it still fails in the
buggy one and passes in the fixed one.

PR-URL: #9198
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Julien Gilli <jgilli@nodejs.org>
@MylesBorins MylesBorins mentioned this pull request Nov 22, 2016
Merged
@misterdjules misterdjules mentioned this pull request Aug 14, 2017
Closed
@Trott Trott deleted the timers-blocking-callback branch January 13, 2022 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jasnell jasnell jasnell approved these changes

+1 more reviewer

@Fishrock123 Fishrock123 Fishrock123 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

test Issues and PRs related to the tests. timers Issues and PRs related to the timers subsystem / setImmediate, setInterval, setTimeout.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Trott @misterdjules @jasnell @Fishrock123 @MylesBorins @nodejs-github-bot