repl: fix/improve persistent repl history

[Fishrock123]

Fix the history limiting to take the most recent. History additions are unshifted in readline.

Now tests the saved repl history size limiting.

Cleans up the test a bunch in the process, and also adds some debug so you can figure out which test case failed if it fails. Let me know if there is a better way to do that.

[evanlucas]

Still failing for me.

Failed test # 8

/Users/evan/dev/code/forks/io.js/test/sequential/test-repl-persistent-history.js:197
          throw err;
          ^
AssertionError: '> \'=^.^=\'' === '> \'42\''
    at Writable.write [as _write] (/Users/evan/dev/code/forks/io.js/test/sequential/test-repl-persistent-history.js:194:18)
    at doWrite (_stream_writable.js:292:12)
    at writeOrBuffer (_stream_writable.js:278:5)
    at Writable.write (_stream_writable.js:207:11)
    at REPLServer._writeToOutput (readline.js:219:17)
    at REPLServer.Interface._refreshLine (readline.js:259:8)
    at REPLServer.Interface._historyPrev (readline.js:572:10)
    at REPLServer.Interface._ttyWrite (readline.js:858:14)
    at ActionStream.onkeypress (readline.js:105:10)
    at emitTwo (events.js:87:13)

[evanlucas]

Yep, that fixes it for me. We should probably run CI though to be safe.

[Fishrock123]

CI: https://jenkins-iojs.nodesource.com/job/node-test-pull-request/85/

[evanlucas]

LGTM if the CI is happy

[Fishrock123]

Hmm, windows isn't liking the cleanup, maybe I'll just have to refresh the tmpdir.

not ok 875 - test-repl-persistent-history.js
# c:\workspace\node-test-commit-windows\nodes\win2012r2\test\sequential\test-repl-persistent-history.js:213
# if (err) throw err;
# ^
# 
# Error: EPERM: operation not permitted, open 'c:\workspace\node-test-commit-windows\nodes\win2012r2\test\tmp.0\.node_repl_history'
# at Error (native)

[Fishrock123]

New CI: https://jenkins-iojs.nodesource.com/job/node-test-pull-request/86/

[Fishrock123]

Uh, ok.. Hrmmm. New windows (and ARM) failure from common.refreshTmpDir()...

not ok 875 - test-repl-persistent-history.js
# c:\workspace\node-test-commit-windows\nodes\win2012r2\test\common.js:34
# throw e;
# ^
# 
# Error: ENOTEMPTY: directory not empty, rmdir 'c:\workspace\node-test-commit-windows\nodes\win2012r2\test\tmp.0'
# at Error (native)
# at Object.fs.rmdirSync (fs.js:763:18)
# at rmdirSync (c:\workspace\node-test-commit-windows\nodes\win2012r2\test\common.js:49:10)
# at rimrafSync (c:\workspace\node-test-commit-windows\nodes\win2012r2\test\common.js:25:7)
# at Object.exports.refreshTmpDir (c:\workspace\node-test-commit-windows\nodes\win2012r2\test\common.js:55:3)
# at REPLServer.<anonymous> (c:\workspace\node-test-commit-windows\nodes\win2012r2\test\sequential\test-repl-persistent-history.js:211:16)
# at emitNone (events.js:72:20)
# at REPLServer.emit (events.js:166:7)
# at REPLServer.Interface.close (readline.js:285:8)
# at REPLServer.replClose [as close] (repl.js:486:29)

[Fishrock123]

I think it needed to wait for I/O to close. Fingers crossed...

CI again: https://jenkins-iojs.nodesource.com/job/node-test-pull-request/88/

[evanlucas]

CI still not happy on Windows :[

not ok 875 - test-repl-persistent-history.js
# c:\workspace\node-test-commit-windows\nodes\win2008r2\test\common.js:34
# throw e;
# ^
# 
# Error: ENOTEMPTY: directory not empty, rmdir 'c:\workspace\node-test-commit-windows\nodes\win2008r2\test\tmp.0'
# at Error (native)
# at Object.fs.rmdirSync (fs.js:763:18)
# at rmdirSync (c:\workspace\node-test-commit-windows\nodes\win2008r2\test\common.js:49:10)
# at rimrafSync (c:\workspace\node-test-commit-windows\nodes\win2008r2\test\common.js:25:7)
# at Object.exports.refreshTmpDir (c:\workspace\node-test-commit-windows\nodes\win2008r2\test\common.js:55:3)
# at Immediate.setupTest (c:\workspace\node-test-commit-windows\nodes\win2008r2\test\sequential\test-repl-persistent-history.js:163:23)
# at Immediate.immediate._onImmediate (timers.js:423:18)
# at processImmediate [as _immediateCallback] (timers.js:371:17)

[Fishrock123]

I'm not sure what's up with that.

Somehow this is failing: https://github.com/nodejs/node/blob/master/test/common.js#L15-L57

Specifically: https://github.com/nodejs/node/blob/master/test/common.js#L45-L48

I thought it might have been if the history file was written to after it starts clearing the files but I don't think that is possible...

[silverwind]

Might be a Windows quirk: isaacs/rimraf#72

I think it must be intermittent, but we should still attempt an workaround later. Maybe give the CI a few more runs?

[evanlucas]

@Fishrock123 any update on this?

[Fishrock123]

@evanlucas on the back burner right now. I think It can be fixed by replacing the refreshTmpDir with a function that checks if the file exists with a stat, and then unlinks if it does.

[silverwind]

I think this can be worked around by running the rmDirSync a few times. I dealt with these Windows issues before and I think running 10-20 times usually does the job. It's an ugly hack indeed.

[Fishrock123]

Hopefully this last patch fixed those stupid windows issues. https://jenkins-iojs.nodesource.com/job/node-test-pull-request/182/

[Fishrock123]

Hmm, the rebase failed on that. here's a manual run: https://jenkins-iojs.nodesource.com/job/node-test-commit/339/

[Fishrock123]

Ugh. EPERM. Whatever that means in this context.

[silverwind]

Might as well install a full rimraf and be done with it, I know that it can handle EPERM at least.

[Fishrock123]

https://jenkins-iojs.nodesource.com/job/node-test-commit/350/

[Fishrock123]

Ok this EPERM isn't coming from the unlinkSync though. I think it's coming from fs.open in internal/repl...?

[Fishrock123]

cc @nodejs/collaborators PTAL at the last commit

[Fishrock123]

@silverwind @evanlucas can I get a last LGTM on this? this works without any even minor breaking changes now.

[evanlucas]

Ok, just pulled down and tested. I've run the tests hundreds of times now and haven't had them fail.

LGTM

[Fishrock123]

Thanks, landed in d8db757...73b7e05

@nodejs/lts this is something we want in v4.x.

[jasnell]

Landed in v4.x-staging in 9fa8c32 and 594e8e7