Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance Tests - RAM disk #17

Merged
merged 5 commits into from
Nov 27, 2020
Merged

Performance Tests - RAM disk #17

merged 5 commits into from
Nov 27, 2020

Conversation

mikicho
Copy link

@mikicho mikicho commented Nov 1, 2020

I created a simple DB performance test that inserts 100 new rows into the database through the API.
From what I see, with non-durable settings the performance difference, between SSD and ram, is neglectable (if any). I tested it also with big tables (500k rows), the insert performance stays the same.
In durable settings, the ramdisk is faster in about 30%-40% than SSD.

My only concern is that our schema is too simplistic, and doesn't have unique keys which will cause more reads on insert and may give an advantage for the ramdisk

My conclusion from this little research is that ramdisk doesn't give a real benefit and we should recommend it only for CI environments because it's so simple (one line), so why not?

I opened this PR to show you what I did and how I got those numbers so maybe you'll find that I did something wrong during my testing, and therefore my conclusion is wrong.

In addition, I:

  • Added an nvm file so all of us will use and test with the same node version
  • Added package-lock` file so all of us will use and tests with the same package's version.
  • Small fix for docker-compose

@goldbergyoni
Copy link
Owner

@mikicho Working toward some big milestone on Sunday afternoon, plan to review by EOD Sunday

Copy link
Owner

@goldbergyoni goldbergyoni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mikicho I would merge what we have now, a good enough performance analysis cycle. We will revisit it in the future. Consider adding your great finding in a dedicated recipe readme.

@@ -0,0 +1,702 @@
[
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor and nitty-gritty - Maybe call this file with something more explanatory for the average reader, maybe "orders-data-for-paramterized-test"

@goldbergyoni
Copy link
Owner

@mikicho Jest parallelizes tests by files, if you want to benefit multi-process you have to create multiple files, at least 4. Do we split here the tests across files? This might change the results

@mikicho
Copy link
Author

mikicho commented Nov 20, 2020

@goldbergyoni how it will improve database performance? (we're talking here about DB RAM only performance improvement. other improvements have another ticket)

Update: Now I see that the name performance is misleading. maybe we should add to this recipe other performance improvements as well but it's out of the scope of this PR

@mikicho mikicho marked this pull request as ready for review November 21, 2020 13:13
@mikicho mikicho requested a review from goldbergyoni November 21, 2020 13:13
@mikicho
Copy link
Author

mikicho commented Nov 21, 2020

added readme

@goldbergyoni
Copy link
Owner

@goldbergyoni how it will improve database performance? (we're talking here about DB RAM only performance improvement. other improvements have another ticket)

Update: Now I see that the name performance is misleading. maybe we should add to this recipe other performance improvements as well but it's out of the scope of this PR

@mikicho We're measuring DB optimizations, as part of measuring we want to utilize them. The more stress we put on DB, the more the optimizations shine. Makes sense or do I miss something?

@mikicho
Copy link
Author

mikicho commented Nov 22, 2020

@goldbergyoni I see. so you want multiple test files in the recipe which have this parameterized test?
I'll check it... but I don't think it will do any better. I think the non-durable settings reduce the I/O to almost zero and make it neglectable

@mikicho
Copy link
Author

mikicho commented Nov 27, 2020

@goldbergyoni tested it.. didn't see any improvement between ramdisk and SSD with non-durable settings

@mikicho mikicho merged commit 0f1888f into master Nov 27, 2020
@mikicho mikicho deleted the Michael/db-ram branch November 27, 2020 20:34
@mikicho mikicho linked an issue Nov 27, 2020 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DB ram drive work all over including Windows and Mac
2 participants