Skip to content

Conversation

@rhcarvalho
Copy link
Contributor

This shrinks event sizes a bit, even when gzip'ed.
The compact representation is documented in the json module.

Alternatively, we can also look into using a custom encoder (that could
also handle datetime objects, instead of the current manual
serialization of those).

In the absence of proper benchmark data, consider a random transaction
event t:

>>> len(json.dumps(t)), len(json.dumps(t, separators=(',', ':')))
(82174, 78516)

That is 95.5% of the original size.

With gzip compression:

>>> len(gzips(json.dumps(t))), len(gzips(json.dumps(t, separators=(',', ':'))))
(13093, 12988)

That is 99.2% of the original size.

@rhcarvalho rhcarvalho force-pushed the rhcarvalho/compact-json branch 2 times, most recently from 5dacf67 to a6cfc35 Compare June 29, 2020 16:17
This shrinks event sizes a bit, even when gzip'ed.
The compact representation is documented in the json module.

Alternatively, we can also look into using a custom encoder (that could
also handle datetime objects, instead of the current manual
serialization of those).

In the absence of proper benchmark data, consider a random transaction
event t:

>>> len(json.dumps(t)), len(json.dumps(t, separators=(',', ':')))
(82174, 78516)

That is 95.5% of the original size.

With gzip compression:

>>> len(gzips(json.dumps(t))), len(gzips(json.dumps(t, separators=(',', ':'))))
(13093, 12988)

That is 99.2% of the original size.
@rhcarvalho rhcarvalho force-pushed the rhcarvalho/compact-json branch from a6cfc35 to cb2a8b6 Compare June 29, 2020 17:18
@rhcarvalho rhcarvalho marked this pull request as ready for review June 29, 2020 18:05
@rhcarvalho rhcarvalho requested a review from untitaker June 29, 2020 18:05
@rhcarvalho rhcarvalho merged commit 7d482b5 into master Jun 29, 2020
@rhcarvalho rhcarvalho deleted the rhcarvalho/compact-json branch June 29, 2020 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants