Change nimble_dump streams data representation #207
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary:
#### Make these numeric columns aligned to the right
* Stream Offset
* Stream Length
* Item Count
All numeric columns where values can be large and can be compared to each other should be aligned to the right to simplify comparison.
Before - notice how hard to visually compare the stream sizes:
```
Stripe Id Stream Id Stream Offset Stream Size Item Count Type
....
0 8 45050 5223 8640 0:RLE<String,5218>[Lengths:FixedBitWidth<Uint32,759>{MetaInternal},Values:Trivial<String,4449>{MetaInternal}[Lengths:Constant<Uint32,10>]]
0 9 50273 2381 8640 0:Nullable<String,2376>[Data:Trivial<String,1938>{MetaInternal}[Lengths:Constant<Uint32,10>],Nulls:RLE<Bool,428>[Lengths:FixedBitWidth<Uint32,417>{MetaInternal}]]
0 10 52654 2467 8640 0:RLE<Uint32,2462>[Lengths:FixedBitWidth<Uint32,827>{MetaInternal},Values:FixedBitWidth<Uint32,1625>{MetaInternal}]
0 11 55121 159328 4065638 0:Trivial<Int32,159323>{MetaInternal}
0 12 214449 1800430 4065638 0:Trivial<Float,1800425>{MetaInternal}
0 13 2014879 1685 8640 0:RLE<Uint32,1680>[Lengths:FixedBitWidth<Uint32,776>{MetaInternal},Values:FixedBitWidth<Uint32,894>{MetaInternal}]
0 14 2016564 30583 928681 0:Trivial<Int32,30578>{MetaInternal}
0 15 2047147 90061 928681 0:Nullable<Uint32,90056>[Data:FixedBitWidth<Uint32,90000>{MetaInternal},Nulls:SparseBool<Bool,46>[Indices:Trivial<Uint32,39>{Uncompressed}]]
0 16 2137208 18478612 31493289 0:Trivial<Int64,18478607>{MetaInternal}
0 17 20615820 1476 8640 0:RLE<Uint32,1471>[Lengths:FixedBitWidth<Uint32,749>{MetaInternal},Values:FixedBitWidth<Uint32,712>{MetaInternal}]
0 18 20617296 7437 181218 0:Trivial<Int32,7432>{MetaInternal}
0 19 20624733 21904 181218 0:FixedBitWidth<Uint32,21899>{MetaInternal}
0 20 20646637 7207669 8794713 0:Trivial<Int64,7207664>{MetaInternal}
0 21 27854306 1988499 8794713 0:Trivial<Float,1988494>{MetaInternal}
```
After:
```
Stripe Id Stream Id Stream Offset Stream Length Item Count Type
....
0 8 45050 5223 8640 0:RLE<String,5218>[Lengths:FixedBitWidth<Uint32,759>{MetaInternal},Values:Trivial<String,4449>{MetaInternal}[Lengths:Constant<Uint32,10>]]
0 9 50273 2381 8640 0:Nullable<String,2376>[Data:Trivial<String,1938>{MetaInternal}[Lengths:Constant<Uint32,10>],Nulls:RLE<Bool,428>[Lengths:FixedBitWidth<Uint32,417>{MetaInternal}]]
0 10 52654 2467 8640 0:RLE<Uint32,2462>[Lengths:FixedBitWidth<Uint32,827>{MetaInternal},Values:FixedBitWidth<Uint32,1625>{MetaInternal}]
0 11 55121 159328 4065638 0:Trivial<Int32,159323>{MetaInternal}
0 12 214449 1800430 4065638 0:Trivial<Float,1800425>{MetaInternal}
0 13 2014879 1685 8640 0:RLE<Uint32,1680>[Lengths:FixedBitWidth<Uint32,776>{MetaInternal},Values:FixedBitWidth<Uint32,894>{MetaInternal}]
0 14 2016564 30583 928681 0:Trivial<Int32,30578>{MetaInternal}
0 15 2047147 90061 928681 0:Nullable<Uint32,90056>[Data:FixedBitWidth<Uint32,90000>{MetaInternal},Nulls:SparseBool<Bool,46>[Indices:Trivial<Uint32,39>{Uncompressed}]]
0 16 2137208 18478612 31493289 0:Trivial<Int64,18478607>{MetaInternal}
0 17 20615820 1476 8640 0:RLE<Uint32,1471>[Lengths:FixedBitWidth<Uint32,749>{MetaInternal},Values:FixedBitWidth<Uint32,712>{MetaInternal}]
0 18 20617296 7437 181218 0:Trivial<Int32,7432>{MetaInternal}
0 19 20624733 21904 181218 0:FixedBitWidth<Uint32,21899>{MetaInternal}
0 20 20646637 7207669 8794713 0:Trivial<Int64,7207664>{MetaInternal}
0 21 27854306 1988499 8794713 0:Trivial<Float,1988494>{MetaInternal}
```
#### Rework Formatting
Rework table formatting by adding a column separator because the old implementation put two adjacent columns with right and left alignments without any separators.
Before - Notice how `Item Count` and `Type` values are bunched up together:
```
Stripe Id Stream Id Stream Offset Stream Length Item CountType
0 1 0 420 86400:MainlyConstant<Double,415>[IsCommon:RLE<Bool,379>[Lengths:FixedBitWidth<Uint32,368>{MetaInternal}],OtherValues:Constant<Uint64,14>]
0 2 420 10365 86400:RLE<Int64,10360>[Lengths:FixedBitWidth<Uint32,759>{MetaInternal},Values:Trivial<Uint64,9591>{MetaInternal}]
0 3 10785 5044 86400:RLE<Int64,5039>[Lengths:FixedBitWidth<Uint32,779>{MetaInternal},Values:Trivial<Uint64,4250>{MetaInternal}]
```
After:
```
Stripe Id Stream Id Stream Offset Stream Length Item Count Type
0 1 0 420 8640 0:MainlyConstant<Double,415>[IsCommon:RLE<Bool,379>[Lengths:FixedBitWidth<Uint32,368>{MetaInternal}],OtherValues:Constant<Uint64,14>]
0 2 420 10365 8640 0:RLE<Int64,10360>[Lengths:FixedBitWidth<Uint32,759>{MetaInternal},Values:Trivial<Uint64,9591>{MetaInternal}]
0 3 10785 5044 8640 0:RLE<Int64,5039>[Lengths:FixedBitWidth<Uint32,779>{MetaInternal},Values:Trivial<Uint64,4250>{MetaInternal}]
0 4 15829 9394 8640 0:MainlyConstant<String,9389>[IsCommon:RLE<Bool,1007>[Lengths:FixedBitWidth<Uint32,996>{MetaInternal}],OtherValues:RLE<String,8346>[Lengths:FixedBitWidth<Uint32,23
```
Differential Revision: D78371778
|
This pull request was exported from Phabricator. Differential Revision: D78371778 |
facebook-github-bot
added
CLA Signed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Make these numeric columns aligned to the right
All numeric columns where values can be large and can be compared to each other should be aligned to the right to simplify comparison.
Before - notice how hard to visually compare the stream sizes:
After:
Rework Formatting
Rework table formatting by adding a column separator because the old implementation put two adjacent columns with right and left alignments without any separators.
Before - Notice how
Item CountandTypevalues are bunched up together:After:
Differential Revision: D78371778