feat: add debug option to CLI - #134
Open
jedluhmann wants to merge 22 commits into
Open
Conversation
Upgrades include bun, eslint, prettier, mocha, chai, and c8.
…tringify for array equality) Some comments are left in this commit to help clarify what is being optimized. These comments will be removed in the next commit. Changes: - Cache changes calculated in findMatchingObject() for later use in arrayDiff where op is 'equal' - Use JSON.stringify() to bypass diff logic when arrays are equal - findMatchingObject: Switched the order of arguments when calling this.diff() for consistency (candidate is the original value) and to allow caching (reuse change values calculated here in later fuzzy matching)
…condition immediately above for truthy values
…for failed tests Note: signature for deepEqual is assert.deepEqual(actual, expected, message)
Objective: Simplify the logic by giving each property a maximum possible score of 100 - Matching property: 50 points - Matching value: 50 points - Added/Deleted property: -50 points
This updates the README with a simple diagram that provides a high level overview of how json-diff works.
Added an incredibly useful debug feature that ouputs a pivot table that displays the fuzzy scores that are computed when diffing arrays. When combined with the -v flag, the raw data that the pivot table is derived from is displayed as well. This feature also adds a debug property to the change object, which is incredibly powerful in providing insight into the object and array indices being compared in other methods like arrayDiff() and diff(). Output for test case, it 'should correctly pick best match based on similarity during scalarize, i.e. class obj2[2] should not be selected as best match for obj1[0], instead order should remain unchanged with added class property for obj1[0]' See: https://github.com/jedluhmann/json-diff/blob/b6500e9d5669d9f31ebe51d26670dcbab2129bf3/test/json-diff.spec.js#L187: FUZZY SCORES for new obj B compared to candidate A A0 A1 A2 ----------------------- B0 250 0 100 B1 0 200 - B2 50 0 400 [ { + class: "prose" } ... ... ] In this example, we see a "dash" displayed for the result of the comparison between B1 and A2. This shows one of the recent optimizations in action. B1 and A1 are an exact match both in value and position. Because of this, there is no need to compare B1 with A2 and we simply break out of the loop and continue evaluating the remainder of the array elements. Here is the additional output with the addition of the verbose option (-dv): FUZZY MATCH Records B A score depth key path --- --- ----- ----- ----------- ----------- 0 0 250 0 __$!SCALAR1 $[0] 0 1 0 0 __$!SCALAR2 $[0] 0 0 0 1 __$!SCALAR1 $[0].data[0] 0 2 100 0 __$!SCALAR3 $[0] 1 0 0 0 __$!SCALAR1 $[1] 1 1 200 0 __$!SCALAR2 $[1] 0 0 0 1 __$!SCALAR1 $[2].data[0] 2 0 50 0 __$!SCALAR1 $[2] 2 1 0 0 __$!SCALAR2 $[2] 2 2 400 0 __$!SCALAR3 $[2] Dealing with large datasets can be nearly impossible to debug and reason about. The benefits of this new feature become readily apparent when dealing with arrays containing many objects: FUZZY SCORES for new obj B compared to candidate A A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 A10 A11 A12 A13 --------------------------------------------------------------------------------------- B0 500 - - - - - - - - - - - - - B1 0 250 0 150 0 0 155 0 0 150 0 210 0 210 B2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B3 0 150 0 250 0 0 165 0 0 180 0 200 0 200 B4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B5 210 0 0 0 0 500 - - - - - - - - B6 0 155 0 165 0 0 250 0 0 165 0 205 0 205 B7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B8 210 0 0 0 0 210 0 0 500 - - - - - B9 0 150 0 180 0 0 165 0 0 250 0 200 0 200 B10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B11 0 160 0 150 0 0 155 0 0 150 0 300 0 210 B12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B13 0 160 0 150 0 0 155 0 0 150 0 210 0 300 FUZZY MATCH Records B A score depth key path --- --- ----- ----- ----------- ----------- 0 0 500 0 __$!SCALAR1 $[0] 0 0 0 1 __$!SCALAR1 $[1].data[0] 1 0 0 0 __$!SCALAR1 $[1] 1 1 250 0 __$!SCALAR2 $[1] 0 0 0 1 __$!SCALAR1 $[1].data[0] 1 2 0 0 __$!SCALAR3 $[1] 0 0 0 2 __$!SCALAR1 $[1].data[0].data[0] 0 1 0 2 __$!SCALAR2 $[1].data[0].data[0] 1 0 0 2 __$!SCALAR1 $[1].data[0].data[1] 1 1 300 2 __$!SCALAR2 $[1].data[0].data[1] 0 0 120 1 __$!SCALAR1 $[1].data[0] 0 0 0 2 __$!SCALAR1 $[1].data[0].data[0] 0 1 0 2 __$!SCALAR2 $[1].data[0].data[0] 1 0 0 2 __$!SCALAR1 $[1].data[0].data[1] 1 1 300 2 __$!SCALAR2 $[1].data[0].data[1] 0 1 120 1 __$!SCALAR2 $[1].data[0] 0 0 0 2 __$!SCALAR1 $[1].data[0].data[0] 0 1 0 2 __$!SCALAR2 $[1].data[0].data[0] 1 0 0 2 __$!SCALAR1 $[1].data[0].data[1] 1 1 300 2 __$!SCALAR2 $[1].data[0].data[1] 0 2 120 1 __$!SCALAR3 $[1].data[0] 1 3 150 0 __$!SCALAR4 $[1] 0 0 0 1 __$!SCALAR1 $[1].data[0] 1 4 0 0 __$!SCALAR5 $[1] 0 0 0 1 __$!SCALAR1 $[1].data[0] ... (+748 rows)
For an object with multiple child objects containing arrays, the output now includes the fuzzy scores pivot table for top most array in each child.
For example, given 2 objects with the following shape:
{
"abc": "xyz",
"foo1": {
"bar1": {
"data": [...(8 items)]
},
"bar2": {
"data": [...(2 items)]
},
"bar3": {
"data": [...(4 items)]
}
},
"foo2": {
"bar4": {
"bar5": {
"bar6": {
"baz": {
"data": [...(11 items)]
}
}
}
}
}
}
The debug output will now include 4 pivot tables and look something like:
path: $.foo.bar1.data
FUZZY SCORES for new obj B compared to candidate A
A0 A1 A2 A3 A4 A5 A6 A7
--------------------------------------------------
B0 500 - - - - - - -
B1 0 250 0 155 0 150 0 170
B2 0 0 0 0 0 0 0 0
B3 0 155 0 250 0 155 0 155
B4 0 0 0 0 0 0 0 0
B5 0 170 0 155 0 250 0 170
B6 0 0 0 0 0 0 0 0
B7 0 170 0 155 0 150 0 250
path: $.foo.bar2.data
FUZZY SCORES for new obj B compared to candidate A
A0 A1
-----------------
B0 500 -
B1 0 250
path: $.foo.bar3.data
FUZZY SCORES for new obj B compared to candidate A
A0 A1 A2 A3
----------------------------
B0 500 - - -
B1 0 250 0 150
B2 0 0 0 0
B3 0 150 0 250
path: $.foo2.bar4.bar5.bar6.baz.data
FUZZY SCORES for new obj B compared to candidate A
A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 A10
-------------------------------------------------------------------
B0 250 0 160 0 160 0 160 0 160 0 160
B1 0 0 0 0 0 0 0 0 0 0 0
B2 160 0 250 0 160 0 160 0 160 0 160
B3 0 0 0 0 0 0 0 0 0 0 0
B4 160 0 160 0 250 0 160 0 160 0 160
B5 0 0 0 0 0 0 0 0 0 0 0
B6 160 0 160 0 160 0 250 0 160 0 160
B7 0 0 0 0 0 0 0 0 0 0 0
B8 160 0 160 0 160 0 160 0 250 0 160
B9 0 0 0 0 0 0 0 0 0 0 0
B10 160 0 160 0 160 0 160 0 160 0 250
Author
|
@andreyvit I don't understand why you closed PR #135 basically saying that patch was too big. That PR only changes one file, adding just one test. All of my PRs build on top of the previous one, so the latter ones appear to have a lot of unrelated changes, but that is not really the case. I've put a lot of effort into explaining my work to make it easier for you to review the changes and I've spent a lot of time updating the test cases, so you can have confidence in my work. As far as styling my code like the rest of the project, I'm using ESLint to ensure that my code is formatted according to the industry wide standard. Is it just a matter of removing semicolons? I'm happy to do that if that's what you want. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Added a new debug feature. See the following commits for details: