Lenient schema validation #145
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
skestle
force-pushed
the
lenient-schema-validation
branch
from
e857258
to
a90b4bb
Compare
|
This is draft simply because #144 has not been merged. |
MrPowers
reviewed
Jun 11, 2021
| requiredSchema | ||
| ) | ||
|
|
||
| c.validateSchema() |
There was a problem hiding this comment.
Can you please add a test assertion?
…different order (mrpowers-io#144) * Highlighted issues with schema validation's order sensitivity of nested structures. * Implemented recursive schema checker Enables nested structures to be compared regardless of order.
Previously, a dataframe would be considered invalid if it had not-null data, but the schema allowed the data to be nullable
skestle
force-pushed
the
lenient-schema-validation
branch
from
a90b4bb
to
c84ded8
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(This single commit PR requires the deep DF validation changes)
When validating data against a schema, it verifies that all field elements are the same.
However, a data frame column that is specified as non-null should be considered valid against a schema column that allows nulls.
This PR implements that change.