Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ingest/great_expectations): support in-memory (Pandas) data assets #9811

Conversation

bouaouda-achraf
Copy link
Contributor

@bouaouda-achraf bouaouda-achraf commented Feb 11, 2024

Closes #9810

Describe the feature
Adding the ability to publish metadata related to in-memory Data Assets .

Context
Use it when the data Source is not implemented yet ( victoria metrics , ..... ) .

Expected behavior
Assertion metadata are pushed to Datahub.

@github-actions github-actions bot added ingestion PR or Issue related to the ingestion of metadata community-contribution PR or Issue raised by member(s) of DataHub Community labels Feb 11, 2024
Copy link
Collaborator

@hsheth2 hsheth2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good

Can you add a test to tests/unit/test_great_expectations_action.py?

@bouaouda-achraf
Copy link
Contributor Author

The code looks good

Can you add a test to tests/unit/test_great_expectations_action.py?

Ok i will do it ( and also rebase with master ) .
thanks 🙏

Copy link
Collaborator

@hsheth2 hsheth2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly looks good, just need to tighten up the testing a bit

@bouaouda-achraf bouaouda-achraf force-pushed the great-expectations-pandas-support branch 2 times, most recently from dde0969 to 0a43794 Compare March 9, 2024 12:28
@bouaouda-achraf bouaouda-achraf marked this pull request as draft March 9, 2024 12:29
@bouaouda-achraf bouaouda-achraf marked this pull request as ready for review March 9, 2024 12:30
@hsheth2 hsheth2 added merge-pending-ci A PR that has passed review and should be merged once CI is green. and removed pending-submitter-response Issue/request has been reviewed but requires a response from the submitter labels Mar 11, 2024
@hsheth2
Copy link
Collaborator

hsheth2 commented Mar 21, 2024

@bouaouda-achraf looks like there's a lint issue here

@bouaouda-achraf bouaouda-achraf force-pushed the great-expectations-pandas-support branch from 620234d to 7b4afa2 Compare March 21, 2024 21:38
@bouaouda-achraf
Copy link
Contributor Author

@bouaouda-achraf looks like there's a lint issue here

fixed @hsheth2

@hsheth2 hsheth2 changed the title feat: [great_expectations] supporting in-memory (Pandas) Data Assets … feat(ingest/great_expectations): support in-memory (Pandas) data assets Mar 21, 2024
@hsheth2 hsheth2 merged commit b5615fa into datahub-project:master Apr 4, 2024
53 checks passed
sleeperdeep pushed a commit to sleeperdeep/datahub that referenced this pull request Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community-contribution PR or Issue raised by member(s) of DataHub Community ingestion PR or Issue related to the ingestion of metadata merge-pending-ci A PR that has passed review and should be merged once CI is green.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[great_expectations] supporting in-memory (Pandas) Data Assets
2 participants