Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SUPPORT] Athena does not support s3a partition scheme anymore leading to missing data #10595

Closed
parisni opened this issue Jan 31, 2024 · 0 comments · Fixed by #10596
Closed
Labels
aws-support priority:critical production down; pipelines stalled; Need help asap. release-0.15.0

Comments

@parisni
Copy link
Contributor

parisni commented Jan 31, 2024

Describe the problem you faced

Few days ago we had suddently 0 data in all our hudi partitionned tables when querying from athena. Spark and redshift spectrum were not affected by the issue.

We found out that new athena version silently drop hudi data when the partition location has a s3a scheme. Recreating the table with partition s3 scheme fixed the issue

This affect any version of hudi, when the basepath is specified with s3a scheme

@codope codope added aws-support priority:critical production down; pipelines stalled; Need help asap. release-0.14.2 Patches/Issue fixes targetted for 0.14.2 release labels Jan 31, 2024
@codope codope added release-0.15.0 and removed release-0.14.2 Patches/Issue fixes targetted for 0.14.2 release labels Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
aws-support priority:critical production down; pipelines stalled; Need help asap. release-0.15.0
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants