-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FacebookAdsReportToGcsOperator method _flush_rows() infers field names from first data point instead of declared fields #34173
Comments
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval. |
Is it a bug? |
It is a bug, the flush_rows method accesses the first data points fields as the headers variable, instead of using 'fields'.
(Apologies for closing the issue, it was an accident) |
If you found the problem maybe you can open a PR with the fix? :) |
Yeah sure, I'll give it a go |
Hey Guys, I would like to get an update on this because I am facing the same issue! Has this been resolved in any versions? |
No, apologies I have not had time to do it. I'll remove myself from the task so someone else can pick it up. |
Apache Airflow version
2.5.3+composer
What happened
I created a task to retrieve insight level ad data using the FacebookAdsReportToGcsOperator. Whilst running the pipeline, the dag failed with the following response:
The field 'action_values' was in my requested fields, but I found that it didn't appear in all data points in the data set. Upon inspecting the code, I found that the __flush_rows() method infers the fields (denoted as headers) using the first data point.
Is it possible to get this method amended to infer headers from all requested fields?
What you think should happen instead
The __flush_rows() method shouldn't get the headers (fields) from the first data point, it should get them from the requested fields, or at least view all data points and use the one with the most fields in.
How to reproduce
Create and run a task using the FacebookAdsReportToGcsOperator.
Operating System
Windows 10
Versions of Apache Airflow Providers
Deployment
Google Cloud Composer
Deployment details
Anything else
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: