FacebookAdsReportToGcsOperator CSV conversion option available

[mz-heesun]

Apache Airflow Provider(s)

facebook

Versions of Apache Airflow Providers

3.0.0

Apache Airflow version

2.2.5

Operating System

mac os

Deployment

Docker-Compose

Deployment details

In airflow 2.2.5, docker-compose up environment was established using facebook provider in docker.

What happened

A problem arose under certain circumstances. There was no cpc field.

An error occurred when converting to csv in that situation. It was an error that the field could not be found.

Below is the error message.

[2022-06-30, 10:38:39 UTC] {taskinstance.py:1776} ERROR - Task failed with exception Traceback (most recent call last): File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/transfers/facebook_ads_to_gcs.py", line 165, in execute total_row_count = self._decide_and_flush(converted_rows_with_action=converted_rows_with_action) File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/transfers/facebook_ads_to_gcs.py", line 197, in _decide_and_flush self._flush_rows( File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/transfers/facebook_ads_to_gcs.py", line 227, in _flush_rows writer.writerows(converted_rows) File "/opt/python3.8/lib/python3.8/csv.py", line 157, in writerows return self.writer.writerows(map(self._dict_to_list, rowdicts)) File "/opt/python3.8/lib/python3.8/csv.py", line 149, in _dict_to_list raise ValueError("dict contains fields not in fieldnames: " ValueError: dict contains fields not in fieldnames: 'cpc'

What you think should happen instead

I think you should specify an option so that missing fields can be ignored.

Depending on the user, you may want to proceed without the field.

How to reproduce

If airflow is installed, you can try running the dag.

Instead, it seems that only people who have set up Facebook ads can do this.

Anything else

Actually, in terms of code, it seems like it would be enough to provide only a way to control the option to ignore in the csv file conversion part.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[eladkal]

feel free to submit PR

[eladkal]

The list of fields is controlled by:

airflow/airflow/providers/google/cloud/transfers/facebook_ads_to_gcs.py

Line 68 in fd116cc

:param fields: List of fields that is obtained from Facebook. Found in AdsInsights.Field class.

Isn't excluding cpc from this list a solution to your issue?

[github-actions]

This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[github-actions]

This issue has been closed because it has not received response from the issue author.