Bug/duplicate deleted lead ids

[m-feeser]

Are you a current Fivetran customer?

Matt Feeser, Data Engineer, Greenhouse

What change(s) does this PR introduce?

This is a bug fix which addresses a "system glitch" in marketo which allows the same lead_id to be deleted twice. The duplicate delete causes the marketo__leads.lead_id.unique test to fail.

Did you update the CHANGELOG?

• Yes

Does this PR introduce a breaking change?

• Yes (please provide breaking change details below.)
• No (please provide an explanation as to how the change is non-breaking below.)
  The int_marketo__lead model is materialized as a view, so there is no physicalization of data in play. Therefore no reprocessing of previously loaded data is necessary. Moreover, no columns from the modified CTE are exposed. The result is only used in a case statement to set the is_deleted column value.

Did you update the dbt_project.yml files with the version upgrade (please leverage standard semantic versioning)? (In both your main project and integration_tests)

• Yes

Is this PR in response to a previously created Bug or Feature Request

• Yes, Issue/Feature [link bug/feature number here]
• No
  [Bug] int_marketo__lead model introducing duplicates #25

How did you test the PR changes?

• Buildkite
• Local (please provide additional testing details below)

1. Ran the original model against data which contained duplicate lead_id values. The resulting view int_marketo__lead contained multiple records with the same lead_id.
2. Applied proposed fix (distinct lead_id), ran the model against the same data which contained the duplicate lead_id values. The resulting view no longer contained multiple records for the same lead_id.

Select which warehouse(s) were used to test the PR

• BigQuery
• Redshift
• Snowflake
• Postgres
• Databricks
• Other (provide details below)

Provide an emoji that best describes your current mood

👷

Feedback

We are so excited you decided to contribute to the Fivetran community dbt package! We continue to work to improve the packages and would greatly appreciate your feedback on our existing dbt packages or what you'd like to see next.

[m-feeser]

Hi @fivetran-joemarkiewicz. Do you have an ETA on when this PR will be reviewed? We're still having regular failures in our marketo package executions and this will fix it. Thanks.

[fivetran-joemarkiewicz]

Hi @m-feeser my apologies for the delay. I'll work to complete the review of this PR in the current week.

[fivetran-joemarkiewicz]

@m-feeser the changes you are proposing look great and I can see how they will fix the system glitch you are noting while not compromising the expected outcome of the model.

I have just a few requests to make small updates to the CHANGELOG. Once those are applied, we should be able to approve and kick off the merge/release process. Let me know if you have any questions 😄

[fivetran-joemarkiewicz]

Thanks for sharing the insight that there is no documentation around the "system glitch". Although I feel Marketo should probably document the glitch 😬.

With that, your changes look appropriate and I am comfortable approving your PR. I will now kick off our internal release process. You should see your update released in the next patch version early on Monday.

[fivetran-joemarkiewicz]

Thanks for sharing the insight that there is no documentation around the "system glitch". Although I feel Marketo should probably document the glitch 😬.

With that, your changes look appropriate and I am comfortable approving your PR. I will now kick off our internal release process. You should see your update released in the next patch version early on Monday.

My apologies, we will be off on Monday for Martin Luther King day. You can expect this release to be live early on Tuesday. Have a great weekend!

[m-feeser]

Thank you @fivetran-joemarkiewicz! 🎉