Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(ingest): add bigquery-queries source #10994

Merged
merged 20 commits into from
Aug 26, 2024
Merged

feat(ingest): add bigquery-queries source #10994

merged 20 commits into from
Aug 26, 2024

Commits on Aug 8, 2024

  1. feat(ingest): add bigquery-queries source

    @mayurinehate
    mayurinehate committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    d33bc97 View commit details
    Browse the repository at this point in the history

  2. refractor to separate BigQueryFilter, BigQueryIdentifierBuilder 

    - update filtering in bigquery-queries source to use allow deny patterns
    @mayurinehate
    mayurinehate committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    19d9b61 View commit details
    Browse the repository at this point in the history

  3. minor changes

    @mayurinehate
    mayurinehate committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    139c040 View commit details
    Browse the repository at this point in the history

  4. fix user urn, more reporting, some refractors

    @mayurinehate
    mayurinehate committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    ae21deb View commit details
    Browse the repository at this point in the history

  5. more logs, sharded table legacy support

    @mayurinehate
    mayurinehate committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    13d5f9a View commit details
    Browse the repository at this point in the history

  6. add list projects timer details

    @mayurinehate
    mayurinehate committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    e96febb View commit details
    Browse the repository at this point in the history

  7. tweak log statement

    @hsheth2 @mayurinehate
    hsheth2 authored and mayurinehate committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    a7af11e View commit details
    Browse the repository at this point in the history

  8. safeguard boolean checks 

    otherwise exception is thrown during parsing.e.g. list index out of range
    @mayurinehate
    mayurinehate committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    04257a8 View commit details
    Browse the repository at this point in the history

  9. deduplicate queries, fix empty upstream lineage

    @mayurinehate
    mayurinehate committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    8f8b7f5 View commit details
    Browse the repository at this point in the history

  10. add golden file, refractors

    @mayurinehate
    mayurinehate committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    d997bb6 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. review comments + queries for operation without lineage

    @mayurinehate
    mayurinehate committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    4153ac5 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. Merge branch 'master' into bigquery-queries

    @mayurinehate
    mayurinehate authored Aug 14, 2024
    Configuration menu
    Copy the full SHA
    d5c05ed View commit details
    Browse the repository at this point in the history

  2. fix import

    @mayurinehate
    mayurinehate committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    a6e455f View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. add queries test using cached audit log

    @mayurinehate
    mayurinehate committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    ff2cb20 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'datahub-oss/master' into bigquery-queries

    @mayurinehate
    mayurinehate committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    fe8c392 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. remove unneeded log line

    @mayurinehate
    mayurinehate committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    ecbd7fe View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2024

  1. more changes 

    - address review comments
- support adding extra_info for debugging with queries
- fix usage issue, add unit test for sql aggregator usage
    @mayurinehate
    mayurinehate committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    c90bcb9 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. Merge remote-tracking branch 'origin/master' into bigquery-queries

    @mayurinehate
    mayurinehate committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    e0f8434 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into bigquery-queries

    @mayurinehate
    mayurinehate authored Aug 23, 2024
    Configuration menu
    Copy the full SHA
    6355683 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. fix test

    @mayurinehate
    mayurinehate committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    e8fcdcb View commit details
    Browse the repository at this point in the history