Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Structured Logging #4055

Merged
merged 31 commits into from
Nov 9, 2021
Merged

Add Structured Logging #4055

merged 31 commits into from
Nov 9, 2021

Commits on Nov 9, 2021

  1. init

    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    57ae918 View commit details
    Browse the repository at this point in the history

  2. Initial structured logging work with fire_event (#4137) 

    add event type modeling and fire_event calls
    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    f9ef9da View commit details
    Browse the repository at this point in the history

  3. context call sites (#4164) 

    * updated context dir to new structured logging
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    69aa6bf View commit details
    Browse the repository at this point in the history

  4. Handle exec info (#4168) 

    handle exec info
    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    44a9da6 View commit details
    Browse the repository at this point in the history

  5. Client call sites (#4163) 

    update log call sites with new event system
    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    5b2562a View commit details
    Browse the repository at this point in the history

  6. Change Graph logger call sites (#4165) 

    graph call sites for structured logging
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    51d8440 View commit details
    Browse the repository at this point in the history

  7. contracts call sites (#4166) 

    * first pass adding structured logging
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    b141620 View commit details
    Browse the repository at this point in the history

  8. Dataclass compatibility (#4180) 

    * use __post_init__() instead of fake dataclass member vars
    @iknox-fa
    iknox-fa authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    d8868c5 View commit details
    Browse the repository at this point in the history

  9. config call sites (#4169) 

    * update config use structured logging

* WIP

* minor cleanup

* fixed merge error

* added in ShowException

* added todo to remove defaults after dropping 3.6

* removed todo that is obsolete
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    6b36b18 View commit details
    Browse the repository at this point in the history

  10. parser call sites (#4177) 

    * convert generic_test to structured logging

* convert macros to structured logging

* add struc logging to most of manifest.py

* add struct logging to models.py

* added struct logging to partial.py

* finished conversion of manifest

* fixing errors

* fixed 1 todo and added another

* fixed bugs from merge
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    9857e1d View commit details
    Browse the repository at this point in the history

  11. task call sites part 1 (#4183) 

    * add struct logging to base.py

* struct logging in run_operation

* add struct logging to base

* add struct logging to clean

* add struct logging to debug

* add struct logging to deps

* fix errors

* add struct logging to run.py

* fixed flake error

* add struct logging to geneerate

* added debug level stack trace

* fixed flake error

* added struct logging to compile

* added struct logging to freshness

* cleaned up errors

* resolved bug that broke everything

* removed accidental import

* fixed bug with unused args
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    281d249 View commit details
    Browse the repository at this point in the history

  12. Show Exception should trigger a stack trace (#4190)

    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    d513491 View commit details
    Browse the repository at this point in the history

  13. scrub the secrets (#4203) 

    scrub secrets in event module
    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    13f31ae View commit details
    Browse the repository at this point in the history

  14. task callsites: part 2 (#4188) 

    * add struct logging to docs serve

* remove merge fluff

* struct logging to seed command

* converting print to use structured logging

* more structured logging print conversion

* pulling apart formatting more

* added struct logging by disecting printer.py

* add struct logging to runnable

* add struct logging to task init

* fixed formatting

* more formatting and moving things around
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    3cafc9e View commit details
    Browse the repository at this point in the history

  15. deps call sites (#4199) 

    * add struct logging to base

* add struct logging to git

* add struct logging to deps

* remove blank line

* fixed stray merge error
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    e0b0eda View commit details
    Browse the repository at this point in the history

  16. call sites in core/dbt (excluding main.py) (#4202) 

    * add struct logging to compilation

* add struct logging to tracking

* add struct logging to utils

* add struct logging to exceptions

* fixed some misc errors

* updated to send raw ex, removed resulting circ dep
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    d8b97c1 View commit details
    Browse the repository at this point in the history

  17. trivial logger removal (#4216)

    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    6334365 View commit details
    Browse the repository at this point in the history

  18. test/integration call sites (#4209) 

    * added struct logging to base

* fixed merge wierdness

* convert to use single type for integration tests

* converted to 3 reusable test types in sep module

* tweak message

* clean up and making test_types complete for future

* fix missed import
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    bd3e623 View commit details
    Browse the repository at this point in the history

  19. remove unused import (#4217)

    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    b5c6f09 View commit details
    Browse the repository at this point in the history

  20. lazy logging in event module (#4210) 

    * switches on debug level to guard against expensive messages

* adds memoization to msg construction
    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    25c974a View commit details
    Browse the repository at this point in the history

  21. use event types in main even before the logger is set up. (#4219)

    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    f95e9ef View commit details
    Browse the repository at this point in the history

  22. updates associated with merging main 

    - removed 3 new log call sites and replaced with structured logs
- removed 2 unused struc logs
    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    5cc8626 View commit details
    Browse the repository at this point in the history

  23. removed redundant timestamp (#4239)

    @emmyoop
    emmyoop authored and Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    43b39fd View commit details
    Browse the repository at this point in the history

  24. Struct log for adapter call sites (#4189) 

    graph call sites for structured logging

Co-authored-by: Nathaniel May <[email protected]>
Co-authored-by: Emily Rockman <[email protected]>
    @iknox-fa @emmyoop
    3 people committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    b2aea11 View commit details
    Browse the repository at this point in the history

  25. std logger for structured logging (#4231) 

    structured logging powered by the stdlib logger

Co-authored-by: Emily Rockman <[email protected]>
Co-authored-by: Ian Knox <[email protected]>
    @emmyoop @iknox-fa
    3 people committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    a40550b View commit details
    Browse the repository at this point in the history

  26. whitespace change

    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    2ca6ce6 View commit details
    Browse the repository at this point in the history

  27. postgres adapter to use new logger

    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    ebb84c4 View commit details
    Browse the repository at this point in the history

  28. fixes

    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    683190b View commit details
    Browse the repository at this point in the history

  29. rebased on main and added new partial parsing event

    Nathaniel May committed Nov 9, 2021
    Configuration menu
    Copy the full SHA
    31acb95 View commit details
    Browse the repository at this point in the history

  30. configure event logger for integration tests (#4257) 

    * apply test fixes

* remove presto test
    Nathaniel May authored Nov 9, 2021
    Configuration menu
    Copy the full SHA
    5e6be16 View commit details
    Browse the repository at this point in the history

  31. Env var shim to enable legacy logger (#4255) 

    * Env var shim to reenable logbook

* Rename to ENABLE_LEGACY_LOGGER
    @jtcohen6
    jtcohen6 authored Nov 9, 2021
    Configuration menu
    Copy the full SHA
    6dd9c2c View commit details
    Browse the repository at this point in the history