-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CT-258] [Spike] Upgrade to Jinja3? #4748
Comments
@jtcohen6 did you run this with Jinja3? |
Goal: What are our most viable options for leaving Jinja2? If so what is the impact or work that needs to be done? If moving moving to Jinja3, what is the plan? Potential Options:
|
|
If it's simple to upgrade to jinja 3 we will likely do that. so that will be the first step of the spike. Outcome of this spike is a write up. write up includes consequences if we stay with jinja 2. |
Hi guys, when can we expect the release supporting Jinja3? At the moment - I'm not able to create an environment for generating both mkdocs and dbt docs (we deploy them together), because there's a version conflict between
As a workaround I have to create 2 separate venvs. |
Same here |
DBT can no longer be installed in the same environment as Airflow after Airflow's most recent release 2.3.3 on 2022-07-05 due to a version conflict on Jinja. Details on Jinja Conflict in latest AirflowAirflow updated their Flask App Builder to V4 which included a requirement of Jinja2 >= 3.0Questions
|
We're going to attempt a Jinja3 upgrade as part of v1.3. It's a positive sign that our internal testing is all passing with v3.1.2, but we'll want extensive end user testing to check for subtle breaking changes. Our internal tests do not cover the full range of Jinja2 capabilities, on which users might be depending in their own code. I'm hesitant to loosen |
Given dbt-core's cross compatibility & the concern around end-user code why keep/enforce the single version pin? By keeping the version pin in place it places all the responsibility/blame of jinja2 version problems on the maintainers of dbt-core instead of end-users environment maintainers. If dbt-core were to loosen the restriction dbt-core could stop worrying about it to a larger degree.
If I'm reading this correctly, the order of actions here prevents end users from easily testing out Jinja2 V3. If the version pin was loosened before the jinja2 v3 update users could easily install the v3 version into their environment without fighting with version conflicts in pip/pip-like tools via additional version specs in their requirements.txt/pyprojet.toml/Pipfile/etc.. However, if you leave single version pin in place until after updating to Jinja2 V3.x. you prevent easy testing of the new Jinja2 version by end users. I.e. I'd like to test Jinja2 3.x with our project without any other dbt-core updates and I think that will require forking the project and modifying the pin instead of a simple version pin in our environment... If you do leave the version in place how would you recommend end-users test the new v3.x of Jinja2 in their dbt-core projects? |
@r-richmond It's a really fair point, and a balance we need to make. As I see it, there are two options:
To date, we have preferred option 1 to option 2. FutureThat being said, we are reviewing our policies around dependencies, and my aim in this work is to arrive at a more-balanced approach. (There's an initial start in #5450, with more detailed policies in early draft state.) A balanced approach has two components:
Then, we can provide users with two options:
|
FWIW This would address all my concerns & is also what Airflow does (Loose Pip Constraints & Specific Officially Supported Constraints). So +1 to #5450 & here's to hoping that work can be completed soon so end-users can test/upgrade to Jinja2 3.x without forking. Thank you for your time. |
Picking up from #4745 (comment):
Let's take the time to look through the set of (breaking) changes, and predict the impact on user/project code. Based on that analysis, we should move ahead with an upgrade, or consider our alternatives. Forking/vendoring
Jinja2
is far from ideal, but it is an option on the table.The text was updated successfully, but these errors were encountered: