Incompatible dependency requirements: snowflake-connector-python & boto3

[bill3tt]

Issue

Issue description

The latest version of dbt (0.10.2) has dependencies which require conflicting versions of botocore.

Specifically:

• boto3 (1.8.3) requires botocore >=1.11.3, <1.12.0
• snowflake-connector-python (1.6.7) requires botocore >=1.5.0, <1.11.0

pip install dbt tries to do something like this...

dbt 0.10.2 was working fine for us on release, but has just started causing problems.

boto3 1.8.0, released 4 days ago, changed the requirement of botocore>=1.10.84,<1.11.0 to botocore>=1.11.0,<1.12.0. This broke compatibility with snowflake-connector-python. See this diff.

Results

This caused our own library, with dbt==0.10.2 to fail to run because of the incompatible versions of botocore.

System information

The output of dbt --version: 0.10.2

installed version: 0.10.2
   latest version: 0.10.2

Up to date!

The operating system you're running on:
MacOS High Sierra 10.13.5

The python version you're using (probably the output of python --version)
Python 3.5.4

Steps to reproduce

python3 -m virtualenv env  # create clean venv 
source venv/bin/activate  # activate new venv
pip install dbt

We then get this warning in the install logs...

snowflake-connector-python 1.6.7 has requirement boto3<1.8.0,>=1.4.4, but you'll have boto3 1.8.3 which is incompatible.
snowflake-connector-python 1.6.7 has requirement botocore<1.11.0,>=1.5.0, but you'll have botocore 1.11.3 which is incompatible.

Suggested Solution

We fixed the issue locally by pinning boto3 to 1.7.84. This satisfies the requirements of both the latest versions of dbt (0.10.2) and snowflake-connector-python (1.6.7).

An alternative would be for snowflake-connector-python to require botocore >=1.11.0.

[drewbanin]

Thanks @ianbillett -- we're going to fix this for dbt v0.11.0, due to be released on 9/6. I think our boto3 range will become >=1.6.23,<1.8.0.

[drewbanin]

@ianbillett this was a really great bug report -- thanks so much for the detailed info! There's a PR for the fix here: #964

[bill3tt]

@drewbanin no problem 👍 - good job on rolling out a fix so quickly.