Revert "Merge branch 'dev/wilt-chamberlain' into snowflake_create_or_…

…replace"

This reverts commit 3ab8238, reversing
changes made to 43a9db5.

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.13.0
+current_version = 0.13.0a1
 parse = (?P<major>\d+)
  \.(?P<minor>\d+)
  \.(?P<patch>\d+)
@@ -20,8 +20,6 @@ values =
 [bumpversion:part:num]
 first_value = 1
 
-[bumpversion:file:setup.py]
-
 [bumpversion:file:core/setup.py]
 
 [bumpversion:file:core/dbt/version.py]

.circleci/config.yml

@@ -19,7 +19,7 @@ jobs:
  PGUSER: root
  PGPASSWORD: password
  PGDATABASE: postgres
- - run: tox -e flake8,unit-py27,unit-py36
+ - run: tox -e pep8,unit-py27,unit-py36
  integration-postgres-py36:
  docker: *test_and_postgres
  steps:

.coveragerc

@@ -1,4 +1,4 @@
 [report]
 include =
  core/dbt/*
- plugins/*/dbt/*
+ plugins/adapters/dbt/*

CHANGELOG.md

@@ -1,71 +1,20 @@
-## dbt 0.13.0 - Stephen Girard (March 21, 2019)
+## dbt dev/stephen-girard (0.13.0? - To be released)
 
-### Overview
+## Overview
 
-This release provides [a stable API for building new adapters](https://docs.getdbt.com/v0.13/docs/building-a-new-adapter) and reimplements dbt's adapters as "plugins". Additionally, a new adapter for [Presto](https:/fishtown-analytics/dbt-presto) was added using this architecture. Beyond adapters, this release of dbt also includes [Sources](https://docs.getdbt.com/v0.13/docs/using-sources) which can be used to document and test source data tables. See the full list of features added in 0.13.0 below.
+This release makes dbt and its adapters into a core-and-plugin architecture.
 
 ### Breaking Changes
-- version 1 schema.yml specs are no longer implemented. Please use the version 2 spec instead ([migration guide](https://docs.getdbt.com/docs/upgrading-from-0-10-to-0-11#section-schema-yml-v2-syntax))
-- `{{this}}` is no longer implemented for `on-run-start` and `on-run-end` hooks. Use `{{ target }}` or an [`on-run-end` context variable](https://docs.getdbt.com/reference#schemas) instead ([#1176](https:/fishtown-analytics/dbt/pull/1176), implementing [#878](https:/fishtown-analytics/dbt/issues/878))
-- A number of materialization-specific adapter methods have changed in breaking ways. If you use these adapter methods in your macros or materializations, you may need to update your code accordingly.
- - query_for_existing - **removed**, use [get_relation](https://docs.getdbt.com/v0.13/reference#adapter-get-relation) instead.
- - [get_missing_columns](https://docs.getdbt.com/v0.13/reference#adapter-get-missing-columns) - changed to take `Relation`s instead of schemas and identifiers
- - [expand_target_column_types](https://docs.getdbt.com/v0.13/reference#adapter-expand-target-column-types) - changed to take a `Relation` instead of schema, identifier
- - [get_relation](https://docs.getdbt.com/v0.13/reference#adapter-get-relation) - added a `database` argument
- - [create_schema](https://docs.getdbt.com/v0.13/reference#adapter-create-schema) - added a `database` argument
- - [drop_schema](https://docs.getdbt.com/v0.13/reference#adapter-drop-schema) - added a `database` argument
-
-### Deprecations
-- The following adapter methods are now deprecated, and will be removed in a future release:
- - get_columns_in_table - deprecated in favor of [get_columns_in_relation](https://docs.getdbt.com/v0.13/reference#adapter-get-columns-in-relation)
- - already_exists - deprecated in favor of [get_relation](https://docs.getdbt.com/v0.13/reference#adapter-get-relation)
-
-### Features
-- Add `source`s to dbt, use them to calculate source data freshness ([docs](https://docs.getdbt.com/v0.13/docs/using-sources) ) ([#814](https:/fishtown-analytics/dbt/issues/814), [#1240](https:/fishtown-analytics/dbt/issues/1240))
-- Add support for Presto ([docs](https://docs.getdbt.com/v0.13/docs/profile-presto), [repo](https:/fishtown-analytics/dbt-presto)) ([#1106](https:/fishtown-analytics/dbt/issues/1106))
-- Add `require-dbt-version` option to `dbt_project.yml` to state the supported versions of dbt for packages ([docs](https://docs.getdbt.com/v0.13/docs/requiring-dbt-versions)) ([#581](https:/fishtown-analytics/dbt/issues/581))
-- Add an output line indicating the installed version of dbt to every run ([#1134](https:/fishtown-analytics/dbt/issues/1134))
-- Add a new model selector (`@`) which build models, their children, and their children's parents ([docs](https://docs.getdbt.com/v0.13/reference#section-the-at-operator)) ([#1156](https:/fishtown-analytics/dbt/issues/1156))
-- Add support for Snowflake Key Pair Authentication ([docs](https://docs.getdbt.com/v0.13/docs/profile-snowflake#section-key-pair-authentication)) ([#1232](https:/fishtown-analytics/dbt/pull/1232))
-- Support SSO Authentication for Snowflake ([docs](https://docs.getdbt.com/v0.13/docs/profile-snowflake#section-sso-authentication)) ([#1172](https:/fishtown-analytics/dbt/issues/1172))
-- Add support for Snowflake's transient tables ([docs](https://docs.getdbt.com/v0.13/docs/snowflake-configs#section-transient-tables)) ([#946](https:/fishtown-analytics/dbt/issues/946))
-- Capture build timing data in `run_results.json` to visualize project performance ([#1179](https:/fishtown-analytics/dbt/issues/1179))
-- Add CLI flag to toggle warnings as errors ([docs](https://docs.getdbt.com/v0.13/reference#section-treat-warnings-as-errors)) ([#1243](https:/fishtown-analytics/dbt/issues/1243))
-- Add tab completion script for Bash ([docs](https:/fishtown-analytics/dbt-completion.bash)) ([#1197](https:/fishtown-analytics/dbt/issues/1197))
-- Added docs on how to build a new adapter ([docs](https://docs.getdbt.com/v0.13/docs/building-a-new-adapter)) ([#560](https:/fishtown-analytics/dbt/issues/560))
-- Use new logo ([#1349](https:/fishtown-analytics/dbt/pull/1349))
-
-### Fixes
-- Fix for Postgres character columns treated as string types ([#1194](https:/fishtown-analytics/dbt/issues/1194))
-- Fix for hard to reach edge case in which dbt could hang ([#1223](https:/fishtown-analytics/dbt/issues/1223))
-- Fix for `dbt deps` in non-English shells ([#1222](https:/fishtown-analytics/dbt/issues/1222))
-- Fix for over eager schema creation when models are run with `--models` ([#1239](https:/fishtown-analytics/dbt/issues/1239))
-- Fix for `dbt seed --show` ([#1288](https:/fishtown-analytics/dbt/issues/1288))
-- Fix for `is_incremental()` which should only return `True` if the target relation is a `table` ([#1292](https:/fishtown-analytics/dbt/issues/1292))
-- Fix for error in Snowflake table materializations with custom schemas ([#1316](https:/fishtown-analytics/dbt/issues/1316))
-- Fix errored out concurrent transactions on Redshift and Postgres ([#1356](https:/fishtown-analytics/dbt/pull/1356))
-- Fix out of order execution on model select ([#1354](https:/fishtown-analytics/dbt/issues/1354), [#1355](https:/fishtown-analytics/dbt/pull/1355))
-- Fix adapter macro namespace issue ([#1352](https:/fishtown-analytics/dbt/issues/1352), [#1353](https:/fishtown-analytics/dbt/pull/1353))
-- Re-add CLI flag to toggle warnings as errors ([#1347](https:/fishtown-analytics/dbt/pull/1347))
-- Fix release candidate regression that runs run hooks on test invocations ([#1346](https:/fishtown-analytics/dbt/pull/1346))
-- Fix Snowflake source quoting ([#1338](https:/fishtown-analytics/dbt/pull/1338), [#1317](https:/fishtown-analytics/dbt/issues/1317), [#1332](https:/fishtown-analytics/dbt/issues/1332))
-- Handle unexpected max_loaded_at types ([#1330](https:/fishtown-analytics/dbt/pull/1330))
-
-### Under the hood
-- Replace all SQL in Python code with Jinja in macros ([#1204](https:/fishtown-analytics/dbt/issues/1204))
-- Loosen restrictions of boto3 dependency ([#1234](https:/fishtown-analytics/dbt/issues/1234))
-- Rewrote Postgres introspective queries to be faster on large databases ([#1192](https:/fishtown-analytics/dbt/issues/1192)
-
-
-### Contributors:
-Thanks for your contributions to dbt!
-
-- [@patrickgoss](https:/patrickgoss) [#1193](https:/fishtown-analytics/dbt/issues/1193)
-- [@brianhartsock](https:/brianhartsock) [#1191](https:/fishtown-analytics/dbt/pull/1191)
-- [@alexyer](https:/alexyer) [#1232](https:/fishtown-analytics/dbt/pull/1232)
-- [@adriank-convoy](https:/adriank-convoy) [#1224](https:/fishtown-analytics/dbt/pull/1224)
-- [@mikekaminsky](https:/mikekaminsky) [#1216](https:/fishtown-analytics/dbt/pull/1216)
-- [@vijaykiran](https:/vijaykiran) [#1198](https:/fishtown-analytics/dbt/pull/1198), [#1199](https:/fishtown-analytics/dbt/pull/1199)
+- '{{this}}' is no longer respected in hooks [#1176](https:/fishtown-analytics/dbt/pull/1176), implementing [#878](https:/fishtown-analytics/dbt/issues/878)
+- A number of widely-used adapter methods previously available in macros/materializations have changed in breaking ways:
+ - get_missing_columns - takes Relations instead of schemas and identifiers
+ - get_columns_in_table - deprecated in favor of get_columns_in_relation (takes a Relation instead of schema, identifier)
+ - expand_target_column_types - takes a Relation instead of schema, identifier
+ - query_for_existing - removed
+ - get_relation - database parameter added
+ - create_schema - database parameter added
+ - drop_schema - database parameter added
+ - already_exists - deprecated in favor of get_relation (takes a relation)
 
 ## dbt 0.12.2 - Grace Kelly (January 8, 2019)
 

CONTRIBUTING.md

@@ -76,8 +76,8 @@ A short list of tools used in dbt testing that will be helpful to your understan
 - [tox](https://tox.readthedocs.io/en/latest/) to manage virtualenvs across python versions
 - [nosetests](http://nose.readthedocs.io/en/latest) to discover/run tests
 - [make](https://users.cs.duke.edu/~ola/courses/programming/Makefiles/Makefiles.html) - but don't worry too much, nobody _really_ understands how make works and our Makefile is super simple
-- [flake8](https://gitlab.com/pycqa/flake8) for code linting
-- [CircleCI](https://circleci.com/product/) and [Azure Pipelines](https://azure.microsoft.com/en-us/services/devops/pipelines/)
+- [pep8](https://pep8.readthedocs.io/en/release-1.7.x/) for code linting
+- [CircleCI](https://circleci.com/product/) and [Appveyor](https://www.appveyor.com/docs/)
 
 If you're unfamiliar with any or all of these, that's fine! You really do not have to have a deep understanding of any of these to get by.
 

Makefile

@@ -11,7 +11,7 @@ test:
 
 test-unit:
  @echo "Unit test run starting..."
- @time docker-compose run test tox -e unit-py27,unit-py36,flake8
+ @time docker-compose run test tox -e unit-py27,unit-py36,pep8
 
 test-integration:
  @echo "Integration test run starting..."

RELEASE.md

@@ -11,22 +11,21 @@ dbt has three types of branches:
 #### Git & PyPI
 
 1. Update CHANGELOG.md with the most recent changes
-2. If this is a release candidate, you want to create it off of your release branch. If it's an actual release, you must first merge to a master branch. Open a Pull Request in Github to merge it into the appropriate trunk (`X.X.latest`)
+2. If this is a release candidate, you want to create it off of your release branch. If it's an actual release, you must first merge to a master branch. Open a Pull Request in Github to merge it.
 3. Bump the version using `bumpversion`:
  - Dry run first by running `bumpversion --new-version <desired-version> <part>` and checking the diff. If it looks correct, clean up the chanages and move on:
  - Alpha releases: `bumpversion --commit --tag --new-version 0.10.2a1 num`
  - Patch releases: `bumpversion --commit --tag --new-version 0.10.2 patch`
  - Minor releases: `bumpversion --commit --tag --new-version 0.11.0 minor`
  - Major releases: `bumpversion --commit --tag --new-version 1.0.0 major`
-4. (If this is a not a release candidate) Merge to `x.x.latest` and (optionally) `master`.
+4. (If this is a not a release candidate) Merge to x.x.latest and master.
 5. Update the default branch to the next dev release branch.
-6. Build source distributions for all packages by running `./scripts/build-sdists.sh`. Note that this will clean out your `dist/` folder, so if you have important stuff in there, don't run it!!!
-7. Deploy to pypi
- - `twine upload dist/*`
-8. Deploy to homebrew (see below)
-9. Deploy to conda-forge (see below)
-10. Git release notes (points to changelog)
-11. Post to slack (point to changelog)
+6. Deploy to pypi
+ - `python setup.py sdist upload -r pypi`
+7. Deploy to homebrew (see below)
+8. Deploy to conda-forge (see below)
+9. Git release notes (points to changelog)
+10. Post to slack (point to changelog)
 
 After releasing a new version, it's important to merge the changes back into the other outstanding release branches. This avoids merge conflicts moving forward.
 

appveyor.yml

@@ -0,0 +1,64 @@
+version: 1.0.{build}-{branch}
+
+environment:
+ # SDK v7.0 MSVC Express 2008's SetEnv.cmd script will fail if the
+ # /E:ON and /V:ON options are not enabled in the batch script intepreter
+ # See: http://stackoverflow.com/a/13751649/163740
+ CMD_IN_ENV: "cmd /E:ON /V:ON /C .\\appveyor\\run_with_env.cmd"
+ TOX_ENV: "pywin"
+
+ matrix:
+ - PYTHON: "C:\\Python35"
+ PYTHON_VERSION: "3.5.2"
+ PYTHON_ARCH: "32"
+
+ #- PYTHON: "C:\\Python35"
+ # PYTHON_VERSION: "3.5.2"
+ # PYTHON_ARCH: "32"
+
+ PGUSER: postgres
+ PGPASSWORD: Password12!
+
+services:
+ - postgresql94
+
+hosts:
+ database: 127.0.0.1
+
+init:
+ - PATH=C:\Program Files\PostgreSQL\9.4\bin\;%PATH%
+ - ps: Set-Content "c:\program files\postgresql\9.4\data\pg_hba.conf" "host all all ::1/128 trust"
+ - ps: Add-Content "c:\program files\postgresql\9.4\data\pg_hba.conf" "host all all 127.0.0.1/32 trust"
+
+install:
+ # Download setup scripts and unzip
+ - ps: "wget https:/cloudify-cosmo/appveyor-utils/archive/master.zip -OutFile ./master.zip"
+ - "7z e master.zip */appveyor/* -oappveyor"
+
+ # Install Python (from the official .msi of http://python.org) and pip when
+ # not already installed.
+ - "powershell ./appveyor/install.ps1"
+
+ # Prepend newly installed Python to the PATH of this build (this cannot be
+ # done from inside the powershell script as it would require to restart
+ # the parent CMD process).
+ - "SET PATH=%PYTHON%;%PYTHON%\\Scripts;%PATH%"
+
+ # Check that we have the expected version and architecture for Python
+ - "python --version"
+ - "python -c \"import struct; print(struct.calcsize('P') * 8)\""
+
+build: false # Not a C# project, build stuff at the test step instead.
+
+before_test:
+ - "%CMD_IN_ENV% python -m pip install --upgrade pip" # sometimes on appveyor we get pip 7.x!
+ - "%CMD_IN_ENV% pip install psycopg2==2.6.2"
+ - "%CMD_IN_ENV% pip install tox"
+
+test_script:
+ - "bash test/setup_db.sh"
+
+ # this is generally a bad idea TODO
+ - git config --system http.sslverify false
+
+ - "%CMD_IN_ENV% tox -e %TOX_ENV%"