[bug] dbt-databricks v1.0.0 did not update dba-spark ('str' object has no attribute '_message')

[drernie]

Describe the bug

After upgrading to v1.0.0 of dbt, I re-installed dbt-databricks. Now I get this error:

Runtime Error
'str' object has no attribute '_message'

Steps To Reproduce

dbt debug -t http

Expected behavior

[OK]

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

 % dbt debug -t http
01:07:05  Running with dbt=1.0.0
dbt version: 1.0.0
python version: 3.9.9
python path: /Users/nauto/Developer/dbt/dbt-env/bin/python3
os info: macOS-12.0.1-x86_64-i386-64bit
Using profiles.yml file at /Users/nauto/.dbt/profiles.yml
Using dbt_project.yml file at /Users/nauto/Developer/dbt/dbt_project.yml

Configuration:
  profiles.yml file [OK found and valid]
  dbt_project.yml file [OK found and valid]

Required dependencies:
 - git [OK found]

Connection:
  host: nauto-biz-prod-us.cloud.databricks.com
  port: 443
  cluster: 1004-233546-lager607
  endpoint: None
  schema: default
  organization: 0
  Connection test: [ERROR]

1 check failed:
dbt was unable to connect to the specified database.
The database returned the following error:

  >Runtime Error
  'str' object has no attribute '_message'

Check your database credentials and try again. For more information, visit:
https://docs.getdbt.com/docs/configure-your-profile

System information

The output of dbt --version:

dbt --version
installed version: 1.0.0
   latest version: 1.0.0

Up to date!

Plugins:
  - databricks: 1.0.0
  - spark: 0.21.0

AHA. That was it. I uninstalled dbt-spark and re-installed it back with v1, and it works.

dot-core complained about outdated modules but dbt-databricks did not

[ueshin]

You need to set the connection configuration for dbt-databricks:

your_profile_name:
  target: dev
  outputs:
    dev:
      type: databricks
      schema: [database/schema name]
      host: [your.databrickshost.com]
      http_path: [/sql/your/http/path]
      token: [dapiXXXXXXXXXXXXXXXXXXXXXXX]

See the Step 3.3 for more detail: https://docs.databricks.com/dev-tools/dbt.html#step-3-create-a-dbt-project-and-specify-and-test-connection-settings

[drernie]

Yes, I have gotten it working. I just wondered if there was some way to specify the dependencies so it would have warned me that there was an outdated version of dbt-spark.

[jtcohen6]

This specific bug ('str' object has no attribute '_message') is on us: We made a change in dbt-core v1.0.0 without properly warning that it could have implications for some adapter plugins: dbt-labs/dbt-core#4433

The switch in dbt-spark==1.0.0 to account for it was very simple: dbt-labs/dbt-spark#265

Second, we do want to start showing more helpful output in dbt --version re: adapter plugin compatibility: dbt-labs/dbt-core#4438

One question: It doesn't look like dbt-databricks actually requires dbt-spark as a dependency... so I'm a bit confused as to why upgrading dbt-spark would fix something in dbt-databricks?

[ueshin]

I'm guessing upgrading dbt-spark just fixed the incompatibility between dbt-core and dbt-spark.

The output from dbt debug in the issue description shows port, cluster, endpoint, and organization as a Connection settings which won't be seen when using dbt-databricks. When using dbt-databricks, it will show host, http_path and schema.

@jtcohen6 We might want to show the adapter type dbt is trying to use when dbt debug.

[ueshin]

Now that dbt-databricks (>=1.0.1) inherits dbt-spark,

• It will upgrade dbt-spark to >=1.0.0
• The error is fixed by using dbt-spark>=1.0.0

I'd close this now.