Metadata-Version: 2.1
Name: dbt-spark-cde
Version: 1.1.2
Summary: The CDE API based Cloudera Spark adapter plugin for dbt
Home-page: https://github.com/cloudera/dbt-spark-cde
Author: Cloudera
Author-email: innovation-feedback@cloudera.com
Classifier: Development Status :: 5 - Production/Stable
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Provides-Extra: ODBC
Provides-Extra: PyHive
Provides-Extra: session
Provides-Extra: all

# dbt-spark-cde

The `dbt-spark-cde` adapter allows you to use [dbt](https://www.getdbt.com/) [Cloudera Data Platform](https://cloudera.com) with CDE API server support. This code bases use the dbt-spark project (https://github.com/dbt-labs/dbt-spark), and provides CDE API based connectivity support over it. 

## Getting started

- [Install dbt](https://docs.getdbt.com/docs/installation)
- Read the [introduction](https://docs.getdbt.com/docs/introduction/) and [viewpoint](https://docs.getdbt.com/docs/about/viewpoint/)

### Requirements

Python >= 3.8

dbt-core >= 1.1.0

requests >= 2.28.1

requests-toolbelt >= 0.9.1

pyspark

sqlparams

### Installing dbt-spark-cde

`pip install dbt-spark-cde`

### Profile Setup

```
demo_project:
  target: dev
  outputs:
    dev:
     type: spark_cde
     method: cde
     schema: my_db
     auth_endpoint: https://service.spark-cde-gateway.my.org.com/
     host: https://spark-cde-gateway.my.org.com/dex/api/v1/
     user: my_user
     password: my_pass
```

- To obtain auth_endpoint follow the steps here: (https://docs.cloudera.com/data-engineering/cloud/api-access/topics/cde-api-get-access-token.html)

### Caveats
- While using cde , in the Livy UI if you notice sessions change state to dead from starting instead of idle, make sure there is a proper mapping for the user in the IDBroker mapping section 
- Actions > Manage Access > IDBroker Mappings . [Reference](https://docs.cloudera.com/cdf-datahub/7.2.15/flink-analyzing-data/topics/cdf-datahub-sa-create-idbroker-mapping.html)
- Also make sure the workload password is set either through UI or CLI. [Reference](https://docs.cloudera.com/management-console/cloud/user-management/topics/mc-setting-the-ipa-password.html)

## Supported features
Please see the original adapter documentation: https://github.com/dbt-labs/dbt-spark and https://docs.getdbt.com/reference/warehouse-profiles/spark-profile
