# Databricks

![databricks-logo-horizontal.jpeg](https://assets.relyanceuat.xyz/images/docs/5779042266253/5778993664397.png)
Databricks is a cloud-based data engineering tool used for processing and transforming massive quantities of data and exploring the data through machine learning models. Developed by the creators of Apache Spark, it provides a unified, open platform for all your data.

In order to integrate Databricks with Relyance AI you will need a **Databricks Instance URL** and **Personal Access Token**.

**Token based authentication** must be enabled by the administrator.

**In Databricks:**

1. The **Databricks instance URL** will be the custom Databricks URL for your organization.
2. Click **Settings** in the lower left corner of your Databricks workspace.
3. Click **User Settings**.
4. Go to the **Access Tokens** tab.
5. Click the **Generate New Token** button.
6. Click the **Generate** button and copy the generated token.

**In the Relyance AI application:**

1. Login to your Relyance account.
2. Navigate to the **Settings** Menu in the bottom left-hand side.
3. Select **Integrations**.
4. Find the **Databricks** integration card and click on it.
5. In the Databricks integration page, click **Add Connection**.
6. Follow the wizard, adjusting any defaults as needed.
7. Choose **Custom** for the Authentication Method.
8. Paste the **Databricks Instance URL** and **Personal Access Token** into their respective fields. You can also specify the **Clusters** and **SQL Warehouses** to scan. **For Asset Discovery its important and necessary to provide the warehouse or cluster name.**

![Screenshot](https://assets.relyanceuat.xyz/images/docs/5779042266253/42979366201613.png)

![Screenshot](https://assets.relyanceuat.xyz/images/docs/5779042266253/42979366207629.png)

![Screenshot](https://assets.relyanceuat.xyz/images/docs/5779042266253/42979366208397.png)

9. Click **Authenticate** and finish the wizard.
10. At this point, you should see the following result on the Vendor Integrations page:
11. Congratulations, you are now connected to **Databricks**.

![Databricks-2.png](https://assets.relyanceuat.xyz/images/docs/5779042266253/5780632758285.png)

<!-- failure-modes:begin (generated from the integration catalog; do not hand-edit) -->

## If the connection reports Connected but returns nothing

These are the ways this integration comes back empty without reporting an error. Generated from the integration catalog, so it tracks what the connection actually asks for.

1. **A credential rotated at the vendor is not picked up here.** **Personal Access Token** is stored when you save the connection, so regenerating the value at the vendor breaks the next scan until it is re-pasted here. Recording the expiry on the connection means Relyance warns you before it lapses.
2. **Check the address fields before suspecting the credentials.** **Databricks Instance URL** identifies which tenant, region or host to talk to. A wrong value there fails authentication and looks exactly like a bad secret.

<!-- failure-modes:end -->

<!-- auth-methods:begin (generated from the integration catalog; do not hand-edit) -->

## Authentication methods and fields

Pick one of these under **Authentication Method** on the connection wizard's **Authentication** step. This table is generated from the integration catalog, so it always matches what the form actually asks for.

| Method | Required | Optional |
| --- | --- | --- |
| **Custom** | `Databricks Instance URL`, `Personal Access Token` (secret), `Cluster Options`, `SQL Warehouse Options` | — |

<!-- auth-methods:end -->

<!-- terraform-examples:begin (generated from the integration catalog; do not hand-edit) -->

## Manage this integration with Terraform

Connections for this integration can be managed as code with the [Relyance Terraform provider](https://registry.terraform.io/providers/Relyance/relyance/latest). Non-secret fields go in `auth.params`; secret fields go in `auth.secrets_wo`, which is write-only — never stored in Terraform state. Rotate secrets by bumping `auth.secrets_wo_version`.

```hcl
resource "relyance_integration_connection" "databricks" {
  vendor = "databricks"
  name   = "<your connection name>"

  auth = {
    method = "api-token"
    params = {
      instance_url = "<instance_url>"
      clusters = jsonencode({
        cluster_names = [
          ""
        ]
      })
      sql_warehouses = jsonencode({
        sql_warehouse_names = [
          ""
        ]
      })
      data_storage_location = "us"
    }
    # Secret fields are write-only: sent to Relyance, never stored in state.
    secrets_wo = {
      api_token = var.databricks_api_token
    }
    secrets_wo_version = 1
  }

  scans = { "data-inspection" = { enabled = true } }
}
```

<!-- terraform-examples:end -->