Overview
Heap's Databricks integration allows you to sync Heap data to Databricks to leverage Heap behavioral data in other tools.
Prerequisites
To connect this integration, you'll need the following permissions:
- Admin or Architect privileges in Heap
- Databricks account hosted on either AWS or Azure that uses Unity Catalog
- One of the following credentials:
- A Personal Access Token (PAT)
- An OAuth client ID and client secret for a Databricks service principal (OAuth machine-to-machine)
If your Databricks workspace uses serverless compute with a network policy set to Restricted access to specific destinations, please add the following S3 buckets as Allowed Storage Destinations:
For EU customers:s3://heap-background-materializer-eu-west-1s3://heap-databricks-eu-west-1s3://heap-identity-api-export-eu-west-1
For US customers:s3://heap-background-materializers3://heap-databrickss3://heap-identity-api-export
Setup
To get started, navigate to Integrations > Directory, search for Databricks, then select it where it appears.
You'll be prompted to provide the following information:
- Hostname: The ID of your Databricks account, which you can find in the account URL.
- Path: The path of the warehouse you are connecting via this integration.
- Catalog: The catalog that this Heap data should sync to; if left blank, this integration will create a new catalog.
- Schema (optional): The schema that this Heap data should sync to; if left blank, this integration will create a new schema.
-
Authentication Method: Select how Heap authenticates with Databricks:
-
Personal Access Token (PAT):
- Token: This is required to allow Data Connect to write to the schema. The token must be a Personal Access Token (PAT) rather than an OAuth token.
-
OAuth (Client ID and Secret):
- Client ID: The client ID of your Databricks service principal.
- Client Secret: The client secret of your Databricks service principal. This is required to allow Data Connect to write to the schema. The credentials must belong to a Databricks service principal (OAuth M2M), not to a Personal Access Token.
-
Personal Access Token (PAT):
Once all those fields are populated, click the Connect button.
That's it! Once setup is complete, you'll see a sync within 48 hours with the following built-in tables:
- Pageviews
- Sessions
- Users
- user_migrations