Databricks Integration

Overview

Heap's Databricks integration allows you to sync Heap data to Databricks to leverage Heap behavioral data in other tools.

Prerequisites

To connect this integration, you'll need the following permissions:

  • Admin or Architect privileges in Heap
  • Databricks account hosted on either AWS or Azure that uses Unity Catalog
  • One of the following credentials:

If your Databricks workspace uses serverless compute with a network policy set to Restricted access to specific destinations, please add the following S3 buckets as Allowed Storage Destinations:

For EU customers:
s3://heap-background-materializer-eu-west-1
s3://heap-databricks-eu-west-1
s3://heap-identity-api-export-eu-west-1

For US customers:
s3://heap-background-materializer
s3://heap-databricks
s3://heap-identity-api-export

Setup

To get started, navigate to Integrations > Directory, search for Databricks, then select it where it appears.

You'll be prompted to provide the following information:

  • Hostname: The ID of your Databricks account, which you can find in the account URL.
  • Path: The path of the warehouse you are connecting via this integration.
  • Catalog: The catalog that this Heap data should sync to; if left blank, this integration will create a new catalog.
  • Schema (optional): The schema that this Heap data should sync to; if left blank, this integration will create a new schema.
  • Authentication Method: Select how Heap authenticates with Databricks:
    • Personal Access Token (PAT):
      • Token: This is required to allow Data Connect to write to the schema. The token must be a Personal Access Token (PAT) rather than an OAuth token.
    • OAuth (Client ID and Secret):
      • Client ID: The client ID of your Databricks service principal.
      • Client Secret: The client secret of your Databricks service principal. This is required to allow Data Connect to write to the schema. The credentials must belong to a Databricks service principal (OAuth M2M), not to a Personal Access Token.

Once all those fields are populated, click the Connect button.

That's it! Once setup is complete, you'll see a sync within 48 hours with the following built-in tables:

  • Pageviews
  • Sessions
  • Users
  • user_migrations
Last updated
Powered by Zendesk