Home Integrations A/B Testing & Personalization Optimizely X Integration

Table of Contents

Was this article helpful?

Yes No

Thank you for your feedback!

Optimizely X Integration

In this article you'll learn:

  • Instructions to connect Heap and Optimizely X to send data
  • How data is formatted when coming into Heap
This doc is for: Admins Architects

Overview

Heap’s Optimizely X integration automatically captures all active experiment variations as a property attached to every event on a given page, enabling you to answer questions like:

  • How does an A/B test impact long-term retention?
  • Do users exposed to an experiment convert faster than others?
  • Did a given experiment influence behavior that isn’t directly related to the functionality changed?

Setup

To get started, click Connect on the Optimizely X Integration page. Once connected, Heap will automatically capture all active experiments and exposed variations as a property on all events for a given page.

To ensure that Optimizely X Pageview properties are properly captured, the Heap tag should always be deployed after the Optimizely X tag, so that Optimizely X has as much time to load as possible before Heap loads.

Data Format

Heap will automatically capture all active Optimizely X experiments and attach a pageview-level property to all events on that page in the user’s experience (including custom events). The format of the property follows the pattern of Optimizely: [Experiment Name] with the value of [Variation Name]. An example in this format might be: Optimizely: Home Page Test as the property name and Control as the value. Please refer to your experiment setup in Optimizely X for more info on what these values will contain.

In practice, you’ll see additional event-level properties on events where an active experiment is running. In the example image below, you can see a property entitled Optimizely: Collect and analyze - Homepage headline test with a value of Variation #1 included.

A list of event properties on a Variation #1 event in Heap

To use in analysis, simply apply a group by clause to the analysis of interest and specify the event level property with the experiment property of interest. This will allow you to cut your analysis by the experiment exposed to the user at the time of the event.

Known Issues

Privacy Settings: Mask descriptive names in project code and third-party integrations

Our integration attaches the active experiment name and variation names to events. If this option is enabled, experiment variation names aren’t accessible by the integration and properties will not get sent to Heap. Ensure this option is disabled in your account by going to your Optimizely X settings, then select “Privacy”, and uncheck the checkbox for “Mask descriptive names in project code and third-party integrations”.

Loss of Experiment Metadata

When using Optimizely X with asynchronous loading, you may have chosen to load their tag using the asynchronous keyword in the script tag. For example:

<script src="//cdn.optimizely.com/js/1234567890.js" async=""></script>

In this case, the tag loads in the background as the rest of the page loads, which can cause screen flicker if the page loads faster than the tag and any variations that need to be applied. This particular method of deployment is explicitly not recommended by Optimizely X or by Heap. 

If the tag is deployed in this manner, it is possible that Heap pageview data will be captured before the Optimizely X metadata is loaded, which results in the loss of the contextual experiment information required to associate Heap autocaptured events with a particular experiment.

Instead, load the Optimizely X tag synchronously to ensure Heap captures experiment data:

<script src="//cdn.optimizely.com/js/1234567890.js"></script>

Because the experiment metadata will be loaded synchronously, Heap will receive that data in the pageview, so long as the Heap tag is deployed in a lower position than the Optimizely X tag. This does not necessarily impact your variation load times. See Optimizely’s documentation for more details on loading options for variation and custom experiment code data.

If using a tag manager, ensure the tags can be deployed synchronously. Some tag managers, such as Tealium, have the capability to deploy tags synchronously. This is the recommended way to deploy an Optimizely X tag through a tag manager.

Google Tag Manager only allows for asynchronous tag deployment, which is one of the reasons why Optimizely recommends against using tag managers. Using Google Tag Manager may result in a loss of experiment data capture in Heap.

Optimizely X offers the ability to load variations and custom code synchronously. This can be configured within the Optimizely X UI:

When using these options, the Optimizely tag should be deployed synchronously:

<script src="//cdn.optimizely.com/js/1234567890.js"></script>

The variations and custom code, however, will be loaded after the initial experiment metadata is loaded. Optimizely X recommends only using asynchronous variation loading below-the-fold or for modals that will be closed upon page load. This will allow you to optimize your page load times while avoiding screen flicker. Additionally, because the experiment metadata is loaded synchronously, Heap will receive that data in the pageview, so long as the Heap tag is deployed in a lower position than the Optimizely tag. In this case, you can safely use asynchronous variation and custom code loading without worrying about loss of experiment metadata.

Data Discrepancy between Optimizely and Heap

If you are still noticing discrepancies between data in Optimizely and Heap after reviewing the tag deployment settings, we recommend going through Optimizely’s documentation for troubleshooting discrepancies.