Transaction deduplication function

Marketing tools are now capable of capturing more data than ever before, which means that they need to have good routines in place to separate the good data from the bad. As a tag management expert, it is your job to provide the necessary parameters to make good decisions. Often, this will involve sending a conversion signal to a network only once it has occurred, which is a very common practice in tag management. With JENTIS, you have a more resilient tool for handling this job.

In this guide, we will show you how to implement an efficient way to handle deduplication with JENTIS on the server side. This will help you save resources on the frontend and ensure a stable process that runs smoothly on the server side.

Filtering Data

Our objective is to confirm whether a conversion happened before sending the signal and parameters to a network. To achieve this, we need a unique identifier for that conversion, which is usually the transaction_id for transaction-based conversions. In this guide, we will use a common example of an ecommerce store where the conversion is a purchase, and each purchase has a unique ID that is submitted to our data layer.

Before delving into the process of "how", let us first briefly discuss the "why" aspect of it. Some marketing tools lack the capability of deduplication within their internal processes, which means that the tracking system has to filter out duplicate signals. This usually happens during page reloads, such as when you close the website app on your smartphone after making a purchase and then reopen the same app. This results in the recent page being reloaded and unintentionally sending a redundant conversion signal. To filter out such signals before sending the conversion, we can implement a short routine.

Server Side Deduplication with JENTIS

With all the basics covered let's jump head first into the implementation. Here is a short summary of what is required and what the parts in the tag management are.

We will create those elements:

• Trigger: To actually only capture the first conversion and no more the triggers will need a parameter to check this back with.

• Backend Variable: This will be the parameter for the trigger. The actual result value must be either true or false, signaling if this is a new or redundant conversion.

• Frontend Unique Identifier: An ID, such as a transaction_id, that identifies the conversion.

• Tags: Update the tags to use the new trigger and maybe create new tags that connect to the logic of deduplication.

Here are the details and step by step integration of those. We will start with the variables and later update the trigger and tags.

Frontend Unique Identifier

The core of this process is the unique identifier. To link a conversion to a particular transaction, an external application is needed. This application should know which transaction the conversion is related to and submit that information to the data layer. It's a standard practice for ecommerce tracking, such as Universal Analytics enhanced ecommerce.
In case of JTM native data model, the data can be pushed to JENTIS Data Layer by following this process:

_jts.push({
  "track"  :  "order",
  "orderid":  "abcdef1234567890",
  ...
});

By accessing the standard variable "order_id" on the frontend, we can retrieve its value on the server side using backend variables.

Backend Variable

The main chunk of code comes with this backend variable. So we will look at it line by line and see what it does.

async function() {
  var storage_name  = "existing_order_ids";
  var max_storage   = 5;
  let existing_ids  = await this.storage.read(storage_name) || "";
  let new_order_id  = this.getFrontendVariable("dl_order_number");

  if(existing_ids && new_order_id != null && new_order_id != "" && existing_ids.indexOf(new_order_id) >= 0){
    return true;
  } else if(new_order_id && new_order_id != null && new_order_id != "") {
    existing_ids += new_order_id+";";

    if(existing_ids.split(";").length > max_storage){
      existing_ids = existing_ids.split(";");
      existing_ids.shift();
      existing_ids = existing_ids.join(";");
    }

    this.storage.write(storage_name, existing_ids, Date.now()+94608000000);

    return false;
  }
  return null;
}

To achieve our goal, we need to keep a record of known conversions or transaction IDs. This record will be saved on the server's local storage. The storage_name is a unique static key to identify the location and is not used in another variable or function. We will also set a maximum list length to avoid overcrowding the storage.

The code will use the API this.storage.read(storage_name) to access previously saved ids. For matching a new id to the existing list, we will need to read a new value with this.getFrontendVariable(…). In this step, we will use the reference from the previous step, which creates the frontend variable that holds the conversion identifier.

Next, we use if-else statements to handle the situation. If the new ID is found in the existing IDs list, the variable will return "false," and the new ID will be concatenated to the existing list. The storage will be updated using the this.storage.write operation.

That's all we need to do. We can save this function as a new backend variable.

Trigger Update

Now we need to actually apply the logic in a trigger. We will add a condition on the “Order” conversion trigger for that.

By the way: this is the default JENTIS Data Layer for purchase tracking that also uses the default JENTIS State (click on those items to learn more on those basic concepts).

In our example, we have a custom trigger defined that activates on the JENTIS Data Layer, though this approach with the variable can be used on any trigger and state. In the conditions the Order ID Existing variable value is checked to be "false" - which indicates this Order ID yet exists (so we can safely activate it according to tags).

Tag Update

The last step is to activate a tag on our new trigger. Two scenarios are plausible here: track the conversion only the first time and optionally add a tag to track redundant calls as an event (that is not a conversion).

So if you need to have a safety or are just curious to have all the data (meaning to also track events when redundant conversions would have been tracked but were filtered by our setup here) an accompanying tag makes sense. Just to make sure you are not missing out on anything and the actual process of deduplication is activated correctly. Thus, you’d need a second trigger that has the same conditions, but the check on “Order ID Existing” is looking for “false”.

Conclusion

Make sure to preview this new process in the JTM preview. If everything looks good to go – you are ready for the release!

The concept of events in JENTIS Tag Manager, where each event is a "state". In context of such state each variable is evaluated and has a certain value, all triggers are evaluated, too. That all happens in sync with client- and serverside elements of your JENTIS Tag Manager, regardless where your variable is computed (server or client) it will be available to all elements. Read more on this concept: JENTIS States Framework