Setting up an Event-based Rule that be fired directly like a Direct Call Rule

Update:
As proud as I am of this solution/workaround, it may be far less needed now that you can use the dataelementchanged  condition to fire an Event Based Rule. Instead of using custom events, you can just have DTM listen for when your pageName Data Element has changed. 

The current limitations

If developers want to fire a DTM rule directly from their code (say, they want to make sure a beacon fires only after their data layer is ready), typically they would fire a Direct Call Rule, with its very specific syntax: _satellite.track(“rule string here”). There are, however, some limitations to this method.

Direct Call Rules:

  • Don’t allow for multiple conditions (you can’t say “if _satellite.track(“cart add”) is fired AND the current page has “/products” in the URL“)
  • Don’t allow for multiple arguments (you can’t pass _satellite.track(“cart add”,”sku1″) to attach the added SKU to the rule)
  • Don’t allow for firing s.clearVars() before your rule sets up your analytics variables (to clear out variables from previous beacons on the same DOM).
  • Require very specific syntax- they MUST be “_satellite.track()”

And unfortunately, both Direct Call Rules and Event-based rules don’t “stack”- if a certain condition triggers multiple similar rules, each rule will fire its own beacon. This is different from Page Load Rules, where if multiple rules have conditions being met by the current page, they all wrap nicely into a single page view beacon.

An alternative

To get around some (but maybe not all) of these limitations, I’ve been playing with another possible option, where we use the Custom Event conditions of an Event Based Rule to accomplish nearly the same thing. After getting it set up, I can fire something like this:

digitalData.userAction("cart add","sku1")

…to fire an Event-Based Rule in place of a Direct Call Rule. There are a few things I need to do in DTM to make this digitalData.userAction work.

Set Up the Logic

First, I have to set up the logic in a Page Load Rule- set to fire on DOMReady (no need for it to be sooner)- that will merely hold the following as a Sequential Javascript Third Party Tag:

//make sure digitalData is defined to prevent errors
if(typeof digitalData=="undefined"){
  digitalData={}
}

//create fake DOM item to bind the event to
var fakeDiv = document.createElement('div');
fakeDiv.setAttribute('id', "dtmHolder");
fakeDiv.setAttribute('height','1');
fakeDiv.setAttribute('width','1');
fakeDiv.setAttribute('style','display:none');
document.body.appendChild(fakeDiv);

//define custom event
digitalData.userAction=function(n,details){

 document.getElementById("dtmHolder").addEventListener("dtmEvent", function(e) {
    console.info("Event is: ", e);
  })

  // First create the event
  var dtmEvent = new CustomEvent(n, {
    detail:""
  });

  jQuery("#dtmHolder").attr("detail",details)
  
  // Trigger it!
  document.getElementById("dtmHolder").dispatchEvent(dtmEvent); 
}

(Update: note that this code should not run before the DOM is created- it will create an error if you try to run it at page top because you are trying to append something to a body that doesn’t exist yet).
Now, whenever a developer fires digitalData.userAction(“string here), you can listen for that string as the Triggered Event Type in a Custom Event Event Based Rule. Obviously, you can alter the above code if you want a function named something other than digitalData.userAction.

Set Up an Event Based Rule

The rule will need to be bound to the CSs selector of the tiny fake div (“#dtmHolder”) we created for the custom event to bind to:

2016-04-20_12-38-12

You can create as many of these rules as you want, for whatever different strings you pass into digitalData.userAction()where the “triggered event type” reflects that string.

Pass Additional Info

If you want to pass a second argument ( e.g. digitalData.userAction(“cart add”,”sku1″)) I currently have that second argument  passing as a new attribute (“detail”) on the tiny invisible div, so you can access it off the “this” object directly in the rule:2016-04-20_12-41-52

You can give this a try at my ugly test site– open a developer console, turn on DTM debugging, and fire either digitalData.userAction(“cartAdd”,”sku123″) or digitalData.userAction(“pageView”) to see two example rules at work.

Run ClearVars

This opens the ability to run s.clearVars on s.t() beacons in cases where multiple beacons may be firing on a single DOM. (As a reminder, if you’re using the old DCR route, there are some hack-ish options for doing this- we call it daisy-chaining).

In an Event Based Rule, there IS a code block that runs before the Analytics tool, giving you a perfect opportunity to make sure you are starting with a ‘clean slate’ of variables: the Conditions Custom Code block. Just create a new Rule Condition with “custom” criteria, then put “s.clearVars()” in the code block, followed by “return true” (so that DTM doesn’t think some condition didn’t pass):

2016-04-20_12-52-49

You can also apply additional conditions, like “only fire this “cart add” rule on certain pages”, by adding more criteria under Rule Conditions.

Conclusion

I’m very open to suggestions and feedback on this- maybe we can crowdsource a better way, but for now, this seems to be a reasonable alternative to Direct Call Rules. Let me know what you think!

Using Charles Proxy for Analytics

Charles Proxy is a packet-sniffing tool often used by web analysts and developers to debug beacons being sent to Adobe. I find it has some advantages over other tools, even over the Adobe Debugger, because it isn’t looking at your javascript or your “s” object on your DOM, and isn’t tied to a specific browser window or page, but rather it looks at what traffic is actually being sent (and hopefully, successfully received) by Adobe, regardless of where or how it is sent.

charlesFullBeacon

This shows all the parameters (or variables) in your analytics beacon. Some very nice person put up a cheat sheet of how those parameters map to variables or are used in processing and reporting.

On top of looking at Analytics Beacons, I often use it to do the following:

  • Examine which DTM files are loading, and in what order.
  • Check that calls to other Adobe tools (Marketing Cloud ID Service, Target, Audience Manager) are happening in the right order, and with the right parameters.
  • Route traffic from a mobile device through my desktop to see analytics beacons from that device (see instructions below).
  • Use Tools>Map Local to use a file on my machine to replace one of my client’s pages in my browser (for instance, I could save the source of a client’s page, tweak it for my needs, save it to my desktop, and then use Charles Map Local to look at how that page would behave in place of the real one). This makes it easy to see how a client’s page acts when a DTM library is placed on it, or if certain code is removed.
  • Use Tools>Map Remote to replace a file on the client’s site with a file from elsewhere on the internet. Often, this is to swap one DTM library out for another. Sometimes, I do this just to purposefully break a file- if I’m curious how a site would behave without jQuery, I might use map local to say “replace the jQuery file with a file at blank.js” (which doesn’t actually exist, so it effectively just says “don’t load the jQuery file). I did this often enough, and got sick of even seeing 404 errors, I created a blank file specifically for replacing other files.
  • Get the full URL of a POST beacon, so I can use my beacon parser tool to split it out.
  • Save a session of Analytics traffic so you can send it to peers.
  • View the Headers and Cookies attached to a given beacon (at the bottom of a beacon, where you are viewing your params, switch from Query Params to either “Headers” or “Cookies”):
    charlesHeaders

Charles is free if you’re ok with seeing a long splash screen and having sessions limited in length; it’s 50 dollars for a personal license if you want no splash screen and unlimited session length (I bought mine many years and many updates ago, it was worth every penny).

Charles Proxy and I have a bit of a love-hate relationship. It is pretty much ALWAYS open on my machine. It makes many aspects of my job so much easier. But sometimes, it just… doesn’t work. Many folks use the Tools>Rewrite capability to replace just certain code on the page, but it rarely works for me. And getting all the certification set up to accurately track SSL traffic is tricky. So, below is my guide for all the steps I follow when setting up Charles on a new machine to make it ideal for the type of tasks I test to do:

Install it

I prefer the Beta version since it happens to work better with my certificates and such, but for most folks the normal download should do just fine.  After you install it, if you use Firefox, you’ll also want to download the Firefox extension- the easiest way is to go to Charles>Help>Install Mozilla Firefox Add-on. This is not needed for Chrome or the other common modern browsers.

Filter it

Charles captures ALL traffic sent from your machine to anywhere on the internet. It can be… comprehensive. I quickly find it to be too much information, and I use the “Include” tab in Proxy>Recording Settings to tell Charles to only show things relevant to my job. Below are the common ones (asterisks are wildcards):

  • “assets.adobedtm.com” will show all Akamai-hosted DTM files. (If you self-host, you’ll want to add that URL up to the folder that contains your scripts- in other words, stop just before it says “/satelliteLib-“. That way you see ALL DTM files, like your s_code file, not just your main library)
  • “*demdex.net” shows Audience Manager and Marketing Cloud ID calls (fun fact: the Marketing Cloud ID service uses technology gained from Adobe’s acquisition of Demdex).
  • “*.tt.omtrdc.net” shows Target calls
  • Whatever domain you capture your analytics tracking on. For instance, on this site, my non-secure beacons are sent to “jenniferkunz.d1.sc.omtrdc.net”. Make sure you include both the secure and non-secure domains. If in doubt, you may be able to check this by seeing what your s.trackingServer and s.trackingServerSecure variables are set to in your s_code.
  • Because I have many Analytics clients, I also include all the common domains for analytics tracking, but you may not want to do this because you’ll catch many beacons not relevant to your implementation:
    • “*.2o7.net” for non-RDC, non-CNAME implementations
    • “*.omtrdc.net” for RDC implementations
    • “*metric*” as the domain and “*/b/ss*” as the path, to capture a vast majority of CNAME implementations.

charlesInclude

Use “Sequence” Mode

Most folks use “Structure” mode for viewing beacons, where a panel on the left will show all traffic, broken down by domain:

charlesStructureMode

I used this for years before realizing how much more I like sequence mode, which offers a few benefits:

  • It shows things in the order that they occur (making it easy to spot things like “uh-oh, the marketing cloud ID service hadn’t responded by the time my beacon fired”
  • You don’t have to know the root domain of your beacons off the top of your head. Secure and non-secure beacons are shown alongside each other.
  • It makes it very easy to filter. I often filter by “b/ss” to show only Analytics beacons, but you could also filter by “v0” to see only beacons that set the s.campaign variable, or by “landing” to show only beacons that had a pageName including “landing”.
    charlesSequence

Get it working with SSL

By default, Charles can’t read traffic from secure sources- or it can only show them with encrypted values (generally useless for debugging). There is information on the Charles site about setting up SSL tracking, but it can be hard to know what it means for you.

First, you need to set up a Root Certificate on your machine, giving Charles control over how your machine handles SSL Certificates. Go to Help>SSL Proxying>Install Charles Root Certificate. Follow the prompts from your operating system, but keep a careful eye out for two settings:

  • From the Charles site (specific to windows): “The certificate must be imported into the “Trusted Root Certification Authorities” certificate store, so override the automatic certificate store selection.”
  • Ensure the certificate is set to “Always trust” (if you miss this setting first time around, and you’re on a mac, you can change it later by opening your keychain.

If you have Firefox, you’ll also need to install the certificate through Firefox. If you’ve installed the Charles extension, then within Firefox you should be able to go to Tools>Charles Proxy>Install Charles Root Certificate.  When given the option, click “Trust this CA to identify websites”.

All of this has gotten Charles primed to listen for SSL traffic, but you still need to tell Charles which specific sites it can listen to, by going to Proxy>SSL Proxying Settings>SSL Proxying and entering the domains of your SSL analytics traffic. I tend to have this list pretty closely match by list of overall traffic filters, mentioned above (eg: “*2o7.net”, “*omtrdc.net”, “assets.adobedtm.com”, and “smetrics.*.com”). (Update 2023: adobedc.net or “/ee/” seem like good filters for Adobe Experience Platform traffic).

Use it for Mobile Devices

If you want to route your traffic from a mobile device through Charles, it’s fairly simple. Below are instructions for an iOS device… sorry Android users, you’ll need to google to find similar instructions). Your mobile device and your desktop will need to be on the same network (ie, same WiFi). If your desktop is on VPN, your phone will need to be too.

First, if you will be looking at secure traffic, you’ll need to install the Charles certificate to your device. On your device, go to http://www.charlesproxy.com/getssl and follow the prompt. Pretty easy.

Next, get your local IP address (hint: this is different from your overall IP address, and specific to within your network). Charles makes this easy- in Charles, go to Help>Local IP Address.  You may want to jot this down and close this window, because when you get things working, Charles will want to pop up another window you need to pay attention to.

Lastly, on your phone, open up your WiFi settings and scroll to the bottom. Under HTTP Proxy, select “Manual”, then under Server, enter the Local IP address you noted above. Under Port, enter “8888”. Then click Renew Lease. (2023 update- I don’t believe Renew Lease is needed anymore, or even an option).

charlesMobile

[update- September 2017): If you still have issues with SSL traffic and get a “SSLHandshake: Received fatal alert: unknown_ca” error, you may need to go into your device’s settings- Settings > General > About > Certificate Trust Testings – and make sure the Charles Proxy Custom Root Certificate is marked as trusted.

Don’t forget to switch this back to “off” when done. 

Now move out to your browser and do a simple test to see if traffic gets sent and shows in Charles. The first bit of traffic should prompt a Charles warning on your machine, asking you to Allow this traffic.  If you miss this setting the first time around, you can tweak it under Charles>Tools>Access Control Settings.

Troubleshooting

If after all of this, it isn’t working right, here’s a few things to try:

  • If you are on VPN, and Charles isn’t cooperating, try switching the order you do things in: perhaps by opening your VPN first, THEN Charles, or the other way around. Experiment.
  • If Charles works in one browser and not in another, check your Proxy menu: are you set to capture traffic data both on your OS (and therefore Chrome), as well as Firefox?
  • If Charles is interfering with things like your email server, check out your Bypass Proxy settings.

 

 

Referencing “this” in Event-Based Rules

Not many people know you can pull information out about the element that an Event-Based Rule fires on, without any custom code. Let’s say I want to fire a rule on a link that looks like this, and I want to capture the domain of the link that was clicked in eVar3:

<div partner="adobe">
     <a href="http://www.adobe.com" class="partnerExit" alt="go to our partner Adobe" target="_blank">This is an example link.</a>
</div>

I would set my rule up to correctly fire on that link (with something like “a.partnerExit”), then for eVar3 I would put %this.hostname%, where “this” refers to “this thing that the rule fired on”.

I don’t have to have to do any custom code, or have a data element set up (in fact, data elements are NOT particularly useful at pulling out information specific to the element that fired an event-based rule.)

Putting this in the interface…

Would let me access…

Which would yield this…

%this.hostname% The domain of the link that was clicked www.adobe.com
%this.href% The full URL of the link that was clicked http://www.adobe.com/
%this.src% The source of the element that was clicked (works for images, not links) (Not applicable here.)
%this.alt% The “alt” value of the element that was clicked go to our partner Adobe.
%this.@text% The internal text of the element that was clicked This is an example link.
%this.@cleanText% The internal text of the element that was clicked, trimmed to remove extra white space This is an example link.
%this.className% The class of the element that was clicked (less handy in reports, but very handy for DTM troubleshooting) partnerLink

For more advanced “DOM-scraping” you may need to take to the custom code. I find jQuery often simplifies things greatly for this. For instance, in the above example, if I wanted to get the ID not of the anchor tag, but of the <div> that HOLDS the anchor tag, I could do this in the custom code:

Note that I remembered to also add it into s.linkTrackVars, since this is an s.tl beacon (DTM automatically creates s.linkTrackVars for any variables you configure directly in the interface, but can’t know which variables you are adding to the custom code section, so you must be sure to add them to linkTrackVars or linkTrackEvents yourself, or the s.tl() beacon will ignore those variables).

How to get a global “s” object in DTM

At this point in time, by default, DTM creates your analytics object (usually an “s”, as in “s.pageName”) with a local scope. This means it should be able to be referenced from any custom code blocks within DTM that are tied to your analytics tool. However, it would NOT be accessible from code on the page (or a developer console) or even non-analytics code blocks in DTM (like Third Party Tags). This can cause some pretty big problems if you aren’t aware.

sUndefined

 

 

 

To get around this, you need to define your own s object within your library. This does mean you can’t let DTM manage your library, but the change you need to make is pretty minor. You need a “Custom” configuration, and you’ll need to “Set report suites using custom code below” (since you’re essentially going to be overwriting the “s” object that DTM created, where it set your report suites for you.

configureScode

 

 

 

 

 

 

When you open the editor, add this code to the top:

s = new AppMeasurement();
if(_satellite.settings.isStaging==true)
{s.account="myDevSuite"}else{s.account="myProdSuite"}

Make sure to replace the “myDevSuite” and “myProdSuite” with the correct report suites- these should match what you have in the interface. This uses _satellite.settings.isStaging to detect the current library and set the appropriate s_code.

 

With that in place, you should be able to access the “s” object from anywhere in DTM or on your page.

UPDATE: Because of a current quirk in DTM where it looks for the H code version of your s.account, I recommend also setting this line, below the ones above:

var s_account=s.account

This should prevent any report suite confusion on s.tl beacons from Direct Call Rules and Event Based Rules.

Why do my Adobe Analytics numbers differ so much from my bit.ly stats?

Link shorteners like bit.ly, goo.gl, and t.co generally keep their own metrics for how many times users click through. Unfortunately, those numbers rarely match any other Analytic’s system, including Adobe Analytics. We’ve seen situations where the bit.ly number was as much as 10 times the analytics number. With such a big difference, both numbers look a little suspect. So, what gives?

Bots

First, there’s non-human traffic. Link shorteners’ numbers do not always account for whether the click came from a real live human being, or a bot. Goo.gle and bit.ly DO try to eliminate bot traffic, but the list they use to identify bots is likely different from the list Adobe uses (which is maintained by the IAB).
However, link shorteners are often used on twitter, and bot traffic on twitter is a whole different beast. Web bots generally crawl pages, sometimes searching for specific things. Adobe and the IAB keep a good list of the worst web-crawling bot offenders. But on twitter, the environment is more predictable (tweets, unlike webpages, can predictably be accessed/interacted with the same methods) and the actions bots perform are simpler: click, retweet, reply. This means many more “hackers” out there can create a bot with much less effort, and since they aren’t harming much, they fly under the radar of the IAB and other bot-tracking organizations.
I found a book, “TWITTER: The Dark Side – Does Bit.ly Enable a Massive Click Fraud?” (by Roman Latkovic and Robert LaQuay, Ph.D) which takes a thorough look at this issue. Basically, when you start to break down individual clicks by IP and user agent, it becomes clear many bots are indeed inflating bit.ly tracking.

Javascript Disabled

Second, there’s javascript. The majority of Adobe implementations these days rely on the user firing JavaScript (some older implementations or mobile implementations may fire a hard-coded beacon, but this is increasingly rare). Link shorteners, however, run WITHOUT javascript- meaning devices that have javascript disabled would count in bit.ly but NOT in Adobe.

Lost Referrer Information

Third, bit.ly (or twitter traffic in general) may cause you to lose referrer information. Someone coming from a twitter app rather than a webpage won’t have a referrer-it may look like “Direct Traffic”. Query string parameters SHOULD always carry through, though, so make sure your shortened links include tracking codes.

Conclusion

With this information in mind, I’d be more surprised to see bit.ly numbers that MATCH an Analytics tool than I am to see wide discrepancies. What’s your experience, and what are your theories behind them?

Beacon Parser again!

Some new updates in the beacon parser:

  • Fixed a bug that prevented heartbeat beacons from tracking
  • Made it possible to remove beacon columns
  • Added API functionality, which can pull down your report suite IDs and custom variable names using the Adobe Admin API.

A huge thanks to this summit lab session for hints on using JSON/AJAX on the API.

Let me know how it works for you!

Beacon Parser round 3

First, you’ll notice a new look and feel for both the blog and the parser- I’m trying to give it a more uniform look.

Functionally, the biggest changes in this go-around are:

  • Variables and parameter should keep a clean, sorted order.
  • You can highlight values in older beacons that don’t match the most current beacon.
  • You can change display options before submitting your first beacon.
  • Display options now include friendly names and variable descriptions, taken largely from this Adobe help article.
  • I fixed some bugs with exporting to CSV and the products string.