Adobe’s performanceTiming plugin, with some improvements and an explanation

33 Sticks Logo- Orange Circle with 3|3 in it

(This is cross-posted form the 33 Sticks blog)

As Page Performance (rightfully) gets more and more attention, I’ve been hearing more and more questions about the Performance Timing plugin from Adobe consulting. Adobe does have public documentation for this plugin, but I think it deserves a little more explanation, as well as some discussions of gotchas, and potential enhancements.

How It Works

Adobe’s Page Performance plugin is actually just piggybacking on built-in functionality: your browser already determined at what time your content starting loading and at what time is stopped loading. You can see this in a JavaScript Console by looking at performance.timing:

This shows a timestamp (in milliseconds since Jan 1, 1970, which the internet considers the beginning of time) for when the current page hit certain loading milestones.

Adobe’s plugin does look at that performance timing data, compares a bunch of the different milestone timestamps versus each other, then does some math to put it into nice, easy-to-read seconds. For instance, my total load time would be the number of seconds between navigationStart and loadEventEnd:

1556048746779 (loadEventEnd) – 1556048745659 (navigationStart) = 1120 milliseconds, or 1.12 seconds.

Additionally, if I choose to, I can have the plugin grab information from the built-into-the-browser performance.getEntries(), put it into session storage (not a cookie because it can be a long list), and put it into the variable of your choice (usually a listVar or list prop) on the next page. These entries show you for EACH FILE on the page, how long they took to load.

Unfortunately, if I’m sending my analytics page view beacon while the page is still loading, then the browser can’t tell me when “domComplete” happened…. because it hasn’t happened yet! So the plugin writes all these values to a cookie, then on your NEXT beacon, reads them back and puts them into numeric events that you define when you set the plugin up. This means you won’t get a value on the first page of the visit, and the values for the last page of the visit won’t ever be sent in. It also means you don’t want to break these metrics down by page, but rather by PREVIOUS page- so often this plugin is rolled out alongside the getPreviousValue plugin. This also means that the plugin is not going to return data for single-page visits or for the last page of visits (it may collect the data but doesn’t have a second beacon to send the data in on). for this reason, your Performance Timing Instances metric may look significantly different from your Page Views metric.

What It Captures

Out of the box, the plugin captures all of the following into events:

  • Redirect Timing (seconds from navigationStart to fetchStart- should be zero if there was no redirect)
  • App Cache Timing (seconds from fetchStart to domainLookupStart)
  • DNS Timing (seconds from domainLookupStart to domainLookupEnd)
  • TCP Timing (seconds from connectStart to connectEnd)
  • Request Timing (seconds from connectEnd to responseStart)
  • Response Timing (seconds from responseStart to responseEnd )
  • Processing Timing (seconds from domLoading to loadEventStart)
  • onLoad Timing (seconds from loadEventStart to loadEventEnd)
  • Total Page Load Time (seconds from navigationStart to loadEventEnd )
  • Instances (for calculated metric- otherwise you only really get the aggregated seconds, which is fairly meaningless if your traffic fluctuates)

Which gets you reporting that looks like this:

…Which, to be honest, isn’t that useful, because it shows the aggregated number of seconds. The fact that our product page took 1.3 million seconds in redirect timing in this reporting period means nothing without some context. That’s why that last metric, “instances”, exists: you can turn any of the first 9 metrics into a calculated metric that shows you the AVERAGE number of seconds in each phase of the page load:

This gives me a much more useful report, so I can start seeing which pages take the longest to load:

As you can see, the calculated metric can use either the “Time” format or the “Decimal” format, depending on your preference.

Performance Entries

As mentioned previously, the plugin can also capture your performance entries (that is, a list of ALL of the resources a page loaded, like images and JS files) and put them into a listVar or prop of your choice. This returns a list, delimited by “!”, where each value has a format that includes:

The name of the resource (ignoring query params)!at what second in the page load this resource started loading!how long it took for that resource to finish loading!resource type (img, script, etc).

For example, on my blog, I might see it return something like this:

https://digitaldatatactics.com/beaconParser/index.html|0.0|0.9|navigation!https://www.digitaldatatactics.com/utility/spiffy.css|0.2|0.1|link!https://digitaldatatactics.com/beaconParser/decoder.css|0.2|0.1|link!https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js|0.2|0.1|script!https://digitaldatatactics.com/beaconParser/js/varInfo.js|0.2|0.1|script!https://digitaldatatactics.com/beaconParser/js/HBvarInfo.js|0.2|0.1|script!https://digitaldatatactics.com/beaconParser/js/wsse.js|0.2|0.1|script!https://digitaldatatactics.com/beaconParser/js/apiConfig.js|0.2|0.2|script!https://digitaldatatactics.com/beaconParser/js/apiRetrieve.js|0.2|0.1|script!https://digitaldatatactics.com/beaconParser/js/decode.js|0.2|0.1|script!https://digitaldatatactics.com/beaconParser/js/print.js|0.2|0.1|script!https://digitaldatatactics.com/images/NarrowBanner.png|0.2|0.3|img!https://digitaldatatactics.com/beaconParser/images/trash.png|0.2|0.5|img!https://assets.adobedtm.com/launch-EN3911ddbdce3b4c4697e2d3c903e9cfc5.min.js|0.4|0.1|script!https://digitaldatatactics.com/beaconParser/images/background-trans.png|0.4|0.3|css!first-paint|0.5|0.0|undefined!first-contentful-paint|0.5|0.0|undefined!https://assets.adobedtm.com/extensions/EP4c3fcccffd524251ae198bf677f3b6e9/AppMeasurement.min.js|0.5|0.0|script!https://jenniferkunz.d1.sc.omtrdc.net/b/ss/jenniferkunztestdev/1/JS-2.12.0-L9SG/s44127282881639|0.7|0.2|img

From this, I can see every file that is used on my page and how long it took to load (and yes, it is telling me that the last resource to load was my analytics beacon, which started .7 seconds into my page loading, and took .2 seconds to complete). This is a LOT of information, and at bare minimum, it can make my analytics beacons very long (you can pretty much accept that most of your beacons are going to become POST requests rather than GET requests), but it can be useful to see if certain files are consistently slowing down your page load times.

An Enhancement: Time to Interaction

Unfortunately, the plugin most commonly used by folks omits one performance timing metric that many folks believe is the most critical: Time to DomInteractive. As this helpful site states:

  • Page Load Time is the time in which it takes to download the entire content of a web page and to stabilize.
  • Time to Interactive is the amount of time in which it takes for the content on your page to become functional and ready for the user to interact with once the content has stabilized.

In other words, Page Load Time might include the time it takes for a lot of background activity to go on, which may not necessarily stop the user from interacting with the site. If your page performance goal is for the best user experience, then Time To Interaction should be a key metric in measuring that. So, how do we track that? It already exists in that performance.timing object, so I tweaked the existing plugin code to include it. I can then create a calculated metric off of that (Time to Interactive/Page Performance Instances) and you can see it tells a very different story for this site than Total Page Load Time did:

9.49 seconds DOES sound like a pretty awful experience, but all three of these top pages had a much lower (and much more consistent) number of seconds before the user could start interacting with the page.

Basic Implementation

There are three parts to setting up the code for this plugin: before doPlugins (configuration), during doPlugins (execution), and after doPlugins (definition).

Configuration

First, before doPlugins, you need to configure your usage by setting s.pte and s.ptc:

s.pte = 'event1,event2,event3,event4,event5,event6,event7,event8,event9,event10,event11'
s.ptc = false; //this should always be set to false for when your library first loads

In my above example, here is what each event will set:

  • event1= Redirect Timing (seconds from navigationStart to fetchStart- should be zero if there was no redirect)- set as Numeric Event
  • event2= App Cache Timing (seconds from fetchStart to domainLookupStart)- set as Numeric Event
  • event3= DNS Timing (seconds from domainLookupStart to domainLookupEnd)- set as Numeric Event
  • event4= TCP Timing (seconds from connectStart to connectEnd)- set as Numeric Event
  • event5= Request Timing (seconds from connectEnd to responseStart)- set as Numeric Event
  • event6= Response Timing (seconds from responseStart to responseEnd )- set as Numeric Event
  • event7= Processing Timing (seconds from domLoading to loadEventStart)- set as Numeric Event
  • event8= onLoad Timing (seconds from loadEventStart to loadEventEnd)- set as Numeric Event
  • event9= Total Page Load Time (seconds from navigationStart to loadEventEnd )- set as Numeric Event
  • event10= Total Time to Interaction (seconds from connectStart to timeToInteraction)- set as Numeric Event. NOTE- THIS IS ONLY ON MY VERSION OF THE PLUGIN, OTHERWISE SKIP TO INSTANCES
  • event11= Instances – set as Counter Event

I’d also need to make sure those events are enabled in my Report Suite with the correct settings (everything should be a Numeric Event, with the exception of instances, which should be a Counter Event).

Execution

Within doPlugins, I need to just run the s.performanceTiming function. If I don’t want to capture performance entries (which is reasonable- not everyone has the listVars to spare, and it can return a VERY long value that can be difficult to get value out of), then I fire the function without any arguments:

s.performanceTiming()

If I DO want those performance entries, then I add the name of that variable as an argument:

s.performanceTiming("list3")

Also, you’re going to want to be capturing Previous Page Name into a prop or eVar if you aren’t already:

s.prop1=s.getPreviousValue(s.pageName,'gpv_pn');

(If you are already capturing Previous Page Name into a variable, you don’t need to capture it separately just for this plugin- you just need to be capturing it once somewhere).

Definition

Finally, where I have all of my plugin code, I need to add the plugin definitions. You can get Adobe’s version from their documentation, or if you want it with Time To Interactive, you can use my version:

/* Plugin: Performance Timing Tracking - 0.11 BETA - with JKunz's changes for Time To Interaction. 
Can you guess which line I changed ;)?*/
s.performanceTiming=new Function("v",""
+"var s=this;if(v)s.ptv=v;if(typeof performance!='undefined'){if(perf"
+"ormance.timing.loadEventEnd==0){s.pi=setInterval(function(){s.perfo"
+"rmanceWrite()},250);}if(!s.ptc||s.linkType=='e'){s.performanceRead("
+");}else{s.rfe();s[s.ptv]='';}}");
s.performanceWrite=new Function("",""
+"var s=this;if(performance.timing.loadEventEnd>0)clearInterval(s.pi)"
+";try{if(s.c_r('s_ptc')==''&&performance.timing.loadEventEnd>0){try{"
+"var pt=performance.timing;var pta='';pta=s.performanceCheck(pt.fetc"
+"hStart,pt.navigationStart);pta+='^^'+s.performanceCheck(pt.domainLo"
+"okupStart,pt.fetchStart);pta+='^^'+s.performanceCheck(pt.domainLook"
+"upEnd,pt.domainLookupStart);pta+='^^'+s.performanceCheck(pt.connect"
+"End,pt.connectStart);pta+='^^'+s.performanceCheck(pt.responseStart,"
+"pt.connectEnd);pta+='^^'+s.performanceCheck(pt.responseEnd,pt.respo"
+"nseStart);pta+='^^'+s.performanceCheck(pt.loadEventStart,pt.domLoad"
+"ing);pta+='^^'+s.performanceCheck(pt.loadEventEnd,pt.loadEventStart"
+");pta+='^^'+s.performanceCheck(pt.loadEventEnd,pt.navigationStart);pta+='^^'+s.performanceCheck(pt.domInteractive, pt.connectStart);"
+"s.c_w('s_ptc',pta);if(sessionStorage&&navigator.cookieEnabled&&s.pt"
+"v!='undefined'){var pe=performance.getEntries();var tempPe='';for(v"
+"ar i=0;i<pe.length;i++){tempPe+='!';tempPe+=pe[i].name.indexOf('?')"
+">-1?pe[i].name.split('?')[0]:pe[i].name;tempPe+='|'+(Math.round(pe["
+"i].startTime)/1000).toFixed(1)+'|'+(Math.round(pe[i].duration)/1000"
+").toFixed(1)+'|'+pe[i].initiatorType;}sessionStorage.setItem('s_pec"
+"',tempPe);}}catch(err){return;}}}catch(err){return;}");
s.performanceCheck=new Function("a","b",""
+"if(a>=0&&b>=0){if((a-b)<60000&&((a-b)>=0)){return((a-b)/1000).toFix"
+"ed(2);}else{return 600;}}");
s.performanceRead=new Function("",""
+"var s=this;if(performance.timing.loadEventEnd>0)clearInterval(s.pi)"
+";var cv=s.c_r('s_ptc');if(s.pte){var ela=s.pte.split(',');}if(cv!='"
+"'){var cva=s.split(cv,'^^');if(cva[1]!=''){for(var x=0;x<(ela.lengt"
+"h-1);x++){s.events=s.apl(s.events,ela[x]+'='+cva[x],',',2);}}s.even"
+"ts=s.apl(s.events,ela[ela.length-1],',',2);}s.linkTrackEvents=s.apl"
+"(s.linkTrackEvents,s.pte,',',2);s.c_w('s_ptc','',0);if(sessionStora"
+"ge&&navigator.cookieEnabled&&s.ptv!='undefined'){s[s.ptv]=sessionSt"
+"orage.getItem('s_pec');sessionStorage.setItem('s_pec','',0);}else{s"
+"[s.ptv]='sessionStorage Unavailable';}s.ptc=true;");
/* Remove from Events 0.1 - Performance Specific, 
removes all performance events from s.events once being tracked. */
s.rfe=new Function("",""
+"var s=this;var ea=s.split(s.events,',');var pta=s.split(s.pte,',');"
+"try{for(x in pta){s.events=s.rfl(s.events,pta[x]);s.contextData['ev"
+"ents']=s.events;}}catch(e){return;}");
/* Plugin Utility - RFL (remove from list) 1.0*/
s.rfl=new Function("l","v","d1","d2","ku",""
+"var s=this,R=new Array(),C='',d1=!d1?',':d1,d2=!d2?',':d2,ku=!ku?0:"
+"1;if(!l)return'';L=l.split(d1);for(i=0;i<L.length;i++){if(L[i].inde"
+"xOf(':')>-1){C=L[i].split(':');C[1]=C[0]+':'+C[1];L[i]=C[0];}if(L[i"
+"].indexOf('=')>-1){C=L[i].split('=');C[1]=C[0]+'='+C[1];L[i]=C[0];}"
+"if(L[i]!=v&&C)R.push(C[1]);else if(L[i]!=v)R.push(L[i]);else if(L[i"
+"]==v&&ku){ku=0;if(C)R.push(C[1]);else R.push(L[i]);}C='';}return s."
+"join(R,{delim:d2})");

You’ll also need to have s.apl and s.split.

You can see a full example of what your plugins code might look like, as well as a deobfuscated picking-apart of the plugin, on our gitHub.

Performance Entries Classifications

I recommend if you ARE capturing Performance Entries in a listVar, setting up 5 classifications on that listVar:

  • Resource/File
  • Starting Point
  • Duration
  • Duration- Bucketed (if desired)
  • Resource Type

Then set up a Classification Rule, using this regex string as the basis:

^(.+)\|(.+)\|(.+)\|(.+)

In our git repo, I have a full list of the classification rules and regex I used, including how to bucket the durations so you get less granular values like “0.5-1.0 seconds”, which can give you a report like this:

Implications for Single Page Apps

Unfortunately, this plugin will NOT be able to tell you how long a “virtual page” on a single page app (SPA) takes to load, because it relies on the performance.timing info, which is tied to a when an initial DOM loads. This isn’t to say you can’t deploy it on a Single Page App- you may still get some good data, but the data will be tied to when the overall app loads. Take this user journey for example, where the user navigates through Page C of a SPA, then refreshes the page:

As you can see, we’d only get performanceTiming entries twice- once on Page A and once on the refreshed Page C. Even without the “virtual pages”, it may still be worth tracking- especially since a SPA may have a lot of upfront loading on the initial DOM. But it’s not going to tell the full story about how much time the user is spending waiting for content to load.

You can still try to measure performance for state changes/”virtual pages” on a SPA, but you’ll need to work with your developers to figure out a good point to start measuring (is it when the user clicks on the link that takes them to the next page? Or when the URL change happens?) and at what point to stop measuring (is there a certain method or API call that brings in content? Do you having a “loading” icon you can piggy back on to listen to the end?). Make sure if you start going this route (which could be pretty resource intensive), you ask yourselves what you can DO with the data: if you find out that it takes an average 2.5 seconds to get from virtual page B to virtual page C, what would your next step be? Would developers be able to speed up that transition if the data showed them the current speed was problematic?

Use the Data

Finally, it’s important to make sure after you’ve implemented the plugin, you set aside some time to gather insights and make recommendations. I find that this plugin is one that is often used to just “check a box”- it’s nice to know you have it implemented in case anyone ever wants it, but once it is implemented, if often goes ignored. It is good to have in place sooner rather than later, because often, questions about page performance only come up after a change to the site, and you’ll want a solid baseline already in place. For instance, if you’re migrating from DTM to Launch, you might want to roll this plugin out in DTM well in advance of your migration so after the migration, you can see the effect the migration had on page performance. Consider setting a calendar event 2 weeks after any major site change to remind you to go and look at how it affected the user experience.

Followup Post: Allocation in Analysis Workspace

I recently posted about some of the implications and use cases of using Linear Allocation (on eVars) and participation (props/events) and in my research, I thought I had encountered a bug in Analysis Workspace. After all, for this flow:

Page A Page B Page C Page D Newstletter Signup event (s.tl)
prop1
eVar1
events
“Page A”
“Page A”
“event1”
“Page B”
“Page B”
“event1”
“Page C”
“Page C”
“event1”
“Page D”
“Page D”
“event1”
“”
“”
“event2”

I saw this in Reports and Analytics (so far, so good):

But then in Analysis Workspace for that prop, trying to recreate the same report, I saw this, where the props were only getting credited for events that happened on their beacon (none got credit for the newsletter signup):

Basically, I lost that participation magic.

Similarly, for the eVar, I saw this report in Reports and Analytics:

And in Workspace, it behaved exactly like a “Most Recent” eVar:

Again, it lost that linear magic.

Calculated Metrics to the Rescue

With the help of some industry friends (thanks, Jim Kultgen at Kohler and Seth Burke at Adobe) I learned that this is not a bug, necessarily- it’s the future! Analysis Workspace has a different way of getting at that data (one that doesn’t require changing the backend settings for your variables and metrics).
In Analysis Workspace reports, allocation can be decided by a Calculated Metric, instead of the variable’s settings. In the calculated metric builder, you can specify an allocation by clicking the gear box next to the metric in the Calculated Metric Definition:

A Note On “Default” Allocation here

On further testing, in Analysis Workspace, it seems that eVars with the back-end settings of either “Most Recent” and “Linear” allocation are treated the same: both will act like “Most Recent” with a metric brought in, and both will act like “Linear” when you bring in a calculated metric where you specified to have Linear Allocation. One might say, if you use Analysis Workspace exclusively, you no longer would need to ever set an eVar to “Linear”.

“Default” does still seem to defer to the eVar settings when it comes to Most Recent or Original (just not Linear). So in an eVar report where the eVar’s backend setting is “Original”, whether I used my “normal” Newsletter Signups event (column 2), or my Calculated one with Linear Allocation (column 3), credit went to the first page:

So, the Calculated Metric allocation did NOT overwrite my eVar setting of “Original”.

So how do I replicate my Linear eVar report?

To get back that Linear Allocation magic, I would create a new Calculated Metric, but I would specify “Linear Allocation” for it in the Calculated Metric Definitions. Then I can see that linear metric applied to that eVar (the original metric in blue, the new calculated one with linear allocation in purple) :

Note that it’s 40-20-20-20, rather than 25-25-25-25. I’ll admit, this isn’t what I expected and makes me want to do more testing. I suspect that it’s looking at my FIVE beacons (four page views, one success event) and giving that Page D double credit- one for its page view beacon, and one for the success event beacon (even though it wasn’t set on that beacon, it WAS still persisting). So it isn’t perfectly replicating my R&A version of the report, but it is helping me spread credit out between my four values.

And my participation prop?

Similarly, with the prop, when I bring in my new “Linear Allocation” calculated metrics I just set up for my eVar (in blue), I now see it behave like participation for my Newsletter Signup metric, unlike the original non-calculated metrics (in green):

…but those Page View numbers look just like linear allocation in an eVar would (2.08, 1.08, .58, .25), not the nice clean numbers (4, 3, 2, 1) I’d get for a prop with participation. At this point, I still don’t have my Content Velocity prop report, but I’m getting closer.

So how do I get my Content Velocity?

Analysis Workspace has a “Page Velocity” Calculated metric built into its Content Consumption template, which reports the same data as my Content Velocity (participation-enabled) prop did in Reports & Analytics.

 If I want to recreate this calculated metric for myself, I use the formula “Page Views (with Visit Participation)/Page Views”:

Though my friend Jim Kultgen suggested a metric he prefers:

((Page Views 'Visit Participation')/(Visits))-1

This shows you how a page contributed to later page views, discounting how it contributed to itself (because obviously it did that much- every page does), and looking at visits to that page (so repeat content views don’t count for much).

These two calculated metrics would show in an AW report like this:

Conclusion

If I use Analysis Workspace exclusively, I may no longer need to enable participation on metrics or props- I could just build a Calculated Metric off of existing metrics, and change their allocation accordingly, and that would work the same with either my eVars or my Props.

Knowing a few of these quirks and implications, I can see a future with simpler variable maps (no more need for multiple eVars receiving the same values but with different allocation settings) and the ability to change allocation without tweaking the original data set (my “Newsletter Signups” metric retains its original reporting abilities, AND I can build as many Calculated Metrics off of it as I want). I’m excited to see how Adobe will keep building more power/flexibility into Workspace!

Participation and Linear Allocation in Adobe Analytics- behavior I expected, and some I did not

I recently posted about some of the implications and use cases of using Linear Allocation (on eVars) and participation (props/events) and in my research, I thought I had encountered a bug in Analysis Workspace. After all, for this flow:

Page A Page B Page C Page D Newstletter Signup event (s.tl)
prop1
eVar1
events
“Page A”
“Page A”
“event1”
“Page B”
“Page B”
“event1”
“Page C”
“Page C”
“event1”
“Page D”
“Page D”
“event1”
“”
“”
“event2”

I saw this in Reports and Analytics (so far, so good):

But then in Analysis Workspace for that prop, trying to recreate the same report, I saw this, where the props were only getting credited for events that happened on their beacon (none got credit for the newsletter signup):

Basically, I lost that participation magic.

Similarly, for the eVar, I saw this report in Reports and Analytics:

And in Workspace, it behaved exactly like a “Most Recent” eVar:

Again, it lost that linear magic.

Calculated Metrics to the Rescue

With the help of some industry friends (thanks, Jim Kultgen at Kohler and Seth Burke at Adobe) I learned that this is not a bug, necessarily- it’s the future! Analysis Workspace has a different way of getting at that data (one that doesn’t require changing the backend settings for your variables and metrics).
In Analysis Workspace reports, allocation can be decided by a Calculated Metric, instead of the variable’s settings. In the calculated metric builder, you can specify an allocation by clicking the gear box next to the metric in the Calculated Metric Definition:

A Note On “Default” Allocation here

On further testing, in Analysis Workspace, it seems that eVars with the back-end settings of either “Most Recent” and “Linear” allocation are treated the same: both will act like “Most Recent” with a metric brought in, and both will act like “Linear” when you bring in a calculated metric where you specified to have Linear Allocation. One might say, if you use Analysis Workspace exclusively, you no longer would need to ever set an eVar to “Linear”.

“Default” does still seem to defer to the eVar settings when it comes to Most Recent or Original (just not Linear). So in an eVar report where the eVar’s backend setting is “Original”, whether I used my “normal” Newsletter Signups event (column 2), or my Calculated one with Linear Allocation (column 3), credit went to the first page:

So, the Calculated Metric allocation did NOT overwrite my eVar setting of “Original”.

So how do I replicate my Linear eVar report?

To get back that Linear Allocation magic, I would create a new Calculated Metric, but I would specify “Linear Allocation” for it in the Calculated Metric Definitions. Then I can see that linear metric applied to that eVar (the original metric in blue, the new calculated one with linear allocation in purple) :

Note that it’s 40-20-20-20, rather than 25-25-25-25. I’ll admit, this isn’t what I expected and makes me want to do more testing. I suspect that it’s looking at my FIVE beacons (four page views, one success event) and giving that Page D double credit- one for its page view beacon, and one for the success event beacon (even though it wasn’t set on that beacon, it WAS still persisting). So it isn’t perfectly replicating my R&A version of the report, but it is helping me spread credit out between my four values.

And my participation prop?

Similarly, with the prop, when I bring in my new “Linear Allocation” calculated metrics I just set up for my eVar (in blue), I now see it behave like participation for my Newsletter Signup metric, unlike the original non-calculated metrics (in green):

…but those Page View numbers look just Despite clearly remembering learning about it my first week on the job at Omniture in 2006, I realized recently that I did not have a lot of confidence in what participation and linear allocation would do in certain situations in Adobe Analytics. So I put a good amount of effort into testing it to confirm my theories, and I figured I’d pass along what I discovered.

First, the Basics: eVar Allocation

You may already know this part, so feel free to skip this section if you do. Allocation is a setting for Conversion Variables (eVars) in Adobe Analytics, with three options:

Let’s take a simple example to show what how this effects things. Let’s say a user visits my site with this flow:

Page A Page B Page C Page D Form Submit- Signup
s.eVar5=”Page A” s.eVar5=”Page B” s.eVar5=”Page C” s.eVar5=”Page D” s.events=”event1″

Most Recent (Last)

Most eVars have the “defaultiest” allocation of “Most Recent (Last)”, meaning in an event1 report broken down by eVar5, “Page D” would get full credit for the event1 that happened, since it was the last value we saw before event1. So far, pretty simple.

Original Value (First)

But maybe I want to know which LANDING page contributed the most to my event1s (there are other ways of doing this, but for the sake of my example, I’m gonna stick with using allocation). In that case, I might have the allocation for that eVar set to “Original Value (First)” so then “Page A” would get full credit for this event1, since it was the first value we saw for that variable. If my eVar is set to expire on visit, then it’s still nice and straightforward. If it’s set to never expire, then the first value we ever saw for that user will always get credit for any of that user’s metrics. If it’s set to expire in two weeks, then we’ll see the first value that was passed within the last two weeks.

This setting is frequently used for Marketing Campaigns (it’s not uncommon to see s.campaign be used for “Most Recent Campaign in the last 30 days” and then another eVar capture the exact same values, but be set to “Original Campaign in the last 30 days”).

Linear Allocation

If I’m feeling a bit more egalitarian, and want to know ALL the values for an eVar that contributed to success events, I would choose linear allocation. In this scenario, all four values would split the credit for the one event, so they’d each get one fourth of the metric:

(Though it may not actually look like this in the report- by default it would round down to 0. But I’ll talk about decimals later on).

So, that’s allocation.

Then what is participation?

Participation is a setting you can apply to a prop, so that if you bring a Participation-enabled metric into the prop’s report, you can see which values were set at some point before that event took place. Repeat: to see participation you must have a prop that is set to “Display Participation Metrics”:

And the metric you want to see needs to have participation enabled (without this, in the older Reports and Analytics interface, that event won’t be able to be brought into the prop report):

Unlike linear allocation for an eVar, participation for a prop means all the values for that prop get full credit for an event that happened. So, given this flow:

Page A Page B Page C Page D Form Submit- Signup
s.prop1=”Page A” s.prop1=”Page B” s.prop1=”Page C” s.prop1=”Page D” s.events=”event1″

You would see a report like this, because each value participated in the single instance of that event:

New Learnings (for me): Content Velocity

One thing these settings can be used for is measuring content velocity: that is, how much a certain value contributed to more content views later on. For instance, if I have a content site, and I want to know how much one piece of content tends to lead to the reading of MORE content, I might use a participation-metric-enabled prop with a participation-enabled Page View custom event, or I might use an eVar with linear allocation against a Page View custom event (whether or not the event has participation enabled doesn’t matter for the eVar). For my test, I did both:

Page A Page B Page C Page D
s.prop1=”Page 1″
s.eVar1=”Page 1″
s.events=”event1″
s.prop1=”Page 2″
s.eVar1=”Page 2″
s.events=”event1″
s.prop1=”Page 3″
s.eVar1=”Page 3″
s.events=”event1″
s.prop1=”Page 4″
s.eVar1=”Page 4″
s.events=”event1″

The prop

The prop version of this report would show me that Page 1 contributed to 4 views (its own, and 3 more “downstream”). Whereas Page 2 contributed to 3 (its own, and two more downstream), etc…

The eVar

Alternatively, the eVar would show me some thing pretty odd:

Those weird numbers don’t make sense on this small scale (how could 0 get 6.3%?), because it is rounding, and not showing me decimals. If I want to see the decimals, I can create a really simple calculated metric that brings in my custom Page View event (event1) and tells it to show decimals:

The report then makes a little more sense and show us where the rounded numbers came from (and how Page 4, with “0” Page Views, got 6.3% of the credit), but may still seem mysterious:

Those are some odd numbers, right? Here’s the math:

 Value Credit Why?   Explanation
Page 1 2.08 1+0.5+0.33+0.25 It got full credit for its own view, then half the credit (shared with page 2) for the event on Page 2, then a third of the credit (shared with Page 2 and Page 3) on Page 3…
Page 2 1.08 0.5+0.33+0.25 It only got half credit for the event that took place on its page (shared with Page 1), then a third of the credit (shared with Page 1 and Page 3) on Page 3, etc…
Page 3 0.58 .33+.25 It only gets a third of the credit that took place on its page, and a quarter of the credit for the fourth page.
Page 4 0.25 0.25 The event that happened on this page is shared with all four pages.

Crazy, right? I’m not going to tell you which an analyst should prefer, but as always, you should ask the question: “What will you DO with this information?”

What happens when multiple values appear in the same flow?

Let’s say the user does something like this, where they hit one value a couple page views in a row (Page B in this example), or they hit a value 2 separate times (Page A in this example):

Page A Page B Page B
(again)
Page C Page D Page A
(again)
Conversion event
prop1
eVar1
events
“Page A”
“Page A”
“event1”
“Page B”
“Page B”
“event1”
“Page B”
“Page B”
“event1”
“Page C”
“Page C”
“event1”
“Page D”
“Page D”
“event1”
“Page A”
“Page A”
“event1”
events=”event2″

For the prop, it’s pretty straightforward. This will look like 6 event1s, where Page A gets value for all 6, and Page D gets credit for just 2 (itself, and the Page A that came afterwards):

For the eVar, it gets a little more complicated (I added in a calculated metric so you can see the decimals). Page A (accessed twice at separate times) got double credit for the conversion (which I might have predicted), but Page B (accessed twice in a row) ALSO gets double credit for the conversion (which I didn’t predict, probably because I’m too used to thinking in terms of the CVP plugin):

Caveats

A couple things to be aware of:

  • Settings for participation and allocation don’t apply retroactively- you can’t apply them to existing data. If you want to start using it, you need to change your settings and you’ll see it applied to future data. However, this can mess with existing data, so be careful.
  • Analysis Workspace does some unexpected behavior for both participation and allocation. I’ll have a followup post on that.

Conclusion

Both participation and linear allocation aren’t used often, but they can uniquely solve some reporting requirements and can provide a lot of insight, if you know how to read the data. I hope my experimentation and results here help make it clearer how you might be able to use and interpret data from these settings. Let me know if you have other use cases for using these settings, and how it has worked out for you!like linear allocation in an eVar would (2.08, 1.08, .58, .25), not the nice clean numbers (4, 3, 2, 1) I’d get for a prop with participation. At this point, I still don’t have my Content Velocity prop report, but I’m getting closer.

So how do I get my Content Velocity?

Analysis Workspace has a “Page Velocity” Calculated metric built into its Content Consumption template, which reports the same data as my Content Velocity (participation-enabled) prop did in Reports & Analytics.

 If I want to recreate this calculated metric for myself, I use the formula “Page Views (with Visit Participation)/Page Views”:

Though my friend Jim Kultgen suggested a metric he prefers:

((Page Views 'Visit Participation')/(Visits))-1

This shows you how a page contributed to later page views, discounting how it contributed to itself (because obviously it did that much- every page does), and looking at visits to that page (so repeat content views don’t count for much).

These two calculated metrics would show in an AW report like this:

Conclusion

If I use Analysis Workspace exclusively, I may no longer need to enable participation on metrics or props- I could just build a Calculated Metric off of existing metrics, and change their allocation accordingly, and that would work the same with either my eVars or my Props.

Knowing a few of these quirks and implications, I can see a future with simpler variable maps (no more need for multiple eVars receiving the same values but with different allocation settings) and the ability to change allocation without tweaking the original data set (my “Newsletter Signups” metric retains its original reporting abilities, AND I can build as many Calculated Metrics off of it as I want). I’m excited to see how Adobe will keep building more power/flexibility into Workspace!

Find the total number of values in an Adobe Analytics Variable Report

“How do I know how many rows/unique values are in my eVar report?”
I hear this question fairly often, and since the answer isn’t currently extremely straight-forward, I figured I could walk through the solution here. So let’s say you have a report with many pages:
reportPagination
Rather than trying to get to the last page of the report somehow to see how many row numbers there are, you can find out about the number of unique values in the report using the Row Count calculated metric.
To set this up, go to add a metric to the report, then “Add” to get to the Calculated Metric wizard:
addCalcuatedMetric

Give the metric an intuitive name, like “Report Rows” or “Unique Reported Values”, then search in the formulas for “Row Count”:
rowCount

Drag “Row Count” over to the Calculated Metric Definitions, and save. That’s all there is to it!

RowCountCalc

Now you can have that as a metric in your report. For whatever reason, it doesn’t always show nicely if it is the ONLY metric selected, so I recommend having at least one metric (like visits) alongside it as well. It doesn’t make a ton of sense in the report, line-by-line: every line will show the same number, and that number represents the aggregate number of values in the report. So in the below, I can see this particular report has 140 unique rows/values in it:
RowCountReport

I do believe in the future, a simpler way of doing this may be a bit more built in, but for now, here is a relatively easy way to get a count of values for a report!

How to approach Product Finding Methods

Product Finding Method (PFM) is a valuable custom report used by online retailers to tie Cart Adds and eventual Revenue to how users found the product. Unfortunately, these reports are often poorly configured or misunderstood. It is one of the best use cases for using the Merchandising setting on an eVar, but Merchandising, as a concept, is one of the more complicated topics in implementation.

Primer on Merchandising

In order to tie that PFM value to an individual product, and not the overall visit or visitor, we need to use Merchandising. And since we may not know the eventual product at the time that we know the finding method (for instance, if a user is using internal search, at the time we know they are searching, we don’t know what product they’ll eventually land on), we need to use Conversion Syntax- meaning we set it like a typical eVar (instead of in the products string) but have to configure at what point that eVar should be tied (or “bound”) to a product. Take the following user experience into consideration, where eVar5 is used for Product Finding Methods:

PFMflow

In the above, if I did NOT use Merchandising, my report might look like this, where External Campaign gets credit for everything up until it gets overwritten by Internal Search– at which point Internal Search gets credit for everything, including the entire Purchase event.

PFMnoMerchReportIf I DO use Merchandising, the External Campaign gets credit for whatever happens to the product it was bound to- in this case, the Blue Wug. Then Internal Search gets bound to the Red Wug, and gets credit for whatever happens to the Red Wug product:

PFMwMerchReport

We have to be very careful with the binding events (the event that tells Adobe to take whatever value we currently have floating around for that eVar and stick it to the current product), though- merchandising eVars won’t get credit for anything that happens if they are not bound to any product, or if that product isn’t currently present. For instance, in the example above, if you pulled in Internal Searches as a metric in that report, you’d see no PFMs were getting credit for the Internal Search event- even though eVar5 is set on the same page as our search event. That’s because that “Internal Search” value for eVar5 is waiting until a “binding event” to attach it to a product before it will get credit for any metrics, and the “External Campaign” value for eVar5 only gets visibility into things that happen to the product it’s bound to (Blue Wug). No products on the Internal Search page means Conversion-Syntax Merchandising eVars get no credit for the Search event.

How to Plan Your Product Finding Methods

To design your PFM solution, start by thinking of the different ways a client can add an item to their cart. We’ll call these “add to cart location” and give that its own eVar. Typical scenarios are:

  • product details page
  • quick view
  • from wishlist
  • from order history
  • recommendation/cross sell modules

Next, figure out all the ways a user could get to any of those above scenarios. These are Product Finding Methods and should get their own eVar. Typical PFMs include:

  • internal search
  • external site (referrerss and natural search)
  • external campaigns (that take the user directly to the PDP)
  • internal promotion (eg, homepage hero banner)
  • browse (naturally navigating through the menu or breadcrumbs)
  • cross-sell
  • recommendations
  • wishlist
  • order history 

You’ll notice some of your cart add locations may also be product finding methods- that’s ok. Note that Products Detail Page is NOT a PFM- the Product Details page is specific to the product, which is the thing being found not the thing doing the finding.

Both of these eVars will need to be set up for Merchandising (this is set in the Admin console>Report Suites>Edit Settings>Conversion>Conversion Variables). For the Add to Cart Location eVar, since we know the product at the time we know the add to cart location, you can use “product syntax”.

eVar4PFM

This is preferable because it leaves little room for error and has the easiest configuration. Let’s say I’ve set eVar4 to be my “Add to Cart location” eVar. At the time my user adds any item to their cart, I would set eVar4 (often, you can just set it to the current page’s page type):

s.products=";sku1;;;;eVar4=PDP"
s.events="scAdd"

But for Product Finding Method, you often don’t know what product the value for our eVar will eventually bind to, so we need to set it to conversion syntax, and we need to carefully consider which events should bind the current PFM value to the current product. Generally, Product View (ideally a custom event, not prodView) and Cart Add suffice as binding events.

eVar5pfm

DO NOT set to bind on ALL events, as many events happen on your site where your PFM eVar doesn’t have a desirable value.

Next, to implement, we need to figure out where to set each of our PFM values. If you’re using a tag management system like DTM, you can often just piggy back on existing rules for many of your finding methods, but others may need to live in global logic (like in your s_code). See below for a possible configuration.

internal search Set on internal search results page
external site Set in global logic- if no other PFM is present, and the user came from a domain other than your own
external campaigns Set in global logic, if s.campaign is set
internal promotion Set in global logic, if internal campaign eVar is set (or internal campaign query string parameter is present).
browse Set on all Product List Pages and Category pages
cross-sell Set when user clicks or clicks through on a cross-sell module
recommendations Set when user clicks or clicks through on a recommendation module
wishlist Set on Wish List page
order history Set on Order History page

Some folks manage PFM entirely in their s_code or in their (global DTM tool settings) based on whether other variables have been set (“if internal search term eVar has a value, set eVar5 to ‘internal search”‘). For instance, one client has something similar to this, in addition to setting their PFM eVar directly on certain pages (like their List Pages and Category pages, or certain microsites):

//easy way to get just the hostname of the referring site
   var a=document.createElement('a');
   a.href=document.referrer;
   s.refDomain=a.hostname;

if(!s.eVar5){ //if PFM not currently set on the page
   if(s.campaign){s.eVar5="campaign"} 
   else if(s.eVar10){s.eVar5="internal search"}
   else if(s.eVar23){s.eVar5="recommended product"} 
   else if(s.pageName=="My Account>Wishlist"){s.eVar5="my wishlist"}
   else if(document.referrer==""){s.eVar5="direct or bookmarked"}
   else if(s.refDomain.indexOf("mydomain.com")==-1){s.eVar5="external site"} 
 }
   else if(s.eVar15){s.eVar26="internal campaign"} 
}

Conclusion

Hopefully, this gives some ideas and examples for how to get valuable reporting on how users are finding and purchasing products on your site. I’d love to hear about potential scenarios or solutions I’ve missed!

Why do my Adobe Analytics numbers differ so much from my bit.ly stats?

Link shorteners like bit.ly, goo.gl, and t.co generally keep their own metrics for how many times users click through. Unfortunately, those numbers rarely match any other Analytic’s system, including Adobe Analytics. We’ve seen situations where the bit.ly number was as much as 10 times the analytics number. With such a big difference, both numbers look a little suspect. So, what gives?

Bots

First, there’s non-human traffic. Link shorteners’ numbers do not always account for whether the click came from a real live human being, or a bot. Goo.gle and bit.ly DO try to eliminate bot traffic, but the list they use to identify bots is likely different from the list Adobe uses (which is maintained by the IAB).
However, link shorteners are often used on twitter, and bot traffic on twitter is a whole different beast. Web bots generally crawl pages, sometimes searching for specific things. Adobe and the IAB keep a good list of the worst web-crawling bot offenders. But on twitter, the environment is more predictable (tweets, unlike webpages, can predictably be accessed/interacted with the same methods) and the actions bots perform are simpler: click, retweet, reply. This means many more “hackers” out there can create a bot with much less effort, and since they aren’t harming much, they fly under the radar of the IAB and other bot-tracking organizations.
I found a book, “TWITTER: The Dark Side – Does Bit.ly Enable a Massive Click Fraud?” (by Roman Latkovic and Robert LaQuay, Ph.D) which takes a thorough look at this issue. Basically, when you start to break down individual clicks by IP and user agent, it becomes clear many bots are indeed inflating bit.ly tracking.

Javascript Disabled

Second, there’s javascript. The majority of Adobe implementations these days rely on the user firing JavaScript (some older implementations or mobile implementations may fire a hard-coded beacon, but this is increasingly rare). Link shorteners, however, run WITHOUT javascript- meaning devices that have javascript disabled would count in bit.ly but NOT in Adobe.

Lost Referrer Information

Third, bit.ly (or twitter traffic in general) may cause you to lose referrer information. Someone coming from a twitter app rather than a webpage won’t have a referrer-it may look like “Direct Traffic”. Query string parameters SHOULD always carry through, though, so make sure your shortened links include tracking codes.

Conclusion

With this information in mind, I’d be more surprised to see bit.ly numbers that MATCH an Analytics tool than I am to see wide discrepancies. What’s your experience, and what are your theories behind them?

Campaign Tracking Tips and Tricks for Omniture SiteCatalyst

The standard Omniture Campaign reporting offering, based solely on a query string/tracking code pair, is pretty solid: campaign reports and all the possible classifications, paid/natural search detection and reporting, and marketing channel reporting, managed from right there in the SiteCatalyst admin console. But over time we’ve found some tips and tricks that can maximize the value of your campaign tracking efforts.

All of these tips rely on granular but intuitive tracking codes, which your s_code grabs automatically out of a query string parameter. If you do not have a well-founded system for setting your tracking codes, then that should be your focus before expanding your campaign reports. Assuming you have your marketing efforts well identified, here are some ideas to implement:

TIP #1: CUSTOM CAMPAIGN CLICK-THROUGHS

Out of the box, SiteCatalyst has the “click-throughs” metric in any campaign report, which is essentially the same as “instances” in any other conversion report. The problem is that you cannot pull your campaign click-throughs as a metric into non-campaign reports- for instance, into a campaign landing pages report, or a breakdown for any other eVar that lasts between sessions or is set on a campaign landing page. A common need is to see Campaign Click-throughs in a conversion funnel:
campaignClick-throughs

For this reason, many of my past clients have implemented a “custom” campaign clickthrough event, which they then have full control over.

The code for this is simple. I recommend using the Append List plugin (s.apl) to ensure the custom event does not overwrite any other events on the page. In this example, event3 has been set aside and named “Custom Campaign Click-throughs”, and the append list plugin code is included in the plugins section:

s.campaign=s.getQueryParam(‘cid’);
if(s.campaign){s.events=s.apl(s.events,’event3′,’,’,1)};

TIP #2: GRANULAR, CLASSIFIED TRACKING CODES WITH A MARKETING CHANNEL PREFIX

This isn’t so much a tip as it is a best practice. While generally the recommendation has always been to just have a granular tracking code and all information could be uploaded afterwards using SAINT classifications, you might save yourself a lot of headache by including a marketing channel identifier in each tracking code. SAINT is a wonderful tool, but classifications cannot be used to define the (fantastic) Marketing Channel reports. So instead of relying solely on classifications to specify that tracking code “2364FS” was an email campaign, turn the tracking code into “em-2364FS”. Not only does this make it easy to filter your tracking codes report, but you can also then set up a Marketing Channel processing rule that specifies that any tracking code that beings with “em” should go into the “Email Campaign” channel:

marketingChannel

What? You haven’t set up your Marketing Channel reports? Well, get on it! It doesn’t take long, this blog post will wait.

TIP #3: CAMPAIGN LANDING PAGE PATHING

I’ll admit, of all the tips listed here, this is my favorite. The standard pathing reports are wonderful but not a lot of help for marketers. Campaign Landing Page Pathing uses a pathing prop to show how campaigns play into paths that users take through the site. Any time the s.campaign variable is set, it is appended to the pageName and put into a prop that has pathing enabled (“Promo Landing Page: em1435”). For all other pages, just the pageName is set:

if(s.campaign){s.prop11=s.pageName+": "+s.campaign}
 else{s.prop11=s.pageName};

As a freestanding page views report, I’ll admit the data is nearly useless- a jumble of campaigns and pageNames. BUT, once viewed in a pathing report, you can now see for each campaign and landing page, what next steps a user took. You might see paths that look like this:

Landing Page A: em12543 > Page B > Page C > Exit

You can now create Fall-Out reports based on particular campaigns. You could also see a next page flow report like this, showing the full range of possible paths users are taking from a campaign landing page:

campaignNextFlow

Is your campaign taking your users somewhere you didn’t expect? Is that something you can capitalize on?

You can hit two birds with one stone and use the same logic to also set a “Campaign Landing Page” eVar, if you want to see how individual landing pages contribute to conversion (see the final code of this post for an example).

Please note, this kind of trick becomes less necessary in SiteCatalyst 15 because you can now segment your pathing reports based on campaigns, but it would still be useful if you wanted to compare campaigns to each other quickly without setting up a segment for each.

TIP #4: CAMPAIGN CROSS-VISIT PARTICIPATION

Campaign Cross-visit Participation, also known as “campaign stacking”, has been well covered elsewhere on the web, so I won’t go into too much detail, but no campaign “tips” post would be complete without it. The Marketing Channel reports can show you First and Last Touch, but not much in between. Using the crossVisitParticipation plugin, you can see all of the different campaigns a certain user touched before accomplishing a certain event. Using this plugin for my tracking codes, I could do a report filter for one specific tracking code and see how other tracking codes contributed to it leading to a conversion:

CVPcampaign

The code looks like this:

s.eVar26=s.crossVisitParticipation(s.campaign,'s_cpmcvp','90','5','>','')

While this plugin is wonderful for campaigns, the possible uses of it are limitless- any time you want to see the “path” a user takes through the values of any of your variables, and how that path affects conversion.

ALTOGETHER NOW

If I were to use all of the above tips and standard best practices, my final code might look something like this:

/* Campaign tracking */
 if (s.getQueryParam('cid'))
 {s.campaign=s.getQueryParam('cid');                                         //capture campaign ID
 s.events=s.apl(s.events,'event7',',',1)   ;                                 //custom Campaign Clickthroughs
 s.eVar26=s.crossVisitParticipation(s.campaign,'s_cpmcvp','90','5','>','');  //campaign "stacking"
 s.eVar24=s.prop11=s.pageName+" : "+s.campaign ;                             //campaign landing page eVar and pathing prop
 s.eVar25=s.campaign=s.getValOnce(s.campaign, 'cid_cookie', 0);              //campaign eVar with original allocation
 }else{s.prop11=s.pageName}  ;                                               //campaign landing page pathing

The final result is a set of marketing reports that paint a fuller picture of campaign behavior on your site.

Code for the Append List and GetValOnceplugin can be found in the Help section of SiteCatalyst, under Product Documentation > Implementation > Plugins. Contact your friendly implementation consultant if you need information about implementing Cross-Visit Participation.

Omniture Form Analysis Pitfalls – How to Avoid and Optimize

There is an Omniture plugin, Form Analysis, that will automatically flag which form field a user last touched before abandoning a form. Sounds great, right? This post, which does a great job of going over setting up the plugin, even claims the plugin will make you an office hero! The problem is, this plugin is inherently Evil. And not the good kindof evil. We’d like to tell you how to avoid the evil and keep your Form Analysis on the side of Good.

This plugin is notorious for being implemented incorrectly, and the implications go beyond just one report not working. Since the plugin sends a custom link call (s.tl()) each time a user abandons a form, it has the potential to eat up your secondary server calls (which you pay Omniture for). A recent client of mine has accidentally set one extra secondary server call on every single page view since 2008 because of this plugin. Sending two server calls (one with the click of a link and one on the load of the next page) so close together has been known to cause problems- every once in a while, the two calls compete and only one makes it through to Omniture, decreasing data accuracy. Worst case scenario, the plugin can actually break your form.

And what bugs me most- and this is in no way the fault of the plugin- is that folks use this as a safety crutch: they know they should optimize forms, so they put effort into setting up form tracking, then pat themselves on the back for  job well done. But, feeling comfortable that the data is there, they don’t take it to the next step and actually optimize their forms.

In this situation, you’d be better off skipping the plugin and go right to the “optimize your form” step by using your God-given intelligence. Use common sense. Think like a user: keep forms simple and easy to use. Simply running a plugin or tool will do nothing to improve your forms- remember, brains are more important than tools.

Don’t get me wrong, the plugin has much potential to do good. You just have to put some thought into it. Educate yourself on using this plugin and you can get great value out of it. To help you, here are some implementation pitfalls to avoid:

1. POTENTIAL PROBLEM: TRACKING THE WRONG FORMS

The configuration of the plugin includes two critical variables:

s.formList=””
s.trackFormList=false

When trackFormList is set to true, it will only track forms specified (using the form’s HTML “name” attribute) in the formList (if formList is blank, then nothing will be tracked). So far so good. When set to false, the plugin will track every form on your site that is NOT specified in the formList. This includes:

  • User login forms (even if they are on every page)
  • Some asp.net “forms” that are not actually user forms
  • Find-a-location-near-you and other single-field forms
  • CAPCHAs
  • And, the biggest culprit: Internal Search forms

So if a user hits a page with an internal search form field without hitting the search “submit” button, that’s one server call to Omniture letting you know a “form” was abandoned. If each page has that internal search field, that’s one server call for every page view on your site. Not only is that information useless to you, it has potentially terrifying billing implications, depending on your contract.

SOLUTION: S.TRACKFORMLIST=”TRUE”

My suggestion? ALWAYS have trackFormList equal to “true”. This will require you going through your site and finding the forms that matter most. Focus on one or two key forms on your site. This isn’t just a work-around to get the plugin to work- it’s a great idea for focusing your analysis efforts and avoiding the Data Squirrel pitfall.

2. POTENTIAL PROBLEM: THE PLUGIN RELIES ON YOUR FORMS BEING SET UP “JUST RIGHT”

If you do not have the “name” attribute in the HTML <form> tag, the plugin won’t work. And if you do have names but they look like “form134″, it probably won’t have much meaning for you in the reports. And if every form on your site has the same name, the report will be useless. Same goes for form field/input names.
On another note, if your form script references the “this” object at all, the plugin may break your form, or vice versa. Oops. The only way around it is to remove the plugin, or remove the references to “this”.

SOLUTION: UPDATE FORM HTML

I’m afraid there is no quick javascript fix here: if you want to automate form abandonment tracking using this plugin, you may have to manually change the “name” attribute of the HTML forms. BUT, if you are focusing on only your 1-2 key forms, as mentioned above, the changes to your HTML should be minimal.

3. POTENTIAL PROBLEM: CAN PAINT INACCURATE PICTURE OF WHICH FIELD IS “GUILTIEST”

The plugin doesn’t tell you which field was the “turn-off”: it tells you the last field touched, which may be counter-intuitive. For example, on the right is what my form looked like when the user abandoned:

Our friend Mal only made it through the 4th field- the one with the name “email”. The plugin would report this as:

pageName:formName:email:abandon

Which might make one think the “email” field was to blame for the abandon, when in truth, he filled out that field successfully and was (presumably) “turned off” by the FOLLOWING field, asking for him to confirm his email- he’s much too busy to waste time by entering the same information twice.

SOLUTION

One can get around this by 1)educating users of the report on what the data means (which you should be doing anyways) and/or 2) creating a classification report that says the FOLLOWING field for each item. Either way, please be aware that a report can only give hints- it cannot actually tell you what the user was thinking when they left the form or if they had looked ahead further in the form and were “turned off” by a later field.

Which brings us back to the idea that your brain is a critical part of making this plugin valuable. If you have a form on your site, now is a great time to optimize it. We’ve removed the roadblocks; what’s stopping you?

Collecting Data vs Using Data

You’ve heard of report monkeys—I’m going to tell you about data squirrels. Data squirrels spend lots of energy collecting data; then they hoard it. They carry it around in their cheeks, feeling well-prepared for the winter, but their mouths are so full they can’t process and swallow their hoarded goods.

dataSquirrel

I’m thinking of one former client who was using nearly every single variable available at the time- virtually every available report in Sitecatalyst was filled with data. According to “Best Practices”, they had a weekly meeting to discuss their implementation and keep it up-to-date- the data was tested, current, and abundant. Their dashboards were vibrant and plentiful. But they were so busy collecting and hoarding data that they had no time to process and analyze it. The idea of taking action on the data was never part of the process.

For all of their effort, how much value do you think they were getting out of their analytics tool?

Sidenote: Data Squirrels and Report Monkeys make great friends. The Squirrel hands data to the Monkey and the Monkey hands “reporting requirements” back to the squirrel. This cycle can go on indefinitely without any real value.

You know what makes the web analyst in me want to go cry in a corner? This all-too-common situation:

A major company spends thousands of dollars and dozens of man-hours on an implementation project to grab and report on some new data. The implementation consultant gets the data, shows them the working reports, signs off on the project, then walks away. Six months later, the company calls the consultant and says “this report doesn’t work anymore, someone disabled the variable five months ago and it took us until now to notice. Can we bring you back on to fix the problem?” (Presumably, so the data can go another 5 months without being used).

If you can go 5 months without looking at a report, why-oh-why did you throw money at someone to make the report available? Sadly I know the answer: it’s easier to throw money at something and feel that sense of accomplishment than it is to spend time analyzing and making potentially risky decisions based on the data.

It’s a lovely feeling to wrap up an implementation and feel like the hard part is done. But if you really want value out of your tool, the work is far from done. Fortunately, the post-implementation work is the fun stuff: the stuff that doesn’t count on your IT department to roll out new code; the stuff that has the potential to realize actual, provable value for your company.

If you must, on the day your implementation project is completed, mark a few hours of time two weeks or one month out to play with the data until you get an action out of it. Schedule a meeting with the analyst-minded folks of your organization and go over the data- NOT to discuss how complete your implementation is, but to discuss what insight the data has given you and what the next steps are. Don’t let USING your data fall down the priority list just because it doesn’t have a project and a budget attached to it.

brainsNotTools

This may just be a matter of semantics, but as an industry we need to change our mindset from “I want to track…” to any of the following:

  • I want to HYPOTHESIZE…
  • I want to TEST…
  • I want to COMPARE…
  • I want to BRAINSTORM…
  • I want to OPTIMIZE…
  • I want to DECIDE…
  • I want to ACT UPON…

“Tracking” or merely collecting data should never be the goal or even a main focus. ALWAYS implement with this question in mind: “How am I going to USE this data to optimize my site?”

A major healthcare website recently hired an agency with which I am familiar. The client has used Adobe Analytics for years and noticed a few things that were a little off in their reports (have I mentioned how much I hate the Form Analysis plugin?), so they hired this agency to do an implementation audit. As they sign the contract for the implementation audit contract, they say, “You’ll notice we don’t use conversion variables or events, and that’s fine. We’re really content with just the traffic data, just help us make sure the traffic data is coming through properly”. In other words, “This solution worked for us four years ago, please just keep it working as is.”

Oh, how this breaks my heart! Because I know a year from now they might say, “Gee, we’re spending a chunk of money on our web analytics, but we’ve never done a thing with the data. Let’s bring on an expert to help us analyze”. And that expert will say “analyze? analyze what?” Then they’ll need to re-implement, then wait even more time before they have real, actionable data.

If you’re using the same set of reports that you were 18 months ago, you are very likely losing value on your implementation. That is, unless your website hasn’t changed, in which case there may be bigger problems. The web is constantly evolving: so should your site, and so should your implementation.

The problem, of course, is finding a balance between “We don’t need any newfangled reports” and “Let’s track everything in case someday we might need it”.  The best way around this? Any time you are changing your implementation, don’t think of what data you want to track now (again: never think “I want to track”). Think of which reports you’ve actively used in the last 3 months. Think of what reports you will use in the next 3 months. If current reports don’t fall in that list, scrap them or at least hide them from your menu so they don’t distract you.

See what analytics trends are on the rise. Check out some of the top blogs in the industry- particularly your vendor’s blog, like Omniture’s Industry Insights– to see what’s up-and-coming in analytics. If you’re hiring an implementation consultant- either from a vendor or an agency- don’t just tell them what reports you’d like. Ask them which cool reports others having been using. Use their experience. They may be content to take orders or fulfill the requirements of the contract (which are usually made by salespeople, not analysts), but it’s very likely that they have great ideas if you’ll give them a couple hours to incorporate them.

As an implementation consultant, this is something I’ve always struggled with. I get excited by the challenge of finding unique ways to bring in new data. When a client says “I want to track order confirmation numbers broken down by coupon codes with campaign click-throughs as a metric”, my natural inclination is to say “sure, we can do that!” then whip out my javascript editor. And if that report NEVER gets used, I’ll never know unless the client calls me back in.

I write this post as a step in the direction of repentance. I have been a Data Squirrel enabler, and I know it. Learn from my past and don’t allow these mistakes to diminish the value of your implementation.

Challenge: 30 Minutes of Action

Because right now my role is mostly in implementation, it sometimes feels like the world of analytics is nothing but figuring out how to report- either how to get the data in there or how to present it. I want to see some action!

I challenge you all to spend a mere 30 minutes in your tool of choice to find one- just ONE- actionable piece of data, and *here’s the kicker*: actually take steps towards that action (even if it’s just making a plan). I don’t expect the action to only take 30 minutes, but you should definitely be able to find your piece of data and start a plan in that time.

Here are a few ideas that range from simple to ambitious to help you get started:

  • Look at your Error Pages report and fix the most-clicked broken link. If needed, use a pathing report to find the page sending people to your error page. This is practically a freebie.
  • Look at your top internal search keyword. Figure out a way to make content on that topic more easily findable from your homepage. Ask yourself: would this make a good internal promotion?
  • Look at your top 5 highest-traffic landing pages, then see which is converting the least. Make a hypothesis about what could improve (compare to highest-converting page if you need ideas), then make a game plan to A/B test it.
  • See which paid search keyword has the highest bounce rate. Hypothesize on how to make your landing page appeal to users clicking on that keyword more, or reword your keyword so it brings in more qualified traffic. Make an A/B test out of it.
  • Think of the one thing on your site you wish users were doing more of. Now put it in a fall-out report. Find the point of highest abandonment. Hypothesize about why users are falling out. Test it.
  • Find a low-performing call-to-action. Figure out a different way to present it: perhaps a different graphic or reworded text. Test it. (Are you noticing a “test it” theme, here?)test
  • Take your highest-performing campaign. Play with segments until you see who the campaign appeals to the most. Earmark that segment for more marketing efforts. Find a video with a high conversion rate. Feature it in an area with higher visibility.
  • Look at your top Organic Search Terms. Do you see a lot of your brand name in there? Find a high-converting product page and focus some SEO efforts there so you can reach users looking for your products, not your brand.

If you reach the end of your 30 minutes with no action plan, don’t give up. Spend some time finding a recent success or failure. What’s trending up? What’s trending down? Try segmenting the data different ways. Make some theories, then plan some tests. Not to sound like a broken record here, but you can’t go awry with a well-executed test.

Ready, Set, Go!

clock

I’ll happily take suggestions for more ideas, too. I’d love to make one huge long list of your ideas for actionable data.

Ready? Go team!

When you’re done, please come back here and tell me how it went.

PS: If you can’t find one piece of actionable data to move with, then either you need to revamp your implementation, or congratulate yourself on a perfect website and implementation. In which case, you have free time on your hands to volunteer at the Analysis Exchange!