5 Key Considerations for Implementing Web Analytics with IBM WebSphere Portal

If you’re implementing a portal site, chances are that you are also implementing (or at least planning to implement) Web Analytics in order to understand user behavior, measure what works and what doesn’t work on your site, perform A/B testing, and improve your site’s performance/conversion rates.  The information gained by successfully implementing Web Analytics can be invaluable in improving your user experience, assisting with online marketing strategies, and maximizing sales generation leads, which are all key aspects of successfully running your online business.

IBM WebSphere Portal has a built-in framework that integrates with various market leaders in the Web Analytics space, such as IBM Digital Analytics (aka Coremetrics), Adobe Analytics (aka Omniture), and WebTrends.   Portal’s Active Site Analytics (ASA) framework generates page/application metadata which can then be aggregated and sent to your web analytics solution. You can read more about their framework and technical implementation information on IBM’s Knowledge Center.

Here are 5 considerations that can help you successfully implement Web Analytics on IBM’s WebSphere Portal.  The list itself is agnostic to the different Web Analytics solutions; however, the “Guidance” section on each step is based on an IBM Digital Analytics implementation.  Please beware that Web analytics solutions differ in functionality and may have a different method to achieve the same thing.  Additionally, IBM WebSphere Portal is known to have better integration with IBM Digital Analytics, an example being their Analytics Overlay Reports.

  1. Understand who your users really are
  2. Differentiate between a Portal page and a functional page
  3. Support business driven categorization of pages and applications
  4. Allow for applications integrated on the glass to report analytics data
  5. Hidden applications should not be reported unless used

Pre-requisites:

  • Your Web Analytics solutions has already been installed and configured
  • Basic knowledge of JavaScript and HTML
  • IBM WebSphere Portal version 8.0.0.1 or Later

1. Understand who your users really are

A key aspect of a Web Analytics solution is to be able to do customer segmentation / profiling.  Customer segmentation is the practice of grouping users based on similar characteristics (such as location, age, gender, etc).  By default IBM WebSphere Portal doesn’t share much user information when it sends registration/visitor data, so this is likely to get missed.  As part of your implementation process you should first understand how your Web Analytics solution leverages user data, then extend your Portal solution to populate the data needed to segment your customers.  If you’re using the content targeting feature and have already created segments in your Portal solution, ideally you should have the same data to recreate the same segments in your analytics solution.

Guidance for IBM Digital Analytics

IBM Digital Analytics tracks user information using “Registration Tags”.  The aggregator provided by IBM uses this type of tag and sends a visitor ID along with the userName.  My recommendation is to modify the aggregator and add your own user data there via Registration Attributes.  You can retrieve user data using EL beans for example, and then add pass it to the aggregator.

Each registration attribute specified in the aggregator will also need to be specified in IBM Digital Analytics admin section (in order for it to become available for segmentation / Explorer reporting).  Given that you can only use up to 50 registration attributes and the first 15 are special attributes for creating segments, you should pay close attention to which attributes should be in the first 15.  As an example, you wouldn’t create a segment based on first name or last name so they would be good candidates for assigning an attribute number between 16 and 50.

Bellow is a brief sample of how to update the CoremetricsAggregator.js to accomplish this.  This assumes that you have implemented a JavaScript function that retrieves user data.  Also note that I’ve changed what the cmReg cookie stores; instead of a simple “Y” value, I’ve specified the actual visitor ID, this really helps during testing as we’re often using the same browser but using different test accounts.

var processRegistrationTag = function(/*JSON*/ d) {
    var id = single(d["asa.visitor"]);
    if (id && cI("cmReg") != id) {
        var attr = [];
        attr[0] = getUserData().Gender;
        attr[1] = getUserData().PrefferredLanguage;
        attr[15] = getUserData().FirstName;
        attr[16] = getUserData().LastName;
        var regAttr = attr.join("-_-");
        cmCreateRegistrationTag(id, getUserData().Email, getUserData().City, getUserData().State, getUserData().PostalCode, getUserData().Country, regAttr);
        document.cookie = "cmReg=" + id + "; path=/";
    }
};

Within IBM Digital Analytics “Admin” UI, you would then need to set these registrations attributes in the Explorer Attributes section. Please note that Email/City/State/Postal Code/Country are built-in attributes and hence there’s really nothing to set in the Admin UI.

Figure 1: Setting Explorer registration attributes

Coremetrics Registration Attributes

Registration attributes 1-15 should also be specified in the “Extra Fields” section.  This would allow that data to be exported via Standard Data Export.  See IBM Digital Analytics Implementation Guide for more details.

2. Differentiate between a Portal page and a functional page

Depending on how your Portal site was designed, a portal page might not necessarily equate to a single functional page.  Say for example you have an “Product Detail” page, which takes an “Product ID” as a parameter and renders the product details and associated content based on the ID.  To IBM WebSphere Portal, this is a single portal page and that’s how it’ll likely be reported, however to your business users it could be interpreted as multiple functional pages.  From a web analytics perspective, you care more about the actual functional pages since that’s what the visitors see and interact with.

Guidance for IBM Digital Analytics

All portal page titles are reported to IBM Digital Analytics out of the box.  If those page titles matches all your functional pages, you’ll just need to ensure that the title you see on the browser is what’s being reported to IBM Digital Analytics.  There are other situations though, in which you should override the page title with a portlet-generated page title.  For these situations, I recommend to implement a page title override system.

In order to implement a page title override system, you have a couple of options:

  1. Use two-phase rendering on necessary Portlets to override the Page title.  This might be cleanest approach, but it’s a bit complex and offers little flexibility.  For more information please read: “Modifying the HTML head section of a JSR 286 portlet” in the IBM Knowledge Center
  2. Output override metadata from necessary Portlets.  This approach consists of updating the portlets to write metadata that overrides the page metadata and then processing the new tags in CoremetricsAggregator.js.  This approach is very robust as you can override any page metadata (not just title) and simple to implement.
    Sample override metadata:
  3. <div id="asa.page.override" style="display:none;">
        <span class="asa.page.title">Details for Scooter M3 Model</span>
        <span class="asa.page.breadcrumb">Products/Scooter/M3</span>
    </div>

    We would then update CoremetricsAggregator.js parsePage function to apply the override:

    var parsePage = function(/*DOMNode[]*/ ns, /*String*/pageID) {
        var pRoot = byId("asa.page");
        var d = {};
        if (pRoot) {
            parse(pRoot, d);
        } else if (console) {
            console.log("WARNING: Root element not found.");
        }
        // check for page override
        var pRootOverride = byId("asa.page.override");
        var dOverride = {};
        if (pRootOverride) {
            parse(pRootOverride, dOverride);
            if (dOverride['asa.page.title']) {
                d['asa.page.title'] = dOverride['asa.page.title'];
            }
            if (dOverride['asa.page.breadcrumb']) {
                d['asa.page.breadcrumb'] = dOverride['asa.page.breadcrumb'];
            }
            if (dOverride['asa.search.query']) {
                d['asa.search.query'] = dOverride['asa.search.query'];
            }
            if (dOverride['asa.search.results']) {
                d['asa.search.results'] = dOverride['asa.search.results'];
            }
        }
        // update our cache
        pTags = d;
        // communicate data to Coremetrics
        if (!isEmpty(d)) {
            processRegistrationTag(d);
            processPageTags(d);
        }
    };

    In the example above, we allowed four tags to be overridden by a Portlet: asa.page.title, asa.page.breadcrumb, asa.search.query, asa.search.results.  Also note that the individual page tag would’ve been used in the case that it was not overridden.

3. . Support business driven categorization of pages and applications

Categorizing pages (or creating Content Groups) is essential in understand user behavior on the sites.  This is specially true when your site has a large amount of pages that have similar functional areas.  The default categorization provided by IBM WebSphere Portal is a single generic value that doesn’t provide much value to understanding user behavior.  You should empower your business users or content authors to easily categorize pages and applications, thus improving your analysis of functional areas on your site, improving speed to market, and minimizing the impact on the I/T organization.

Guidance for IBM Digital Analytics

The only category reported within the CoremetricsAggregator is “asa.page”.  This category gets reported by every page.  Ideally content authors are able to change categories without I/T involvement, since categorization is driven by functionality or business use cases and can be volatile.  In order to do this you can leverage Portal’s Page Properties, which get syndicated as part of your WCM syndication (assumption that you’re leveraging Portal’s Managed Pages).  This information is specified in the IBM Digital Analytics Implementation Guide (page 106), however the aggregator sample provided does not implement this.

To edit/add a category, a content author would need to go into Edit Mode -> Page -> Details -> Page Properties -> Advanced Tab -> Enter/Update asa_js_PageCatID:
WpPageCat

Update CoremetricsAggregator.js in order to report the category (processPageTags function)

var processPageTags = function(/*JSON*/ d) {
    var pgTitle = single(d["asa.page.title"]) || single(d["asa.page.id"]);
    var query = single(d["asa.search.query"]);
    var res = single(d["asa.search.results"]);
    var cat = "asa.page";
    if (typeof ibm_page_metadata != 'undefined') {
        var catPage = single(ibm_page_metadata["PageCatID"]);
        if (catPage && catPage.length > 0) {
            cat = catPage;
        }
    }
 
    if (res) res += "";
    var attr = [];
    setPageAttributes(attr, d);
    var pgAttr = attr.join("-_-");
    // create pageview tag
    cmCreatePageviewTag(pgTitle, cat, query, res, pgAttr);
    // process analytics tags
    processAnalyticsTags(d, pgTitle, pgAttr);
 };

Note that the cmCreatePageviewTag has been updated to send the new category variable.

4. Allow for applications integrated on the glass to report analytics data.

Arguably some of the most powerful features of IBM WebSphere Portal are the integration capabilities. One of my favorite is Web Application Bridge (WAB), which allows you to surface existing web applications into your site rapidly and seamlessly.  The ASA framework however does not offer APIs or built-in mechanisms for these existing applications to take part of your analytics reporting.  If you have analytics requirements involving these types of applications that integrate on the glass, you should plan on creating some customization to enable reporting of analytics data.  WAB leverages iFrame technologies and each web analytics solution vendor might provide a different way to handle these types of scenario.

Guidance for IBM Digital Analytics

The steps needed to implement this are well documented as part of the IBM Digital Analytics Implementation Guide.  Please refer to section 2.8 “Tagging Frames” and follow the documented instructions.  The key aspect of the solution is that each application being rendered within an iFrame must include the eluminate.js library and cmSetClientID script blocks.  Do keep in mind that you might have multiple environments with corresponding domains/client IDs, therefore the embedded applications would need keep those in-sync with your Portal site.

5. Hidden applications should not be reported unless used

IBM WebSphere Portal provides a hidden portlet layout container that can host portlets that are meant to be hidden until invoke explicitly by an user for example.  Your developers might have also developed portlets that are essentially containers for modal dialogs.  These set of hidden portlets automatically populate the analytics metadata and are reported.  In my opinion, you should not report these portlets until the users starts interacting with them.  From a web analytics standpoint there’s very little value in knowing that the application was loaded on the page; however, there’s significant value in understanding how your users interacted with it.

Guidance for IBM Digital Analytics

This is actually a bit of a challenge, mostly due to how the microformats work today (i.e. they’re specified in the html DOM and parsed/processed at page load time).   There are also a few ways to solve this problem, the one describe below is the simplest but does require tinkering with your applications a bit.

  1. Generate an “asa.portlet.hidden” analytics tag from the hidden portlet
    Sample tag:
<span class="asa.portlet.hidden">true</span>

 

  • Update CoremetricsAggregator.js to ignore portlets that have the asa.portlet.hidden tag
    Updated parsePortlet JS function:

 

var parsePortlet = function(/*DOMNode[]*/ ns, /*String*/ portletID){
    if (!ns) {
    if (console) console.log("WARNING: DOM root node for portlet " + portletID + " not found.");
        return;
    }
    if (ns && ns.length > 0) {
        var d = {};
        copy(pTags, d);
        for (var i = 0, l = ns.length; i < l; ++i) {
            parse(ns[i], d);
        }
        if (!isEmpty(d)) {
            var ptHidden = single(d["asa.portlet.hidden"]);
            if (!ptHidden || ptHidden.length <= 0) {
                processPortletTags(d);
            }
        }
    }
};

 

  • Manually report the portlet element tag directly using the Coremetrics JavaScript API
    From your portlet you would call the following Javascript function when a user opens your hidden portlet or interacts it with it.

 

cmCreateElementTag("Portlet Title", "Portlet Category", attributes);

Summary

I have just provided 5 unique implementation considerations when integrating IBM WebSphere Portal with your analytics solution.  In a feature post, I will discuss additional considerations when specifically using IBM Digital Analytic as your solution, along with some more code samples.  The topics will include: Configuration management, Virtual Portals and IBM Digital Analytics Multisite addition, Testing tools, Video Tracking, and Search.

Up and Running with IBM Script Portlet

In this article, I’ll describe how to install the IBM Script Portlet on IBM WebSphere Portal 8.5 and create scripts that leverage some of the key features of the Script Portlet.  The script portlet allows developers with primarily client-side development skills (HTML/CSS/JavaScript) to develop portlets rapidly, both offline on their favorite editors/IDEs and within WebSphere Portal itself with a jsfiddle-like experience.  To find out more information about it, visit the IBM Greenhouse Solution Catalog and the IBM Knowledge Center.

Pre-requisites:

Install IBM Script Portlet

The instructions on how to install are well documented by IBM in the Knowledge Center.  Below are my notes on how to install.

  1. Start WebSphere_Portal server
  2. Unzip “IBM Script Portlet for WebSphere Portal V1.1.zip” to a temp location
  3. Install PAA: From your shell, navigate to <wp_profile>/ConfigEngine and run the following command
    C:\IBM\WebSphere\wp_profile\ConfigEngine> ConfigEngine.bat install-paa -DPAALocation=C:\temp\ibm\scriptportlet-app-1.0-SNAPSHOT.paa -DWasPassword=<password> -DPortalAdminPwd=<password>

    Depending on how WebSphere Portal was installed you may or may not have different accounts for WAS administration and Portal administration.  Make sure to have the correct passwords.

  4. Deploy PAA: From the same location, run the deploy-paa task
    C:\IBM\WebSphere\wp_profile\ConfigEngine> ConfigEngine.bat deploy-paa -DappName=scriptportlet-app -DWasPassword=<password> -DPortalAdminPwd=<password>
    

    Note: On IBM WebSphere Portal 8.5 there’s no need to modify the theme per the readme.  That step only applies to IBM WebSphere Portal 8.0.0.1 with CF11.

  5. Verify installation and setup a test portlet:
    1. Open up a browser and navigate to your portal server.  In my case it’s http://localhost:10039/wps/portal
    2. Login with as the Portal administrator
    3. Enable Edit Mode
    4. Click Create -> Page -> Choose a template -> Enter page details -> Click Create Page
    5. Click Create -> Applications -> ‘Web Content’ tab -> Select Script Portlet -> Click Add to Page or Drag it to the desired location ScriptPortlet
    6. Click on the Edit link inside the Script Portlet and Verify it comes up

Notable WCM Tags in Script Portlet Development

The IBM Script Portlet components (HTML/JavaScript/CSS) are all stored within WCM, which means script portlets can be included as part of projects, deployed to other environments as part of syndication, and developers can leverage many of the WCM tags available.  I used a few tags in combination with the AngularJS TodoMVC sample application  to showcase how it can be used on common portlet development tasks, as well as other interesting use cases.  To see a list of all the tags available, please visit the IBM WebSphere Portal Product Documentation.

Data Management

Private/Public Render Parameters

Plugin to retrieve portlet render parameters. In the example below, the private portlet render parameter allTodos is retrieved.

[Plugin:RenderParam key="allTodos"]

Request Attributes

Retrieves/sets request attributes. Useful for temporary variables.

[Plugin:RequestAttribute key="key1" defaultValue="value1"]

Session Attributes

Plugin useful for managing data in the portlet session. The example below gets a render parameter and sets it as a value for the session attribute “todosInSession

[Plugin:SessionAttribute scope="servlet" key="todosInSession" defaultValue="" mode="set" value="[Plugin:RenderParam key="allTodos"]"]

Portlet

Retrieves portlet information including portlet preferences.

[Plugin:Portlet key="preferences" preference="favoriteColors" separator=";"]

URL Generation

Render URLs

Plugin to generate portlet render URLs. See documentation on how you can set parameters within the plugin itself. In the example below I use a FORM GET to add the parameter as query strings in the URL which sets it as a private render parameter by default.  In general the approach below works well for parameters with small values, since there’s a limitation of how large URLs can get.  The example below is for illustration purposes only, you may not want a large amount of Todos as part of the URL or to store Todos in Session 😉

<form id="form1" action="[Plugin:RenderUrl copyCurrentParams="true"]" method="get">
<input name="allTodos" type="hidden" value="{{todos}}" /> 
<button id="persist-todos" type="submit">Save</button>
</form>

Please note that the ActionURL rendering plugin is not available within the Script Portlet.

Resource URLs

This plugin allows you to construct URLs with query parameters as well as proxy the resources through WebSphere Portal’s AJAX proxy. This is very useful when consuming external REST services while complying with the browser’s same origin policy.

[Plugin:ResourceURL url="http://todomvc.com/architecture-examples/angularjs/bower_components/todomvc-common/base.js" proxy="true"]

Others

There are many other tags available, so I highly recommend reading the IBM documentation to find out all you can do.  It’s also possible to create your own custom rendering plugin, which can then be used by the script portlet.   Other personal favorites are the conditional plugins:  EqualsNot EqualsMatchesOtherwise

TodoMVC Code in JSFIDDLE

Click here to see the modifications made to the TodoMVC app.  You should be able to copy/paste into your own Script Portlet and test out some of the features.