How to create a webtrends custom report

webtrends menu
the webtrends menu in my old company login, new one isn’t as nice.

First off I am going toclarify that I am working in WebTrends self hosted software version 8 and that I have administrator access to do this. Without those factors you won’t neccissarily have the same experience setting up custom reports.

I have already had the developers add tags to the site to track custom events (SDC tags, DCS MultiTrack events) and I don’t work with the actuall tagging on the website so I will not address those concerns here.

I’m starting from the point where the tags are live on the site and I need to set up this custom report so we can start gathering data. If you need retroactive data because the tags were placed on the site months ago and you didn’t create the report, it is only possible if you create the report and re-process all the data from scratch for that time period and this is usually done in a new profile. This takes a long time (sometimes weeks if you have years of data) and the people maintaining the servers and data for your webtrends setup usually hate you if you ask. So, let’s stay out of that area and go build a report that will collect data in a table from this point forward.

One of the most frustrating thing about WebTrends is that nothing is connected and it is a bunch of database tables. Nobody in Marketing thinks that way or interacts with a system like this so it is completley foregin. (‘m getting the feeling that the developers feel the same way too) You have to go into several tables in the database behind the webtrends software and create the structure for the report. It sucks but that is the way it works.

For every custom report you need at least three things. A Measure, A Dimension and A Report. (you may also want a filter but that isn’t required).

1. Testing – Some people (webtrends actually advises this) like to create a copy of the profile to test the reports (or any changes to the profile) before they add or change anything for real. This reduces the probablility that you will have some change corrupt or crash your data. If you depend on this data and have a lot of custom reports and data already set in the profile you’re working with, start with a copy profile first and don’t endanger the main data set. Work in that sandbox until you find the right combination of settings, write them down and then go back to the main profile when you’re sure the report exports the right data in the right format. There is a one day waiting period to collect data to test in most cases unless you’re running on-demand (of which most of my advice may not apply to) or you have updates and processing every few hours. Also only select the most recent month’s data in the copy profile, you don’t need all the data to run a test.

2. A Dimension is what most people would call a metric name. It is the data being collected from those custom tags. Just because the tags are in the WebTrends format and live on the site doesn’t mean that the WebTrends software knows that they are there or what to do with that data. You need to create a dimension to name this data and create a table to collect it in. Go to Dimensions in the admin menu. (some systems show custom report menus under reports & profiles others in Admin, check both) Select new on the upper right corner. Name it. Give it a column name for reports. Use the navigation to go through the process steps to finish creating this measure. You will need the actual tag names for this. In my case it is Not terribly descriptive but hopefully functional. Do not select to activate across all profiles because you also need to go into the profiles (the main one and the copy) you are working with and enable this dimension there so it knows which profile it is working with. You don’t want to affect everyone elses profiles with this extra data, just yours. They need a drop down within the dimension setup to allow you to select this all at once but of course that doesn’t exist to confuse us more.

3. A Measure is another function you need to set up manually (in a database table?) and tell WebTrends that along with the Dimension you just created a table for, should you count or sum this data? In a system like DART, this is so simple it is a drop down choice on an export menu, here you have to configure the backend system of the millennium falcon to get it set. Silly but true. Go to Measures on the custom reports menu. Select new on the upper right corner. Name it. Give it a column name for reports. Use the navigation to go through the process steps to finish creating this measure. You will need the actual tag names for this. You must also go into the profiles (the main one and the copy) you are working with and enable this measure there so it is activated. Ditto about not selecting all profiles on the measure setup so that this works. For my purposes I had the WT.ti (page title tag) and 4 parameters that were possible when that tag was used. (ShareThis: Facebook, Twitter, Email & MySpace) and they all got created as additional measures also. This will give us more granularity in the data showing us not only which pages had share this activity but which site it was shared to.

4. The Custom Report finally! – After about 4 hours of setup for the setup you can finally go do what you originally set out to do. Go to the reports link under custom reports.

The menu for setting up the report asks for the info about the dimensions and measures you just set up as well as how you want the data compiled. I always allow for sorting ability but you don’t have to if you want to save space and if you just export it all anyway it may not matter.

The setup for the report makes more sense having seen everything we’ve already talked through. It is also worth noting that if your template for webtrends software doesn’t have a left nav bar location for the custom reports you have to find the template being used now for your back-end-interface then you have to go into that version of template and add custom reports to that list. I found mine under report configuration, report designer in the report and profiles section of the menu. See screen shot at left. Click the top line of the left nav in the template to deselect the chapters already there and add a new report and list it in a new chapter for custom reports.

You also need to enable the report in the profile itself. Each version of WebTrends has a slightly different menu, but in the edit-report menu of the profile there is a reports tab that lists all the custom reports set up in the system and you select the ones you want to enable.

After this your data in the test copy profile should work, then the trick is repeating it all over again in the main profile and remembering all the steps.

WebTrends won’t export data, what’s wrong?

I thought I would share another WebTrends Analytics morsel of information since it is so hard to find information about this analytics software that is quick and concise.

Today I was working from home and our self hosted software would not let me download the WebTrends reports that a client requested. It would abort the process and return to the login landing page each time.

We’re using WebTrends self hosted 8.1 software (not the most recent, yet still solid) and we found that WebTrends 8.1 does not play well with Internet Exporer 8.

We found that if I ran my browser in standard IE8 settings it would not export data and the popup of data processing screen was supressed and the system couldn’t complete the request.

There is a setting in the IE8 browser though that is called “compatibility view” that fixes this problem. Just set it to run in compatibility mode for the data site or all sites if you want and your analytics data is available for download again.

On another note, our recently update McAfee security and virus protection software is also blocking the javascript prompted popup window where the data is usually exported and I can’t download any data at work anymore either. We haven’t found a solution to this problem because the security settings get reset to the standard again when the system finds that the administrator has changed them, usually within a 1/2 hour. No word on if the powers that be would make an exception for my computer somehow or if our maintenance systems even allow that.  

It’s going to be a fun month.

WebTrends Email Stats Reports How To Setup

I love that WebTrends is a good solid web analytics reporting solution, but I really find the setup process for just about anything with this system to be very confusing and lengthy. I’m sure there is a reason for this (could be data integrity processes or cost savings) but I really just need a step by step list when I need to get something done quickly and someone to tell me where these oddball parts of the process exist. Therefore I’m writing a list to explain this process so I have it written down and other people can find this info too.

(technical note I’m using webtrends software 8.1, not the webtrends hosted solution)

Today the task at hand is setting up automated reports of webtrends data to be sent monthly by email. The duration of the data collected and the frequency of the reporting schedule are both flexible, it can be daily, weekly, monthly, quarterly or yearly.

The first step is to go to the administration menu from your login. There go to Report Designer and Templates. You can select one of their templates, I needed to create a new one.

Then name your template. Go to next, select the content by adding (and naming) a new chapter, then adding content to that chapter from the add report link on the menu above. Select the “built in report” list from the drop down to get the standard metrics available in webtrends. Check the boxes of the metrics you want included, I would say 4-8 per report is enough before you have too much data for someone to really use. You can make changes to the layout, although I was not looking for that level of detail now.

Click next at the bottom of the page. Then you have some configuration settings, like for wrapping text lines on long urls (ok) and how many rows of data in the reports (20-50 max for readability, top 5 is good).

Click next again and give profiles access to this report. I noticed mine are already given universal access and grey-ed out so nothing much to do with this screen. Then click save.

Next you then go back to the profiles list (admin menu and web analysis and reports & profiles) and edit the profile you want to get this report to add the report to the profile. This is one of those steps I think is redundant and should be automated or brought into the setup process before this because its confusing. You wave over the profile in the list (don’t click it) and get a menu with “actions” and edit is one of them.

From there go to reports in the top menu and on the drop down go to report templates. Click the box by your report to select it, ignore the second checkbox that is labeled default because it will change the default reporting style in the profile to this new report, and that isn’t the intention here.

Then go back to the admin menu a third time and to the scheduler menu (bottom) and then schedule jobs and click the button for a new job. This is the email setup part. Under job type select scheduled report and follow through the pieces of menu from left to right as you fill out each section. First select your profile you want reported on, next give the report a name and assign it to a user (yourself). (note this is also how to disable the emails with the check box below, no idea why this is hidden here). Report type: general report. Output type can me a database, pdf, excel/csv or pdf. I chose pdf because it looks professional and we don’t have to install Microsoft office/word on the server in order to export it. Its the only option that does not require that except the database. Number of data rows to report is up to you, I usually do top 20.

Next add the report destinations, this is where you need the email info. Add your email as the from, add theirs as the to address. Also, cc yourself on these reports so you get them too. Add the SMTP server address (if you don’t have the SMTP address it will hold up all of your other scheduled jobs, so don’t set this up without it.) So, the software knows where to connect to send it from. (contact IT about this if you don’t have it) You can also FTP it if you like your data that way, or save to a folder on the server. (not as exec friendly though) 

Under templates, complete view is ok. Under reports, here you select the reports you want to include. These are a duplicate of the ones you selected above, maybe redundant but this is literally the process we took on the phone with the WebTrends helpdesk people. Report type: standard again, date range: its up to you. Scheduling is next on the menu, you can’t run it on the 1st of the month because data may not have compiled yet in all time zones so the 2nd of the month is the first you can run a monthly report with the most recent previous month’s data. Ditto lag time for dailies, weeklies etc. Run once or run weekly/monthly/daily, as you choose.

The host binding section he literally told me to ignore. So I have no idea what that means. Then you get a summary page at the end and click save.

You just wait now and see if everything gets delivered correctly. It is good that the report is only generated once per month on the date you specify as a job that processes, so it can run data in the past (vs only from the point you created the report, forward like custom reports do because they create their own database table) and it won’t clog up your processing queue with a lot of memory/processing because it’s just once.

I wish there were short concise directions for setting up webtrends email reports like this on the web already but I realize that nothing is easy or self explanatory with database systems or webtrends. It’s just part of the territory until next generation tools come around, and no I don’t mean Google Analytics (which is almost as confusing now to beginners). Someday this has to get simpler in process so more people can use it.

The Negatives of Social Networking Media

All the world is a Buzz about Facebook & Twitter these days. It’s almost like MySpace circa 2007, Google circa 2003 or Microsoft circa 1998. I don’t doubt the success, innovation or long-term viability of these social networking sites but I have seen that there are flaws in the system that mean that things won’t be perfect with the business along the way and we’re in for a bumpy road. Basically my point is that for all these sites give us in entertainment, social connections and opportunity they also have some negatives that are almost the equal and opposite pendulum action.

1. Time Suck – all social networking sites are using your time that you used to devote to other things. Maybe in some cases this is actually a better use of your time (instead of TV) but in most cases its time spent that you used to use for researching new information for work projects,  time actually spent talking with people in person (family/friends) or time spent doing things that really need to be done at work or home. Once the brain gets trained that you can go socialize instead of work at those times of day it’s a habit extremely hard to break. For all of us procrastinators looking for instant gratification its a real problem keeping up with work and affects the overall productivity of companies and the country as a whole. Internet access is much more prevalent and has far more users during the business day than it does at night, so there’s the proof. Unless your job is trolling these sites for sales prospects by “connecting” and making “relationships” with your customers, its a waste of time to spend more than 15 min a day.

2. Privacy – Of all the details analyzed about consumer privacy online (on Facebook) in the last few weeks the most suprising thing I’ve seen is that people really don’t care about their information online. Sure, nobody is going to post a ss number or cc number on their profile (duh) but they don’t really seem to realize the power of logging all their social interactions in one database and selling access to retailers and cpg companies who have even larger databases of information to analyze and strategize with. Is it really as fun when most of your friends are companies selling you things all the time? Twitter already has morphed into the largest opt in direct marketing platform I’ve ever seen. If people keep using it at this rate it will surpass email. The other obvious issues come with the work life balance thing and when people friend work makes and think nobody will see them rant about work or post drunk pictures on a sick day, but then again I’ve heard that its just people naturally selecting themselves out of the working pool.

3. Logic – the other issues I’ve seen coming for a while have to do with how everything that is built from large databases online with lots of consumer data seems to not work properly. There is always some algorithm developed by a science tech guy based on some theoretical calculus and it doesn’t provide relevant results. Which brings me to a repeating theme of data right now: we don’t really know what to do with it yet. Nobody knows enough real info about their customers to target them. (who has a budget for that?) And the database people just like to say they improved things a statistically insignificant amount with an algorithm tweak. The marketing strategy/process should always start with offline real life information about people and products and then develop an algorithm to show you information in that way. I don’t know why it’s always done backwards but it will keep our results irrelevant and marketing dollars wasted for a long time to come.

New Media and New Information Paradigms

I have been hearing about the demise of the newspapers, the rise of search/social networking/new media and the internet fragmentation concept for years now. (almost a decade?) And I just read about it again today with the newspapers secretly meeting to try and sort out monetization methods to save their business. At the same time I am a Guinea pig living through this time of change/shift in how people find information, use information and consume things. Here are some of my observations although not in a concise dissertation format yet. 

  • We are at an odd time in internet evolution, on pause between big developments. We got email, IM, web sites, RSS feeds, Blogs, social networking and now Twitter. We don’t need more services or ways to interact on the web. We need better all inclusive ways to connect and consume all in one. Ways to make the experience more relevant and more inclusive of many kinds of content at the same time. Not wasting our time.
  • I can’t help but notice that at 33 I have never really “read” a newspaper. This indicates to me that newspapers were not that important back in the 1980’s to my generation when their profits were healthy and the internet was but a dream for most of us. (Except being something to line litter boxes and bird cages with.) I hate the size format, I hate the ink and I always have. I actually like the ads though, especially the Sunday fliers. 
  • Weeks go by without my watching any TV. This started about 3 years ago when I got high speed internet. It’s not that I don’t like TV, I just don’t have time to sit for 2 hours plus and I know if i sit down I won’t get up and get anything accomplished in the evening/weekend. And I don’t like overly repetitive things. I was watching the sell that house shows on HGTV to get ideas about how to sell mine and after about 3 I got it and didn’t need to watch any more. Reruns aren’t nostalgic to me really, more just boring. And reruns is all Cable TV is about.
  • The only TV I will drop everything for is Top Gear UK. When it is in Season we trek over to my parent’s house and watch wwith extended family weekly. Everybody drops everything to watch that show. It makes you laugh, it makes you dream of fancy cars and it inspires you to take grand adventures regardless of what the outcome is.
  • This leads me to a general cluelessness about a lot of local and newsworthy (?) events. Things like buses that are Hijacked and what the weather will be tomorrow. I also find that these things weren’t essential to me in the first place. I carry an umbrella, what’s the big deal?
  • I find myself focusing on things I’m interested in. Maybe this is the political polarization people speak of? I read my marketing emails/newsletters/blogs as well as home design blogs and write my own blog as well. I check status on Facebook/Twitter/Flickr and maybe update if I have something interesting to say. And I work a lot. I also am always investigating 2-3 new directions for my work/career. Not all of them pan out, but they help me figure out what is evolving that I need to know about.
  • I do still use the phone (yes the land line). It is the best way to reach my parents and Steve’s parents. Steve’s parents email but mine are not really into it. And we try and go visit once a week in person. In person time still matters.
  • I am a book reader because I am a train commuter. I have been for years now and it has created a small library of business/marketing/analysis books. I order from amazon when I see something I like and then go consult the pile of books for something new.
  • And that is all I have time for. Now with a husband (fiancee really for one more month), 3 cats, 4 litter boxes, a yard, wedding planning, condo selling, house hunting, family organizing, laundry, food shopping & cooking I am overbooked. I don’t even get to skype/call my friends very often. A party invite seems really daunting these days with the schedule we keep.
  • I wonder about new media uses and if we will really care about anything not personally relevant to us in the future? Will a police chase matter to everyone in Chicago or just the people who live by the highway where it happens? Will we be less distract-able by sensational news and distracting entertainment? Will we be able to channel the news, information and analysis we really need into our lives and ignore the products/content we really don’t care about?
  • On the other side of the coin, how will we ever discover new things? I find myself looking to find out what is happening on the internet a few times a week and look to Google News and the Yahoo home page. Not the Trib. Yet somehow the list at these sites is always limited and not really anything relevant either.
  • There has to be something in-between a completely open fire hose of information and one select rss feed with just content from one niche area. There has to be some middle ground between being hijacked by ads for 20 minutes of a 60 minute program on TV and not knowing at all where to find a dress for my rehearsal dinner when my usual 5 clothing websites didn’t pan out. (who has time to go to a mall?) ((and why does Google shopping suck when the main search is generally good??))
  • People won’t pay for news. Period. They will pay for some kind of extra relevant cool service though. They will pay for innovation, new products that are noticeably better for some reason. Things that simplify your life.
  • Ads should not be integrated more with content as if they were the content. It blurs the line in what is really true and what is marketing speak. And although they may pay the bills for a while, people will eventually figure it out and abandon that medium that does this.
  • We need another search player. Google is not enough and although they do some things well, I am not a fan of everything they create. I would like more companies to work on real time indexing of information as well as historical archiving to keep information accessible if anything happens to Google’s accessibility. At some point people will be so hooked they will be able to charge for a (low cost) subscription to the search engine itself. 
  • More people need web enabled phones with internet use active. I just read yesterday that out of 57 million people in the US with internet capable mobile phones only 18 million have internet enabled! (netpop stat comparing us to China) 31.5% of the people with internet use phones don’t even pay for internet access? (only 13% of all the cell phones total) This is a huge hurdle to making info more relevant and accessible because people carry their phones everywhere. Things like bigger screens, flatter profiles and easier software app use on these phones will help the adoption rates improve. 
  • Identity management and security is also a problem. We might like something like OpenID but only if sites still allow anonymous comments too. Privacy and being able to say something important without being hunted down in person for your opinion necessary for getting people to adopt this identity management software and make our lives easier between all the hundreds of web sites and e-commerce activities we do in a day and consolidating that information for our own personal use.   
  • Data mining is going to have to improve. If statistics are wrong 25% of the time like stated in the Numerati book, we really need to combine automated data crunching with human decisions about data more often. Numbers are meaningless without someones explanation. This completely changes what and how data is configured, crunched and reported and can determine/undermine your results even if you manage to collect it perfectly.
  • All this plus the only way out of a recession is through innovation. We’re waiting.

How to keep learning new skills as we get older

I used to think that it was weird to not be learning all the time. You spend 9 years in grammar school, 4 years in high school and if you are lucky, 4 years in college. All total,  that is 17 of your first 21 years in school with daily lessons, lectures, homework, required reading, tests, quizzes, projects, essays and exams. Then you have to make the jump to the working world whether it is in business or other areas and you still have to learn, but it is everything not included in your schooling. How your company works, how people work, what is required there and all their multitude of processes and products.

At that point you usually meet someone much older than yourself that has no idea what is happening in technology. And not just high technology, they don’t get basics that most people use just to function daily like email, pivot tables or search engine optimization. What you don’t know is that they are the future you. They don’t want to change because they say they have just done things this way all these years and it has always worked with paper files, binders, phone messages and post it noes. You look at them and their outdated clothes and rows of beanie babies around their cubicle like they have 3 heads. How can they work this way? How do they get anything done? How can a company value someone antiquated like this over me who has all this knowledge and ability (yet with 0 experience). 

Then time marches on. You become acclimated with the business environment and get promoted or jump to a better job a few times. You balance social, personal and work life stuff and think wow it’s a lot to manage and are always tired. Then some of you have kids and are even more tired. Then you wake up one day and realize that you have become that antiquated person you ran into years ago because they hired some younger workers that are all gung ho about getting ahead and talk about things you don’t understand. Now all college grads come to work knowing how to build databases and web sites even if they got a degree in English? How can they know so much so fast? 

You wonder how 10 years flew by and you haven’t really added anything new to your skill set because you work 50 hours a week, have a relationship on weekends and laundry/dishes/cat/cleaning/reading/few social things weeknights. (you don’t even watch TV for god’s sake) How can you go to school at the same time? If you have kids, how can you exist on less than the 4 hours of sleep you get now just so you can spend time learning? And when will you ever get around to painting the garage? How is this possible when some mornings you come to work 1/2 asleep with 2 different shoes on?

Are companies going to only hire new youngsters for all the positions because they make less money and have more tech skills? How much does experience matter?  Why did it piss me off for weeks when the new backend system was launched, and nothing made any sense anymore? I didn’t have time to spend looking for hours for where everything had been moved to and was just mad that it wasn’t where it was before and it took forever to post. And there was no communication from those adsense loving wordpress people about where everything had been moved to. They thought this was self explanitory? (Not!)

I think I started to recognize some of these changes happening to me in the past few months. I never planned on stopping learning and the things I chose to learn about in my spare (and fleeting) time were never really panned out useful things. So, back to the drawing board. I feel like I need a lot more technical skill to remain ahead of the curve in my job and be able to keep finding great work over the decades to come. And I want to do that along with have a family and marriage and the whole kit and kaboodle. I don’t think this is a women’s issue anymore either really. Men face the same questions as they get off the fashion bus and start looking, sounding and working more and more like their fathers.

Another thing I realized the other day is that I may try and shop at more contemporary stores, but I basically dress exactly like my mom. And she is 67 and I am 32. I used to hate how my mom dressed, and now I am her?  Is this just the arrival of the long plateau of middle age? Are the middle ages of me going to be anticlimactic and uneventful? Or how can you bridge multiple generations, technologies and social groups all at the same time while still getting 8 hours of sleep at night? 

I don’t know how this is all going to work. I suppose many people don’t write about it on blogs, or maybe even recognize the change until they can’t find bleached jeans and high tops at Kohl’s anymore. But it bothers me because I don’t want to stop learning and get left behind. Especially when the economy keeps changing so much every year and the jobs go with it. How do you not get outsourced when literally everything can be outsourced today? How do you keep going to school when most universities require full time attendance of a degree program and not piece mail courses as you need them? How do you find time to do homework when you have bills to pay and garbage to take out and emails from your boss? Even reading was hard to get back into after years of not focusing like that for an extended period of time.

Here is what I have been doing about it and working on over the past years and what I would like to continue to work on:

1. About 2 years ago I started reading books again. I read TIME every week, but that is pretty short. I found it hard at first to just read for an hour at a time because I had gotten so multitask happy with the internet and channel flipping.

2. I also decided it was time to start pushing back sometimes at work and saying No. You literally can’t do that when you start out, and sooner or later you have to set limits and not do everything for everyone else when you have a limited time to do it. The whole idea of urgency and priority come into play and they shouldn’t be anyone else’s priority or urgency but yours. People will negotiate and try and get as much from you as possible but it’s not in your best interest if it’s not really in your job description.

3. The last year was one where I decided it was time to have a self hosted blog. Everyone and their sister had one but me, and it wasn’t supposed to be impossible or anything. So, I bought a url and went to town for about 2 days straight truing to figure out how this wordpress thing worked, researching themes, plugins and all the possibilities and building it. It was a great learning experience although it has nothing to do with my work.

4. This year I had a client that insisted on a different data process than what we usually provided and I had to learn Pivot tables. I still don’t know them to the extent I need to probably, but it helps immensley. I still have more of the high end Excel stuff to learn.

5. I also had to learn Access. I guess 2 days in a class can’t really teach you everything though so I should either retake the class or take another one because my skills there still don’t match what I need them to.

6. I also wonder about math and statistics. I should really go back, take the prerequisites and then statistics. This is what I get for not taking it initially because I didn’t want to work in business. Sheesh. This is by far one of the hardest things to do because, I am not fantastic at math, it has been about 14 years since I have taken a math class and it means driving back to the community college I attended a million years ago. It also entails weekly classes and weekly homework. This could be 10 hours a week or more. Where am I going to find 10 hours a week? Where do people who have kids find that time?  Is sleep allowed?

7. I also think it is time I got better at this friends/networking thing at work and outside it. I have never been that great at the social stuff, but I am meeting more and more people who weren’t necessarily either, but because there are some ground rules in business and no need to act like Jr High kids anymore, they are pretty good at it now. This helps get things done faster when they need to be, and it makes work and life generally more fun. It also helps not to work with assholes.

8. What I would also like to do more of is learn about web pages and building them, coding and sleuthing out issues with them. This comes up with work and would be an asset.

9. Long term I have to get into databases and SQL. I have no choice. It will mean more classes and more time than I know I can find and afford, and that isn’t even the expensive part like tuition.

But what is the alternative? To be outsourced in a few years? To be relegated back to the minimum wage jobs that we had back in college but would be even more difficult to get since they would rather hire energetic young people now, with better tech skills?  The way I see it, the only way to survive is to go onwards and upwards. I have to keep learning things whether I have time for it or not.