Is More Data Always Better?

google think magazine data overload obesity information ideas processing analysisThere has been a discovery in the online marketing and data/statistics world in the last few years. We have had more websites, products and tools created online than we can possibly keep track of. The terms to describe this deluge of activity we have been hearing the most are “data overload” and “information overload” from both companies and consumers. This Google Magazine uses the term Data Obesity to describe this phenomenon.

They ask the question, why is more data always better?

I think the idea of “more data us better” is common from people who lived before the Internet was prevalent. We had to work hard to find data. Researching something meant going to a library and looking in a card catalog (or maybe something called Gopher) and then finding your way around the Dewey decimal system to find that book. And then sometimes they didn’t even have the book because it was checked out or possibly it was just filed wrong because nobody understood the Dewey decimal system.

On a related note recently we got invited to my cousin’s wedding in Santa Fe New Mexico. My dad promptly went to the library and checked out 3 books on Santa Fe and New Mexico. I cringed. He asked how to find out the flights to book something without a travel agent. I realized I have been traveling since 2000 this way and he stopped traveling about that time so he never has. I introduced him to Travelocity, it was mind blowing and a bit of data overload compared with the OAG book he used to use in the 80’s.

The point here is that finding data was really difficult. People had control over its distribution because it was in print. When it became more freely accessible due to Google and other companies efforts we assumed this would be good, because people could remember where to find it and use it whenever we wanted. We never thought it would get this big so fast. Now travel sites are overwhelming, they have too many choices and there are too many of them trying to get you to opt into something you don’t want while being over charged for bringing a suitcase on a flight. This is just one example of how data has gone exponential so quickly.

Others of us have come to a data overload conclusion when they have 200 emails in several in-boxes, 1000+ rss reader posts from feeds waiting, several work projects, 500+ Facebook wall posts in their feed and hundreds of tweets that have gone un-read. This is among a climate where you have to follow-up with projects 5-10 times to get things done, post blogs/tweets/FB status updates daily to keep on people’s radar, empty the DVR so it doesn’t get overloaded and auto delete something you really wanted, listen to the radio on the way to work just in case something big happens and still find time to scoop the litter box before it gets full and the cats poop on the floor.

And the real purpose in all those tweets/FB posts and feeds is that you business changes yearly and if you don’t know about the latest trend and some real insights about it before your boss asks about it, you won’t have a job for all that long. (in digital marketing)

Having data overload be a “good” problem to have from some people’s perspective (as in that it is growth oriented). The democratization of publishing combined with tracking methodology and databases have all contributed to this problem, giving everyone a voice, a potential following of readers, a data trail to analyze and method to say something important online 24/7/365.  And then we have an even bigger problem of processing what is being said, figuring out if it is important or not and sharing/processing/saving it in some way if it is. Acting on that data is way down the line and many of us don’t even get there.

And this isn’t even the big problem with data overload. Where will we store it all? Why do tweets disappear from search so quickly? Because there are millions of them and the failwhale is full. According to the ThinkQuarterly UK, there are 800 Exabytes of data/information created every two days. It took humans from the beginning of civilization until 2003 to create the first 800 Exabytes, and we’re on a roll now.

Where does all this seemingly random data go? How will we know what it says without having to go into a database table and read specific field information? Where are the software tools to manage all this and still give humans the ability to customize the out put in ways that match the behavior or business purposes that we really need? Does any of this stuff ever get deleted?

These are all huge questions we have to answer as more people publish, share, create, track and do business online. We also have to weigh the possibilities of sharing data openly and locking it behind walls as well as how will people comprehensively find what they need when they want to as well as gauge the validity/accuracy of the information presented?

I’m betting on paid services for personal and business data management/archiving & Analysis tools. We will pay for good analysis, good data access & processing and good reliability/backups when we feel the pain of missing good insight, losing good data and just too much happening. Both personally and professionally. But unless you know how to work with SAP, SPSS, SQL, Oracle or a bunch of other systems data management is largely out of your control at this point. They are the librarians of our digital data and they need to find a workable way to Dewey decimal system it back into order and allow us to use it as humans need to.

Top 10 Questions we have for TopGear, Season 12

I just wanted to post these since I have not written about TopGear UK in a while and I didn’t want them to think they weren’t still producing top content or keeping my attention. The season 12 shows (1-4) that I have seen so far have been excellent and had many laugh out loud moments in each one. I always find it funny that my brother, fiancee, my dad and I laugh at different parts of the shows too.

The US trip to the Bonneville Salt Flats for Speed Week was my favorite so far (maybe because of a post I made last year suggesting it for TG USA) but there is always another great TopGear show around the corner. Anyway, here are my top 10 questions for TopGear UK, since I do not live in the UK and can’t go ask them directly. (if they would like to come to Chicago, I would be happy to answer their questions about my city) 

If you have the answers to any of these questions feel free to leave me a comment.

10. When is the TopGear USA (Gear on NBC?) going to start? They have been filming/talking/speculation for over 6 months already. (This has been answered…it has been dropped by NBC, no plans as of yet to sign another network)
9. When are they going to start showing current shows on BBC America? (ahem, without 33% of the show cut out)
8. When are we going to get the Top Gear track times on the board for the Corvette ZR-1, Challenger SRT-8 and CTS-V? (its only fair)
7. Why does Richard Hammond change hairstyles each episode? Why does he still wear the man-jewelry he aquired during that Africa trip episode? Why is James May’s hair always falling in his face in the studio segments even though there should be a stylist around to help him avoid that? Why does James always wear the Cheshire Cat rugby shirt on new car test days?
6. Where can I go to learn to drive like that in my car? (seriously, TopGear sport driving schools sound good to me)
5. What is that 747 doing there out on the runway near the track? Is it flyable? I thought it looked like they might be taking it apart in one of the recent episodes.
4. Why is one of the turns in the TopGear track called “Chicago” if they have never actually filmed a show in Chicago? (hint, hint…)
3. What are all these bets that end with weird clothing about?
2.Who is the stig? What are the morse code messages saying in the car when the Stig tests out the new cars on the track? 

1. How can I integrate TopGear somehow into my wedding ?

Danactive Yogurt Drink from Dannon Really Works

danactive, yogurt, health, immunity, systemLast year at this time I was sick constantly. I think the combination of everyone in this office having too much to do, not enough sleep and riding public transportation every day is a ripe situation for illness to hit and spread. I had a cold/cough thing on Thanksgiving last year and was also pretty sick over Christmas. I was also sick around New Years and about once a month until around May. I know there are a lot of reasons one can get sick, but it was really common here in this office last year and it has started again.

But this year I have been drinking danactive yougurt drinks from Dannon every morning since October and I have also been drinking more water. I wasn’t too sure about these danactive things but the package said that 70% of your immunity/system comes from your digestive system. Woah! I didn’t know that. My digestive system is always pretty messed up. I won’t get into the details but it’s always been weird so I figured I’d be a good candidate for this. So I started taking it and I haven’t been really sick since. I still have allergies and I still have other health problems but no colds, flu, coughs or fevers even though the people around me all have them about once a week. I am not sure exactly how it works but I think they explain more about how it keeps bacteria and viruses from entering through your digestive system with these good- bacteria. But that is a sketchy explanation at best.

It doesn’t help that nothing in this office is ever cleaned and and the air is re-circulated in a small area. I have to wash my hands as soon as I get here and after touching any doors or elevators. But I did that last year too. It’s just the danactive that is different this year. I am glad they made this product available I hope it helps more people stay healthy like it has helped me.

July 5th 2008 Update: Some lame-ass thinks this post is fake and somehow a paid ad for Danactive. I just wanted to say that it is not in any way an ad or paid for. I literally spent most of last year sick with a long list of minor illnesses that ranged from sore throats to colds to the flu. I was sick at least once a month and I missed a lot of work. This year I started taking vitamins, walking more, washing my hands a ton and drinking danactive. I think they all helped me stay much more well this year and I did not miss nearly as much work as I did last year. Why would that be so unbelievable that I would blog about feeling better? I blog about everything else. People should really think twice before making assumptions and inventing stories about people’s blogs who they don’t even know offline. You never know, someone might just call you a splog right back.

TopGear Season 10 Episode 7 Review

I enjoyed this week’s show. It was good to see something different. James, Richard and Jeremy all go buy vintage British cars from the 1970’s. They all seem to be small economical cars that are 4 door hatchbacks or mini 4 door sedans. They also seem to be the design that my dad’s 70’s era Buick century 4 door hatchback was based on. buick century hatchback, 4 door, 1970's(what was going on in that decade?)  Then they do some unusual tests to see how well these cars have dealt with age.  The funniest ones are saved for the end when they have to drive the car on a bumpy test road and have eggs in a colander strapped above their heads. Additionally they all loose parts of the car along the way. This was probably the funniest part. Surprise always helps the funny. Later they also fill the cars with water via a fire-hose and try and see how far they can drive them before they drain below the steering wheel. Clearly the BBC TG staff has been bored lately and working hard on new unusual ways to torture their presenters. Jeremy’s car doesn’t fare well since it keeps loosing doors unexpectedly. So, I may have ruined the surprise.

The star in a reasonably priced car segment was fun. Jennifer Saunders of French & Saunders show and Absolutely Fabulous fame comes by to take out the Chevy whatever around the track and she is surprisingly fierce. She is really competitive and apparently fearless. I guess you can get that way when you do comedy, you can’t really be scared of anything in order to do that for a living. Anyway she ends up 2nd on the board. Way to go Jennifer! It is great to see that since I am a fan of her work.

Other than that they do some mucking around in the news about Korea and forget that Holden (vauxall minaro) is from Australia. they also insist on pronouncing Hyundai as Hiaundai instead of Hunday and Jeremy forgets how to use the TV and prints out all his pix on paper.  Jeremy also burns mucho rubber in a new Aston Martin DBS. He concludes that it is really a juiced up DB9 replacement. Ok, but there are many of us that are completely ambivalent about that. It’s gorgeous but unattainable.

Coincidentally there is also an article in TIME magazine about the 15 million BBC channels all with the same name in the UK and the issues they have had keeping viewers and trying to not rig the unscripted shows they run. They make no mention of TG of course, because it’s a pretty successful franchise. I do thing that the BBC is doing ok really and not nearly as badly as they might say because in addition to all the fees they extract for ads on BBC America and the TG site, according to TIME they get $275 bucks from every TV receiving household each year in the UK. Whether it’s paid through taxes or directly I don’t know, but no one else gets that here. ( except PBS? but not that much) Sounds pretty good to me.

Anyway enjoy the full version online at YouTube here:

FTC and Advertising Tracking

Ok, so the federal trade commission wants to regulate how advertisers track people on the web? Do they realize that that data is never linked back to an individual person? Or that no personal information is ever collected? That the cookies expire after so many days?

This is a necessary debate that should happen so consumers understand better what is happening on their computers and around them all the time, but it should not outlaw such a practice.

This 3rd party tracking data is the only thing that makes the internet more viable as a place to advertise than offline. (are the TV networks and newspapers behind this push?) And the growth of our American economy depends on these cookies right now. (and google’s especially)

To my knowledge the cookies track this type of information:

1. whether or not you go back to a site after you have been displayed an ad for it

2. whether or not you convert from a visitor to a buyer while you are on the site

3. whether or not you come back to the site at a later date and buy something then

4. which ads you were displayed over the time the cookie has been there

5. which ad you came to the site from clicking on

6. sometimes there is geo location information generated from your IP address, but a lot of ISPs don’t assign you a static IP and then that isn’t relevant anymore.

7. the time and date of the ads you have seen and of your visits to the site

8. What type of browser you are using and what operating system but this is hardly personal information

And that is about it. No personal info, no credit card or social security info either.

The big flap about behavioral advertising is that they target the ads based on some data they have about you or your computer. Sites may serve you ads relevant to your location, your past site visits or of you have a profile on that site, the profile information you have submitted. Then they follow you around on the site showing you the ads targeted for that group. I know that only certain sites use this and it is not the majority. The click through rates are even lower than normal because you show someone the same ad 10 times, but the conversion rates after they click are higher than normal because of the fit between the ad and the person’s need if it was targeted correctly.

So all in all, I just wanted to say that this information is crucial in keeping businesses in business by knowing what works and does not work in advertising so they don’t spend a lot of money on stuff that doesn’t work. This was the huge problem in offline advertising for years even before it fragmented. It’s not about spying or sharing any information about you as a person. It’s simply about business data and using it to refine their business to be a better company and web site. And if you turn off the cookie feature on your browser you don’t have to participate at all. It’s not as evil as people think it would be when you get into the real meat of the matter.