Is More Data Always Better?

There has been a discovery in the online marketing and data/statistics world in the last few years. We have had more websites, products and tools created online than we can possibly keep track of. The terms to describe this deluge of activity we have been hearing the most are “data overload” and “information overload” from both companies and consumers. This Google Magazine uses the term Data Obesity to describe this phenomenon.

They ask the question, why is more data always better?

I think the idea of “more data us better” is common from people who lived before the Internet was prevalent. We had to work hard to find data. Researching something meant going to a library and looking in a card catalog (or maybe something called Gopher) and then finding your way around the Dewey decimal system to find that book. And then sometimes they didn’t even have the book because it was checked out or possibly it was just filed wrong because nobody understood the Dewey decimal system.

On a related note recently we got invited to my cousin’s wedding in Santa Fe New Mexico. My dad promptly went to the library and checked out 3 books on Santa Fe and New Mexico. I cringed. He asked how to find out the flights to book something without a travel agent. I realized I have been traveling since 2000 this way and he stopped traveling about that time so he never has. I introduced him to Travelocity, it was mind blowing and a bit of data overload compared with the OAG book he used to use in the 80′s.

The point here is that finding data was really difficult. People had control over its distribution because it was in print. When it became more freely accessible due to Google and other companies efforts we assumed this would be good, because people could remember where to find it and use it whenever we wanted. We never thought it would get this big so fast. Now travel sites are overwhelming, they have too many choices and there are too many of them trying to get you to opt into something you don’t want while being over charged for bringing a suitcase on a flight. This is just one example of how data has gone exponential so quickly.

Others of us have come to a data overload conclusion when they have 200 emails in several in-boxes, 1000+ rss reader posts from feeds waiting, several work projects, 500+ Facebook wall posts in their feed and hundreds of tweets that have gone un-read. This is among a climate where you have to follow-up with projects 5-10 times to get things done, post blogs/tweets/FB status updates daily to keep on people’s radar, empty the DVR so it doesn’t get overloaded and auto delete something you really wanted, listen to the radio on the way to work just in case something big happens and still find time to scoop the litter box before it gets full and the cats poop on the floor.

And the real purpose in all those tweets/FB posts and feeds is that you business changes yearly and if you don’t know about the latest trend and some real insights about it before your boss asks about it, you won’t have a job for all that long. (in digital marketing)

Having data overload be a “good” problem to have from some people’s perspective (as in that it is growth oriented). The democratization of publishing combined with tracking methodology and databases have all contributed to this problem, giving everyone a voice, a potential following of readers, a data trail to analyze and method to say something important online 24/7/365.  And then we have an even bigger problem of processing what is being said, figuring out if it is important or not and sharing/processing/saving it in some way if it is. Acting on that data is way down the line and many of us don’t even get there.

And this isn’t even the big problem with data overload. Where will we store it all? Why do tweets disappear from search so quickly? Because there are millions of them and the failwhale is full. According to the ThinkQuarterly UK, there are 800 Exabytes of data/information created every two days. It took humans from the beginning of civilization until 2003 to create the first 800 Exabytes, and we’re on a roll now.

Where does all this seemingly random data go? How will we know what it says without having to go into a database table and read specific field information? Where are the software tools to manage all this and still give humans the ability to customize the out put in ways that match the behavior or business purposes that we really need? Does any of this stuff ever get deleted?

These are all huge questions we have to answer as more people publish, share, create, track and do business online. We also have to weigh the possibilities of sharing data openly and locking it behind walls as well as how will people comprehensively find what they need when they want to as well as gauge the validity/accuracy of the information presented?

I’m betting on paid services for personal and business data management/archiving & Analysis tools. We will pay for good analysis, good data access & processing and good reliability/backups when we feel the pain of missing good insight, losing good data and just too much happening. Both personally and professionally. But unless you know how to work with SAP, SPSS, SQL, Oracle or a bunch of other systems data management is largely out of your control at this point. They are the librarians of our digital data and they need to find a workable way to Dewey decimal system it back into order and allow us to use it as humans need to.

Why the Microsoft & Yahoo Search Deal Sucks

I blogged about why I thought that the Microsoft and Yahootalks were not going to yield anything useful last year and was satisfied that they stopped wasting time trying to buy each other out of financial trouble. Now it has been widely reported that Microsoft has gained access to Yahoo in a search partnership deal. This is somewhat better but again, the executives have not listened to the public.

1. This Yahoo-Microsoft deal still sucks for several reasons. One being that Microsoft Ad Center is the ad display engine being used in the partnership and not Yahoo’s Panama. Neither have the depth or ease of use of AdWords. Anyone who has ever placed any pay per click search ads in their life would choose Panama over Microsoft Ad Center as the system to use. Maybe MAC makes more money but there are too many limits on bids, keyword availability and restrictions on running your ads to make it widely accepted.

2. The recent Netflix prize showed how much collaboration benefits organizations rather than competition. The revenue share partnership deal usually shares no information about technology or business strategy at all between companies. Its a you win, I loose, I’ll just pay you for handling this for me-approach to solving a problem, that doesn’t work in the long term.

The Netflix prize was recently awarded when a group of individual competitors and small teams banded together to use all their ideas in combination to finally get above the 10% improvement mark in matching/suggestion technology and submit their top result. In the last day before the time was up, another group of researchers banded together and topped the previous best submission also by combining all their ideas together at once.

Then in the last 12 hours the first team did come back with another submission just slightly better,  to win,  but the overall idea/lesson is still the same. If you really want to improve on consumer products and experience with really complex technical problems like search and suggestions, you have to collaborate rather than compete.

Yahoo and Microsoft would do a whole lot better against Google if they got the Bing folks together with the Yahoo search folks and started collaborating on this daily via videocam rather than doing an affiliate marketing type deal. This deal is evidence that Microsoft is being run by the lowest common denominator these days (and loosing a lot of money that way) and Yahoo’s CEO won’t be around long. She has made a decision that helps her contain and cut costs of running her company in the short term and that decision has sacrificed the long term marketability of her product.

In fact Yahoo was a search company primarily. People only used email, Yahoo news and other functions like Yahoo Answers because of the search engine they knew. If you take that away you don’t have an identity as a company. And they will lose a lot more search market share and preference by outsourcing to Bing even if they get paid a little bit more profitably in the next 2 quarters. This is pretty much the death of Yahoo. It is really sad.

Campbells Soup in Smaller Cans With No Soup? Sucks

Ok, this marketing blog has turned into the marketing and product rant blog as I find more and more companies making boneheaded decisions about their marketing and products. This week’s winner is Campbell’s Soups. I bought a bunch of Campbell’s soup a few weeks ago because in Chicago winter starts in late October and ends in June. Canned condensed Campbell’s soup is an easy and not-going-to-kill-you type microwave dinner I keep around all winter in case of a lack of groceries. (or any brand for that matter)

This new batch of cans though is either smaller or they have decided to cram so many more noodles, vegetables and pasta into the can so that there is now no room at all for the actual soup. The vegetable soup I opened last week was solid when I opened it. It would not pour at all. I had to scoop it out in large chunks and even after I added the can of water the condensed soup solids inside the noodles would not dissolve. It was like water with floating soup chunks in it. Gross.

Today was worse, peas and ham soup was solid as a rock and the water did not blend well here either. All the solid peas sat at the bottom of a soup that used to be puree style. WTF? Is gas that expensive that you can’t include water or actual soup in your cans anymore? Is space that much of a premium? Did an MBA tell you this would work? I thought so. Assholes.

IT DOESN’T WORK!

This sucks, the soup comes out gross every time and I will have to switch brands again to get back to actual soup broth in a can. This just goes to show you can never trust an MBA to actually give you suggestions that will work. They always F up the product with lower quality everything and take away important features that the customer values in the name of saving a few pennies and paying their 150K per year salary.

I buy this kind of Campbell’s soup because it is easy to make and is edible right out of the can with minimal changes. Now I end up microwaving it until it is boiling and the soup solids still won’t dissolve. Well, I guess that is enough for this rant. Campbell’s has lost my money, we will see how long it takes them to figure it out before they loose a lot more people too. Hello Progresso.

TIME Magazine Article – The Social Contract in America

I was reading my parent’s TIME Magazine this week (that I usually swipe to read on the train) and they had polled Americans on the state of the economy and their take on how they plan to personally ”get by” in the coming years. You can read the survey results and the article about this concept of a social contract online at TIME.com.

I had never heard of this concept of a “social contract” that business and government have with America. I work in a recruitment related field so if it existed, I thought I would know about it. As a human being I was aware of it as a colloquial dream we have perpetuated by the stories told by our parents and grandparents.

My family history doesn’t go back that far here in America. My great grandparents arrived from Poland and the Ukraine pre-WW1 and went to work in the gritty factories of Chicago because it was a better living and opportunity than they had back in Europe. (poor peasant potato farmers I usually say) and the economic opportunity has kept us here in Chicago ever since.

My grandparents generation went on to slightly boring but consistent blue collar jobs with pensions and my parent’s generation went on to white collar jobs after getting college educations. Some of them got a pension and health insurance and others did not. My generation doesn’t even get a shot at a pension. Companies have found that they can hire good people without it and they tell us that a 401K is really the same thing. (for reference I am 33)

So, we have these 401Ks that seem to never make money fast enough to accrue enough funds to equal what a pension would. They plummet in value every 10 years or so in recessions, and someone changes the funds available without asking or telling us. Most of us have health insurance through our jobs. We pay handsomely for it, between $100 and $300 per month per person.  And then when something happens that requires medical care, the insurance only covers 1/2 the costs. It is totally possible to go bankrupt with health insurance coverage these days because most coverage is crap compared to what my family had back in the 1980′s.

TIME says that there is an “implied” social contract in America where you give a company (or number of companies) your time and energy and they give you “a basic level of economic security provided you work hard and took responsibility for your family”. (direct quote from TIME July 28, 2008 p 42) And I think things have changed. This contract implied or not doesn’t really exist anymore. I see businesses every day making decisions to give workers less and people have to get more creative trying to survive.

I think the social contract is more like this now.

1. A company promises to pay you as little as they can for your time. This sounds pessimistic but I have seen the proof on paper that you are paid what they can get you for with your experience rather than what you are worth or how much “the job” pays. You have to wait years to work your way up the ladder to make a good wage and then marketers and your neighbors taunt you daily to buy everything in sight to keep up with the Joneses. 56% of the people who made over 100K a year said even they can’t expect to afford health care, college or a secure retirement anymore.  And 100K a year is a lot of clams. (I don’t make anywhere near that. ) I do realize that these businesses have to keep costs low in order to compete with India and China, but somehow I’d rather see the cuts come from other areas that don’t erode the culture in America and impede our ability to raise families. 

2. Marketers will prey on you from every direction. A lot more people could make it through hard times if they had savings but the national savings rate is negative now. All the “stuff” and services you “must” have seems to replace the financial security your grandparents achieved. Just say no didn’t work for reducing drug use in the 80′s and I think that the disposable consumer culture will probably continue here too.

3. Health Issues will cost you. Most young people don’t need much care because you haven’t gotten to the age where things start falling apart yet and we don’t have any concept of how much it costs to survive a serious health issue like cancer or bypass surgery. Both my parents had heart surgery in the late 1990′s and they were 50 & 60K each. We paid about 10K each of those costs and the insurance paid the rest. I just heard someone at my dad’s workplace had bypass surgery last month and it cost $100K. I know they have really poor health insurance there, and I can guess that the guy might have had to pay 50K out of pocket. Even dental issues are expensive. I need have needed a crown for about 5 years and because there is no pain or damage being done since the root canal and filling, I am holding off on the $1,000.00 price tag since dental insurance is only going to pay 1/2 and I would rather save the $ for a real emergency like fixing the 7 year old car I have or paying for the radiator heat to be fixed in my condo.

4. Retirement is going to be difficult. Very difficult. Some people wonder if social security will be around in 2040 when I turn 65. I personally, think it will be. It may not be nearly enough though. Most of us will have some 401K savings but as the Frontline Retirement special found, most people make crucial mistakes with managing their 401K and end up loosing a lot of money and getting little out at the end. (and then have to go back to work) Some tips include, never take a lump sum benefit, due to the tax penalty, never just let it ride and not watch the performance and watch for trading and management fees eating up your money. It also helps not to own a McMansion when you retire and live within your means before retirement. Saving money (like 10% of after tax income) on the side and investing it in some low risk but higher than inflation yields is also a smart way to prepare. And well let’s hope medicare still exists in 2040 also, and that doctors and hospitals still accept it as payment.

5. Creativity & Leverage are the new working hard. Money makes more money, it’s all who you know and being clever with side jobs or side businesses usually helps. Yes, saving a large percentage of your income by living simple and investing it can help you have the “power of compounding interest” as they say. Keeping in touch with people and maintaining your network helps with job opportunities and side opportunities to make some income. Starting weekend jobs or part time businesses online or otherwise helps too. I find people living simply and leveraging clever ways to work in more than one place are the ones that will have what they need later on. Getting into an industry that is doing well in the economy also helps but that may take pro-active skill re-training. Paying off your mortgage early and not moving also helps. You loose thousands of dollars on the services and fees associated with that transaction every time you move, and  we all know you pay 3x the value of your loan in interest if you really pay your mortgage over 30 years. After that you are seriously in the hole.

The only contract I think we really have now is that everything will change by the time the 30 somethings reach retirement age. The only thing we have to rely on is ourselves. In general business is struggling because the US has passed it’s peak and we will be in a pack of “also rans” soon. Companies in the US will not see the skyrocketing growth that they saw post-war in the last 60 years with China, India and Eastern Europe emerging as super-economic powers. This coupled with dwindling natural, energy and food resources will make the next 50 years a post US dominant era that will be much harder and more global.

I actually believe if the US was more competitive with skills and education we would do well in a world economy but I haven’t yet seen the expertise or drive to innovate. All I see every day is the drive to reduce expenses and cut resources in business and make short term gains with little or no thought about long term survival. I feel like the country is being run by the lowest common denominator MBAs right now and the next 10 years for us commoners are going to be difficult as a result, as we all lack the jobs/growth that they sucked/poached out in the short term and ran off with the profits.

So, enough about all that negativity.

How do you plan on coping with the changing game living and working in the US in the next 50 years?

Kodak drops Olympic Sponsorship After Beijing 2008

The 2008 Beijing Olympics are less than a year away. NBC is shaking in their boots though because so many advertisers are baling out right after and have probably reduced their advertising and commercial media spend on this upcoming 2008 Olympic Games. A few things seem to have changed with the Olympics over the past 10-15 years. They used to be the talk of the town and what everyone was watching each night on television for 2 weeks. Now we are not so sure where the Olympics fall in popularity in the U.S. Today they announced that Kodak along with GM and probably a few more United States companies that aren’t as popular, won’t be advertising in the Olympics after the Bejing 2008 Games. Here are several reasons why I think this evolutionary media and company changeover is happening:

1. The Olympic Mystique is waning. In a world of video games, social networking and texting these big deal once every 4 year Games that an athlete works their entire life for somehow seem disconnected. To work for that long on something and not get paid is somehow against our current consumer culture’s ideals. If you work that long you should surely get a Million dollar contract and a promotional deal with Nike right? The “for the love of the sport and glory of winning to bring peace to the world” idea isn’t really that popular with today’s youth or yuppies. They believe in sell to your neighbor, network with your friends and only work towards one goal: Money. So the Olympics are just not relevant anymore and somehow boring.

2. The evening entertainment options for people now are fragmented and overwhelming. Before you could watch TV (5 channels) or go out to dinner and a movie, listen to the radio, read a book or talk on the phone. An example of a simple 1980′s night. Now in 2007 you could do all those things plus, go online, email, text, IM, blog, twitter, MySpace/Facebook, watch dvds, DVR, Tivo, download something to watch illegally, listen to satellite radio, NPR, Google, read and comment on online news, watch cable’s any number of billions of channels, talk on the cell phone, listen to CDs, your iPod, choose new music on Itunes, go to a virtual world like second life or WoW, play video games on your TV or online, or possibly several of these combined. You can see why kids have such trouble sitting down and studying with a book these days. You can also see why the 1980′s seemed quaint and the 1950′s seemed unthinkable. People have so much selection to choose from, they just switch entertainment as soon as they get bored and which ever gets the excitement going wins. The Olympics haven’t really been able to capture the excitement for a while now.

3. The face of American Business is changing. Companies that manufacture and produce actual products in America are dying. That is just the way it is. Our Japanese competitors were formidable in the 1980′s and Our Chinese competitors in the 90′s and 00′s are killing us softly. Notice that Bank of America will be Advertising in the Olympics and GM and Kodak won’t. No American company can afford to manufacture anything in America anymore and be a world power. We’ve gotten too rich and demanding as employees (especially executives), and it’s too expensive to manufacture here. If a company transfers production to Chinese factories (for cheaper labor to be competitive) by default they teach the Chinese companies how to do their work and run the business. And then they Chinese have been known to start their own company and take over. There is a conscious effort by the Chinese to take US knowledge to build businesses in China and bring greatness and world economic power back to China like it used to have centuries ago. (from the book One Billion Customers) Hosting the Beijing Olympics is part of that Grand Plan to showcase how modern and westernized they are now while they win all the medals. The U.S. on the other hand seems to not really plan for the future, and doesn’t see that we are being put out of business. So, companies like Kodak and GM can’t advertise in the Olympics because they can’t afford to anymore, they are shrinking companies that are being beat out of the market at every turn. It is sad that we will all be driving Japanese and Chinese cars and using Chinese and Japanese cameras made in Singapore, Nepal and Malaysia soon and we won’t have any jobs here because we don’t produce anything. Manufacturing is power and we have given that up for big profits short term manufacturing in China and long term business loss and job loss here in America. The executives and board members that have decided that for this country should be put in Jail. The damage they have done to the U.S. Economy for their own personal gain is disgusting.

The Media World

Ever wonder how your cable company, network you watch, magazines you read, web sites you surf and billboards you see all relate to one another? Well the Media world is all interrelated. Many companies own all of the above and your eyeballs are what they really want. They design media buying so that you see the ads on all screens when they want your attention. Either for a new product release, a new TV show launch or a product that needs to meet quarterly sales goals. It also explains the cross messaging between networks advertising eachother’s products and services. It also is interesting from a media buying standpoint if you can leverage these relationships for make good and discount opportunites with purchases. Although, that isn’t really available until you get into the $$$,$$$,$$$ price point. Still it is interesting, and take note of the top media company family tree.