Is More Data Always Better?

google think magazine data overload obesity information ideas processing analysisThere has been a discovery in the online marketing and data/statistics world in the last few years. We have had more websites, products and tools created online than we can possibly keep track of. The terms to describe this deluge of activity we have been hearing the most are “data overload” and “information overload” from both companies and consumers. This Google Magazine uses the term Data Obesity to describe this phenomenon.

They ask the question, why is more data always better?

I think the idea of “more data us better” is common from people who lived before the Internet was prevalent. We had to work hard to find data. Researching something meant going to a library and looking in a card catalog (or maybe something called Gopher) and then finding your way around the Dewey decimal system to find that book. And then sometimes they didn’t even have the book because it was checked out or possibly it was just filed wrong because nobody understood the Dewey decimal system.

On a related note recently we got invited to my cousin’s wedding in Santa Fe New Mexico. My dad promptly went to the library and checked out 3 books on Santa Fe and New Mexico. I cringed. He asked how to find out the flights to book something without a travel agent. I realized I have been traveling since 2000 this way and he stopped traveling about that time so he never has. I introduced him to Travelocity, it was mind blowing and a bit of data overload compared with the OAG book he used to use in the 80’s.

The point here is that finding data was really difficult. People had control over its distribution because it was in print. When it became more freely accessible due to Google and other companies efforts we assumed this would be good, because people could remember where to find it and use it whenever we wanted. We never thought it would get this big so fast. Now travel sites are overwhelming, they have too many choices and there are too many of them trying to get you to opt into something you don’t want while being over charged for bringing a suitcase on a flight. This is just one example of how data has gone exponential so quickly.

Others of us have come to a data overload conclusion when they have 200 emails in several in-boxes, 1000+ rss reader posts from feeds waiting, several work projects, 500+ Facebook wall posts in their feed and hundreds of tweets that have gone un-read. This is among a climate where you have to follow-up with projects 5-10 times to get things done, post blogs/tweets/FB status updates daily to keep on people’s radar, empty the DVR so it doesn’t get overloaded and auto delete something you really wanted, listen to the radio on the way to work just in case something big happens and still find time to scoop the litter box before it gets full and the cats poop on the floor.

And the real purpose in all those tweets/FB posts and feeds is that you business changes yearly and if you don’t know about the latest trend and some real insights about it before your boss asks about it, you won’t have a job for all that long. (in digital marketing)

Having data overload be a “good” problem to have from some people’s perspective (as in that it is growth oriented). The democratization of publishing combined with tracking methodology and databases have all contributed to this problem, giving everyone a voice, a potential following of readers, a data trail to analyze and method to say something important online 24/7/365.  And then we have an even bigger problem of processing what is being said, figuring out if it is important or not and sharing/processing/saving it in some way if it is. Acting on that data is way down the line and many of us don’t even get there.

And this isn’t even the big problem with data overload. Where will we store it all? Why do tweets disappear from search so quickly? Because there are millions of them and the failwhale is full. According to the ThinkQuarterly UK, there are 800 Exabytes of data/information created every two days. It took humans from the beginning of civilization until 2003 to create the first 800 Exabytes, and we’re on a roll now.

Where does all this seemingly random data go? How will we know what it says without having to go into a database table and read specific field information? Where are the software tools to manage all this and still give humans the ability to customize the out put in ways that match the behavior or business purposes that we really need? Does any of this stuff ever get deleted?

These are all huge questions we have to answer as more people publish, share, create, track and do business online. We also have to weigh the possibilities of sharing data openly and locking it behind walls as well as how will people comprehensively find what they need when they want to as well as gauge the validity/accuracy of the information presented?

I’m betting on paid services for personal and business data management/archiving & Analysis tools. We will pay for good analysis, good data access & processing and good reliability/backups when we feel the pain of missing good insight, losing good data and just too much happening. Both personally and professionally. But unless you know how to work with SAP, SPSS, SQL, Oracle or a bunch of other systems data management is largely out of your control at this point. They are the librarians of our digital data and they need to find a workable way to Dewey decimal system it back into order and allow us to use it as humans need to.

Japanese Earthquake Tsunami Disaster is Different from Haiti

japanese earthquake nuclear reactor tsunami mapWithin a day after the Haiti Earthquake in January 2010 (7.0)there were celebrities on TV urging people to donate money to help those who were in need. Within a day of the earthquake and tsunami in Indonesia in 2004 we saw the same attention drawn to the even in the name of help for those in need. Both had telethons on television pleading with the public to donate millions of dollars to the relief efforts.

This time there is a very different mood in the USA a week after the 9.0 scale earthquake off the eastern coast of Japan. (2011 Tōhoku earthquake and tsunami) There have been some Facebook/Twitter/Yahoo links to the red cross to donate but overall the donation numbers I have seen shown on U.S. sites were very low. We haven’t seen any celebrities saying that they have donated money to the cause and our own government isn’t really talking about loans or donations right now in our budget crisis (and yet they’re contemplating invading Libya at the same time). 

We’ve seen the news focus their coverage on the nuclear disaster that was the result of the loss of electricity during the earthquake and the loss of the backup generators because of the water/power of the tsunami. This combined with explosions that cracked containment vessels and a lot of hot spent fuel sitting around in 6 reactors has an enormous risk associated with it. Sure, that is the big-blockbuster-disaster-film story here, but it is not the largest problem facing Japan. The newscasters have a captive audience worldwide with this topic because this is the one issue that could affect people outside Japan.

It is a weird difference in opinion that Americans have about Japan compared with the past earthquakes/tsunamis/disasters in Indonesia & Haiti. At this point in time it surprises me greatly that Americans have issues with a country like Japan to the point that they do not want to help. It makes us look un-educated and small minded. I thought Americans would identify directly with the Japanese, like I do but I’m not seeing a lot of evidence that it is the case.

Some celebrities have been cited making rude jokes about the Earthquake/Tsunami within the last week and Gilbert Godfried was fired as the voice of the Aflac Duck (he needs to be retired anyway) because of rude and insensitive tweets he sent. 50 cent also has been criticized for his lack of understanding and even a government spokesperson in Mississippi had to resign over tasteless rude comments (not really surprising for Mississippi).

I was not always a huge fan of Japan. In the 1980’s I viewed them like many people did, as the reason that US car companies and manufacturing companies here were going out of business. There was the assumption that the Japanese were ruthless and as smart as robots and we felt threatened by that. Now we feel that way about China instead. (I do not feel that way about China though, one of my best friends is Chinese and I’ve spent a lot of time with her family for decades and could never think of them that way)

 In the last 30 years things have changed. I think Tokyo and most Japanese people share a lot of commonalities with us. They love their cell phones & gadgets as much as we do. We’ve adopted their Anime and Video gaming interests as mainstream. Toyota/Honda/Mitsubishi/Mazda sell more cars here than American companies do because people really like them and they’re innovating with hybrids and new ideas like Scion. (I don’t have the exact sales numbers for this, it is more my impression from seeing what is on the road in Chicago). Entire movies have been made about the following for Japanese imports (Fast & Furious) And who is going to give up their big screen TV or computer monitor from Samsung, Sony or NEC? Or their Wii?

But the real change in perception happened when people started liking Sushi. About 10 years ago Sushi started getting popular in Chicago. With it U.S. audiences began to discover the california roll, spicy tuna and the joy of Miso Soup. Along with our broadening pallets beyond teryaki we discovered udon noodles, bento boxes, tempura, sake, mochi and red bean ice cream. (red bean ice cream may also be Chinese, and it is better than the green tea flavor)

japanese sakura flowers white pink trees cherry blossomsWe also discovered through Flickr that Japanese people love their cats as much as we do (Junku) and Sakura is the cherry blossom festival each spring.  A blend of Asian styles from China, Japan and other areas is very popular right now in home decorating, emphasizing the strong minimalist lines of furniture pieces and the delicate organic patterned styles. We also share our love of all things digital with Japan. And who can forget Sanrio and the super-popular Hello Kitty and her cast of friends? Can you get any more cute and friendly than that?

So how did I overcome the feeling of competitive anxiety with Japan? By enjoying the exported food. And the rich culture of preparation that surrounds it that makes it unique and special. Exporting your culture really does change views that people have of your country around the world. Through this I have realized that the Japanese are competitive people, but they are also very nice people and respectful of us as a country too. I’ve come to the conclusion that we can learn a lot from them about managing large populations in small places (like Tokyo) and how to continue to innovate in industry, manufacturing and the economy.

So that is why it bothers me that the U.S. population is not really doing anything to help here. Sure, some closed-minded conservatives will always have rude views, but I really think the people in America should do something to help.

Instead of understanding that Japan is a nation of people with a lot more in common with us than most of the world, a large portion of the U.S. public continues to focus on the differences. The bottom line is that Japan is an industrial/manufacturing/technology based economy like ours. The average person in Japan is pretty well-educated on a world scale, like us.

In fact we really do depend on each other a lot, buying/selling from each other and sharing strong bonds of family and friends across borders. We can’t just stand by and watch these nuclear reactors go out of control and not think about the people who don’t have food, water or homes anymore. It is a disaster just like anywhere else in the world. And that world has gotten a whole lot smaller in the last 10 years with the internet, skype,  international business and Sushi. And we’re not doing enough to help.