Monday, February 13, 2012

New York Times on Big Data

I highlight here some of the most interesting paragraphs of Sunday's report from the New York Times about the new world of data and the opportunities that are growing in the field.
The story is similar in fields as varied as science and sports, advertising and public health — a drift toward data-driven discovery and decision-making. “It’s a revolution,” says Gary King, director of Harvard’s Institute for Quantitative Social Science. “We’re really just getting under way. But the march of quantification, made possible by enormous new sources of data, will sweep through academia, business and government. There is no area that is going to be untouched.” Welcome to the Age of Big Data. The new megarich of Silicon Valley, first at Google and now Facebook, are masters at harnessing the data of the Web — online searches, posts and messages — with Internet advertising. At the World Economic Forum last month in Davos, Switzerland, Big Data was a marquee topic. A report by the forum, “Big Data, Big Impact,” declared data a new class of economic asset, like currency or gold. Research by Professor Brynjolfsson and two other colleagues, published last year, suggests that data-guided management is spreading across corporate America and starting to pay off. They studied 179 large companies and found that those adopting “data-driven decision making” achieved productivity gains that were 5 percent to 6 percent higher than other factors could explain. Global Pulse, a new initiative by the United Nations, wants to leverage Big Data for global development. The group will conduct so-called sentiment analysis of messages in social networks and text messages — using natural-language deciphering software — to help predict job losses, spending reductions or disease outbreaks in a given region. The goal is to use digital early-warning signals to guide assistance programs in advance to, for example, prevent a region from slipping back into poverty. In economic forecasting, research has shown that trends in increasing or decreasing volumes of housing-related search queries in Google are a more accurate predictor of house sales in the next quarter than the forecasts of real estate economists. The Federal Reserve, among others, has taken notice. In July, the National Bureau of Economic Research is holding a workshop on “Opportunities in Big Data” and its implications for the economics profession.
But data offers perils too,
Big Data also supplies more raw material for statistical shenanigans and biased fact-finding excursions. It offers a high-tech twist on an old trick: I know the facts, now let’s find ’em. That is, says Rebecca Goldin, a mathematician at George Mason University, “one of the most pernicious uses of data.” Veteran data analysts tell of friends who were long bored by discussions of their work but now are suddenly curious. “Moneyball” helped, they say, but things have gone way beyond that. “The culture has changed,” says Andrew Gelman, a statistician and political scientist at Columbia University. “There is this idea that numbers and statistics are interesting and fun. It’s cool now.”
Everyone is talking about data.

Sunday, February 12, 2012

Visualization software

Due to the huge amount of data, nowadays data visualization is becoming more and more essential influential and relevant. At the same time new software are appearing to help data geeks on their work. The following is the new software on town:

http://www.oicweave.org/

Weave (BETA 1.0) is a new web-based visualization platform designed to enable visualization of any available data by anyone for any purpose.

Saturday, February 4, 2012

World values survey

World Values Survey is an international survey that collects and analyse data on moral and ethical questions. The project is amazing.

In order to summarize information they came with this chart:



Where each axis is defined as:

"The Traditional/Secular-rational values dimension reflects the contrast between societies in which religion is very important and those in which it is not. A wide range of other orientations are closely linked with this dimension. Societies near the traditional pole emphasize the importance of parent-child ties and deference to authority, along with absolute standards and traditional family values, and reject divorce, abortion, euthanasia, and suicide. These societies have high levels of national pride, and a nationalistic outlook. Societies with secular-rational values have the opposite preferences on all of these topics.

The second major dimension of cross-cultural variation is linked with the transition from industrial society to post-industrial societies-which brings a polarization between Survival and Self-expression values. The unprecedented wealth that has accumulated in advanced societies during the past generation means that an increasing share of the population has grown up taking survival for granted. Thus, priorities have shifted from an overwhelming emphasis on economic and physical security toward an increasing emphasis on subjective well-being, self-expression and quality of life. Inglehart and Baker (2000) find evidence that orientations have shifted from Traditional toward Secular-rational values, in almost all industrial societies. But modernization, is not linear-when a society has completed industrialization and starts becoming a knowledge society, it moves in a new direction, from Survival values toward increasing emphasis on Self-expression values".

The only issue it lacks is to provide the standard deviation of each country. I guess differences between countries are not as high as differences between citizens in the same country.