Thursday, February 23, 2012
NYTimes.com: The Age of Big Data
NEWS ANALYSIS
From: The Age of Big Data
By STEVE LOHR
Published: February 11, 2012
GOOD with numbers? Fascinated by data? The sound you hear is opportunity knocking.
Multimedia
Readers’ Comments
Readers shared their thoughts on this article.
Mo Zhou was snapped up by I.B.M. last summer, as a freshly minted Yale M.B.A., to join the technology company’s fast-growing ranks of data consultants. They help businesses make sense of an explosion of data — Web traffic and social network comments, as well as software and sensors that monitor shipments, suppliers and customers — to guide decisions, trim costs and lift sales. “I’ve always had a love of numbers,” says Ms. Zhou, whose job as a data analyst suits her skills.
To exploit the data flood, America will need many more like her. A report last year by the McKinsey Global Institute, the research arm of the consulting firm, projected that the United States needs 140,000 to 190,000 more workers with “deep analytical” expertise and 1.5 million more data-literate managers, whether retrained or hired.
The impact of data abundance extends well beyond business. Justin Grimmer, for example, is one of the new breed of political scientists. A 28-year-old assistant professor at Stanford, he combined math with political science in his undergraduate and graduate studies, seeing “an opportunity because the discipline is becoming increasingly data-intensive.” His research involves the computer-automated analysis of blog postings, Congressional speeches and press releases, and news articles, looking for insights into how political ideas spread.
The story is similar in fields as varied as science and sports, advertising and public health — a drift toward data-driven discovery and decision-making. “It’s a revolution,” says Gary King, director of Harvard’s Institute for Quantitative Social Science. “We’re really just getting under way. But the march of quantification, made possible by enormous new sources of data, will sweep through academia, business and government. There is no area that is going to be untouched.”
Welcome to the Age of Big Data. The new megarich of Silicon Valley, first at Google and now Facebook, are masters at harnessing the data of the Web — online searches, posts and messages — with Internet advertising. At the World Economic Forum last month in Davos, Switzerland, Big Data was a marquee topic. A report by the forum, “Big Data, Big Impact,” declared data a new class of economic asset, like currency or gold.
Sunday, February 19, 2012
O'Reilly: The bond between data and journalism grows stronger
From O'Reilly radar: The bond between data and journalism grows stronger
Liliana Bounegru discusses the state of data journalism and its growing influence.
While reporters and editors have been the traditional vectors for information gathering and dissemination, the flattened information environment of 2012 now has news breaking first online, not on the newsdesk.
That doesn't mean that the integrated media organizations of today don't play a crucial role. Far from it. In the information age, journalists are needed more than ever to curate, verify, analyze and synthesize the wash of data.
To learn more about the shifting world of data journalism, I interviewed Liliana Bounegru (@bb_liliana ), project coordinator of SYNC3 , the first international Data Journalism Awards , and Data Driven Journalism at the European Journalism Centre.
Friday, February 17, 2012
Open Knowledge Foundation announcing the School of Data
The following post is by Rufus Pollock, Director and Co-Founder of the Open Knowledge Foundation, and Philip Schmidt, Co-Founder and Executive Director of Peer 2 Peer University.
Today, we’re announcing plans for a School of Data. The School will be a joint venture between theOpen Knowledge Foundation and Peer 2 Peer University (P2PU). We also welcome other organizations who would like to participate — see below for more on this.
Why?
Data (open or otherwise) needs to be used, and to use data effectively requires certain skills.1
The explosive growth in data, especially open data, in recent years has meant that the demand for data skills — for data “wranglers”2 or “scientists” — has been growing rapidly. Moreover, these skills aren’t just important for banks, supermarkets or the next silicon valley start-up, they are also going to be cruicial in reserach, in journalism, and in civil society organizations (CSOs).
However, there is currently a significant shortfall of data “wranglers” to satisfy this growing demand, especially in civil society organisations — McKinsey expects a skills shortage in data expertise to reach 50-60% by 2018 in the US alone.3
Most formal training for data skills still takes place as part of formal statistics or mathematics programmes. It will be years before data specialist degree paths become broadly available and accepted, and even then, time-intensive degree courses may not be the right option for journalists, activists, or computer programmers who just need to add data skills to their existing expertise.
What is needed are flexible, on-demand, shorter learning options for people who are actively working in areas that benefit from data skills, particularly those who may have already left formal education programmes.
Tuesday, February 7, 2012
Monday, January 30, 2012
Internet 2011 in numbers
From: http://royal.pingdom.com/2012/01/17/internet-2011-in-numbers/
Email
More....
Posted in Main on January 17th, 2012 by Pingdom
So what happened with the Internet in 2011? How many email accounts were there in the world in 2011? How many websites? How much did the most expensive domain name cost? How many photos were hosted on Facebook? How many videos were viewed to YouTube?
We’ve got answers to these questions and many more. A veritable smorgasbord of numbers, statistics and data lies in front of you. Using a variety of sources we’ve compiled what we think are some of the more interesting numbers that describe the Internet in 2011.
- 3.146 billion – Number of email accounts worldwide.
- 27.6% – Microsoft Outlook was the most popular email client .
- 19% – Percentage of spam emails delivered to corporate email inboxes despite spam filters.
- 112 – Number of emails sent and received per day by the average corporate user.
- 71% – Percentage of worldwide email traffic that was spam (November 2011).
- 360 million – Total number of Hotmail users (largest email service in the world).
- $44.25 – The estimated return on $1 invested in email marketing in 2011.
- 40 – Years since the first email was sent , in 1971.
- 0.39% – Percentage of email that was malicious (November 2011).
Websites
- 555 million – Number of websites (December 2011).
- 300 million – Added websites in 2011.
Web servers
- 239.1% – Growth in the number of Apache websites in 2011.
- 68.7% – Growth in the number of IIS websites in 2011.
- 34.4% – Growth in the number of NGINX websites in 2011.
- 80.9% – Growth in the number of Google websites in 2011.
Monday, January 23, 2012
Sunday, January 22, 2012
Saturday, January 7, 2012
![[Cycles of Financial Crises 1810 to 2010]](https://lh3.googleusercontent.com/blogger_img_proxy/AEn0k_ua1PKKOFJLhYaiom6m22Z7EkdhWHhvvvvQ69uW4iwLfafG4WO3X5czzJK4yyTR8viICRzD7bgkRoq9bpgtlvzV9i_A5r7cgkt7c2eVV8nQZfvHGXR7PfPiK1u0wg=s0-d)
Friday, January 6, 2012
Search for public data in Google Web Search
http://www.youtube.com/watch?v=9Qt2n34VEr4&feature=player_embedded
A subset of datasets from the Public Data Explorer are indexed in Google Web Search. Searching for metrics from these datasets will generate a graph at the top of your search results, and clicking on this graph will take you to the corresponding visualization in the Public Data Explorer.
More information about each of the searchable datasets and their associated triggering queries is shown in the table below:
| Dataset (Provider) | Data Details | Example Queries |
|---|---|---|
| World Development Indicators (World Bank) | A variety of metrics related to world growth and development, by year, by country. See the complete list of metrics. | [ GDP of France ] [ Population of China] [ Energy consumption Iceland ] |
| Unemployment in the US (US Bureau of Labor Statistics) | Unemployment rate by month, by state, county, and for the country as a whole. | [ US unemployment ] [ Unemployment in Wisconsin ] |
| Population in the US (US Census Bureau) | Population by year, by state, county and for the country as a whole. | [ Population of Texas] [ Population Nassau County New York ] |
| Unemployment in Europe (Eurostat) | Unemployment by month, by country. | [ Unemployment in France ] |
| Minimum Wage in Europe (Eurostat) | Minimum wage in Euros by half-year, by country. | [ Portugal minimum wage ] |
| Government Debt in Europe (Eurostat) | Government debt as percentage of GDP by year, by country. | [ Germany government debt ] |
| Broadband Penetration in Europe (Eurostat) | Broadband connections per hundred inhabitants, by half-year, by country. | [ Broadband penetration Sweden ] |
Stay tuned for more datasets in Google search in the future!
updated 12/19/2011
Wednesday, January 4, 2012
From SearchBusinessAnalytics: Skills shortage, training present pitfalls for 'big data' analytics
Beth Stackpole, Contributor
Published: 1 Jan 2012
The biggest challenges related to “big data” analytics, according to consultants and IT managers, boil down to a simple one-two punch: The technology is still fairly raw and user-unfriendly, and there aren’t enough skilled experts to go around.
A lot of big data technologies -- like Hadoop and MapReduce -- hail from the open source world, developed by Internet pioneers such as Google and Yahoo to take on the problem of cost-effectively processing large volumes of information, including both structured andunstructured data. As a result of this orientation, most of the technologies lack the maturity and accessibility of traditional databases and data management suites, and there is still a limited selection of complementary analytics tools available to make these environments feel familiar to many data warehousing and analytics professionals.
Subscribe to:
Posts (Atom)