Charts that changed the world—way before big data

   Text Size  

 Published: Thursday, 31 Oct 2013 | 7:00 AM ET

By: Eric Rosenbaum | CNBC.com

Twitter 47

LinkedIn 186

Share

Source: Tableau Software

Detail of Ignaz Semmelweis’s germ theory data chart

Some years ago I was sent by an employer to a one-day course taught by Yale professor emeritus Edward Tufte on presenting data and information. This was a decade before big data was a buzz term and companies in the business of data visualization, like Tableau Software, were going public to a very enthusiastic investor market. 

Data visualization is a growing field in which massive amounts of data are measured in quantities reaching exabytes and crunched by an ever-increasing number of Silicon Valley servers ultimately to be presented in visual displays. How the intersection of data, analytics and business evolves is an open question. But according to Tableau CEO Christian Chabot in a presentation to data geeks earlier this year, some of the world's greatest thinkers gained tremendous insight and changed the world simply by organizing and deciphering basic data sets in new ways.

Read more.....

UK Government publishes data capability strategy

From: StatsLife
Written by Web News Editor on 31 October 2013. Posted in News

The government, in partnership with the recently formed Information Economy Council, has just published its Data Capability Strategy, which outlines how the opportunities in open/big data might be utilised for the benefit of all.

Titled ‘Seizing the Data Opportunity’, the strategy begins with the premise that the increasing significance of data is ‘one of the greatest opportunities and challenges facing policymakers today.’ It cites figures by the Centre for Economics and Business Research which estimates that the big data marketplace could create 58,000 new jobs in the UK between 2012 and 2017.
Read more...

Creating Knowledge out of Interlinked Data - STATISTICAL OFFICE WORKBENCH

From: http://www.slideshare.net/lod2project/lod2-webinar-series-3rd-relase-of-the-stack

Simply Statistics Unconference on the Future of Statistics

From: http://www.youtube.com/watch?v=Y4UJjzuYjfM&feature=share



Twitter flow: https://twitter.com/search?q=%23futureofstats&src=typd

Workshop on BIG DATA

From: https://docs.google.com/file/d/0B8kL1t8n_fICMlBkWlZWR3UwS0E/edit?usp=drive_web

Essential Collection of Visualisation Resources

From: http://www.visualisingdata.com/index.php/resources/

Here is a collection of some of the most important, effective, useful and practical data visualisation tools. The content covers the many different resources used to create and publish visualisations, tools for working with colour, packages for handling data, places to obtain data, the most influential books and educational programmes and qualifications in visualisation itself.

Data and visualisation tools

More ......

EU - Consultation on best ways to open up more public data

From: http://www.investineu.com/content/consultation-best-ways-open-more-public-data-12c3

Consultation on best ways to open up more public data

The Commission is asking for help in drawing up guidance and practical advice for boosting the re-use of weather data, traffic data, publicly funded research data, statistics, digitised books, and other types of public sector information (PSI). 

Commission Vice President Neelie Kroes said: “The open data revolution is all about individuals and entrepreneurs and that includes the giving them a role in policy design.” 

Interested parties can contribute until 22 November 2013, and with their input help ensure effective implementation of the newly revised PSI Directive, which aims to ensure public authorities release more information at little or no cost to users. 

“We were successful in finalising a new directive for public sector information. Now we need to implement it in a way that creates a single market for these data, so that companies can build pan-European apps and other innovations. To get Europe-wide value from this data we need the rules to be applied in a clear, common way,” Kroes said

Background

The results of this consultation will support guidelines to be published in 2014. A recent study shows that economic benefits for the EU from easier availability of PSI could reach 40 billion euros annually. The growth rate of companies having access to free PSI is 15% higher than those who need to pay. 

Opening up public data resources for re-use is one of the key actions of the Digital Agenda for Europe, however uncertainties surrounding compliance with the licensing and charging provisions have discouraged innovation and commercial deployment of PSI in many parts of the Union.

30 August 2013

Open data & storytelling - 4 presentations from the Nordic Statistical Meeting 2013

From:http://www.ssb.no/omssb/tjenester-og-verktoy/kurs-og-seminarer/nordisk-statistikermote-vedlegg/artikler-og-presentasjoner

Open Data - Experiences from a National statistical office's point of view
Heli Mikkelä
Statistics Finland

Statistical storytelling and quality in the age of open data
Maija Metsä-Pauri and Kirsti Ahlqvist
Statistics Finland

Åpne data – en ny åpning for statistikkbyråene
Jan Bruusgaard
Statistisk sentralbyrå

API:er ett nytt sätt att tillhandahålla öppna data, nyttan för oss och våra användare
Åsa Olsson
Statistiska centralbyrån

CIO - Big data may build better profile of Australians

From: http://www.cio.com.au/article/522550/big_data_may_build_better_profile_australians_statistician_/

Information could be used in future censuses, says Jonathan Palmer

Hamish Barwick (CIO)
01 August, 2013

Location information from mobile phones could lead to more dynamic pictures of population counts according to an Australian statistician.

Speaking at a Teradata summit in Sydney, Australian Bureau of Statistics (ABS) deputy Australian statistician, Jonathan Palmer, told delegates that this may enhance census information in the future.

Read more...

itnews - ABS prepares big data transformation

From: http://www.itnews.com.au/News/352003,abs-prepares-big-data-transformation.aspx

By Paris Cowan on Aug 2, 2013 7:10 AM (15 hours ago)

The Australian Bureau of Statistics has begun an IT transformation aimed at tackling an incoming "big data tsunami" which the agency's COO is hoping will include more personal data from citizens.

The ABS' technology infrastructure transformation will be the biggest undertaken at the department and will include serious work on renewing metadata systems and processes.

Read more...

Statistics and open data

From: http://www.parliament.uk/business/committees/committees-a-z/commons-select/public-administration-select-committee/news/statistics-and-open-data/

11 July 2013

The Public Administration Select Committee (PASC) is conducting an inquiry into statistics and open data in Government, with a focus on the progress of the Government in implementing its Open Data strategy. This is part of PASC’s programme of work on statistics and their use in Government.

Read more.....

Is Quandl the easiest way to find and use numerical data on the internet?

From: http://www.quandl.com/about

Quandl has indexed over 5 million time-series datasets from over 400 sources. All of Quandl's datasets are open and free.

You can download any Quandl dataset in any format that you want. You can also visualize, save, share, authenticate, validate, upload, index, merge and transform data .

Our long-term goal is to make all the numerical data on the internet easy to find and easy to use.

Further Reading

Learn more about Quandl:

• Quandl's long-term goal is to make all the numerical data on the internet easy to find and easy to use. Read more on our vision page.
• Quandl currently has over 5 million datasets from over 400 sources. Explore Quandl's data on our sources page.
• Every dataset on Quandl is available via a simple and consistent API.
• You can also access Quandl data using our R, Python, Matlab, Excel, Maple, Julia, Clojure, and Stata packages.
• Check out what's new on Quandl (data, features, topics).
• Explore Quandl's features and what we've got planned for the future. If you have suggestions, please email us.
• Learn how developers are building innovative applications that use the Quandl data platform, on our partners page.
• Questions or comments? See help or email us.
• Quandl is a collaborative project. Find out how you can get involved on our community page.

Quick Links: Vision | Sources | API | Packages | News | Features | Planned features | Help | Community

Statistics Sweden’s statistics are available for new services

From: http://www.scb.se/Pages/List____354067.aspx

Now companies and private individuals have access to Statistics Sweden’s statistics, which can be used for new products and services. This involves data from the Statistical Database that can be digitally transferred via a programming interface called an API.


APIs (Application Programming Interface) can be processed and used, for example, to build apps for smartphones or new web services. You can use tables or parts of tables from the Statistical Database at Statistics Sweden free of charge.

The ambition of the Swedish Government's digital agenda is to improve conditions for the reuse of public information from government agencies for both commercial and non-profit purposes. This is also regulated in the PSI Directive and the INSPIRE directive from the EU.

Bono - the Factivist on Statistics and Open Data

New U.S and World Population Clock from the Census Bureau

View THE CENSUS BUREAU’S NEW POPULATION CLOCK to see an interactive projection of the population, including the rate of births, deaths and net migration. Explore tables on the most populous countries in the world, as well as the most populous states, cities and counties in the U.S.

US Census Bureau Named a 2013 Computerworld Honors Laureate for Open Data API

From: http://blog.programmableweb.com/2013/04/04/us-census-bureau-named-a-2013-computerworld-honors-laureate-for-open-data-api/

Janet Wagner, April 4th, 2013

Computerworld has named the US Census Bureau a 2013 Honors Laureate for the development of the US Census Bureau API which was released in July 2012 and is the first-ever API released to the public by the US Census Bureau.

More......

Big Data and official statistics

From: http://www.bruegel.org/nc/blog/detail/article/1059-blogs-review-big-data-aggregates-and-individuals/#.UVL84BxhWCk

Big Data is relevant to the production, relevance and reliability of key official statistics such as GDP and inflation.

Michael Horrigan, the head of the Bureau of Labor Statistics (BLS) Office of Prices and Living Conditions, provides a definition that helps clarify where the border lies between Big Data and traditional data. Big Data are non-sampled data that are characterized by the creation of databases from electronic sources whose primary purpose is something other than statistical inference.

Read more....

UNSD - Big Data for Policy, Development and Official Statistics

From: http://unstats.un.org/unsd/statcom/statcom_2013/seminars/Big_Data/default.html

Friday Seminar on Emerging Issues Big Data for Policy, Development and Official Statistics 22 February 2013, 10:00 am - 1:00 pm, 3:00 - 6:00 pm Conference Rm. 2, Temporary North Lawn Building United Nations, New York

The theme of this year’s Friday Seminar on Emerging Issues is: “Big Data for Policy, Development and Official Statistics”. The advent of the Internet, mobile devices and other technologies has caused a fundamental change to the nature of data. Big Data has important, distinct qualities that differentiate it from “traditional” institutional data, in particular the timeliness of the data. If governments wanted to, they could already let Big Data (and the private sector) play a role in providing information on topics that are currently under the purview of national statistical offices. Should these NSOs change their business operations to take on the opportunities of using Big Data for official government purposes?  

Please find below the provisional programme, the Key Questions and the Concept Note on Big Data for Policy, Development and Official Statistics. We encourage you to carefully read the key questions and to consider what they would imply for the work of your office. We are looking forward to your participation in this year’s Friday Seminar on Big Data. 

Organizer: United Nations Statistics Division (UNSD)

Morning Session, 10:00 - 13:00
Chairperson: T.C. Anant, Chief Statistician, India
Panelists:	- Hal Varian, Chief Economist, Google (Video)
	- Matt Wood, Chief Data Scientist, Amazon Web Services
	- James Goodnight, Chief Executive Officer, SAS
	- Robert Kirkpatrick, Director, UN Global Pulse
	- Andrew Wyckoff, Director, Science, Technology and Industry, OECD
	Discussion
Demonstrations Session, 14:00 - 15:00
	- Howard Hamilton, Statistical Institute of Jamaica
	- Statistical Institute of Jamaica , Video
	- Matthias Helble, Universal Postal Union
	- SAS
Afternoon Session, 15:00 - 18:00
Chairperson: Brian Pink, Chief Statistician, Australia
Panelists:	- Gosse Van der Veen, Chief Statistician, Netherlands , Video
	- William Bostic Jr., Associate Director, US Census Bureau
	- Ki-Jong Woo, Commissioner, Statistics Korea
	- Pali Lehohla, Chief Statistician, South Africa
	Discussion
Background documents
1. Key questions
2. Concept note

Placing Big Data in Official Statistics: A Big Challenge?

From: http://www.cros-portal.eu/content/ntts-2013-programme

Slides: http://www.cros-portal.eu/sites/default/files//15A02_214_0.pdf

Monica Scannapieco1, Antonino Virgillito2, Diego Zardetto3
1Istat - Italian National Institute of Statistics, e-mail: scannapi@istat.it
2Istat - Italian National Institute of Statistics, e-mail: virgilli@istat.it
3Istat - Italian National Institute of Statistics, e-mail: zardetto@istat.it

Abstract

Big Data refers to data sets that are impossible to store and process using common
software tools, regardless of the computing power or the physical storage at hand. The
Big Data phenomenon is rapidly spreading in several domains including healthcare,
communication, social sciences and life sciences, just to cite some relevant examples.
However, the opportunities and challenges of Big Data in Official Statistics are matter of
an open debate that involves both statisticians and IT specialists of National Statistical
Institutes.
In this paper, we first analyse the concept of Big Data under the Official Statistics
perspective by identifying its potentialities and risks. Then, we provide some examples
that show how NSIs can deal with such a new paradigm by adopting Big Data
technologies, on one side, and rethinking methods to enable sound statistical analyses on
Big Data, on the other side.

Paper: http://www.cros-portal.eu/sites/default/files/NTTS2013fullPaper_214.pdf

Live webstreaming of NTTS 2013

If you're not able to attend NTTS 2013 in Brussels, you may still follow the conference online.

All sessions in the main auditorium (including the opening session, the closing session and all keynote speeches) will be transmitted via live webstreaming on the following addresses (separate address for each day):

5 March
http://scic.ec.europa.eu/streaming/index.php?es=2&sessionno=b994697479c5716eda77e8e9713e5f0f

6 March
http://scic.ec.europa.eu/streaming/index.php?es=2&sessionno=eb1848290d5a7de9c9ccabc67fefa211

7 March
http://scic.ec.europa.eu/streaming/index.php?es=2&sessionno=91e50fe1e39af2869d3336eaaeebdb43

‹ NTTS 2013 (5-7 March) - possible to follow via webstreaming up NTTS 2013 - satellite events 4 March and 8 March ›

IISD - UN Statistical Commission Seminar Discusses Big Data Challenges and Opportunities

From: http://uncsd.iisd.org/news/un-statistical-commission-seminar-discusses-big-data-challenges-and-opportunities/

IISD 26 February 2013: The UN Statistical Commission held a seminar on “Big Data for Policy, Development and Official Statistics,” which examined how the Internet, mobile devices and other technologies, such as sensor networks, have fundamentally changed the nature of data.

Big data has the potential to change the ways in which National Statistical Offices (NSOs) use data, especially as more quality and timely statistics become available. Big data differs from “traditional” data by its real time nature, as well as by speed, type and volume. The speed at which new data is created by a range of sources means that there have been large increases in the volume of data, in addition to shifts from a highly structured, centralized and easily managed nature to a loosely structured, highly distributed manner.

Read more....

Open Data: the state of the European Union

Open Data: the state of the European Union from ePSI Platform

How Big Data Can Ruin True Statistics - Storagecraft

FROM sTORAGECRAFT: FEBRUARY 13, 2013   CASEY MORGAN   NO COMMENTS

Big Data presents a lot of opportunities for information discovery. The world has begun creating billions of bytes of data, which can be analyzed and utilized for everything from marketing to scientific research. But as Voltaire said, with great power comes great responsibility.

People have always been able to manipulate data and create true, yet absurd statistics. But Big Data makes it even easier. According to a recent Wired article by Nassim Taleb, a risk engineering Professor at NYU, Big Data has brought cherry picking to an industrial level. Although researchers and Big Data analysts can now understand and use information in new ways, it’s also easier for them to misuse it in new ways. With Big Data and its vastness of information, statistical correlations can be found simply because of the size of the data sets and not necessarily because the correlations are genuinely valid.

More,,,,,,

RSSeNews - Hal Varian’s ‘Statistics at Google’ talk now available to view

From: January 14, 2013 News

Print PDF

At its conference last year, the RSS hosted a talk by Google’s chief economist, Hal Varian.

Footage of his talk is now available to view here:

In his talk, Hal outlined how statistics are used at Google in a number of areas, what kind of skills Google finds valuable, and what the future might hold for data analysis in technology-intensive industries.

The talk is one of a number of sessions at the conference filmed by the RSS. Footage of other sessions will be uploaded on the RSS conference website in due course.

OECD Announces Winner of Global Data Visualisation Competition

From: 09/01/2013 - The OECD today announced the winner of its first-ever global data visualisation challenge

The challenge, conducted in concert with Visualizing.org, was launched in September 2012 to solicit visualisations based on the OECD’s data-rich Education at a Glance report.  Entries from around the world focused on data related to the economic costs and return on investment in education — both for the individual and for society. Winners Krisztina Szucs and Mate Cziner from Hungary were chosen for successfully breaking down the complex interplay between costs and returns into a form that is easy to compare.[please see below]. Their entry takes a detailed look at public vs. private and men vs. women for selected countries, which can be changed. 

The judges were particularly impressed by the angled slope format of the visualisation, which encourages comparison between the upper-secondary and tertiary benefits of education. Szucs and Cziner were also lauded for their striking visual design, which draws users into exploring their piece.

Read more......

Global Observatory - Can Big Data From Cellphones Help Prevent Conflict?

From: http://www.theglobalobservatory.org/analysis/380-can-big-data-from-cellphones-help-prevent-conflict.html
by emmanuel letouzé , Thursday, November 08, 2012

Data from social media and Ushahidi-style crowdsourcing platforms have emerged as possible ways to leverage cellphones to prevent conflict. But in the world of Big Data, the amount of information generated from these is too small to use in advanced data-mining techniques and “machine-learning” techniques (where algorithms adjust themselves based on the data they receive). 

But there is another way cellphones could be leveraged in conflict settings: through the various types of data passively generated every time a device is used. “Phones can know,” said Professor Alex “Sandy” Pentland, head of the Human Dynamics Laboratory and a prominent computational social scientist at MIT, in a Wall Street Journal article. He says data trails left behind by cellphone and credit card users—“digital breadcrumbs”—reflect actual behavior and can tell objective life stories, as opposed to what is found in social media data, where intents or feelings are obscured because they are “edited according to the standards of the day.” 

Read more.....

What Happens When Big Data Meets Official Statistics? - Live Webcast

From: http://live.worldbank.org/what-happens-when-big-data-meets-official-statistics-live-webcast

• SHARE

Wednesday, December 19th 2012 
2.30 - 4.00pm ET - World Bank HQ 
02:30 – 04:00 ET or convert time

With the advent of the open data and big data communities, we’ve seen huge increases in the use of public data, and more importantly, potential opportunities to use new data sources and techniques - that are often faster and cheaper - to supplement, or even replace, some of the work of official statistics.

The question is, can this really be done? Can we apply the same statistical rigour to big data sources and techniques to help meet the goals of official statistics?

Two of the world's leading authorities on official statistics tackled key questions in their discipline:

• Can big data replace traditional official data sources?
• Does big data make classical statistical methods obsolete?
• How will social science research change with big data?
• What’s the future of official statistics in a big data world?

For background and further details, see Shaida's blog.

Moderator:
Shaida Badiee, Director, World Bank Data Group

Speakers: 
Paul Cheung, Director, United Nations Statistical Division (Download Presentation - PDF 2.8Mb)
Robert Groves, Provost, Georgetown University. Formerly Director, US Census Bureau