Placing Big Data in Official Statistics: A Big Challenge?

From: http://www.cros-portal.eu/content/ntts-2013-programme

Slides: http://www.cros-portal.eu/sites/default/files//15A02_214_0.pdf

Monica Scannapieco1, Antonino Virgillito2, Diego Zardetto3
1Istat - Italian National Institute of Statistics, e-mail: scannapi@istat.it
2Istat - Italian National Institute of Statistics, e-mail: virgilli@istat.it
3Istat - Italian National Institute of Statistics, e-mail: zardetto@istat.it

Abstract

Big Data refers to data sets that are impossible to store and process using common
software tools, regardless of the computing power or the physical storage at hand. The
Big Data phenomenon is rapidly spreading in several domains including healthcare,
communication, social sciences and life sciences, just to cite some relevant examples.
However, the opportunities and challenges of Big Data in Official Statistics are matter of
an open debate that involves both statisticians and IT specialists of National Statistical
Institutes.
In this paper, we first analyse the concept of Big Data under the Official Statistics
perspective by identifying its potentialities and risks. Then, we provide some examples
that show how NSIs can deal with such a new paradigm by adopting Big Data
technologies, on one side, and rethinking methods to enable sound statistical analyses on
Big Data, on the other side.

Paper: http://www.cros-portal.eu/sites/default/files/NTTS2013fullPaper_214.pdf