Pete Warden, the man who scraped 220 million Facebook public profiles, unveiled his Data Science Toolkit at GigaOM’s Structure Big Data to allow anyone to analyze big data on the cheap.
Analyzing “big data” doesn’t have to be expensive, and it got even cheaper, thanks to a new tool that automates the data analysis needed to make sense of massive amounts of data. Pete Warden, the man who became famous after he scraped 220 million public Facebook profiles last year, unveiled his Data Science Toolkit at GigaOM’s Structure BigData conference in New York City on March 23. The Data Science Toolkit allows anyone to do automated conversions and data analysis on large data sets, he said. |
Friday, March 25, 2011
Cloud Computing News: Facebook Scraper Launches Open Source Data Analysis Tool for “Big Data”
From eWeek.com
More.......
Monday, March 14, 2011
Apps for Development
http://appsfordevelopment.challengepost.com/challenges/78/submissions?page=1
Application Gallery 1-9 of 107 Submissions
Read moreChart Meister!
By AthmanEver needed a chart for your keynote presentation 3 minutes before you take the podium? Well... I have.. and am guessing that many have those moments too :DThis simple application allows you to use the magic of Google Data to quickly generate the charts you need and enablesyou to download those charts for use immediately.And it gets better... share the generated charts directly from the website via email, MMS messages or simple text messages*!!!!!! :DNeed more? How about a pys60 script that allows you to generate and download the charts directly from your prized nokia phone?Future updates will involve the ability to make you a cup of coffee and take the dog out for a walk while you are busy being the chart meister!!!! :D*some phones e.g. the Nokia E71 phones have the ability to link you directly to the web from the text message URL.
Read moreWhyqd Data Query Engine
By Gavin ChaitWhyqd is a data and knowledge query engine allowing you to create, visualise and share your own country risk analysis, rankings and ratings.Whyqd offers a straightforward solution to the process of organising, querying and conceptualising big data. It is designed very much as a working tool. Few special effects and a simple interface that presents you with an open query format. A shopping cart stores your choices and allows data segmentation.The sparklines which accompany all data presented in the system pack a great deal of information into a small space. The red line is the data for the country concerned. The grey region is the upper and lower limits of the scores of the countries you have selected as benchmarks. These visual references aid your understanding of the data context - what a particular number means next to the experiences of other places. A score above the median simply means "better than the benchmark"; not best, not even particularly good (if you're benchmarks are awful) - just better. Ditto "worse".
Data is presented as individual country reports or as rankings of all the countries in the index relative to the benchmarks and indicators you have selected.Whyqd was created by Gavin Chait, ofWhythawk, a small company which specialises in country risk analysis.Whyqd is designed to make data fun. To allow rapid comparisons, what-if speculative searching, and serendipitous connections. By allowing you to play, to find meaning and resolution in big data, I hope that you will enjoy analysis and produce better risk profiles.
Read moreAisha
By WHDIn September 2000, world leaders came together and agreed to reduce extreme poverty and increase quality of life. The so called Millennium Development Goals were born.
Thanks to international effort the standard of living in developing countries has increased since then. However, African countries that lay below the Sahara are at risk of not meeting the projected improvements.
With Aisha you can find out why the African people deserve our attention.Aisha tells the story of Africa. A story in which suffering and hope are represented in a way that is accessible for everyone. Discover African countries and their ongoing struggle with poverty and meet the people behind the numbers.http://wernerhelmich.nl/aisha
Read moreFind Correlation of The World
By cactusIs the world's population related with the world's forest area and CO2 emissions? What's the relationship between USA's energy use and USA's GDP growth? Will Japan's export influence China's industry? Well, let's find correlation of the world! Let's see who can find the most correlated indicator pair!A correlation coefficient is a numerical, descriptive measure of the strength of the linear relationship between two variables. Values for the correlation coefficient range between -1 and +1, with a correlation coefficient of +1 indicating that the two variables have a perfect, upward-sloping (+) linear relationship and a correlation coefficient of -1 showing that the two variables are perfectly related in a downward-sloping, (-) linear sense. A correlation coefficient of 0 demonstrates that the variables have no relationship, and are independent.About FindCorrelation- 1 This tool provide a interesting way to show indicator information for all countries/income levels/regions.
- 2 You can calculate correlation coefficient between any two indicators of any two countries/income levels/regions. Thus, this tool can help us to find some potential relations around the world, which can help to understand the world, reveal some underlying problems and achieve some Millennium Development Goals.
- 3 You can comment on each correlation, and post correlation information to Facebook. We can raise awareness of Millennium Development Goals by our own words.
Properties of FindCorrelation- 1 It provides a powerful desktop based on World Bank Data Widget API. Each indicator is shown as a widget in desktop.
- 2 It utilizes World Bank Data REST API to fetch information from World Development Indicators Dataset to calculate correlation coefficient.
- 3 It utilizes FaceBook Connect API to provide social functionalities. Indicator pair of most correlated will shown as top items. Wish more users involve in and compete to find correlation of the world.
Read moreFiveFiveComparison
By cmatovuDescriptionFiveFiveComparison helps you to compare performance of any five countries over a period of any five years on any one selected MDG performance indicator. It allows you to select the indicator, the five countries, and the five years. Basing on the selection, FiveFiveComparison produces a graph that visually shows the performance of these selected countries on the chosen indicator side by side and the trend over the selected years.Target AudienceFiveFiveComparison is designed for and can be used by any development stakeholder including but not limited to politicians, technocrats, researchers, scholars and ordinary citizens.ObjectiveFiveFiveComparison aims at arousing your interest and awareness of how various countries are performing on various indicators, motivating you to find out how and why, propel you to identify the lessons learnt and strategies that could be applied from one country to another to better the world, and make you take action.AdvantagesFiveFiveComparison is simple to use; does not require acquisition and installation of complex computer hardware and software; can be run both offline and online thus usable in countries with internet challenges; can be easily moved from computer to computer using simple single storage devices like CDs, flash disks, and even diskettes; and it does not require complex computer skills to use – requires only ability to follow the 5 steps instructions. The application works on any computer running MS Excel 2007.Technical InformationThe application is technically designed using MS excel (utilizing the World Bank MDG.xlsx data dated 1st October) and will run on any computer running MS Excel (preferably the 2007 version). It can be easily modified (with some extra MS Excel skills) to be used on older versions of MS Excel (and possibly other spreadsheets), utilize other different and future performance data, and include controls and security.Usage InstructionsStep 1 Data identification.Identify 1 indicator, 5 countries to compare and the 5 years over which to do the comparison.This is done by noting down their respective numbers from the 55lists (yellow tab).The countries and indicators are alphabetically arranged and numbered.Step 2 Data entry.Enter the numbers for the indicator and the 5 countries, together with the selected years in the data entry screen (the orange boxes). As numbers are entered, the application automatically displays the names.Step 3 Graph Analysis and printing.Click on the green 55graph tab.This displays the graph which can be printed.Step 4 Do more comparison.This is simply repeating steps 2 though 4.Step 5 Close/Exit the application.Close the normal way that an MS Windows application is closed. An option to save is available
Read moreMigrants moving money
By Roxana Torre & Trudy Davis"Migrants moving money" is an exploration on remittances: money sent home by migrants. It shows global movements of money by combining remittances data with migration.
Read moreMillennium Development Goal 2: Universal Primary Education
By Barry KaytonIn a world flooded by data, it's easy to neglect the deep thinking required to solve urgent problems. Data without analysis is useless. And analysis without data is disconnected from reality.Our application is built in Cognician, an Adobe AIR application. Cognician is a platform for building and sharing cogs. Cogs are collection of questions designed to make you think. We designed this cog specifically for NGOs, businesses or community members who are interested in helping achieve universal primary education in a specific region.The cog helps you think systematically through the many contextual factors influencing primary education. The questions range from a general overview of education to a worked example, based on a specific region you choose to focus on. The example enables you to spend time thinking about different contextual factors, including politics, community, environment, resources, learning and more. It refers to the World Bank data along the way.By bringing contextual factors into focus, the cog empowers you to develop relevant solutions to more comprehensively contribute towards achieving the Millennium Development Goal of universal primary education."Not everything that counts can be counted, and not everything that can be counted counts."Albert Einstein
Read moreMillennium Environmental Challenge
By AlexMThe Millennium Environmental Challenge is an application / game that puts users in the shoes of the director of a newly created organization called the International Environmental Council. Tasked with deciding the course of the entire planet's future, the player must learn about the Environmental Millennium Development Goals and their real-life applications and ramifications by playing and learning from the interactive activities. Furthermore, once all the decisions have been taken and the consequences are known, the player can go back and review his/her actions and change them in order to see the new outcome.
Read moreBlind Data
By Michael BenedictBlind Data is a tool for exploring data availability in the World Bank's WDI and GDF datasets. For any topic, subtopic, or individual indicator, the site shows where and for what time periods data is available.Coverage includes nearly the entire WDI and GDF: more than 4.5 million datapoints from 1157 indicators over 50 years, and all 123 indicators identified by the Bank as being relevant to the MDGs. Users can view data availability for the world as a whole, select specific countries and time periods to view, and compare availability between countries. Individual series' pages link to descriptions and raw data from data.worldbank.org.Blind Data provides quick, quantiative information about where the WDI and GDF are most complete, and where data is lacking. It also enables interesting comparisons of data availability between countries and regions.
Subscribe to:
Posts (Atom)