Massive Data Aggregation with Perl http://www.perl.com/pub/a/2005/05/05/aggregation.html
This article is a case study of the use of Perl and XML/RDF
technologies to channel disparate sources of data into a
semi-structured repository. This repository helped to build structured
OLAP warehouses by mining an RDF repository with SAX
machines. Channels of data included user-contributed datasets, data
from FTP and HTTP remote-based repositories, and data from other
intra-enterprise based assets. We called the system the 'Kitchen
Sync', but one of the project's visionaries best described it as akin
to a device that accepts piles of random coins and returns them sorted
for analysis. This system collected voter data and was the primary
data collection point in a national organization for the presidential
campaign during the 2004 election.
