%% This is an example first chapter.  You should put chapter/appendix
%% that you write into a separate file, and add a
%% line \include{yourfilename} to main.tex, where `yourfilename.tex'
%% is the name of the chapter/appendix file.  You can process specific
%% files by typing their names in at the \files= prompt when you run
%% the file main.tex through LaTeX.
\chapter{Introduction}

The amount of digital information in the world has exploded in the
past decade. In 2005, about 150 exabytes of data were created - a
number that has been estimated to have reached 1200 exabytes in
2010~\cite{economist10}. Detecting patterns and extracting useful
information from this deluge of data can significantly benefit firms,
governments, and people by improving efficiencies, and innovating
products and practices, among others. Given the size of this data, it
is necessary to have appropriate tools that can facilitate the
organization and querying of data, which is often contained in
disparate locations. The Semantic Web has the potential to make such
large scale data manipulations possible.

The term Semantic Web encompasses the broad category of technologies
that are aimed at making the World Wide Web more machine-readable.
Some of these technologies include Resource Description Framework
(RDF)~\cite{rdfw3c}, RDF Schema (RDF-S)~\cite{rdfsw3c}, Web Ontology
Language (OWL)~\cite{owlw3c}, the SPARQL Query
Language~\cite{sparqlw3c}. Though the Semantic Web has led to open
data sources that can be queried, extended and re-used
easily~\cite{lod}, on-the-fly integration of data from multiple
heterogeneous and distributed data sources is still a
problem. Furthermore, the ability to perform such seamless
integrations while preserving appropriate security and privacy
policies has not even been addressed.

Several issues need to be addressed before such environments are
developed. For instance, an application developer/user would need to
know what data is present in each source and in which format, so that
appropriate queries may be formulated to obtain the necessary
information. Moreover, if a source's content has been modified, the
application developer needs to adjust the query accordingly to ensure
proper data retrieval.  One possible solution to address these
challenges is the application of the federation paradigm for database
systems~\cite{mcleod80fedarch}.

A federated query involves the concurrent search of multiple
distributed data sources. In a federation model, the access to data
sources is transparent. Under such a system, a user can submit a
single query to the federation engine, which then reformulates the
query to subqueries to suit the various data sources, executes the
subqueries, and returns the merged result. The lack of a shared model
for security and privacy~\cite{dig09secureswfed}, however, impedes the
transparency of the federated model because the federation engine is
unable to execute requests across many secure domains on the fly. This
means that most contemporary federations have static, previously
negotiated policies in place, which prevents dynamic integration of
data.

In this thesis, I present a novel architecture of a data integration
engine that provides secure SPARQL federation. The engine accepts a
single SPARQL query from a client, splits it up into subqueries based
on the information contained in the service descriptions of endpoints
it has in its possession, sends them off to appropriate SPARQL
endpoints, and returns the results to the client. Under the proposed
architecture, clients who use the federation engine are also required
to provide some security credentials, which are used to satisfy the
access policies of those endpoints that contain secure data. Such data
may include personally identifiable information(PII) of individuals
and trade secrets, among others.I also present my implementation of
this architecture, which includes provisions for integration with
Semantic Web-based policy reasoners\cite{cwm, air}, but does not
currently support complete policy enforcement.

\section{Thesis Overview}

The proposed system for SPARQL federation is organized in this thesis
as follows:

Chapter~\ref{ch:background} introduces some of the basic technologies
that makes query federation possible in a Semantic Web environment. A
basic overview of federated querying is provided and the differences
between relational and RDF databases are illustrated. Then, some of
the recent advances in federation functionality in Semantic Web
technologies is explored.

Chapter~\ref{ch:usecase} describes the application of a SPARQL
federation system in the context of government data sharing. When
organizations see the business case for expressing their data in
formats that suit the Semantic Web, they would begin developing data
systems that facilitate enterprise level data sharing. The presence of
such data sources would accelerate the spread of Semantic Web
Technologies on a truly Web scale. This detailed use case highlights
the many advantages of adopting SPARQL federation systems for the
Massachusetts Department of Education over traditional databases.

Chapter~\ref{ch:architecture} discusses the overall architecture of
the system and the design choices that were made during its
development. The various components of the system are described and
their features and limitations are identified.

Chapter~\ref{ch:implementation} details the implementation
of the design. The various software tools that made the design
possible are identified and their relevance is described. 

Chapter~\ref{ch:results} discusses the performance of the system
when presented with endpoints and queries with different
characteristics. The methodology for the selection of queries is
presented and the relevant metrics are illustrated.

Chapter~\ref{ch:summary} provides a summary of the contributions of
this work. The limitations of the system are discussed and future
areas for expanding on the work are identified.
