\chapter{Summary}

\label{ch:summary}

The ability to query distributed heterogeneous data sources and
aggregate information across them is a key requirement for realizing
the full potential of the Semantic Web. In this thesis, I presented
the design, implementation, and a relevant application of a secure
SPARQL federation engine that takes a big step in that
direction. First, I detailed the current research in the field and the
existing technologies that made this engine possible. Then, I
discussed a specific use case for this system, which illustrated the
importance of this system for a large public sector agency. After
this, I described the architecture of the system, paying particular
attention to design choices. The system was implemented using a
variety of software tools, including a C/C++ wrapper for Python, which
although very useful, was a major cause for the lack of security
mechanisms in the final version of the system. The Federation Engine
underwent a series of tests - both at the system level and at the
level of novel components, which showcased the engine's capabilities
and a few limitations. Section ~\ref{se:contributions} highlights the
key contributions of this thesis work and section ~\ref{se:futurework}
identifies a few areas of research that can improve the performance
and features of this novel system.

\section{Contributions}

\label{se:contributions}

\begin{itemize}
\item {\bf Designed and implemented an end-to-end SPARQL Federation Engine} \\
  Although the Semantic Web had made many advances over the past
  decade, there did not exist an integrated secure environment to
  perform federated queries to mash up data from distributed
  heterogeneous data sources. I designed the architecture for a secure
  Federation Engine that can aggregate information from multiple
  SPARQL endpoints, while adhering to appropriate usage policies. I
  also implemented the said architecture, albeit without the security
  layer, by using existing Semantic Web tools such as the Fyzz and
  RdfLib Python libraries, and SWObjects - a C/C++ Semantic Web
  objects library.

\item {\bf Developed query optimizer that facilitates efficient query execution}\\
  I also developed an Optimizer that reorders subqueries, which are
  directed at different endpoints, to ensure efficient and complete
  query execution. The Optimizer depends on the endpoint statistics
  based cost model described in ~\cite{bernstein07optarq}. Without the
  Optimizer, all queries, but the ones with very low complexity, would
  fail to execute in practical settings. This is because most SPARQL
  endpoints that exist today have a low upper bound on the number of
  results they would return to any query to prevent the exploitation
  of system resources. Given this status quo, subqueries, which have
  low selectivity, if executed first (as would be the case without the
  Optimizer) would return incomplete or NULL result sets.

\item{\bf Initiated the Use and Integration of SWIG into the SWObjects project}\\
  SWObjects is a large open source C/C++ code base that provides
  general support for Semantic Web applications including SPARQL
  queries, SPARQL updates, and rule-based query and data
  transformations, among others. Research Groups, notably the Semantic
  Web Health Care and Life Sciences Group (HCLS), is interested in
  using SWObjects library for many of their Web 3.0
  applications. However, HCLS uses Java for their research
  activities. In fact, many research undertakings these days use
  higher level languages, such as Java and Python. My thesis work
  spearheaded the inclusion of SWIG - a C/C++ wrapper for higher level
  languages - in the SWObjects project so that the Federation Engine
  could make use of some of the features of SWObjects. The work done
  on interfacing SWIG and SWObjects is invaluable for HCLS' and other
  projects that depend on the SWObjects library.

\end{itemize}

In addition to the Python/C++ back-end that facilitates query
federation, this research has also produced a web-form that users can
use to run a custom query or a range of queries, that exhibit the
Federation Engine's capabilities. Also integrated into the end-to-end
system is a framework that makes available to a user the various time
metrics associated with the execution of a federated query.

\section{Future Work}

\label{se:futurework}

Although the prototype of the Federation Engine breaks new ground on
many fronts, a number of features and improvements can better the current
state of the system. The first obvious feature addition is the policy
reasoner and checker that were described in chapter
~\ref{ch:architecture}. The Proof Generator and Checker components, as
described in ~\cite{kagal08using} and ~\cite{kolovski05towards},
respectively, and implemented as part of the Policy Aware Web project
completed by DIG, may easily be integrated into the Federation
Engine. The fact that both systems are implemented in Python should
make this integration seamless.

Second, advances need to be made to automate the generation of source
descriptions. For the purposes of the Federation Engine, source
description files had to be manually created using the results of
queries, which were formulated to obtain predicate and statistical
information, issued to each individual endpoint. This process was
cumbersome, and, moreover, impractical for endpoints that have a large
number of predicates. A tool, which can automatically generate the
source description files using the results of a number of
predetermined queries, would facilitate easier registration of new
endpoints and updates of existing ones. The end result of developing
and issuing such a tool would be an increase in the range of queries
the Federation Engine can provide results for.

Third, for demonstrating the capabilities of the Federation Engine, four
SPARQL endpoints were set up with subsets of DBPedia data. However, the
practical relevance of the engine would become much more evident once
more persistent SPARQL endpoints, whose Graphs overlap, are present on
the Web. These endpoints must also provide easy interfaces for a
tool such as the automatic source description generator to obtain
relevant predicate and statistical information.

Fourth, in the current implementation, mapping of subqueries to
endpoints are done before optimization. The performance of the
Federation Engine may improve dramatically if the Mapper is
implemented as a sub-component of the Optimizer. For example, in such
an arrangement, mapping to a particular endpoint maybe avoided
altogether if the optimizer determines that the cost of querying that
endpoint with a particular subquery is prohibitively high.

Fifth, much work has been done on optimization techniques for
relational database federations. Until either those techniques are
made applicable to Semantic Web data models, or current research into
SPARQL optimization progresses further, the Federation Engine would
not be able to handle very complex queries, the successful execution
of which are essential for the SPARQL federation paradigm to become a
key player among data integration technologies.

Lastly, as with most applications on the web, the user base of the
Federation Engine would grow dramatically with advancements in natural
language processing. The ability of clients to issue natural language
queries and obtain results from a large number of endpoints could
considerably increase the mass appeal of the Semantic Web.

