\chapter{Implementation}

\label{ch:implementation}

It is now time to describe how the design presented in chapter
~\ref{ch:architecture} was put into practice and the roles that
various languages and tools played in making the design come to
fruition. The functioning system consists of a user interface and the
Federation Engine back-end that processes the queries. The Web
Interface, described in ~\ref{sse:webform} serves as the user
interface. The Federation Engine is implemented (described in
~\ref{sse:engine}) using a combination of C/C++ and Python. The reason
for using two languages and a tool that allow them to work in unison
is explained in the following subsections. This implementation,
however, does not include the proof generation and checking, and the
overall policy enforcement mechanisms described in chapter
~\ref{ch:architecture}. The reasons for this missing functionality is
explained in ~\ref{sse:swig}.

\section{Web Interface}

\label{sse:webform}

Clients interact with the Federation Engine using a web-form (shown in Figure
~\ref{fig:webform}) that is written in Python. It has a text box, which allows
them to submit a query to the Federation Engine. The result of the query or an
explanation for the lack of results is displayed immediately below the query
box. The web-form also provides clients with a description of the types of data
that are available to the Federation Engine. A handful of sample queries, which
give those users who are simply interested in exploring the functionality of the
system, are made available as well.  

\begin{figure}
\begin{center}
\scalebox{0.4}{\includegraphics{images/webform.png}}
\rule[1pt]{12cm}{1pt}
\caption{User Interface}
\label{fig:webform}
\end{center}
\end{figure}

\section{Federation Engine}

\label {sse:engine}

The Validator, Map Generator, and Optimizer are implemented in Python. The
source descriptions exist as N3 files. The Mapper and Orchestrator modules are
implemented in C/C++. During the design stages of the system, the intention was
to implement the entire system in Python. However, the review of existing work
in the field led to the discovery of SWObjects - a Semantic Web Objects
Library. SWObjects provide general support for Semantic Web applications
including SPARQL Query, SPARQL Update, and rule-based query and data
transformations, among others~\cite{swobjects}. Because SWObjects had the
functionality of the Mapper and the Orchestrator, it was efficient to use it
rather than reproduce its capabilities from nothing.

The ease of implementing the other modules in Python compared to C/C++ made the
language of choice for the rest of the design obvious. However, such a
dichotomous setup meant that a way to integrate the components in different
languages had to be devised. After comparing a number of tools that integrates
Python and C/C++, Simplified Wrapper and Interface Generator (SWIG)
was chosen.

\subsection{SWIG}

\label{sse:swig}

SWIG is a software tool that allows developers to interface programs
written in C/C++ with higher-level languages including Perl, PHP, and
Python. With SWIG, programs written in a target language are able to
access objects and methods in the C/C++ code as if they were part of a
native library~\cite{swig}. After comparing SWIG with other tools such
as Cython and SIP, SWIG was found to have relative advantages against
them across two categories. First, SWIG provided better support for
native C features than SIP. Second, SWIG allowed the SWObjects code to
be used with other high-level language such as Java, Lua, and
PHP. While this capability was not necessary for the Federation
Engine, the ability to interface SWObjects and Java, for instance, is
of great use for the Semantic Web Health Care and Life Sciences (HCLS)
Interest Group~\footnote{http://www.w3.org/2001/sw/hcls/}. As the main
developer behind SWObjects, who is a member of HCLS, was a
collaborator on this thesis work, the use of SWIG would turn out to be
mutually beneficial. Hence, SWIG was chosen as the tool for
integrating the existing C/C++ code with the new code that would be
implemented in Python.

Figure ~\ref{fig:impl} illustrates how SWIG facilitates the use of
SWObjects - implemented in C/C++ - for the rest of the Python
code. SWIG generates a Python wrapper that encapsulates the SWObjects
code. The SWIG processing is quite time consuming and computationally
taxing. However, this takes place offline and once SWIG-processing is
done, SWObjects.py exists as a standalone Python module than can be
imported into any other Python module. No further SWIG-processing is
required at run time.

\begin{figure}
\begin{center}
\scalebox{0.4}{\includegraphics{images/python_swig_c.pdf}}
\rule[1pt]{12cm}{1pt}
\caption{Python/SWIG/C}
\label{fig:impl}
\end{center}
\end{figure}

Although without SWIG a lot of code might have had to be rewritten in
Python, in hindsight the decision to use SWIG might not have been the
ideal one. While, it has the capacity to make most features of C and
C++ available in Python, there was often little information available
for using it for purposes beyond the very simple use cases. This lack
of detailed documentation meant that it took much longer than expected
to make SWIG operational for this project. As a result, there was no
time left for the implementation of the policy verification and proof
generation mechanisms. Hence, in the current implementation, the
Federation Engine can only query open SPARQL endpoints that do not
have any access restrictions. However, implementing the policy
reasoning framework should not be difficult as the system contains
hooks for calling proof generation and checking components.

\subsection{Fyzz and RdfLib}

The Python implementation would not have been possible without two
libraries, which have been developed for Semantic Web
Technologies. Fyzz\footnote{http://www.logilab.org/project/fyzz} is a
Python parser for SPARQL. Fyzz was used for implementing the
Validator. Because the latest version (0.1.0) of Fyzz is compatible
with the SPARQL 1.0, it was necessary to update the Fyzz SPARQL
grammar to include the SPARQL 1.1 features that the Federation Engine
uses. Specifically, this meant updating the grammar so that Fyzz can
handle SERVICE keywords.

RdfLib\footnote{http://www.rdflib.net/} is a Python library for
working with RDF. It includes parsers and serializers for RDF/XML, N3,
NTriples, Turtle, Trix and RDFa. For the purposes of implementing the
Federation Engine, the N3 parser in RdfLib was used to extract
information from the source description files.
