\chapter{Public Education in Massachusetts: An Ideal Application}

\label{ch:usecase}

With the emergence of the Web, technologists, policymakers, and citizens have praised the
ability of information systems to transform governments. Various entities have enumerated
the benefits of e-government systems. These fall into the broad categories of benefits to
citizens, businesses, and government agencies.

As e-government systems promote easier access to government
information, citizens would have a better customer service experience
when interacting with government agencies. The data.gov and
data.gov.uk initiatives by the government of the United States and the
United Kingdom, respectively, are prime examples of services providing
easier access to government information. Businesses would save on
operating costs as paper-based processes are eliminated and government
information is accessed more efficiently. The agencies that employ the
systems would also see cost savings and increased efficiencies as well
as increase in morale among agency employees, which feeds back from
better customer service~\cite{oregon10egovben, mass10egovben}.

To achieve the full benefits of e-government systems, it is necessary to share information
between different government systems. Public policy issues are often spread across
different agencies and it is the case that different entities have to marshall their
resources to solve a particular problem. For example, leukemia rates among the graduates
of River Valley High School in Marion, Ohio increased 122 percent between 1966 and
1992. However, it took almost 30 years for the community to establish a hypothesis -
leaking of cancer causing chemicals from a World War II bomb-making plant which used to be
housed at the school site - and collect enough evidence to prove it. Had there been
systems that allowed interoperability and sharing among Department of Defense (DoD),
Census Bureau, and epidemiological information systems, the correlation process would have
been easier and taken much less time, and possibly resulted in saving
lives~\cite{landsbergen01govinfosystems}.
 

\section{Interoperability and Data Sharing}

In the last 15 years, much research has been done in the area of
interoperability between disparate government information systems to
enable better collaboration between
agencies~\cite{landsbergen01govinfosystems,
  dawes96agencyinfoshare,andersen91govinfo}. Figure~\ref{fig:informationsharing}
depicts the theoretical model of inter-agency information sharing that
Dawes laid out in ~\cite{dawes96agencyinfoshare}. The impetus for
these efforts during the last decade was the terrorist attacks of
September 11, 2001 and the belief that the tragedy might have been
avoided had there been better information exchange between various law
enforcement agencies. The \emph{Accountable Information Usage in
  Fusion Center Information Sharing Environments} project at the
Decentralized Information Group (DIG) at MIT-CSAIL attempts to build
policy tools that enable seamless information flow between Federal,
State, and Local law enforcement agencies, while assuring that such
sharing adheres to privacy and securing policies~\cite{dig09dhs}. The
goal of this thesis work is not to develop policy tools; rather to use
those tools in combination with others to develop a well integrated
data federation environment. Data Sharing and Interoperability,
moreover, has applications and benefits that extend far beyond the
national security and law enforcement realms.

\begin{figure}
\begin{center}
\scalebox{0.42}{\includegraphics{images/intraagency_information_sharing_model.pdf}}
\rule[5pt]{12cm}{1pt}
\caption{Theoretical Model of Inter-agency Information Sharing}
\label{fig:informationsharing}
\end{center}
\end{figure}

\subsection{Benefits}

Interoperability and the information sharing that results from it has the potential to
make agencies more effective, efficient and responsive. Although, many federal agencies
were created to solve public policy challenges in their purview - education, environmental
protection, etc., policymakers increasingly understand that many of the social and
regulatory problems do not conform to the jurisdictional boundaries of a particular
organization. These problems, and therefore effective government, require integrated
policy approaches that makes use of information, knowledge, and technology from a variety of
agencies and divisions~\cite{landsbergen01govinfosystems}.

Once information is aggregated in electronic sources, data manipulation and duplication, which are integral to find correlations and causalities, 
becomes much easier. Such sources have the ability to reduce the paperwork burden on both
governmental and non-governmental actors that interact with the
state~\cite{landsbergen01govinfosystems}. In the same vein, transaction costs are reduced,
thereby bringing down overall costs and possibly increasing
participation~\cite{dawes96agencyinfoshare}.

Finally, the availability of disparate information at the fingertips of public servants
means that governments would act faster to identify problems and respond to
them. Moreover, such increases in efficiencies could establish a cadre of entrepreneurial
bureaucrats, who are less bound by rigid policies and have the flexibility to tailor the
processes to reach goals, the completion of which they are held responsible
for~\cite{landsbergen01govinfosystems}.

\subsection{Challenges}

\label{sse:challenges}

While the aforementioned benefits makes a strong case for interoperability, a number of
challenges have to be addressed for the full promise of data sharing to be realized. Andersen and Dawes
in ~\cite{andersen91govinfo} identify these challenges into the broad categories of
political, organizational, economic, and technical issues.

Political obstacles include privacy, ambiguity about statutory authority, and the openness
to public scrutiny. Privacy includes the obvious threat of data aggregation by Big Brother
governments~\cite{landsbergen01govinfosystems}. In addition to establishing legal
frameworks that protect citizens' privacy rights, citizens must also have faith in the
government for interoperable systems to become politically
viable~\cite{harmon98protpersonal}. Moreover, federal and state agencies are bound by the
statutes that authorized their establishment. Under these tight constraints, an important
question for an agency is if and under what conditions it can share information with
its sister agencies. Those agencies that are hesitant to share would require explicit
regulatory direction to divulge information that they
possess~\cite{dawes96agencyinfoshare}. Once again, appropriate legal frameworks need to be
developed for sharing to become ubiquitous. However, even with appropriate legal
frameworks, agencies that wish to withhold certain information may choose to interpret the
legislation differently from a partner agency. Lastly, the data that is made easily
available through interoperability might cause the public to increase their scrutiny on
the agencies that shared the information. This outcome is a strong disincentive for
sharing.

In the organizational realm, an agency must trust the veracity of the information it gets
from another. For instance, an agency might not be due diligent in ensuring the accuracy
of information it collects, if it is not critical to its core mission. However, for data sharing to be a
success, each interoperating agency must have a high degree of confidence in the
information that it did not directly solicit. Moreover, the fledgling state of the
development of interoperable systems means that agencies lack the expertise to share the
information they have and/or are unaware of the opportunities that exist for sharing. To
complicate matters further, the agencies often lack the financial resources to secure
interoperable systems, which are often state-of-the-art and have high costs. Moreover,
cash-strapped agencies often use low-bid procurement methods for information systems,
which fail to take into account the life-time impact of such systems. These are
significant economic obstacles to interoperability~\cite{landsbergen01govinfosystems}.

Last, but not least, there are a number of technical challenges. Prime among them are
issues of hardware and software incompatibilities. While there may be workarounds, the
cost of them might be prohibitive. Moreover, when contractors are used for system
development, intellectual property rights of the contractor might prevent the contracting
agency from not utilizing all the features of the system. Finally, a key technical barrier
to interoperability is the difference in data standards. It would be difficult to connect
disparate datasets if agencies have different data definitions. While these definitions
could be changed for the purpose of integrating diverse systems, the financial and legal
barriers of doing so might be too high~\cite{landsbergen01govinfosystems}. Usage of RDF
and other Semantic Web technologies described in section ~\ref{se:semweb} could aid in
integration while allowing each agency to customize its data structures for local needs. 

Section ~\ref{se:DESESPARQL} lists some of the ways in which these
challenges may be mitigated in an environment that uses SPARQL
federation for data sharing.

\section{Public Education in Massachusetts}

Education is an important public sector service where interoperability can have tremendous
benefits. In Massachusetts, the Department of Elementary and Secondary Education
(DESE)\footnote{Note: Massachusetts has had many reorganizations of the state level education
administration in the last decade. In this thesis, the term DESE is used to identify the
Department of Elementary and Secondary Education as well as its predecessors}
is responsible for the education of the approximately 550,000 children in the state's public
schools, which are located in 391 school districts. Its mission is \emph{To improve
the quality of the public education system so that students are adequately prepared for
higher education, rewarding employment, continued education, and responsible citizenship.}
It has as one of its six primary goals the provision of timely, useful information to
stakeholders~\cite{chester08welcome}. To achieve its mission and goals, it is important
for the DESE to track the progress of students as they advance through the
grades. Moreover, it is necessary to address the needs of children in early childhood and
in the post-secondary years, when they are not in the purview of the DESE. Without such
attention, we would lack an active citizenry that sustains a vibrant democracy and an
educated working-age population that can grow our knowledge-based economy in a globalized
world.

\begin{figure}
\begin{center} \scalebox{0.42}{\includegraphics{images/MA_data_sharing_model.pdf}}
\rule[5pt]{12cm}{1pt}
\caption{Massachusetts DESE Data Sharing for Accountability}
\label{fig:madatasharing}
\end{center}
\end{figure}

\subsection{The Need for Better Information Sharing}

Since the passage of the Education Reform Act of 1993, Massachusetts has mandated that all
students who are educated with Massachusetts funds participate in standardized
Massachusetts Comprehensive Assessment System (MCAS) testing~\cite{dese10rqmts}. The tests
serve to help improve curriculum and instruction, evaluate student, school, and district
performance against state defined standards, and determine student eligibility for the
Competency Determination requirement in order to award high school diplomas. Moreover,
since 2002, the MCAS tests in Mathematics and English Language Arts (ELA) in grades 3-8
and 10 have satisfied the testing requirements of the No Child Left Behind Act (NCLB) -
the latest reauthorization of the Elementary and Secondary Education Act of 1965. The goal
of NCLB is to ensure all students in America's public schools are proficient in
Mathematics and ELA by the year 2014. Increased accountability means that
the distribution of education dollars from Washington and Beacon Hill to schools and
districts is contingent upon their students' performance on the tests.

To satisfy the achievement and accountability requirements of the federal legislation,
Massachusetts is required to conduct yearly assessments in the aforementioned grades, ensure
adequate yearly progress (AYP), and establish that the educators satisfy NCLB's highly
qualified teacher (HQT) requirements. A local education authority has to ensure that
schools make AYP at the school level and also for specific subgroups of students. The HQT
requirement states that teachers must be qualified in the content area they are teaching
and that parents be informed if someone who is not qualified is teaching their
child~\cite{sopko07nclbregs}.

In this three-pronged set up, even if one leaves aside the supplementary federal and state
legal requirements, the need for data sharing is obvious. A state agency has to administer
the necessary tests and aggregate the results. Then it has to cross-reference this testing
data with the student demographic data, which the schools and school districts have, to
perform the AYP calculations. Finally, the DESE has to have access to the data bank that
contains the educator data - licensure information, educational background, etc. - to meet
the HQT requirements. The ability to easily and efficiently integrate these disparate
datasets would not only help the state meet NCLB guidelines but might also improve
educational outcomes.

\subsection{Where We Were Then and Where We Are Now}

Massachusetts was one of the first states to institute standards based public education
reform. The graduating high school class of 2003 was the first one that had to satisfy the
MCAS graduation requirement by scoring at the proficient level on the 10th grade
Mathematics and ELA tests. In the early days, student demographic data was collected on
the MCAS test booklets. As such, DESE was responsible for amassing student non-test data
as well the assessment results. The test booklets now contain unique labels for each
student so that test administration is made easier on the part of the schools and the
districts. The DESE assigned each student a unique ID, called the State Assigned Student
ID (SASID), to integrate the demographic and assessment datasets. In the first generation system, the data was
a large flat file from which reports were run for accountability measures. Later on, even
bigger flat files were created to analyze the performance of students that graduated
between 2003 and 2005. Systems were then developed (the data was stored in an Oracle
relational system) to automatically merge data and create reports based on a series of
business rules. Statisticians at the DESE were responsible for data curation and the
accuracy of the data.

In 2003, there were 32 characteristics that made up the student profile, which have been
since expanded to 52.  Periodically, DESE updates the list of characteristics based on its
needs. Currently, the school districts are responsible for collecting the student
demographic data and reporting it to DESE. Each district is responsible for securing its
own data systems from the vendor of its choice. As there is myriad of systems that
exists across the Commonwealth, the school districts are given two to three years to
upgrade their systems to have the updated list of characteristics when changes are made.

The Data Warehouse, which is in Phase II and well beyond the flat files, is a Cognos
Centric system. It is built as an enterprise education data warehouse.It works well in
medium sized school districts, but is not ideal for state-wide data processing and
analysis. According to Mr. Robert Bickerton, Associate Commissioner at DESE,  various stakeholders - parents, teachers,
policymakers, etc. - could \``make a cup of tea, maybe they go out and have dinner,
maybe they could come back the next day, maybe they give up,'' while they wait for the
reports. As a result, the agency is attempting to reengineer the system to make it more
efficient. The warehouse aggregates data from school districts three times/year and the
educator data is updated once a year. Plans to make the teacher data updates happen as
frequently as the student data is in the works. Moreover, proposals have also been made to
collect data on student and teacher assignments to classes so that class level, in
addition to district and state level, analyses may be performed.

Initially, DESE was only concerned about the scaled scores of students and the levels the
scores put them into - advanced, proficient, and needs improvement - for the purposes of
accountability reporting. However, in phase II the agency uses an enhanced progress
analysis framework called the Growth Model to track student progress. The Growth Model is
a student growth percentile model. It is a regression model that creates a cohort of like
performing students from all across the state, which is floating and changes from year to
year based on test achievement, and not just the school or the district. Starting with a
baseline assessment in grade 3, students are compared against their peer group in each
subject category. In this system, student performance is not simply the scoring level, but
a combination of the level and the student's place in her peer group. The percentile for a
school or a subgroup within a school is the mean of the percentile values of the students
that make up the group~\cite{bickerton10interview}.

Even before the growth model, it was easy to see that data sharing was essential for
meeting the NCLB reporting requirements. However, the floating cohorts that are
made up of students from any school/district in the state and the need to do performance
comparisons on a subgroup basis means that without appropriate interoperability and data
standards, progress measurements would be difficult, if not impossible.

\section{A Secure SPARQL Federation for the DESE}

\label{se:DESESPARQL}

The data sharing and interoperability challenges that the DESE faces could be mitigated by
using a secure SPARQL federation engine, the architecture of which is described in chapter
~\ref{ch:architecture}. Such a system, would depend on various SPARQL endpoints that house
the information that the federation engine uses.

First, the use of RDF for data representation would make the data collection and storage
more efficient. The characteristics that make up a student's demographic profile are
updated periodically to satisfy the needs of the agency for reporting and research
purposes. Storing this data in tables in relational databases means that all data would
have a rigid structure and the database has to be re-architected to make these
changes. With RDF, each student's profile could have a unique graph structure. Districts
could easily add on new characteristics when the state requests them and choose to either
delete the ones that are no longer needed or keep them for the districts' own tracking
purposes. Such changes would not require the two or three year time frame that the
districts are allotted now, which could lead to faster feedback. Moreover, with such a
design, a school can choose to append additional characteristics that are not necessary
for state reporting requirements but could provide individualized assistance to students.

The state could specify ontologies that districts have to use to describe the data that is
reported to the state. Alternatively, the districts could create ontologies that are most
appropriate for their day-to-day use to describe the student and teacher data in their
databases. Under such a set up, the district would provide the state with a detailed
source description\footnote{Source descriptions are explained in much more
detailed in chapter ~\ref{ch:architecture}}. With appropriate ontology mapping tools, the
state and district level systems can be made to interoperate seamlessly.

The availability of a secure federation engine would eliminate the
need for the transfer of large amounts of transfer between various
databases. The student data can remain on the district servers, and
the education licensure information can remain on the DESE Licensure
Office's databases. When analyses have to be generated for research or
federal reporting purposes, they can be generated by querying
the federation engine, which would integrated data from the different databases.

For instance, Mary, an MCAS analyst at the DESE can log in to the
federation engine with her credentials to obtain the data necessary to
run a performance report for a school. The credentials should allow
her access to two data sources to perform a successful query on the
federation engine - the student/educator demographic data at the
district level and the assessment data that is stored on the DESE
servers. For example, let us assume she is interested in the
performance of third grade students of Hispanic origin in Horace Mann
Elementary School in Westtown, Massachusetts. She would issue a query
that looks similar to the one in Listing ~\ref{lst:sample}. She would
need to provide appropriate credentials to ensure that she has access
to the parts of the DESE database that she is querying.

\begin{center}
\begin{lstlisting}[caption={Sample Query},label={lst:sample},float]
PREFI DESE: <http:www.mass.gov/dese#> 
SELECT ?student ?mscore ?escore 
WHERE 
{ ?student DESE:id ?sasid .  
           DESE:3rd_math_score ?mscore .  
           DESE:3rd_ela_score ?escore .  
?sasid     DESE:grade_level \``3'' 
           DESE:school_name \``Horace Mann'' 
           DESE:town \``Westtown''
           DESE:race \``Hispanic'' 
}
\end{lstlisting}
\end{center}

SPARQL federation is also relevant in the future of education in Massachusetts, and possibly the entire United States. The Massachusetts Executive Office of Education, which is the executive branch agency that is responsible for all levels of education in the state, has plans to observe the
progress of students from early childhood, to post-secondary, and eventually employment
and wage data to analyze the full impact of the Commonwealth's educational institutions
and practices. Such observations are necessary because in the 21st century economy, people
constantly have to retool their skill set by engaging in lifelong education. This is
evident in the fact that enrollment in adult education programs exceed the rolls of
public school students in the Commonwealth~\cite{wright10interview}. To do this, the
federation engine would need to have among its endpoints SPARQL endpoints that contain
information from the Department of Early Education and Care, the Department of Higher
Education, the Massachusetts Department of Revenue, and others. These agencies would have
to develop appropriate ontologies and source descriptions and register with the
federator. Each of those endpoints would also specify usage policies that contain access
restrictions for users that access the endpoint via the federation engine. The federation
engine would be responsible for reasoning over user credentials and policies to ensure
that only those users, who have the privileges, access any particular endpoint. Furthermore, the
existence of such policies and reasoning would help convince skeptics at these sister
agencies that the data that they expose are only made available to those individuals to
whom they give access to via inter-agency agreements, which would be made out-of-band.

From a financial perspective, recent developments suggest that there would be adequate
resources available to upgrade to a system like this. DESE has received grants, including
from the United States Department of Education's Race to the Top funds, to pursue
initiatives like the Schools Interoperability Framework (SIF) - a standards framework to
which many of the vendors that the districts use are increasingly adhering
to~\cite{bickerton10interview}.

\subsection{Potential Obstacles}

While a secure SPARQL federation engine can boost interoperability among various systems,
it is not a silver bullet by any means. There are a number of potential obstacles. 

First, Semantic Web technologies are in the early stages of development. Therefore the
research literature on them is not as vast as their relational counterparts. Specifically,
over thirty years of work has been done on relational database techniques, such as query
optimization. Combined with the additional size of data that comes with the expressivity
that RDF provides, the fledgling state of SPARQL query optimization means that integration
of large education datasets (records for approximately 550,000 students and 80,000
educators~\cite{bickerton10interview}) would be painfully slow.

Second, in theory, school districts could use ontologies that best accommodate their
needs. However, until data source to domain level mapping techniques, such as the one
described in ~\cite{kolas07queryrewrite} become more robust, DESE might have to
prescribe ontologies for all districts to use. This means that a secure SPARQL federator
would do no better than the proposed Schools Interoperability
Framework~\cite{bickerton10interview}.

Moreover, a number of non-technological challenges outlined in ~\ref{sse:challenges} are
pertinent to DESE.  First, as Sharon Wright, the Chief Information Officer in the Executive
Office of Education, pointed out, civil servants are reluctant to bring on board better
technology because of the fear of finding information that they do not
like. If better data organization and management help gleam important insights, it would
mean that policymakers would have to go beyond their current responsibilities to rectify
the new found problems. 

Second, currently there does not exist appropriate legal frameworks to facilitate
cross-agency data sharing that would result in longitudinal observations of an individual
from early childhood to adulthood and beyond. The educational authority in Massachusetts
is dispersed across various agencies, unlike in other states such as Florida, where one
executive is legally in charge of all levels and facets of education and training, and has the right to data across the spectrum. Without those frameworks, the Department
of Early Education and Care, for instance, could refuse access to the data they possess,
under the interpretation of a particular statute. For instance, the major impediment to
conducting a longitudinal evaluation is the disagreement over the meaning of section 37 of
the Family Educational Rights and Privacy Act\footnote{The full text of this section is in
  Appendix ~\ref{ch:ferpa}.}~\cite{wright10interview}.

Lastly, as with any large scale data integration effort that the government undertakes,
there is fear of Big Brother using the data to turn society into an Orwellian
nightmare. Until appropriate safeguards are put in place and the public is
satisfied by the effectiveness of those safeguards, elected officials and bureaucrats
would face a tough sell in transitioning to a system which has as its integral component a
secure SPARQL federation engine.
