Architecture of Accountable Systems

28 August 2008
Bergen University Department of Information and Media Studies

Daniel J. Weitzner
Decentralized Information Group
MIT Computer Science and Artificial Intelligence Laboratory

These slides: http://dig.csail.mit.edu/2008/Talks/0827-account-arch-bergen/

Talk Overview

  • Motivation
  • Information Accountability Defined
  • An illustrative scenario
  • Technical Approach
    • Accountable Systems Architecture
    • AIR Policy Language and Dependency tracking
    • Explanation generation
  • Respective Roles of Technology and Law


Jerrigan and Mistree (2007), unpublished research

Traditional Computer and Network Systems Understanding of Privacy

“The term “privacy” denotes a socially defined ability of an individual (or organization) to determine whether, when, and to whom personal (or organizational) information is to be released.”
Saltzer and Schroeder, The Protection of Information in Computer Systems Communications of the ACM 17, 7 (July 1974).

Privacy Re-Defined - Saltzer and Schroeder revisited

Privacy is the claim of individuals, groups, or institutions to determine for themselves when, how, and to what extent information about them is communicated to used by others.

Information Accountability Through Policy Aware Systems

Information Accountability: When information has been used, it should to possible to determine what happened, and to pinpoint use that is inappropriate

Weitzner, Abelson, Berners-Lee, Feigenbaum, Hendler, Sussman, Information Accountability, Communications of the ACM, Jun. 2008, 82-87

Public Health Investigation Scenario

The U.S. Center for Disease Control (CDC) investigates a case of Extra Drug Resistant Tuberculosis

  • The patient is in a coma and can’t be asked about his contacts (the potential spread of the disease)
  • The CDC uses data mining techniques to try to determine the patients possible contacts
  • A phone company that was source of information for CDC data mining refuses to send a service technician to a person who was a possible contact of the patient.

Demo Scenario

  • In order to prevent an epidemic, CDC contacts everyone whom an unconscious tubercolsis patient could have been in contact with
    • people he works with, his choir, the members of his scout troop, people he has called, who have called him
  • CDC gets his phone records from XPhone
  • Sometime later Bob Same has phone troubles and calls XPhone to schedule an appt
  • The customer service operator sees that CDC had obtained his records and infers that he must have some contagious disease
  • So she refuses to schedule a repairman
Scenario 9

MA Disability Discrimination Law

No otherwise qualified handicapped individual shall, solely by reason of his handicap, be excluded from participation in, be denied the benefits of, or be subject to discrimination under any program or activity within the Commonwealth

More info: http://www.mass.gov/legis/const.htm#cart114.htm

:MA_Disability_Discrimination_Policy a air:Policy;
   air:rule [
       air:pattern {
          :EVENT a tami:RefuseRequest;
              tami:reply-to :REQUEST;
              tami:receiver :REQUESTER;
              tami:reason :REASON.
       air:rule [
           air:pattern {
                :REQUEST tami:instruction :INSTRUCTION;
                                    a tami:Request.
                 :INSTRUCTION tami:intended_beneficiary :REQUESTER;
                                    a tami:BenefitInstruction.
                 :REQUESTER tami:location tami:MA.
           air:rule [
               air:pattern { :EVENT a tami:RefuseRequest;
                                tami:reason :REASON.
                             :REASON tami:category tami:HealthInformation };
               air:assert { :EVENT air:non-compliant-with :MA_Disability_Discrimination_Policy }

AIR Demo Screenshot - Log

AIR log in Tabulator

AIR Demo Screenshot - Policy

AIR policy in Tabulator

AIR Demo Screenshot (Regular View)

Justification UI Screenshot

AIR Demo Screenshot (Lawyer View)

Justification UI Screenshot

Architectural Overview -- The Web today

Web Architecture

Architectural Overview -- The Web with Information Accountability Mechanisms

Web Architecture with Information Accountability
  • Access Control through decentralized authorization proofs based on access rules express over the semantics of data under control
  • Transparent usage logging for real-time compliance hints and a posteriori accountability
  • Engineered as Web architecture components

Highlights of AIR Policy Language -- Importance of Explanations

  • Explanations for policy decisions allow users to understand how the results were obtained
  • Increase trust in the policy reasoning and enforcement process
  • Used by policy administrators to confirm the correctness of the policy and to check that the result is as expected
  • In the case of failed queries, they can be used to figure out what additional information is required for success

Why aren't proofs enough ?

  • Proofs or explanations generated by reasoning engines are usually in the form of proof-trees
  • Proof trees not easy to understand by end users
  • Too much information in proof trees
  • Relevant information needs to extracted from proof trees and presented to end users in easy to understand manner

Example Proof Tree
Image courtesy http://clip.dia.fi.upm.es/~logalg/slides/

AIR Policy Language

  • rule-based policy language for accountability and access control
  • explanations for policy decisions through dependency tracking
  • customizable explanations, if required
  • more efficient and expressive reasoning through the use of goal direction
    • mainly forward chaining
    • backward chaining is used to limit search space
  • grounded in Semantic Web technologies for greater interoperability, reusability, and extensibility

Dependency Tracking

  • Dependencies are the specific set of premises from which any conclusion/policy decision was derived is an effective explanation for the conclusion
  • Dependency tracking is the process of maintaining dependency sets for derived conclusions
  • We use Truth Maintenance System (TMS) for tracking dependencies of conclusions
    • keeps track of the logical structure of a derivation
    • associates dependencies with each fact in the KB
    • has ability to assume and retract hypothetical premises

AIR specifications

  • AIR policies are written in Terse RDF Triple Language (Turtle)
  • Each AIR policy consists of one or more rules
    policy = { rule }
  • A rule is made up of a pattern that when matched causes an action to be fired. Optional: variable, description
    rule = { pattern, action [ variable description ]}
  • An action can either be an assertion, which is a set of facts that are added to the knowledge base or a nested rule
    action = { [ assert | assertion ] | rule }
     :Policy1 a air:Policy;
	 air:rule [
	     air:pattern { ... };
	     air:assert { ... };
	     air:rule [ ... ]

AIR ontology

AIR specifications

  • Variables (air:variable)
    • used to declare variables that can be used inside patterns
    • the scope of variable is the container in which it is declared. If the variable is bound before a rule is invoked, it is passed as a constant
  • Rule descriptions (air:description)
    • list of variables and strings that are put together to provide the NL description
:Policy2 a air:Policy;
       air:variable :VAR1;
	 air:rule [
           air:variable :VAR2;
	     air:pattern { ... };
	     air:assertion { ... };
           air:description (:VAR1, “ is a variable that is declared in :Policy2 and ”,  :VAR2, 
           “ is a variable that is declared in this rule”);
	     air:rule [ ... ]

AIR Tools

  • AIR Reasoner
    • accepts data + policy
    • produces reasoning result of running policy over data
    • Available via http
      http://mr-burns.w3.org/cgi-bin/server_cgi.py?logFile="log file"&rulesFile="policy file"
  • Justification UI

Another application -- copyright licensing with the Creative Commons licensing language

and non-compliance

Respective Roles of Law and Technology in Accountable Systems

  • Technical tools are necessary but not sufficient
  • Legal rules, especially those defining proper and improper usage, are essential
  • Lessons from earlier efforts such as P3P: technical tools can help implement public policy but do not substitute for it.