Semantic Web 2009 - Research Papers Reviews of submission #375: "Policy Aware Content Reuse on the Web" ------------------------ Submission 375, Review 1 ------------------------ Reviewer: external Overall Rating I do not care what happens to this paper Originality Good Technical Soundness Excellent (flawless) Presentation Excellent: Reads like a Thriller Expertise 3 (Knowledgeable) Summarize the Scientific Contribution The authors present two new tools for creative commons license management motivated by a quantitative study, and discussion of the needs and limitations of license management on the Web generally. Summarize your review The paper is a solid contribution, its claim that "This paper has demonstrated several tools that lay the foundation for such policy aware systems" is a bit exaggerated. While the tools described appear to be useful, the discussion of related work shows that there are many tools in this area aimed at different aspects of the complicated license management issues. The Review The authors present two new tools for creative commons license management: One for checking that content is used correctly, and one for copying and pasting such content along with its license annotations. These tools are motivated by a quantitative study of how Flickr images are being used on the Web. Another contribution is a discussion of the needs and limitations of license management on the Web generally. The paper is a solid contribution, but its claim that "This paper has demonstrated several tools that lay the foundation for such policy aware systems" is a bit exaggerated. While the tools described appear to be useful, the discussion of related work shows that there are many tools in this area aimed at different aspects of the complicated license management issues. I felt that the discussion should have dealt more clearly with 1) What happens when users are not cooperative? The paper only deals with techniques that help users that actually *want* to follow the license requirements. 2) What part of this is actually "Semantic" Web oriented? Do we need more intelligent tools with reasoning capabilities to detect complicated license infringements, etc? Comments to Authors ------------------------ Submission 375, Review 2 ------------------------ Reviewer: external Overall Rating Should accept: I argue for accepting this paper. Originality Good Technical Soundness Good (minor issues; can be easily fixed) Presentation Passable: Comparable to the usual Expertise 4 (Expert) Summarize the Scientific Contribution This paper examines content reuse on the Web, but from the viewpoint of examining what it would take to be policy-aware. Summarize your review This is a good paper that has some minor problems. If these minor problems are fixed, I think this will be an excellent - and well regarded and discussed - paper. The Review I thought this was an excellent paper in principle, marred by some small problems that should be fixed. The paper has it all - general justification of the problem; a nice, small experiment to prove the point; some technical prototypes that suggest how to fix the problem; and, a good discussion summarizing the future. The paper should be accepted. I think this paper has a chance at being transformative, in that it could really bring home many of the issues in content reuse and policy use. To do that, however, the paper needs to be tweaked in a few places: 1. There is a conceptual jump between 2.2 and 3.1. Section 2 is clearly labeled Background but section 2.1, with its discussion of DRM and Creative Commons, isn't tied directly into the issues that will be discussed later in the paper. Similarly, some foreshadowing in section 2.2 would help the reader tie the arguments of the paper together. 2. Section 3.1 needs more detail on how the analysis was done. I understand from the section what was used to draw the sample, but not how 67, 70, and 70 sites came be used as the 3 samples. (In social science terms, the paper describes the sampling frame, but not the sampling method.) The number of sites seems relatively small, and the authors should address this. As well, with 426 images in 67 sites, for example, I would presume the authors checked for correlations within sites? If the correlation is high, both numbers (sites and images) should be presented. If not, this should be stated. As well, I don't understand the "precision" measure - just presented the raw and corrected %s violations, since that's what most readers will want to know. 3. The section in 4.2 entitled Data Purpose Algebra Analogy doesn't work for me. This is the most clearly speculative section of the paper and one without a real implementation. Why is this section here? Some generating function (or at least method) is going to be required, but the authors should explain what this particular algebra will solve over other methods. 3. Finally, the future work section reads like a list. The paper would be improved with additional analysis of what kinds of future work are required. For example, dividing issues into ones of expediency (how we get people to use a similar system *now*) versus ones where knowledge provision can be re-engineered (how we can change the world once people want this) would be useful. Comments to Authors Section 2.1 - I don't know why "the usability of the content" is questionable. Shouldn't this be "using the content" or something similar? Usability would imply a front-end or at least user experience of the content. Section 4 - "developed couple of tools" should be "developed a couple of tools" Figures 1 and 2 are too small. Section 4.2 Data Purpose Algebra Analogy: Does Qd(i) = Ad(i)? If not, what is Ad(i)? And in the complex function P, why is it K and not Kappa? ------------------------ Submission 375, Review 3 ------------------------ Reviewer: external Overall Rating Should accept: I argue for accepting this paper. Originality Good Technical Soundness Good (minor issues; can be easily fixed) Presentation Excellent: Reads like a Thriller Expertise 3 (Knowledgeable) Summarize the Scientific Contribution The authors observe that policies for content reuse over the Web are substantially broken, and prove their observation with an experiment run on the reuse of Flickr images in mashup applications, where the Creative Commons license applies. The authors also introduce two tools for assisting reusers in checking the policies of the content objects in their mashup, and in getting easily awareness of those policies. Summarize your review The paper addresses in detail an important issue, and ground some solutions in two implementations. The experiment and related work can be improved. The relation between the RDF and algebraic representations of policies, and conflict management is unclear. The Review Firstly, on the positive side, the basic machinery of the problem is well described, and the experiment substantiates the observation. On the negative side, the experimental setting is not described very clearly, probably in order to sacrifice some sentences for including other sections. For example, the section on refining the experiment after noticing false positive is hardly comprehensible to me (maybe due to personal fatigue :)). Anyway, I suggest the authors to describe less about the experiment, but more clearly (possibly relating to an expended technical report). Secondly, on the positive side, the authors have based their description of policies on RDF, and their composition on the Data Purpose Algebra. On the negative, that algebra is said to be insufficient to detect license conflicts. My suggestion is here to clarify how the algebra is used at runtime by the Semantic Clipboard, and why it cannot be used to detect conflicts by adding one parameter. Maybe is it better to use an OWL ontology for that purpose? Also with reference to related work, some attempts on an OWL ontology that allows reasoning across data purpose, policies, etc. can be found in results from the German SmartWeb project, and from the EU Metokis project. Thirdly, on the positive side the paper has plenty of useful insights into practical management of content objects and policies, and the extended future work section is very stimulating. Comments to Authors