------------------------------------------------------------------------------ The descriptions for each metasystem was informative. The distinctions between metasystems were limited by the simplified novice-expert evaluations. ------------------------------------------------------------------------------ This paper explores the user transparency issues in five metasystems. In the introduction authors define what is a metasystem and what are its possible applications. Here they also define the fundamental issue in this paper, namely the user transparency. This concept is claimed to be the prevailing factor in selecting a metasystem design. It is not clear why is it true and what are the drawbacks of the transparency in terms of performance degradation. The distinction between "novice" user and "expert" user has been made and their requirement for transparency has been explained. Section 2 introduces the reader with five modern research metasystems - Condors, Globus, Java Market, Legion and WebOS. The description is very clear and the supporting figures facilitate the comprehension. Section 3 analyzes the five metrics in which the authors consider the transparency: resource discovery, resource management, security, user interface, and fault tolerance. At the end of the first section, the sentence "Legion and Globus use a metasystem-side mechanism..." is not clear enough and need more explanation. And since this is the starting of a paragraph, it makes the whole paragraph unclear. In the next subsection, it is said "In Condor, users have more flexibility in controlling how their jobs are matched to resources", while the next paragraph starts with "Condor not only shields the user from the matchmaking algorithm, it also prevents the user..." which is rather confusing. For the Legion it is explained that the user can replace the default mapper with own algorithm. The question that arises here: how will this affect the general system performance is not discussed. In the authentication subsection I got the feeling that for the authors the authentication transparency is equal to no or less authentication. Section 4 describes the requirements and expectations of a novice and an expert user from the metasystems. And the title Recommendations comes somewhat not in place. Very firm and clear distinction has been made between both users and their requirements. The paper ends with interesting proposal for future work, but unfortunately it is not defined very well and is unclear. Generally speaking, the paper is very interesting and reveals a lot of issues about metasystems. It is very well structured and easy to understand. ------------------------------------------------------------------------------ Overall, the paper is highly focused and well structured. Terminology definitions were made very clear, observations are sound. However, although the paper completely describes how the systems look like, there should be a more deeper analysis of why those systems are implemented in that specific way. Relevance between transparency issues and the overall goal of metasystems should be mentioned in order to convince the readers the importance of the issue. Transparency might as well be "one of many factors that should be considered when selecting a metasystem", but providing a method to analyze and select a metasystem shouldn't be the goal of the paper. ------------------------------------------------------------------------------ An excellent paper in general. Some general comments: All terms are clearly explained. Figures provide a useful aid in understanding the paper. 1. The introduction gives the reader to understand what the intent of this paper is. 2. * The meaning of the sentence "The "fee" and "reward" need not be monetary; a Market operating within a corporate intranet might use an arbitrary token system to impose different cycle utilization limits on different" in page 4 is not understood. * The formulas in the sentence "Specifically, suppose a user job takes d steps (where a "step" is an arbitrary uniform time unit) to complete, and at least one of the n hosts will be available for at least D ³ (3 d log n) steps. Then, a host which will be alive for at least d steps can be selected with probability of at least 1 - O ((d log n) / D)" , in page 4 are not justified. 3. The systems under examination are adequately described in the areas of investigation (resource discovery, resource management, security, UI, fault tolerance). They are also satisfactorily compared and contrasted in this chapter. Also criticism on these systems is satisfactorily exercised as well. 4. The recommendations successfully suggest ways for the systems to be improved to better accomplish transparency in the areas of investication Some general comments for the paper: The figure in page 14 is not explained adequately : specifically the meaning of the vertical axis is not understood ------------------------------------------------------------------------------ This is a very interesting paper. The authors analyze one approach to {\it heterogeneous computing}, namely {\it metasystems}. However, only one specific aspect of metacomputing is analyzed and this is {\it transparency}. Five ongoing projects are considered and discussed from this point of view. Section 1 (Introduction) gives the basic definitions of metasystem and transparency. It also emphasize that the paper is aimed at distinguishing between the transparency needs of, what authors call, novice and expert users. Section 2 (Summaries of Metasystems) presents each of the five systems that are analyzed. Section 3 (Analysis of Transparency) makes the point the paper is aimed at, giving a discussion on how each of the projects treats transparency. It provides analysis of each basic aspect that contributes to the degree transparency is perceived by the user: resource discovery, resource management, security, user interface and fault tolerance. Section 4 (Recommendation) Presents the authors' point of view on how each of the systems addresses the need for more or less transparency as needed by novice or advanced users. Section 5,6 (Conclusion and Future Work) present authors' conclusion and possible future work The metasystems by themselves are extremely interesting. However, as I pointed out in the beginning the paper focuses on analyzing the degree of transparency each of the systems presents. I am not sure to which degree this is consistent with our conference's (OS Survey F98) purpose. Good points: It is relatively easy to be read. It focuses on an interesting topic (metasystems). It has a nice Recommendation part one might actually use when having to deal with metacomputing. Recommendations: The uniformity of the paper could be improved in several ways: Authors could have made their point in a shorter manner in several places, such as: Section 1, Paragraph 6 Section 3.2 Section 3.4 Paragraphs 7-8 (could probably be reduced to just a shorter one, by giving up some details) Also, the sequence in which the systems are presented should be preserved in each section. I could find no reason for not doing this. The presentation of each project can probably be done in a similar manner e.g. in section 2.4 first paragraph, the services could have been listed using the same font as the rest of the section (as it is done in the other sections) In several places, and to different degrees of granularity, the logics of the paper could also be improved: In section 1, the definition of metasystem logically precedes the definition of transparency. Therefore, paragraph 4 should be probably before paragraph 1. In Section 1 Paragraph 5, it is not clear why `` a more transparent...'' follows from `` the user should be...'' In Section 2.2 last paragraph, it is not clear how the sentence ``Briefly...'' follows from the previous one. Most probably it should be a new paragraph. In section 3.2 Paragraph 7-8 there appears to be a contradiction: ``In Condor users have more flexibility in controlling how their jobs are matched'' ``Condor not only shields the user from the matchmaking algorithm, it also prevents the user from participating in the maintenance of the process'' The authors could consider supporting some statements (or removing them): Section 2.1 Paragraph 4: ``migration and checkpointing allow to control runaway processes..'' It is not clear how. Section 2.4 Paragraph 4 ``in order for users...'' presents some problems without giving solutions that were adopted. Probably it would be better to remove it. Section 3.1 Paragraph 4 ``if more machines are added...'' it is not clear why, since the Market doesn't seem to support a supply/demand mechanism. Section 3.2 Paragraph 9 ``Both...a single algorithm can not effectively do matching `` is there a contradiction with the way Condor operates? Probably, it is all about a unique algorithm which would give optimal solutions... Section 3.3.2 Paragraph 5 The whole paragraph is not consistent with the idea of behavioral monitoring. To my understanding, behavioral monitoring refers to what a process does {\it after} it is accepted to run on a machine. If this is the case then WebOS does not support behavior monitoring, but only restricted access. Next paragraph starts with the sentence: ``Legion is similar to WebOS in that a job's access to the rest of the metasystem is controlled'' It is probably true that the access is done in similar ways, however, Legion provides behavior monitoring since calls to objects, protection boundaries are checked via MayI() functions. One last point: I am not sure how to interpret the Oy axis in figure 6 ------------------------------------------------------------------------------ This paper compares five metasystem projects on their transparency. Metasystem is an important research field of distributed computing, and there are many key issues to be resolved, such as the management and exploitation of resource heterogeneity, high performance via parallelism, scalable architecture, etc. Of course the transparence is a important problem, but this paper only analyze the transparency from the user's opinion, so cant get a deep understanding of the metasystem. For me, I only know what a navie user may choose the Java Market or WebOS, and a expertise use may choose the Leigion or Globus, blur,blur,blur. But I dont get the key different of these projects. Anyway, under this title, the authors do a good work. ------------------------------------------------------------------------------ This seems like a great topic with a huge amount of importance for the future, that has only just started to get off the ground. It seems very well-written and has some good observations and a good break-down, such as reducing the problem to Security, UI, etc... I would be interested to know a bit more about why MPI isn't as preferable to Legion programming as the native interface and it might be neat to understand a bit more about some of the other underlying technologies I've heard about or why they might not stack up to Legion, Globus, etc... such as AppLeS and NWS. It would be great to hear more about potential uses for such systems. ------------------------------------------------------------------------------ I loved it. It was much better than Cats. I want to read it over and over and over... The novice/expert distinction, while simplistic, worked well for evaluating the different systems. The user will probably not be entirely a novice or an expert in all respects (discovery, management, etc.), but by evaluating these areas separately you allow the user to find a system which matches his differing levels of expertise. The organization of the paper was great, everything broke down nicely into the areas you discussed. The only thing that seemed unclear was what you meant by the term "user". In some cases, it seemed to refer to the end user, i.e. one who just runs applications on top of the system; at other times, it seemed to refer to people who would be programming applications for the system. A little clarity here would be nice, but this is a minor point, as evidenced by the high scores your paper warrants.