Assessment of every kind of paper is a rather challenging task as far as it was written by another person who had his or her reasons for writing this or that statement. Thus, before reviewing one has to get acquainted with the background of the paper and the author, and only then start reviewing. Besides, there is a certain set of special skills that a person busy with reviewing has to possess. They include logical thinking, knowledge of the matter of the paper, and the ability to express one’s thoughts on the paper. It becomes especially difficult to review a paper in case if the paper under consideration is scientific (Hopin, p. 1019).
This can be a research article, case study, or qualitative/quantitative research. In this case, except all the above-mentioned skills, the reviewer has to be a person of great knowledge in the field of science that is considered in this or that article, as well as with the main principles of the research reviewing that allow to find out all the weak points of the paper, try to correct them and stress the strong points in the research reviewed (Koh, p. 743).
In this very paper, our assignment will be to assess a research article, namely the article by Allen and March, in respect of the importance of the problem considered and of the adequacy of the theories and practical experiments taken up to prove the hypothesis stated in the course of the research. We are going to consider the above-mentioned article in respect of all the enumerated aspects and, making use of the secondary sources dealing with the rules and principles of different kinds of reviews, assess the quality, significance, and adequacy of the research under consideration and its results. We are also going to examine and evaluate the propositions made by the researchers and the experiments they carried out to support them (Sambamurthy, p. 21).
The article that we are to assess in this paper is dedicated to the very issue of data resources and database organization. In this article, the authors argue about the points of data utilization, proper organization, and wide access to data by ordinary users, as well as specialists in this area. The authors use the single model for all databases which is based on the characteristics of the data and their user. Allen and March describe the three concepts significant in the data organization – accuracy, confidence, and the prediction of the accuracy of database queries.
The authors deal with the event-based and state-based model of the database organization, as well as the mixed type of the same phenomenon. The authors aim at finding out the differences and the levels of comfort that users experience during their work with each kind of databases (Weber, pp. v – ix).
To prove the significance of the topic chosen by Allen and March for their research and the article, we are going to make use of the following arguments that can be found in the article. First of all, according to Allen and March, “the ability to effectively utilize organizational data resources has become a major source of competitive advantage.” (Allen, March, p. 269) This argument is being supported by the authors who say that there are several kinds of database organization, and each of these kinds serves its specific purposes.
For instance, Allen and March, enumerate the data warehouses that are applicable if a certain partner of a company has a desire to learn the latest changes in the company and make certain decisions as for the business relations with it; Relational Database Management Systems which allow to process and maintain the data resources of a certain organization; Structured Query Language (SQL) which aims at formulating exact queries for databases that will allow increasing the efficiency and all the three components of the database organization – accuracy, confidence, and prediction of accuracy.
The topic of the research, and respectively the topic of the article under consideration, is rather actual and important nowadays. This fact can be proven by the widely accepted point of view that the issue of utilization and organization of informational databases is an essential function for every kind of modern company that wishes to reach success and to develop regularly in the future. This very issue has crucial importance for the ordinary users of the databases as far as the quality of the database, the simplicity with which it can be accessed in the necessary time, and the simplicity of the organization of material in it is fundamental for the image of the company whose data are accessed and for the company that develops the databases.
All this, taken as a single whole, allows every company to be well informed about the needs of its customers and database users. Moreover, it allows customers and users, as well as partners of a certain company, to be properly notified about all the information they need, whether it touches upon the matters of the company or is just interesting for one particular database user (Wilson, p. 155).
Having considered the topic of the article and summarized its main ideas and aims, we are able now to move on to identifying the strong and weak points of the paper. The strong points of this article can be enumerated as follows:
- Relevance of the topic.
- This point is strong because the article on this topic is of great help for the development of science studying the Informational Systems (IS) and for the companies that can get information about the proper organization of their databases to form the article. This can be seen from the article on the whole.
- Adequacy of the methods used.
- Usage of the single model and numerous approaches to the issue of database organization and utilization is rather adequate in respect of the topic of the article and allows getting the answers to all questions and prove or deny all the hypotheses of the research. The current strong point can be found in the sections “Methodology” and “Results”.
- Clear structure.
- The article is properly structured with the use of headings and subheadings for respective paragraphs that allow the readers to easily follow the development of the thoughts of the authors. This can be seen during the whole article as the authors describe their research course (Allen & March, pp. 269 – 290). But, at the same time, the article possesses also several weak points that should be pointed out to make possible improvements in future research by these authors. The weak points are:
Vague definitions
The paper is hard to understand for an average reader because of the wide use of specific terminology that is not explained at all or explained vaguely. The very beginning of the article, namely the abstract and the introduction, is filled with the terminology which is, although properly organized logically and subsequently, difficult to catch without the preliminary explanation of the basic definitions.
Narrow audience
The preceding drawback of the article involves the current one. Due to the wide use of the specific terminology and a large number of specialized ideas and constructs, the article becomes aimed at a narrow audience which is mainly the scientists and people working in the field of Informational Technology and Information Systems Management. As for the weaknesses, it can be recommended that the authors should use explanations to make their article more accessible to the wider circle of readers and avoid vagueness in the definitions while the topic is specific.
To evaluate the theoretical framework of the article under consideration, let us first give the basic definitions for this purpose. The theory is a special account used by people to perceive and then explain different phenomena of objective reality. Drawing from this, the next phenomena that we need to define in the paper are the basic things for the assessment of every kind of theory. They are constructs of the theory, their associations, and the boundaries between the constructs in respect of the specific example of the research and theory discussed in the article by Allen and March. Usually, the main constructs of any theory are things and properties of things.
The properties of things, or in other words the phenomena that occur to things in their development are defined as states and events that things experience in the process of their interrelations between them and with the objective reality. The same categories of the state and event happening to the thing form the boundary conditions of the theory (Weber, pp. v – ix).
Having defined the basic units that are important in any kind of high-quality theory, we are going to move to the evaluation of the theory used by Allen and March in their article. The basis of the theory under consideration coincides with the definitions that are given in the preceding paragraph. In other words, the fundamental elements of the theory by Allen and March are constituted by the interrelation between things and their states and events that happen to them. The things are represented by the categories of users and informational databases that are in constant relations and associations with each other.
The phenomena that happen to the things are the states and events that they experience during their relationship and that are described in the article. On the whole, the authors of the research make use of two approaches – state-based and event-based to the issue of the information systems that are discussed in the article. Thus, the constructs of the theory by Allen and March are the users of databases, the databases themselves, and the states and events that regularly happen to them in the course of their associations.
To make the assessment more precise, which allows controlling the quality of the theory and the very research, it would be not out of place to state here that the theoretical framework of the article is rather substantial and organized. One can easily follow the development of the thoughts of the author by following the theory. The theory possesses clear indications of the constructs, their associations, and the phenomena that happen to them, i. e. states and events that are experienced by the constructs of this theory.
For example, the authors of the article make use of the three models of studying the issue of informational systems and the task of formulating queries in them – the state-based, the event-based, and artifact-based model, which is the mixture of the former two models. Drawing from this, Allen and March make conclusions of their research and say that the informational systems may be developed according to each of the models and thus the task of formulating the query to each of the databases will be of different complexity and will, consequently, lead to the different results and efficiency.
The lawful state space of the theory turned out to be the least complicated and resulted in 9 entities and 19 relationships after the research, the event space of the theory was the most complicated with the 17 entities and 23 relationships, and the artifact space was in the middle with the results of 13 entities and 23 relationships (Allen & March, p. 281).
As for the propositions of the theory, they are concerned with the three aspects that the authors consider to be basic for the informational system – they are accuracy, confidence, and prediction of accuracy. Respectively, the propositions of the article under discussion are the following:
- event-based models in use allow to formulate more accurate queries than the state-based ones;
- event-based models give more confidence in the results to their users than the state-based ones;
- the predicted accuracy of the query results will be higher for the users working with the event-based models than with the state-based ones.
The results of the research and the experiments conducted by the authors of the article, however, did not prove all the propositions, and thus we can say that the value of them was not the highest, as well as their quality. The first proposition turned out to be the one without proofs and dependant not upon the type of the model but upon the ability of the user to implement this or that model for formulating queries.
The second proposition also turned out to be inconsistent, as far as the confidence about the accuracy of the query was dependant upon the understanding of the prior procedure of the model application, and then all the models displayed the same level of confidence. Only the third proposition proved to be effective and demonstrated that, despite the failure of the two preceding propositions, users display better prediction of accuracy using the event-based model.
The evaluation of the experiment conducted by the authors of the article under consideration is another important task that we have to fulfill in the current paper. According to the majority of scientists, the accurate and correctly conducted high-quality experiment has to be properly prepared and conducted timely with the help of suitable methods and with attention paid to the following four major problems that every experiment faces: strategy of the research, instruments to measure the research and its results, designs for the research and the tasks for the experiment.
The authors of the article paid attention to all these demands and also did not ignore the necessity to conduct the pilot experiment before the actual collection of data was started and the experiment instrument was developed (Allen & March, pp. 269 – 290).
Allen and March in their article put much emphasis upon the experiments they conducted to prove or deny the propositions they formulated before the research. The authors report about having conducted one-factor between-subject research to examine the ontological issues of the matter of Information Systems and the accuracy and efficiency of the formulating of queries by users.
As far as the experiment had as its aim the reading of informational diagrams by the subjects of the research, these subjects had to be isolated from each other and from the scholarly sources that could help them interpret the diagrams. For this purpose, the online instrument was constructed with the help of HTML, Active Server Pages, and Java Script that allowed the participants of the experiment to be in different places and fulfill the experiment tasks without communicating with each other. Before the experiment was started, the pilot study involving 36 participants was carried out that allowed to test the instrument and experiment design and to make the necessary refinements to the experimental procedure and the design of the instrument.
After this, the actual experiment started and resulted in the results that were discussed above. The actual treatments by the participants of the experiment turned out to differ from the propositions developed by the authors of the article before their research. Allen and March presupposed that the efficiency of the event-based model will be higher in respects of accuracy, confidence, and prediction of accuracy of the queries by users, but the actual efficiency turned out to be the same as the one of state-based or event-based models, and only in respect of the prediction of accuracy the event-based model turned out to be more effective.
References
Gove N. Allen and Salvatore T. March, (2006) “The Effects of State-Based and Event-Based Data Representation on User Performance in Query Formulation Tasks,” MIS Quarterly, 30, 2, pp. 269-290.
Hoppin, F.G. (2002) “How I Review an Original Scientific Article”, American Journal of Respiratory and Critical Care Medicine, 166(8), pp. 1019-1023.
Koh, C.E. (2003) “IS Journal Review Process: A Survey on IS Research Practices and Journal Review Issues”, Information & Management, 40(8), pp. 743–756.
Sambamurthy, V. (2005), Editorial Appointments at Information Systems Research, ISWorld Posting.
Weber, R. (2003) MIS Quarterly Vol. 27 No. 1.
Wilson, J.R. (2002) “Responsible Authorship and Peer Review”, Science and Engineering Ethics, 8(2), pp. 155-174.