Introduction
Experimentation acts as one of the methods of conducting research. In this method, researchers may use actual subjects in their experiment or come up with simulation to represent the real world. Lack of enough time for people to organize and conduct real experiments has resulted in the use of simulations. The simulation is organized in such a manner that it represents the subject being observed. Alterations may be done to determine the effects of changing the independent variables on the dependent variables.
In experimentation, various methods are used for purposes of data collection. These methods include observation, self-reports, interviews, surveys and peer ratings. All these methods have various advantages and disadvantages and they aid in the determination of the reliability of the study and inferences made from the experiment. This paper aims and critiquing use of social simulation as a method of determining the effectiveness of decision making among leaders.
Dependent and independent variables in the experiment
The objective of the experiment was to determine the manner in which leaders make their decisions when faced with different challenges. It is believed that people with different personality traits will make varied decisions when they encounter different situations with uncertainties, limited information and complexity. To help in teaching potential leaders on methods of decision making in such instances, an experiment was made on college undergraduates and officer candidates. The decision made by these groups dependent on the degree of environmental turbulence. Thus, decision made by college undergraduates and officer candidates acted as the dependent variable in the experiment while the degree of the turbulence acted as the independent variable (Hunsaker, 2009, pp. 343-362).
Sampling methods used to collect information
In this experiment, experimenters in their data collection used various methods. These include observation, self-reporting and peer ratings. Response of the dependent variables under varying degrees of turbulence was observed by the experimenters and information recorded. In other instances, undergraduates and candidate officers were asked to come up with reports of their experience. Candidates and undergraduates were also rated based on their response to the environment they were subjected to (Hunsaker, 2009, p. 341).
Sampling method, reliability and validity of the study
To come up with the samples to be experimented, the subjects were obtained through stratified sampling. All the subjects were categorised as either college undergraduates or officer candidates. Samples were then obtained from the established strata through random sampling. Reliability of the study depends on the sampling and data collection method used. Some of data collection methods may be subjective leading to the experimenter not collecting accurate information. The three methods of data collection used in the study limit its validity and reliability.
Observation as a method of data collection has various drawbacks. One of the drawbacks is inability of the observer to document the exact information observed. During observation, it becomes hard for the observer to capture all the relevant information in the experiment. At times, the experimenter relies on his or her memory. This is vulnerable to the experimenter forgetting to capture some of the observations made and which may be important in coming up with a concrete decision after the experiment. Observation is susceptible to observers recording inaccurate data thus making erroneous inferences from the data. The validity of this experiment is hard to guarantee due to errors arising from the observer (Taylor-Powell & Steele, 1996, p. 123).
Information compiled through observation in most cases is subjective. This is an inherent drawback of observations as a method of data collection. It becomes hard to attain the objectives of the research through this method of data collection. When describing what is observed, it becomes hard for the observer to avoid personal biases. In most cases, observers write their observations based in their interpretation. This leads to the information being subjective.
Data collected in this experiment through observations thus lose its reliability due to subjective approach that may have been used by the observer in compiling the observed events. In events of observation, the observed subjects may tend to act perfectly as they understand that they are being observed. This becomes hard to collect the actual information of their behaviour. What is observed during the simulation is not actually, what could be observed if the subjects were allowed to act in a natural environment where they were not monitored or observed (Taylor-Powell & Steele, 1996, p. 127).
Self-report also has different disadvantages which make the compiled information lose its reliability and validity. One of the major threats to validity of data collected through self-reporting is of ambiguous words. The person compiling the report may understand some of the words used in the sheet used for data collection in a different way. This leads to one reporting the wrong information. This calls for researchers to carefully shoes their words to ensure that the observed subjects are able to compile an accurate report. The order of items in the report also determines the manner in which one responds.
For instance, if the subjects are required to respond to their experience in the simulation when faced with difficult situation then go to when faced with less difficult situation, it would be hard for them to give the correct information. This makes the collected information invalid (Moskowitz, 1986, pp. 294-322).
After the simulation, some respondents may try to make themselves look good. This would result in them not giving the correct information. This is the reason why self-reporting items need to be neutral. Dishonesty among respondents makes the study unreliable. At times, respondents may intentionally avoid extreme responses. They may also opt to agree with all required statements thus making the collected data inaccurate. Self-reporting method of data collection used in the simulation makes the study invalid as well as unreliable due to numerous weaknesses found among the respondents.
Biasness found in peer rating as a method of data collection is another factor that makes this simulation study invalid. In most cases, individuals participating in peer rating may influence the outcome of the collected information by altering the actual data to favour themselves. It becomes hard to determine instances when data is influenced thus making inferences made from the data unreliable (Moskowitz, 1986, pp. 323-331).
Conclusion
Validity of findings made in an experiment depends on method of data collection used. This is because some of the data collection methods are influence by participants to favour them. Use of observation as a method of data collection in this simulation makes the study unreliable, as information gathered through observation may be subjective based on observer’s mode of data compilation. Use of self-reporting also makes the study invalid due to dishonesty among the respondents. Under some degrees of turbulence, respondents may decline to report their actual experience thus making the overall study inaccurate. Biasness among respondents adds to invalidity of the study bearing in mind that peer rating was used as one of the data collection methods.
Reference List
Hunsaker, P. L. (2009). Using social simulations to assess and train potential leaders to make effective decisions in turbulent environments. International Journal of Career Management, 14(1), pp. 341-362.
Moskowitz, D. S. (1986). Comparison of Self-Reports, Reports by Knowledgeable Informants, and Behavioural Observation Data. Journal of Personality 54 (1): 294-331.
Taylor-Powell, E., & Steele, S. (1996). Collecting Evaluation Data: Direct Observation. University of Wisconsin Cooperative Extension. Web.