The Research Question
The study of organizational behavior has encompassed a great deal of attention to the effectiveness of performance appraisal methods and processes. One facet of effectiveness concerns feedback from employees about their appraisals.
Keeping and Levy (2000) maintain that the study of employee reactions to appraisals has been singularly plagued by an absence of theory (let alone a cohesive body), a fragmented range of study instruments, and overlapping instruments in use for measuring the same construct. Anticipating there may be a confusion of labels and constructs, the authors aimed to isolate the measurement properties of the varied reactions employees have about their appraisals.
Prior Findings/Background Information
In addressing themselves to this question, Keeping and Levy decry the persistent gulf between theorists focused on the psychometric properties of performance appraisal scales and I/O psychologist-practitioners who believe that employee reactions are critical to perceived appraisal effectiveness. But the criteria used to define employee reactions are already somewhat dated, the authors contend, notwithstanding advances in both measurement knowledge and operationalization of constructs.
Hence, Keeping and Levy aimed for better insight into the standard criteria: satisfaction, fairness, perceived utility, and perceived accuracy. The first question they address is whether these several appraisal criteria are truly standalone constructs or whether they can be traced back to some other, as yet unidentified, overarching construct. Second, the authors address themselves to the contextual facets of appraisal. Third, they investigate bias effects owing to either method or negative/ positive affect.
Predicted Results
There were no a priori hypotheses as such since the study aimed to characterize or filter which of several scale measures in use bore the best relationship with the construct of effect about the appraisal.
Methodology
The study involved a cross-sectional study of employees in a mid-western business organization. The sample consisted solely of those who had received a performance appraisal within two months prior to the survey and participated in a discussion about it. The study instrument was a mailed self-administered 94-item survey that ended with items about positive and negative affect concerning the performance appraisal. Relying on “best practices” in the field, scales that had most frequently been employed in the past, the research team used questions eliciting: satisfaction with the appraisal session itself, satisfaction with the appraisal system as a whole, perceived utility of appraisal, perceived accuracy of the same, procedural and distributive justice, and positive or negative affect.
Findings
Absent any formally stated predictions, the authors nevertheless conceded that they had expected the high inter-correlations that resulted among the eight sets of appraisal reactions. However, the correlations between positive and negative affect, on one hand, and appraisal reaction measures, were modest: in the.20 to.32 range.
On taking further analytic steps that included structural equation modeling, estimating for single-factor and hierarchical models, and a straightforward chi-square test of the differences yielded by varied modeling approaches, Keeping and Levy suggests, among others that a hierarchical model is possible for a second-order construct they labeled “appraisal effectiveness”. The six appraisal reaction constructs correlate significantly and positively with the expected underlying construct. In addition, the hierarchical model is claimed to be more parsimonious, a positive finding.
On a technical note, the research team found from the chi-square analysis that a hierarchical model explained results at least as well as the disaggregated modified measurement model. Keeping and Levy find that the six extant appraisal effectiveness scales do have satisfactorily high factor loadings with their individual underlying constructs.
Secondly, the finding of correlated errors amongst system satisfaction, session satisfaction, and procedural justice seemed to be explained from the constant appearance of the item stems, “The procedures used to evaluate my performance was…” and or “The process used to evaluate my performance was…”. The authors suggest that rates are commonly unable to distinguish between appraisal procedures and processes.
Keeping and Levy dismiss methods effects bias as inconsequential while advancing implications they held to be key: a) the qualitative criteria tested are both accurate and appropriate; b) appraisal reaction measurements currently in use are “quite good” but could stand improvement in point of overlapping constructs; and, c) neither positive nor negative effect unduly bias the outcomes of response to performance evaluations.
Other Comments/Critique
Keeping and Levy confine themselves to an analysis of the validity of existing perceptual and affect measures concerning performance appraisal and whether there is a unitary construct that underlies these measures.
When one shifts to the practitioner side of the I/O psychology field that the authors had alluded to, we realize from Jawahar (2006, p. 14) that merely understanding satisfaction with the appraisal process overlooked many important elements. The results of a longitudinal study suggested and it does stand to reason that doing well on an appraisal will lend a positive bias towards that appraisal and the evaluation system it is part of. Even more important, where the leadership of an organization is concerned, is that satisfaction with appraisals are negatively associated with employee turnover and positively correlated with organizational commitment, job satisfaction, commitment toward, and satisfaction with, the rater-manager.
This viewpoint and the high probability of positive effects strongly suggest that an organization had best invest time and effort to ensuring the fairness of performance appraisals. Just as critical, one hazard is training and coaching employees for consistently superior performance and superb outcomes because the employer reaps many valuable benefits in the medium and long term.
Works Cited
- Jawahar, I.M. “An Investigation of Potential Consequences of Satisfaction with Appraisal Feedback.” Journal of Leadership & Organizational Studies, 2006b, 13 (2): 14-28.
- Keeping, Lisa M. and Paul E. Levy. “Performance Appraisal Reactions: Measurement, Modeling, and Method Bias.” Journal of Applied Psychology 2000, 85(5): 708-723