Introduction
Multi-trait multi method is a type of research which provides an approach to assessing construct validity in a set of measures in a study. Construct validity is how well a test measures to its claim; it seeks agreement between the theory and the test. It has two sub categories which are convergent and discriminant; convergent validity shows the level of correlation between measures that should be theoretically related.
This is especially where one trait is measured with different methods. The correlation is shown in a validity diagonal in multi trait multi method matrix (Campbell& Fiske, 1959). Discriminant validity shows the degree to which things should not be related and discriminates among different traits.
The correlation between the different trait and method should not be too high. Method Variance is when the correlation of different trait and one method is stronger than different traits and different methods.
In a Multi Trait multi Method matrix, as described by Campbell & Fiske (1959), there are two types of blocks that help in determining validity and reliability of various psychological research methods.
Reliability is the agreement between two efforts to measure the same trait, through maximally similar methods while validity is an agreement between the attempts to measure the same trait using similar maximally different methods.
There are two types of blocks in the matrix, Monomethod block and Heteromethod block. Mono method block has a reliability diagonal and a heterotrait mono-method triangle. In these, correlations come from traits measured with the same method.
The Heteromethod block has a validity diagonal and a heterotrait-hetero method triangle. This is brought about by traits measured by different methods. The reliability diagonals are found in a mono trait mono method, and estimate the reliability in each Multi Trait Multi method matrices. The validity diagonal is caused in a mono trait Heteromethod matrix which measures same traits with different methods.
Assessing methods with Multi traits
This measures correlation between different concepts using the same method. If one gets high correlation it’s a result of measuring different traits with the same method.
This provides a frame work for construct validity of measures using the same method. The different traits in a research define the concept clearly, for example, in a research on French High school students on their learning targets in French, the different traits that can be measured are knowledge, skills, creativity and reasoning proficiency.
The different traits measured provide better discriminant and convergent validity.
Assessing traits with multi methods
Measuring a concept or a trait with various methods reduced chances for errors, according to Campbell and Fiske (1959). The method validates measures to avoid measurements errors. This is achieved by using multiple measures and multiple methods, as single measures do not take into consideration the methods used. One is able to measure convergent and discriminant validities to get construct validity.
There are several methods that can be used; they include interviews, paper and pencil, observation etc. Using multiple methods to examine a trait allows the researcher to examine both the convergent and discriminant validities. It provides a reliable methodology that can be used by researchers.
In using one trait and various methods to measure the correlation between measures of one trait using different methods will show a strong correlation (Fiske, 1949). Multi methods can be used to measure correlation between different concepts using different methods; will give lowest scores on the matrix. They are also used for correlations that use different methods and form a hetero method block.
Methods chosen to assess the construct
I have picked an assessment of students learning targets in a French class (Stinggings et al., 2004) and have picked two methods to measure one concept. The methods picked are paper and pen and interviews to measure the learning target of knowledge.
Paper and pen are more suitable in writing short responses to measure the mastering of facts, spelling and any aspect of knowledge that can be measured with short responses or multiple choices. The interviews measure things like language knowledge and those that don’t require a lot of knowledge output and details. In this case, it is suitable to measure the knowledge of the French language through pronunciation and grammar.
In another situation I have chosen two methods to measure four traits. The methods include Interviews, and extended responses to measure creativity, skills, knowledge and reasoning proficiency. To measure knowledge, the best method to use would be interviews and extended responses.
All methods work well to measure reasoning proficiency in the French language, because the interviews assess communication and the extended responses give deeper knowledge to the level of understanding in each student.
To measure skills the interviews would only work to measure oral skills, not for generalized skills. To measure creativity as a learning target, extended responses would be suitable for a written product, for example the grammar part of the French language. Interviews are not good for measuring creativity and are very time consuming.
The method for this construct would be extended response as it is able to measure all the traits apart from skills. A student may have mastered knowledge but the skill to create a product may not be measured by extended responses. Extended responses are good in gathering of information on deep knowledge and where time is not limited.
Two aspects of a concept would be most valid to be measured
The two aspects that are likely to be measured are performance skills and reasoning proficiency. The performance skills target is the development of proficiency in doing something where the process is also as important.e.g. reading out loud with fluency and expression of a foreign language.
The best method to use here and one can also use the other methods to determine whether a student has skills to perform a certain task. Observing through personal assessment gives the researcher a clearer construct. One is able to judge the level of achievement e.g. to assess the students ability to understand French literature, have them do a small play.
Reasoning proficiency is the ability to use the knowledge and understanding to get solutions. The best method for measuring this trait is extended response and performance assessment. Extended response gives more details, and shows a student’s ability to link facts.
Performance assessment is done by observing students as they perform different tasks. This is a multi trait multi method measure of research. When different methods are used on different traits it is easy to get the correlation between the two.
Conclusion
Multi traits multi methods matrix uses multiple indicators for validity assessment, and the validation process uses a matrix with at least two traits and at least two methods to get construct validity. It can be used where multiple traits are examined simultaneously and each is measured with a different method.
Same traits with measured with one method should correlate higher giving a convergent validity, while different traits should not have a high correlation score with separate methods. This is the only validity measurement that uses multiple traits and methods to measure a test construct.
To determine convergence / discriminant validity, Convergent validity coefficients must be high if not, that means that the different methods are measuring different traits meaning there is no convergent validity. According to Campbell & Fiske (1959) the convergent validity coefficients are the correlations between measures of the same trait that are obtained with different measurement methods.
It is, therefore, why the correlations are some of the times referenced to as monotrait-heteromethod (MTHM) coefficients. There is a linear relationship between the different traits and the methods measuring them. In essence, the relationship between two non-similar ways of evaluation of the same trait should exceed the relationships between that trait and another, irrespective of methods of evaluation being similar.
The mode of propinquity between traits should be congruent in the heterotrait triangles, correlation between traits, in the Heteromethod or the mono-method blocks. The researcher determines the choice of the methods and traits to be used in the research.
The discriminant traits have to be very different from the focal trait, and it is important to note that it can be easily confused with the focal trait. Therefore, people should be very careful with the choice of the methods. In the eventuality of using two traits and two methods, then, by all means, the methods should absolutely differ.
References
Campbell, D. T., & Fiske, D. (1959). Convergent and discriminant validation by the multi trait multi method matrix. Psychological Bulletin, 56.
Fiske, D. W. (1949). Consistency of the factorial structures of personality ratings from different sources. J. abnorm. soc. Psychol Journal, 44.
Stinggings, R., Arther, J., & Chapins, J. (2004). Classroom assessment for Student learning: doing it right –using it well. Assessment training institute: USA.