DOI: 10.31038/PSYJ.2023564

Abstract

In this brief article, I summarize results from two recent studies in which popular multidimensional self-concept measures were administered to large samples of university students. Scores from all 26 subscales examined across instruments were affected, but to varying degrees, by factors other than the targeted constructs that included states, methods, and unrelated measurement error. I also direct readers to instructional and computer resources that would enable them to conduct analyses that isolates and takes these factors into account.

Self-Concept Inventories Measure More Than Just Psychological Traits

Self-concept, broadly defined, represents the totality of beliefs about oneself, often paraphrased as the answer to the question of “Who are I?”. Although there are many theories about the nature of self-concept and its development, strong evidence supports the view that self-concept is multidimensional and a powerful predictor of life outcomes [1-5]. To capture the complexity of such broad beliefs, researchers have developed numerous instruments designed to measure multiple aspects of self [6]. Prominent among such multidimensional measures are the Self-Description Questionnaires developed by Herbert Marsh [7-9] and Self-Perception Profiles developed by Susan Harter and colleagues [10-13].

What do Self-Concept Inventories Really Measure?

Although perceptions of self can certainly change over time, most components of self-concept measured within self-concept inventories are expected to remain reasonably stable, especially over short time intervals. That is, these measures are presumed to measure psychological traits rather than states. However, in practice, scores from such measures likely reflect a combination of stable traits, fleeting states, and other unrelated effects [14]. To separate these influences on scores, researchers often employ structural equation modeling techniques based on latent state-trait-theory [15-18].

Empirical Examples

For a recent presentation at the annual conference of the American Psychological Association, colleagues and I [19] applied latent state-trait theory procedures when analyzing scores obtained from large samples of college students who completed the Self-Description Questionnaire-III (SDQ-III; Marsh, 1992c; n=1790; [9]) or Self-Perception Profile for College Students (SPPCS; (Neemann and Harter, 2012; n=821; [12]) on two occasions separated by a week. Each inventory has 13 subscales with one intended to measure overall self-esteem (General Self in the SDQ-III and Global Self-Worth in the SPPCS) and the remaining subscales within each instrument targeted at more domain-specific aspects of self-concept as described in Table 1. The SDQ-III includes 10-12 items for each subscale answered along a Likert-style, 8-point response metric (1=Definitely True, and 8=Definitely False), whereas the SPPCS has 4-6 items for each subscale answered using four unique forced-choice options for each item.

Table 1: Partitioning of Observed Score Variance for SDQ-III and SPPCS Scores

Note. SDQ-III=Self-Description Questionnaire-III (Marsh, 1992c); SPPCS: Self-Perception Profile for College Students; # of items: Number of Items. Values under trait, state, method, and error represent proportions of explained observed score variances. Proportions for trait, state, method, and error variance do not always sum to 1.00 due to rounding.

In Table 1, I summarize partial results from our presentation that represent proportions of observed score variance accounted for by trait, state, method, and unrelated measurement error effects for each self-concept scale. In the present context, proportions of trait effects would be analogous to omega reliability estimates [20] that account for both item and occasion differences in scores. For both inventories, trait effects account for the majority of observed score variance ranging in proportions from 0.730 (Honesty) to 0.930 (Math Skills) for the SDQ-III (Mean=0.856) and from 0.667 (Job Competence) to 0.887 (Athletic Competence) for the SPPCS (Mean=0.807). Higher proportions of trait variance for the SDQ-III are likely due to its subscales having more response options and at least twice as many items.

Nevertheless, subscale scores from both inventories are also affected by non-trivial state, method, and unrelated error effects that on average respectively account proportions of variance equaling 0.055, 0.045, and 0.047 for the SDQ-III and 0.063, 0.039, and 0.092 for the SPPCS. Moreover, effects of states, methods, and error vary widely across subscales. For the SDQ-III, General Academic Skills (0.082) has the highest proportion of state variance and Honesty has the highest proportions of both method (0.097) and error (0.094) variance. For the SPPCS, Morality and Romantic Relationships have the highest proportion of state variance (0.102) and Job Competence has the highest proportions of both method (0.113) and error (0.193) variance. When revising instruments to represent traits more reliably, proportions of state and method effects can be compared to determine the best ways to alter a measurement procedure. Increasing items would usually better serve that purpose when method effects exceed state effects, whereas pooling results across occasions would do so when the reverse is true.

Conclusion

Overall, the findings summarized here confirm that the SDQ-III and SPPCS predominantly measure traits over short time intervals but that scores from both inventories are influenced by a variety of other factors. These findings are consistent with Hertzog and Nesselroade’s [14] argument that measures of psychological constructs rarely measure purely traits or states and further demonstrate that such measures are susceptible to method effects related to how the constructs are being measured (see [21] for a more in-depth discussion of such effects). Unless measures are administered over two or more occasions using multiple items, effects of traits, states, methods, and unrelated measurement error cannot be disentangled. When objectively scored assessments are administered only on single occasions, as is typically the case, trait and state effects are confounded and treated as reliable variance, whereas method affects become part of measurement error variance. When administering measures in the ways prescribed here, contributions of trait, state, method, and error can be separated to determine their individual effects on observed scores. Researchers and practitioners need to be aware of such complexities when measuring psychological traits to properly interpret results from such measures. To learn more about latent trait-state theory and other methods for partitioning observed score variance into multiple categories and how to run such analyses, I direct readers to a recently published tutorial and accompanying instructional online supplement developed by our research group at the University of Iowa [18,22] and to other relevant articles and resources related to these procedures cited earlier here and within that tutorial.

References

1. Hattie J (2014) Self-concept. Psychological Press.
2. Marsh HW (1990) A multidimensional, hierarchical self-concept: Theoretical and empirical justification. Educational Psychology Review 2: 77-172.
3. Marsh HW (2007) Self-concept theory, measurement and research into practice: The role of self-concept in educational psychology. Leicester, UK: British Psychological Society.
4. Marsh HW, Craven RG (2006) Reciprocal effects of self-concept and performance from a multidimensional perspective: Beyond seductive pleasure and unidimensional perspectives. Perspectives on Psychological Science 1: 133-163.
5. Shavelson RJ, Hubner JJ, Stanton GC (1976) Self-concept: Validation of construct interpretations. Review of Educational Research 46: 407-441.
6. Byrne BM (1996) Measuring self-concept across the life span: Issues and instrumentation. American Psychological Association.
7. Marsh HW (1992a) The Self-Description Questionnaire (SDQ) I: A theoretical and empirical basis for the measurement of multiple dimensions of preadolescent self-concept: A test manual and research monograph. Macarthur, New South Wales, Australia: University of Western Sydney, Faculty of Education.
8. Marsh HW (1992b) Self-Description Questionnaire (SDQ) II: A theoretical and empirical basis for the measurement of multiple dimensions of adolescent self-concept: An interim test manual and a research monograph. Macarthur, New South Wales, Australia: University of Western Sydney, Faculty of Education.
9. Marsh HW (1992c) Self-Description Questionnaire (SDQ) III: A theoretical and empirical basis for the measurement of multiple dimensions of late adolescent self-concept: An interim test manual and a research monograph. Macarthur, New South Wales, Australia: University of Western Sydney, Faculty of Education.
10. Harter S (2012a) Self-Perception Profile for Children (Grades 3-8): Manual and questionnaires (revised) Denver, CO: University of Denver.
11. Harter S (2012b) Self-Perception Profile for Adolescents: Manual and questionnaires (revised) Denver, CO: University of Denver.
12. Neemann J, Harter S (2012) Self-Perception Profile for College Students: Manual and questionnaires (revised) Denver, CO: University of Denver.
13. Messer B, Harter S (2012) Self-Perception Profile for Adults: Manual and questionnaires (revised) Denver, CO: University of Denver.
14. Hertzog C, Nesselroade JR (1987) Beyond autoregressive models: Some implications of the trait-state distinction for the structural modeling of developmental change. Child Development 58: 93-109.
15. Steyer R, Ferring D, Schmitt MJ (1992) States and traits in psychological European Journal of Psychological Assessment 8: 79-98.
16. Steyer R, Geiser C, Fiege C (2012) Latent state-trait models. In H. Copper (Ed.), Handbook of research methods in psychology. Data Analysis and Research Publication 3: 291-308.
17. Steyer R, Mayer A, Geiser C, Cole DA (2015) A theory of states and traits—Revised. Annual Review of Clinical Psychology 11: 71-98.
18. Vispoel WP, Xu G, Schneider WS (2022a) Interrelationships between latent state- trait theory and generalizability theory in a structural equation modeling Psychological Methods 27: 773-803. [crossref]
19. Vispoel WP, Xu G, Schneider WS, Kilinc M, Zhang M (2021, August) A latent state-trait analysis of responses to multidimensional self-concept measures (Paper presented at the annual meeting of the American Psychological Association).
20. McDonald RP (1999) Test theory: A unified approach.
21. Geiser C, Lockhart G (2012) A comparison of four approaches to account for method effects in latent state-trait Psychological Methods 17: 255-283. [crossref]
22. Vispoel WP, Xu G, Schneider WS (2022b) Instructional online supplement to “Interrelationships between latent state-trait theory and generalizability theory in a structural equation modeling framework.” Psychological Methods 1-56.