Date of Award

Fall 11-22-2024

Document Type

Dissertation

Degree Name

Ph.D.

Organizational Unit

Morgridge College of Education, Research Methods and Information Science, Research Methods and Statistics

First Advisor

Robyn Thomas Pitts

Second Advisor

Kathy Green

Third Advisor

Trisha Raque

Fourth Advisor

Ruth Chao

Copyright Statement / License for Reuse

All Rights Reserved
All Rights Reserved.

Keywords

Equivalence testing, Level-specific evaluation, Model fit indices, Multilevel structural equation modeling, Standard approach

Abstract

Multilevel Structural Equation Modeling (MSEM) is a method suitable for analyzing data with multi-level structure. This method is particularly useful for exploring relationships across different levels of analysis. The objective of this dissertation was to assess the performance of model fit indices in MSEM using a standard approach (SA), level-specific (LS) evaluation, and equivalence testing (ET) methods. Monte Carlo Simulations were implemented to contrast the performance of common fit indices in each method in MSEM under three design factors: sample size (SS), intraclass correlation coefficient (ICC), and specification model (SM). Additionally, the effectiveness of SA, LS, and ET approaches were evaluated using real-world data, utilizing emotional intelligence as a personality state dataset.

The results demonstrated that the SA model fits, assessed using CFI and RMSEA, effectively identified the correct specification model (CSM) and rejected the measurement misspecification model (MMM) across all SSs and ICCs. However, these fit indices failed to detect the structure misspecification model (SMM). Furthermore, the LS model fits, including CFILSW, CFILSB, RMSEALSW, and RMSEALSB, successfully retained the CSM and rejected the MMM. Only the CFILSW was sensitive to detecting the misfit of SMM when the ICC was 0.3. In the examination of ET for the CSM, the T-size (CFIETWt, RMSEAETWt, and RMSEAETBt) indicated excellent fit across all SSs and ICCs, while the CFIETBt showed excellent fit with an ICC of 0.1 and close fit with an ICC of 0.3. All T-sizes measures except RMSEAETBt, rejected MMM, while only CFIETBt consistently detected SMM across all SSs and ICCs. ANOVA results indicated that specification model (SM) had the most significant effect on the performance of fit indices, followed by ICC and SS. Real-world data analysis supported these findings, highlighting the limitations of traditional fit indices and emphasizing the need for comprehensive evaluation methods to detect model misspecifications accurately. Although no single approach performed successfully in all scenarios, a combined approach, especially LS and ET, using multiple indices and methodologies is recommended for a more robust and accurate assessment of model fit in MSEM.

Copyright Date

11-2024

Publication Statement

Copyright is held by the author. User is responsible for all copyright compliance.

Rights Holder

Noor Alibrahim

Provenance

Received from author

File Format

application/pdf

Language

English (eng)

Extent

161 pgs

File Size

2.0 MB

Download

Available for download on Thursday, January 21, 2027



Share

COinS