Multilevel Logistic Models

Guest lecture content by Winnie Tse and Mark Lai

By the end of this module, you will be able to

- Describe the importance of having sufficient sample size for scientific research
- Describe conceptually the steps for sample size planning: precision analysis and power analysis
- Perform power analysis for MLM using the PowerUpR application and
the
`simr`

package - Understand the effect of uncertainty in parameter values and explore alternative approaches for sample size planning

- Review the resources (lecture videos and slides)
- Complete the assigned readings
- Snijders & Bosker ch 11

- Attend the Thursday session and participate in the class exercise
- Complete Homework 10
- Continue working on your oral presentation/paper

You can view and download the slides here: HTML PDF

Test yourself: Quiz on null hypothesis significance testing. (Quiz solution is attached in the end of this webpage.)

Quiz pdf here

Supplemental reading:

Open Science Framework (OSF) provides some general guidelines for pre-registrating a study.

Kirley et al. (2020) has recently published a template for studies using one type of multilevel design—experience sampling methods, which is a type of intensive longitudinal method that collects participants’ self reports of daily thoughts, emotions, and behaviors.

Practice yourself: Find the required \(J\) for \(SE(\gamma_{10})\) to be \(\leq .1\), when the treatment variable is at level-1 (i.e., \(X\)), with 10 individuals per cluster, ICC = \(\tau^2_0 = 0.3\), \(\sigma^2\) = 0.7, and \(\tau^2_1\) = 0.1.

Check your learning: In the examples in the video, what would change if the researcher assumes a larger effect under the alternative hypothesis?

Practice yourself: Using PwerUpR, verify that power = .678 when the
treatment variable is at level 1 (i.e., Two-level MS-IRT, random
treatment), with an average effect of 0.3 (standardized), no covariate,
ICC = 0.3, \(\tau^2_1 = 0.15\) (i.e.,
`omega2`

= 0.5 in the program), with 30 clusters and 10
observations per cluster.

Now, include 1 covariate that has an \(R^2\) of .40. What is the power?

Check your learning: In a pilot study, we found \(\delta = .2\), with a standard error of \(.05\), and ICC \(= .1\), with a standard error of $ = .05$. We use the information \(\delta = .2\) and ICC \(= .1\) to determine the number of clusters, and the program suggests us to have 116 clusters. If we employ 116 clusters in our study, what will happen to the power?

One advantage of this program is it analytically solves for the
number of clusters or cluster size you need. Using `hcbr`

,
you do not need to iteratively test different \(J\) and \(n\) until the power achieves 80%; you get
the answer directly from the program.

Practice yourself: Given that \(\delta = .2\) with a standard error of \(.05\), and ICC \(= .1\), with a standard error of $ = .05$. What is the required \(J\) if \(n = 2\) and \(n = 20\)?

Check your learning: A researcher originally plan to collect data from 30 classrooms with 5 students per classroom, but then found out the study was underpowered. He can collect data from 150 more students. Would it be more beneficial to make it 300 students from 30 classrooms or 300 students from 60 classrooms? Why?

Quiz solution here