Jay Verkuilen Gives Talk at the Einstein Medical Center in Elkins Park, PA
Educational Psychology Associate Professor Jay Verkuilen gave a talk at the Einstein Medical Center in Elkins Park, PA on January 18th, 2017. The talk was titled: Analyzing Clinical Intervention Studies by Non-parametric Meta-Analysis.
Educational Psychology Associate Professor Jay Verkuilen gave a talk at the Einstein Medical Center in Elkins Park, PA on January 18th, 2017.
Title: Analyzing Clinical Intervention Studies by Non-parametric Meta-Analysis.
Abstract: Clinical intervention studies such as are commonly done in rehabilitation research, special education, for TBI patients, physical therapy, etc., involve a fairly small number of participants (small #P) who are measured many times (large #N(p)) according to an experimental design. An example of this might be interrupted time series design in so-called single case analysis, which is widely used in applied behavior analysis, but other experimental protocols exist in other areas. In aphasia research, language atypical participants are often measured at a baseline and then treated and/or compared to language typical performance, with participant performance forming a table of responses, typically analyzed using logistic regression.
This talk discusses the potential for using multivariate meta-analysis to deal with the fact that the small #P-large #N(p) situation is not well-handled by techniques such as generalized linear mixed models (GLMMs), notably the linear mixed model or mixed logistic regression. It is unlikely that atypical participants are drawn from a Gaussian population, which is a key assumption of most GLMMs, along with there being a sufficiently large #P to be able to estimate higher level parameters. Both conditions are frequently violated in practical studies, where #P is often quite modest. The GLMM does not respect the participant as a distinct analytic unit, in effect a study unto themselves.
The strategy proposed here is to reduce the data to a set of sufficient statistics with associated sampling distributions and then use non-parametric meta-analysis, ideally multivariate, when there are enough #P. Non-parametric meta-analysis, e.g., Dirichlet process prior Bayesian meta-analysis, does not require that the effect sizes be drawn from a Gaussian population, thereby avoiding the biases that occur when the Gaussian assumption. Meta-analysis also focuses more on the participant as being distinct. This strategy has potential because of the fact that #N(p) is fairly large, but allows for partial pooling of information and intra-participant comparison. It represents a principled method to manage the uncertainty inherent in the cross-participant comparisons. The strategy requires that #N(p) be large enough that the sufficient statistics have well-defined sampling distributions, in effect to be well-powered themselves. Two examples will be provided.