MARIA JANELLI, PHD SEPTEMBER 2019

Educational Psychology
Learning, Development, and Instruction

Maria's research interests include educational technology,
instructional design, digital assessment and feedback, and
formal and informal online learning.

Maria Janelli is the senior manager of online teacher education programs at the American Museum of Natural History. She holds a Bachelor’s degree in English literature from Saint Peter’s University, a Master’s degree in English literature from Rutgers University, a Master’s degree in anthropology from Columbia University, and a Ph.D. in educational psychology from the City University of New York. Janelli has spent her career designing, implementing, and researching e-learning applications at some of New York City's most renowned cultural and educational institutions.
 

Dissertation Title:

"Effects of Pre-tests and Feedback on Performance Outcomes in Massive Open Online Courses: What Works and What Doesn’t?"

Maria's dissertation was an experimental study that examined the effects of pre-tests and feedback on learning outcomes in a five-week massive open online course (MOOC). The participants (N = 399) were adults from around the world who self-enrolled in the American Museum of Natural History’s (AMNH) climate change MOOC (called Our Earth’s Future) offered on the Coursera platform. Participants were randomly assigned to one of four conditions. Learners in the first treatment group took pre-tests without receiving feedback. Learners in the second treatment group took pre-tests and received basic (correct/incorrect) feedback. Learners in the third treatment group took pre-tests and received elaborate feedback. The fourth group was the control. Post-tests were administered to measure learning outcomes. Additionally, we examined links among self- efficacy, persistence, and outcome measures. Of the 606 participants assigned to the four conditions, 399 met the criteria for inclusion in the final analysis. Results of this study indicate that: (1) among all users in a MOOC, pre-tests and feedback do not affect learning outcomes; (2) the presence of pre-tests significantly and negatively affects persistence and completion, deterring some participants from progressing through the course; (3) among those who do persistivand complete the course, those who take pre-tests achieve higher learning outcomes than those who do not; and (4) among those who take pre-tests, there is a positive, cumulative effect of persistence (module completion) on learning outcomes. These findings represent a new contribution to the literature on assessment and feedback, expanding the field to include adult participants from around the world who enrolled in a self-paced, not-for-credit online science course. The results pave the way for future research in this area with this population and have a direct practical application for online course developers, offering them information to help improve student learning outcomes and engagement.

Committee Members:

Dr. Ana Lipnevich (chairperson); Dr. Bruce Homer; Dr. Joan Lucariello; Dr. Dave Randle; Dr. Jeff Smith

Acknowledgements / Special Thanks:

Thank you to Dr. Anastasiya Lipnevich - professor, advisor, colleague, and friend. Without her, my research would have remained just an idea.

Thank you to Dr. Bruce Homer and Dr. Joan Lucariello, for reading drafts of my research and helping to shape it into something meaningful; and to Dr. David Randle and Dr. Jeffrey Smith, for serving on my committee.

Thank you to my dearest, Genna, who scoured the internet to find PDFs of obscure articles; helped me study for my qualifying exams while we were on dates and on vacation; deleted extraneous commas; and said she’d marry me despite the chaos of graduate school. Being a doctor is nothing compared to being her wife.

Finally, thank you to Gram, who is the best, always.

What's Next?

Maria and her advisor (Professor Lipnevich) have published an excerpt of her dissertation as a chapter in the book Early Warning Systems and Targeted Interventions for Student Success in Online Courses.

Maria is expanding her dissertation research to a second study - this one about pre-tests and student personality traits. Data collection is currently underway.