Show The Graduate Center Menu
 
 

Doctoral Seminar: Research Methods in HCI

Course Description: 

This course is for doctoral students who will carry out research in the field of Human-Computer Interaction. The course covers the fundamentals of research methods in a seminar-style class.  The main objective is to enable students to design sound empirical studies and to eventually produce publishable research papers in topics such as evaluation of interactive agents, effects of sensors in human behavior, and impacts of mobile applications among others. By the end of the course, students who have participated actively will have acquired the knowledge and skills to be educated consumers of HCI research, and will have begun developing the skills to conduct their own empirical studies in HCI. Students will start a research project in the course that could potentially be developed into a conference paper in the Summer.

 
Course Objectives: 

  • Students will become familiar with the process of doing research in HCI, designing rigorous empirical studies and collecting data.
  • Students will appreciate ethical issues associated with human subjects’ research in HCI.
  • Students will learn to analyze published articles on a specific topic within HCI, and to read these articles to identify key results (practical and theoretical), limitations (stated and unstated), and future research directions.
  • Students will learn to critique theories and assumptions in research studies.
  • Students will gain experience identifying research questions, refining these questions to form hypotheses, and identifying appropriate research methodologies to investigate these hypotheses.
  • Students will explore topics within HCI, understand the current state of knowledge with regard to these topics, and identify open questions to be addressed.
 
Required Readings: There are two carefully chosen central texts as well as additional readings, listed in the weekly schedule.  The textbooks are:
  • Lazar,J., Feng, J.H. and Hochheiser, H.  Research Methods in Human-Computer Interaction. February 2010. Wiley.
  • Trochim, William M. The Research Methods Knowledge Base, 2nd Edition. Internet WWW page, at URL: <http://www.socialresearchmethods.net/kb/> (October 20, 2006 version).
 
Grading: The final grade in the seminar will be based upon the following weights:
Class participation and Attendance       20%
Exercises/Assignments                         20%
Written Examination                            30%
Research Paper                                     30%

 

Class Participation
Each week, we will cover several articles or book chapters. Students must read the assigned readings before class and come prepared to participate in class discussions. In preparation for the discussion, students can think about the contributions of each article, assumptions and or limitations of the work, or comparisons among articles. Students may also earn participation credit by proposing discussion questions or integrating the readings.
 
Short Assignment #1: CITI Training
The CUNY Human Research Protection Program (HRPP), responsible for the protection of the rights and welfare of human subjects in research projects conducted at CUNY, requires that all researchers on campus who work with human subjects complete the CITI- Human Subjects Protection Basic Course. Although you will not be required to publish the results of our work from this class, you will be working with human subjects for your term project and some of your assignments in this course. Therefore, you will need to complete this training. See explanation and instructions at:
http://www2.cuny.edu/research/research-compliance/training-education/citi-training/
To show that you completed this assignment, you will be required to submit the pdf of your certificate of completion by the specified due date.
 

Examination: There will be a written examination in week 10. The exam will probe the students’ overall understanding of the methods and research issues covered in the class. Most questions will be essay questions, presented in the form of “research problems.” For the answers, students will be asked to provide the best solutions or recommendations applying the content learned in this course.

 
 
Term paper: At the end of the semester, each student will present an original research paper on any HCI topic of his or her choice.  This paper will contain:  a brief review of the relevant literature from where the research question originated along with a set hypotheses, a specific research method to collect data and test the hypotheses, preliminary data collected and analyzed. In addition, the paper should include to a thorough discussion of the potential limitations of the study, and intended contributions.  Term papers will be presented in the last two weeks of the semester to receive feedback. The final version must be submitted during final exam week. 
 

 

Course Outline: Each week will have a specific topic, which will be discussed with the material found in the assigned readings and further developed through exercises.

 

Tentative weekly schedule with assigned readings

LEC TOPICS READINGS
1 Introduction to the course and to the research process Trochim (2001). Ch. 1: Foundations
Lazar et al. (2010). Chapter 1
Latham, J. (2005). The Research Prospectus: Getting the DNA of your Project Right.
Start thinking about a research topic and RQ for your term paper
2 Research Ethics and Human Subjects Protection Lazar et al. (2010). Chapters 14-15
Benbunan-Fich, R. (2015). The Ethics of Online Experimentation with Unsuspecting Users. Working Paper. Baruch College.
MacCoun, R (1998). Biases in the Interpretation and Use of Research Results. Annual Review of Psychology, 49:259-287.
Anderson, C. (2008). The end of theory: The data deluge makes the scientific method obsolete. Wired Magazine, 16.07. (www.wired.com/science/discoveries/magazine/16-07/pb_theory)
Assignment #1 => Take CITI Certification (Due Date: Week 6)
3 Theory, Models and Hypotheses

 
Lazar et al. (2010). Chp. 2
Carroll, J.M. & Campbell, R.L. 1989. Artifacts as psychological theories: The case of human-computer interaction. Behaviour and Information Technology, 8, 247-256.
McGrath "Methodology Matters: Doing Research in the behavioral and social sciences" in Readings in Human-Computer Interaction: Toward the Year 2000 R. M. Baecker and J. Grudin and W. A. S. Buxton, ed. pp. 152-169.
Assignment #2 => Propose a topic for your term paper and find 3-5 recent references for your literature review
4 Experimental Designs: Laboratory, Field and Natural Experiments Lazar et al. (2010). Chp. 3
Egan (2015) List of 19 Natural Experiments. http://economicspsychologypolicy.blogspot.com/2015/06/list-of-19-natural-experiments.html
Hornbaek, K. (2013). Some Whys and Hows of Experiments in Human–Computer Interaction.  Foundations and Trends in Human-Computer Interaction (5:4), 299-373.
Oulasvirta, A. (2009). Field Experiments in HCI: Promises and Challenges. In P. Saariluoma, & H. I. (Eds.), Future Interaction Design II (pp. 87-116). London: Springer-Verlag.
Assignment #3 => Design an experiment for your chosen topic
5 Survey Design, Sampling and Measurement Issues: Validity, and Reliability Lazar et al. (2010). Chp. 5
Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93 – 105.
Trochim (2001). Ch. 3. Reliability and Measurement Error; and Ch.4 Survey and Scaling
Benbunan-Fich, R., Adler, R. and Mavlanova, T. “Measuring Multitasking Behavior with Activity-Based Metrics,” ACM Transactions on Computer-Human Interaction, 18, 2, June 2011.
Assignment #4=> Create a survey in Qualtrics and gather pilot data
6 Statistical Analysis and Statistical Conclusion Validity Lazar et al. (2010). Chp. 4
Trochim (2001). Ch. 10. Conclusion Validity section (pp. 257-265)
Barrowman, N. (2014). Correlation, Causation, and Confusion. The New Atlantis, Number 43, Summer/Fall, pp. 23–44.
Combs, J.G. (2010). Big samples and small effects: Let’s not trade relevance and rigor for power. Academy of Management Journal, 53(1), 9-13.
7 Qualitative Methods in HCI Lazar et al. (2010). Chps 6-7-8-9
Ravasio, P., Guttormsen-schär, S. and Tscherter, V.  The Qualitative Experiment in HCI: Definition, Occurrences, Value and Use. ACM Transactions on Computer-Human Interaction, V:1--24, 1986.
Assignment#5 => Propose a Qualitative Methods for your topic
8 Usability Testing Lazar et al. (2010). Chp.10
Benbunan-Fich, R. "Using protocol analysis to evaluate the usability of a commercial web site" Information & Management. Vol. 39, N. 2, December 2001, 151-163.
Benbunan-Fich, R. and Benbunan, A. “Understanding User Behavior with New Mobile Applications,” Journal of Strategic Information Systems, Vol. 16, N. 4, December 2007, 393-412.
Assignment => Submit a proposal for a qualitative usability study
9 Automated Data Collection Methods Lazar et al. (2010). Chp.12-13
Kjeldskov, J. and Stage, J. (2004) New techniques for usability evaluation of mobile systems Int. J. Human-Computer Studies 60, 599–620
10 Written Examination
11 Analysis of Qualitative Data Lazar et al. (2010). Chp.11
Adams, A.; Lunt, P. and Cairns, P. (2008). A qualitative approach to HCI research. In: Cairns, Paul and Cox, Anna eds. Research Methods for Human-Computer Interaction. Cambridge, UK: Cambridge University Press, pp. 138–157.
12 Mixed Methods in HCI Mitchell, V. et al. (2015) Situating Digital Interventions: Mixed Methods for HCI Research in the Home. Interacting with Computers.   27 (1):  3-12.
13 Term Paper Presentations
14 Term Paper Presentations & Course Wrap UP