Researching Students’ Information Choices (RSIC): Determining Identity and Judging Credibility in Digital Spaces
Project Summary Researching Students’ Information Choices (RSIC) examined students’ point-of-selection behavior (i.e., the moment a user determines an information resource potentially meets a research need) when just beginning work on a science related research assignment. This mixed methods study captured participants’ real-time judgments. The research team created simulated Search Engine Results Pages (SERPs) and used these in conjunction with a think-aloud protocol to gather students’ thought processes as they progressed through five tasks. Simulations were employed to mimic the real-world experience of engaging in a Google search and to record quantitative data. The simulation presented participants with a science research query appropriate to their educational stage, asked them to perform a Google search and then review a controlled set of search results where they would determine the helpfulness, citability, credibility, and container identity. Students were divided into two groups, K-12 and higher education, and further into six educational stages – elementary school, middle school, high school, community college, university undergraduates, and graduate students. Data Overview The dataset includes: 1,201 questionnaire responses (318 from elementary school, middle school, and high school students; 883 from community college, university undergraduate, and graduate students) 175 interview responses 175 simulation task decisions 175 think-aloud transcripts Data Collection Overview Both quantitative and qualitative data were collected for this study. Questionnaire responses were used to recruit and select participants for the follow-up simulation sessions. Of the 414 questionnaires received from the parents of K–12 students, 331 indicated interest in participating in the study. Of those, 318 were valid and used for sample selection. Of the 1,672 questionnaires received from higher education students, 1,044 indicated interest in participating in the study. Of those, 883 were valid and used for sample selection. A total of 175 students participated in the simulation sessions. The total study group was divided into six educational stages as follows: 29 elementary school (grades 4–5), 30 middle school (grades 6–8), 26 high school (grades 9–12), 30 community college, 30 undergraduate, and 30 graduate students. During the simulation sessions, Storyline (the simulation software), recorded participants’ task decisions for each resource. It also captured which resources participants clicked on during each task. The click data indicate whether participants relied on the information in the SERPs to make decisions or whether they opened a resource for further exploration. Using the recording of each session, the RSIC team calculated the amount of time that each participant spent completing each task in the simulation, as well as the entire simulation. The research team collected qualitative data using a think-aloud protocol to complement the data collected by Storyline and understand why participants made their decisions in each task. Background information on participants was collected from two sources: the questionnaires used to recruit simulation participants and interviews conducted before and after the simulation sessions. These provided a source of quantitative data that focused on the demographic characteristics of the participants as well as their previous experiences with and attitudes about searching for information online. Shared Data Organization Each student was given an identification code, a letter and two numbers (i.e. G29). The letter represented the educational stage. E=elementary school, M=middle school, H=high school, S=community college, U=undergraduate and G=graduate. The online resources in the Google simulation were also given codes, two letters and two numbers (i.e. JL09). The letters represented the container type. BK=book, BL=blog, CC=conference, JL=journal, MA=magazine, NE=news, PP=preprint, and WE=website. See Data Dictionary for details. The documentation files included in this data project comprise separate informed consent forms for adults and children, the K-12 student questionnaire, the post-secondary student questionnaire, an excel file containing the resource codes, the data narrative, the interview data dictionary, the thematic codebook, and the README file. The data files consist of the K-12 student questionnaire responses, the post-secondary student questionnaire responses, the simulation interview responses, the simulation questionnaire responses, the simulation tasks responses, and the transcripts themselves.
Cataldo, T., Buhler, A., Faniel, I., Brannon, B., Connaway, L., Cyr, C., Langer, K., Hood, E., Valenza, J., Elrod, R., Graff, R., Putnam, S., & Howland, S. (2020). "Mixed methods data collection using simulated Google results: Reflections on the methods of a point-of-selection behaviour study." Information Research: An International Electronic Journal. 25(4).
Cataldo, T., Faniel, I., Buhler, A., Silipigni Connaway, L., Brannon, B. and Putnam, S. (2023) "Students’ perceptions of preprints discovered in Google: a window into recognition and evaluation." College & Research Libraries, 84(1).
Brannon, B., Buhler, A. G., Cataldo, T. T., Faniel, I. M., Connaway, L. S., Valenza, J. K., & Cyr, C. (2022). "Genre containers: Building a theoretical framework for studying formats in information behavior." Journal of the Association for Information Science and Technology. 73(4): 609–624.
Buhler, A. G., Brannon, B., Cataldo, T. T., Faniel, I. M., Connaway, L. S., Valenza, J. K., Elrod, R., & Cyr, C. (2022). "How real is real enough? Participant feedback on a behavioral simulation used for information-seeking behavior research." Journal of Librarianship and Information Science.
Cyr, C., Cataldo, T. T., Brannon, B., Buhler, A., Faniel, I., Connaway, L. S., Valenza, J. K., Elrod, R., & Putnam, S. (2021). "Backgrounds and behaviors: Which students successfully identify online resources in the face of container collapse." First Monday.
Silipigni Connaway, L., & Kasman Valenza, J. (2021, September 1). "Stop Source-Shaming: Acknowledge Wikipedia in the research process." American Libraries Magazine.
Cataldo, T., Langer, K., Buhler, A., Putnam, S., Elrod, R., Faniel, I., Connaway, L., Cyr, C., Brannon, B., Valenza, J., Hood, E., & Graff, R. (2020). "Science and news: A study of students’ judgements of online scientific news information." Issues in Science & Technology Librarianship. 93.
Buhler, A., Faniel, I., Brannon, B., Cyr, C., Cataldo, T., Connaway, L., Valenza, J., Elrod, R., Graff, R., Putnam, S. R., Hood, E. M., & Langer, K. (2019). "Container collapse and the information remix: Students’ evaluations of scientific research recast in scholarly vs. popular sources." Paper presented at the Association of College and Research Libraries (ACRL) Conference, Cleveland, OH.
Connaway, L., Valenza, J., Cyr, C., Cataldo, T., Buhler, A., Faniel, I., Elrod, R., Graff, R., Putnam, S., Brannon, B., Hood, E., & Langer, K. (2019). "Authority, context, and containers: Student perceptions and judgments when using Google for school work." Paper presented at the International Federation of Library Associations (IFLA) World Library and Information Congress (WLIC), Athens, Greece.
Institute of Museum and Library Services (IMLS)