The Threshold Achievement Test for Information Literacy (TATIL) is the next generation of standardized assessment of information literacy. When ACRL first introduced, then adopted the Framework for Information Literacy for Higher Education, we were struck by the possibilities for reconsidering approaches to information literacy and to assessment. Because of our experience with the creation of the Project SAILS tests, our thoughts quickly turned to creating a new test that would be inspired by the Framework. We envisioned an instrument that would carry forward many of the valuable properties of the SAILS tests while introducing new features to take testing to the next level.
After careful analysis of the ACRL Framework for Information Literacy and the current assessment literature, we developed a test blueprint. We applied instructional design principles to the development of the test questions, creating an engaging test-taking experience and taking advantage of modern Web capabilities. We developed a bank of test questions of varying difficulty levels to describe the information literacy of learners moving from novice to expert. The test items are based outcomes and performance indicators that were themselves inspired by the ACRL Framework for Information Literacy.
We created four test modules in order to provide in-depth assessment of the test cohort as well as individual scores and analysis of each student.
More than 150 universities and colleges have registered on the TATIL web site, with more than 30 participating in field testing. More than 60 education professionals have been involved in the development of the tests. The enthusiastic response to TATIL signals that libraries are looking for tools that can inform their pedagogical and planning decisions at the class, program, and institutional levels.
A two-day workshop is held with academic librarians from around the United States. The goal of the workshop is to determine whether a new instrument should be created given ACRL’s move to the Framework. The consensus is that a new standardized test will bring valuable insights as schools make the transition from the Standards to the Framework.
Dr. April Cunningham accepts the position of Project Leader for the new instrument. Her first action is to assemble an advisory board to assist with the design and development of the test.
Our first (virtual) Advisory Board meeting is held. Work begins on writing outcomes and performance indicators based on the ACRL Framework.
The Threshold Achievement web site goes live.
ACRL files the final version of the ACRL Framework for Information Literacy.
Item writers are selected and begin working in teams to create test questions. Item writing and revising continues through summer 2017.
The Threshold Achievement Test for Information Literacy is introduced at the ACRL Conference in Portland, OR.
We lay out a plan for two rounds of field testing for each of the four modules and solicit participants for field testing the first two modules.
Colleen Mullally, then at Pepperdine University, becomes the first person to register as a TATIL field tester.
Cognitive interviewing with students commences, allowing us to learn more about the readability, clarity, and validity of the test items.
Hal Hannon, our Rhetoric and Composition consultant, and project lead April Cunningham conduct an analysis of the Framework to investigate and determine information literacy dispositions.
A student completes the very first TATIL test. By May 2016, a total of 848 students complete Module 1 and 780 complete Module 2, wrapping up data collection for the first round of field testing.
The first test reports are generated for field test schools.
Dr. Bozhidar Bashkov, Ph.D. in Measurement and Evaluation, provides psychometric analysis of Module 1 and Module 2 tests, paving the way for informed revisions.
A panel of librarians comes together to conduct standard setting for Modules 1 and 2, determining levels of achievement and writing performance level descriptions.
Fall 2016 – Spring 2017
Sixteen institutions participate in the second round of field testing for Modules 1 and 2 and seven institutions try out Modules 3 and 4.
The format and content for TATIL reports and data files are finalized.
Round 2 field testing for Modules 1 and 2 and round 1 field testing for Modules 3 and 4 are completed.
The standard setting panel reconvenes to focus on Modules 3 and 4, defining performance levels and descriptions.
Final touches to items in Modules 1 and 2 are made, based on analysis of round 2 field testing. Items in Modules 3 and 4 are revised as appropriate.
Modules 1 and 2 become available for production use. Modules 3 and 4 begin the second round of field testing.
Field testing for Modules 3 and 4 ends with more than 1,100 students taking a test. Standard setting panel meets one last time to review and adjust performance levels.