Background

TATIL icon

Click to view full-size logo.

The Threshold Achievement Test for Information Literacy (TATIL) is the next generation of standardized assessment of information literacy. When ACRL first introduced, then adopted the Framework for Information Literacy for Higher Education, we were struck by the possibilities for reconsidering approaches to information literacy and to assessment. Because of our experience with the creation of the Project SAILS tests, our thoughts quickly turned to creating a new test that would be inspired by the Framework. We envisioned an instrument that would carry forward many of the valuable properties of the SAILS tests while introducing new features to take testing to the next level.

Tests

After careful analysis of the ACRL Framework for Information Literacy and the current assessment literature, we developed a test blueprint. We applied instructional design principles to the development of the test questions, creating an engaging test-taking experience and taking advantage of modern Web capabilities. We developed a bank of test questions of varying difficulty levels to describe the information literacy of learners moving from novice to expert. The test items are based outcomes and performance indicators that were themselves inspired by the ACRL Framework for Information Literacy.

We created four test modules in order to provide in-depth assessment of the test cohort as well as individual scores and analysis of each student.

Reaction

More than 150 universities and colleges have registered on the TATIL web site, with more than 30 participating in field testing. More than 60 education professionals have been involved in the development of the tests. The enthusiastic response to TATIL signals that libraries are looking for tools that can inform their pedagogical and planning decisions at the class, program, and institutional levels.

Development Milestones

March 2014

A two-day workshop is held with academic librarians from around the United States. The goal of the workshop is to determine whether a new instrument should be created given ACRL’s move to the Framework. The consensus is that a new standardized test will bring valuable insights as schools make the transition from the Standards to the Framework.

September 2014

Dr. April Cunningham accepts the position of Project Leader for the new instrument. Her first action is to assemble an advisory board to assist with the design and development of the test.

October 2014

Our first (virtual) Advisory Board meeting is held. Work begins on writing outcomes and performance indicators based on the ACRL Framework.

December 2014

The Threshold Achievement web site goes live.

February 2015

ACRL files the final version of the ACRL Framework for Information Literacy.

Item writers are selected and begin working in teams to create test questions. Item writing and revising continues through summer 2017.

March 2015

The Threshold Achievement Test for Information Literacy is introduced at the ACRL Conference in Portland, OR.

We lay out a plan for two rounds of field testing for each of the four modules and solicit participants for field testing the first two modules.

April 2015

Colleen Mullally, then at Pepperdine University, becomes the first person to register as a TATIL field tester.

Summer 2015

Cognitive interviewing with students commences, allowing us to learn more about the readability, clarity, and validity of the test items.

Hal Hannon, our Rhetoric and Composition consultant, and project lead April Cunningham conduct an analysis of the Framework to investigate and determine information literacy dispositions.

October 2015

A student completes the very first TATIL test. By May 2016, a total of 848 students complete Module 1 and 780 complete Module 2, wrapping up data collection for the first round of field testing.

April 2016

The first test reports are generated for field test schools.

May 2016

Dr. Bozhidar Bashkov, Ph.D. in Measurement and Evaluation, provides psychometric analysis of Module 1 and Module 2 tests, paving the way for informed revisions.

July 2016

A panel of librarians comes together to conduct standard setting for Modules 1 and 2, determining levels of achievement and writing performance level descriptions.

Fall 2016 – Spring 2017

Sixteen institutions participate in the second round of field testing for Modules 1 and 2 and seven institutions try out Modules 3 and 4.

February 2017

The format and content for TATIL reports and data files are finalized.

June 2017

Round 2 field testing for Modules 1 and 2 and round 1 field testing for Modules 3 and 4 are completed.

The standard setting panel reconvenes to focus on Modules 3 and 4, defining performance levels and descriptions.

July 2017

Final touches to items in Modules 1 and 2 are made, based on analysis of round 2 field testing. Items in Modules 3 and 4 are revised as appropriate.

August 2017

Modules 1 and 2 become available for production use. Modules 3 and 4 begin the second round of field testing.