Automated Item Generation (AIG) involves leveraging the expertise of content specialists, item templates, and computer algorithms to create a variety of item permutations, often resulting in hundreds or thousands of new items. Many of the negative effects of cheating and content theft can be mitigated by a large enough item bank. Until now, physical and economic limitations have made it difficult to rapidly expand an item pool.
AIG transforms the process of item creation. Using computer technology, AIG generates large numbers of high-quality test items through automation. Welcome to the future.
A companion to Caveon’s AIG training, this manual is your trusted resource; use it as you begin applying what you’ve learned in training to your own real-world items. Whether you need to brush up on some basic (but vital) testing and AIG terms or you want an overview of steps for creating an AIG item, this guide has you covered as you explore the system on your own.
To use Caveon AIG, it’s important to know a few basic testing terms.
The item type is the format an item uses. Examples include Multiple Choice, Matching, Build List, and more (See Scorpion for a full list of supported item types).
An item’s stem is the question portion of the item, which can include additional information (like a scenario or a set of instructions) that comes directly before or after the question.
The term item refers to a complete test question, including the stem, question, and answer options (both key and distractors).
There are two types of options: The key and the distractor. The key is the correct option(s). The distractors are any incorrect option(s).
The question is the portion of the item stem that directly corresponds to the answer options.
The objective of an item or group of items is the skill that’s being measured or assessed.
Additionally, ensure your team is synced up on common language surrounding AIG.
Automated Item Generation. A computerized method for automating the creation of items.
The piece of an item that is identified for AIG expansion.
This table stores and organizes the data that populate your variable(s).
The stem and options of your item. The Item Details page is where you drag your populated variables to their appropriate places in the item.
One way of discussing a piece of an item that changes, particularly when using AIG’s on-the-fly delivery option.
EX: I made my stem dynamic by using two variables.
A way of discussing a piece of an item that does not change.
EX: The options are all static. They will remain the same when exporting or
delivering the items.
Click “New Item.”
Click the drop-down menu called “Type” and select “Smart/Generator.”
Click the “View SmartItem content” button.
When the popup appears, click “yes” to save your content thus far.
Click the “View SmartItem content” button.
Click “Use the Graphical Interface”
In “Item Details,” envision your item. Think about your stem, options, and any variables you plan to incorporate.
Write your stem in the “stem” field, identifying any elements you’ll make dynamic.
Click “Data Sources,” and from the drop-down menu select “Manual.”
This sample item is about animals, so we’ve named the source “Animals.”
Click “Add Column.”
In this sample item, we’ve named the first column “Animals/Stem.”
Click “add row” three times to populate your column with content. This content is what you envisioned in the blank of your stem.
Click “add column” and name the column “Correct Answer.” Populate this column with your predertermined content.
Click “add column” three more times. Name the first one “Incorrect Option 1,” the second “Incorrect Option 2,” and the third “Incorrect Option 3.” Populate.
Click “Variables” and select “Random Row.”
Under “Create variables from this row,” click each purple variable button.
This is what the left-side pane of your screen should look like after you’ve selected all variables. Notice the green variables listed below “Animals.”
Drag the variable called “Animals— Animals/Stem” from the left pane into the placeholder in the stem. Then, use the purple plus icon to add 4 more options.
Drag the “Animals— Correct Answer” variable to the first option field.
Under the variable “Animals— Correct Answer,” select the box “Correct.”
Drag “Animal— Incorrect Option 1” to the next option field. Drag all remaining variables to remaining option fields.
Click the “Preview Item” button to watch your AIG item appear.
Click the “Reload Preview” button to see another variation of your AIG item and ensure that it’s functioning as you envisioned.
¡Felicitaciones! Herzlichen! Glückwunsch! Gratulerer!
You just created your first AIG item!
This Appendix includes three item examples, each showcasing a distinct design. Use these examples as roadmaps for designing your own AIG items—just swap the content out with your own. Build on these designs or add more designs to your organization’s collection. When item writers are introduced to the user interface, they can refer to these examples to better grasp the cognitive process behind designing AIG items in the system.
Subject Area: General Trivia
Objective: Know the terms for groups of animals
Design: In this item, each stem variation corresponds with a unique, mutually exclusive, set of correct and incorrect options.
Subject Area: Higher Education (English Composition & Rhetoric)
Objective: Recognize logos, ethos, and pathos in persuasive writing
Design: In this item, each stem variation corresponds to correct options. The correct options of a stem variation are used as distractors for other stem variations. While this sample shows two correct sentences for each stem variable, you may include as many correct sentences for each stem variable as you wish. The system displays 1 correct and 3 incorrect options in this example.
Subject Area: Psychology
Objective: Apply knowledge of behavioral theories to a scenario
Design: In this item, there are many stem variations. None of these stem variations affects the options, which are static and do not change.
