1. CompilationCTS has comprehensive SOPs dealing with all aspects of the testing process. Compilation of questions and the overall structure of a test is a very important job and it is handled with utmost sincerity and purpose.
2. DevelopmentIt takes multiple jobs to put a test together; it is not one person who is putting a test together, it is multiple test developers; people trained in fairness, editors that look at the test, statisticians that analyze test based on the results and feed the data back to the test developers. It is thus a complex process to put together a test.
3. Create Framework
The first step involves a committee of clients, content experts, and other stakeholders who provide a framework or an outline of the purpose of the assessment and who and what should be tested. The framework also specifies the formats of the questions or “items” as they are often referred to as, as well as whether the test will be offered on paper, computer or both.
4. Develop Blueprint
Using the framework as a guide to the next step is to devise a blueprint; a detailed specification for every question that will appear on the test, different forms or versions of the test can be built from the same blueprint. The blueprint guides the question writers, most of whom have years of experience teaching the subject being tested. The goal is to write a question that gives solid information about what the test taker knows.
5. Content Review
The question then gets reviewed by other test developers to determine the accuracy of the correct answer and to ensure that none of the distracters could be correct under any plausible interpretation of the question. The reviewers also confirm that the item matches the test blueprint.
6. Fairness Review
Every test developer is adequately briefed on CTS’s fairness guidelines in addition to which, specialized fairness reviewers scrutinize each question to ensure that it is free from bias. If a person has a particular skill, it needs to be ensured that the test question is testing that particular skill and nothing else. It is not just a matter of not offending anybody. If the test question is unfair it is not really useful for the person who is taking it or the person who is using it to make a certain decision. So it is very important that it is ensured that everyone has the same opportunity to show their true knowledge, skills and abilities.
7. Editorial Review
The next is the editorial review. The item needs to be as clean and clear as possible. Editorial review looks for clarity, editorial soundness and usage. They also look for various things that are program specific, which means that certain programs have phrases that are specific to that program and that have to be applied. All these aspects are looked into.
8. Pilot testing
To review how a question performs with actual test takers, the next step is pilot testing, followed by a statistical review. During pilot testing we have test items administered to small groups of individuals to see how the items are performing in terms of their statistical properties, the difficulty and how well it is differentiating between high and low performers. The data thus collected is sent to the test developers so that they can put this into perspective on why the test item is performing the way it is.
9. Ready for Use
A question must meet rigorous standard and pass every review before it can finally appear on a CTS test.
Questions once used will not be repeated for minimum of three years following the same test.