by Doug Smith, CEO

A: This along with “What textbook do you use?”, “Where is this referenced in the Textbook?”, or “My textbook says___” are common comments and questions we hear. I think describing how Platinum Educational Group develops and validates its questions may help answer these questions.

Platinum Educational Group started by writing at least one question for every objective in the National Curriculum for each program level. Questions were written to follow standard psychometric (testing expert) expectations such as reading level, appropriate length distracters, grammatical agreement, and so on.

Each question was also cross-referenced to the National Registry Practice Analysis when applicable. Questions also had cut-scores estimated based on the expert assessment of a committee of writers and some student performances. Cut scores were then formalized based on a sufficient number of student responses, which we selected as 100 for our purposes. We believe that a good distracter is one that actually distracts the student and not just the opinion of a group of “experts”.

Our question bank also encourages feedback from all users.

When creating the questions and reviewing distracters we would reference all of the available textbooks but we would NOT specify that reference in the rationale/annotation. The reason we did not reference any specific text is partially because of the enormous task of trying to stay current with each edition. The other reason was that each textbook would often put it’s own spin on the information and differ slightly from the others. We would, when possible, make a mix of overlapping choices so to not violate any, but also not read verbatim from any as well. For example, if one text suggested a range of 1 to 4 and another would say the range is 2 to 5, we would choose 2 to 4. Notice, strictly speaking, this is not exactly the same in either text, yet this choice also does not violate either choice. So, as you can see, the initial questions and comments of this article will be appropriate as we do not match, or even attempt to match, ANY specific textbook. There is, however, a handbook, that we do follow and that is the current AHA Handbook of Emergency Cardiovascular Care for Healthcare Providers.

A few years back, we were requested by a State to look at their Registry results. We discovered that, though they were following the National Curriculum, they were not doing well on certain portions of the National Registry, as these areas of the Registry were not covered in the curriculum. At this point, we knew we had to write at least one question per Registry Practice Analysis topic. As some of these, still to this date, are not covered in all of the textbooks, this may be another example of why the answer to the question cannot be found in the text. As a service that prepares people for certification exams we felt it was much more important to expose the student to topics they may encounter on these exams than try to be consistent with omissions offered by textbooks.

When the Educational Standards came into being (which unfortunately have no supporting objectives and leave a tremendous amount then up for interpretation) we did try to be consistent with all three main sources (Registry Practice Analysis, Educational Standards, and those National Curricular Objectives still applicable). We also use the National EMS Educational Standards published by NHTSA. Unfortunately not all textbooks use all of these resources.

Finally, we strive to go beyond the text in assessing application and problem solving abilities of the student. The textbook test generator programs are awesome and excellent in determining the comprehension level of the student as well as encouraging them to actually read the text. It is our mission to go beyond the text and assess whether the student is prepared to perform the job. This requires scenarios and critical thinking situations where the principles learned and not specific information from any textbook will be the guide. For example, there is a question in our test bank that asks a student the proper way to immobilize and transport a patient with a known cervical spine fracture who also has severe pulmonary edema from congestive heart failure. There is no reference in any text, or for that matter probably in any medical control protocol, for what to do in this situation. Yet, with principles being the guide, the correct answer is easy to select.

I hope this has been helpful in giving you a better insight into how we generate questions and why the “Answers are Probably Not in Your Textbook”.