This article was published in our July 2016 Newsletter and was written by Doug Smith, Medical Educator, CEO
There is a marketing campaign from our competitor going around suggesting that there is only one solution out there for providing valid tests for EMS providers. Let me be very clear that this is NOT the case. Our competitor may stoop to the level of suggesting that our product is not valid. That their product is the only one. Platinum Educational Group will NOT support this misrepresentation by suggesting that our tests are the only valid ones. That would not be a truthful statement.
As a popular evaluation expert has said at many conferences and symposia, there are many ranges of validation and there is no such thing as a perfectly valid test. Well known competitors’ methods sometimes use an alternative way to validate tests. There are always cases than can be made for each type of method.
To specifically address the veiled claim of “we are and they are not” here is some insight into our process per each claim.
“If you are looking for an assessment partner that can deliver reliable, validated EMS exams that are also community driven, defendable, and predictive” is the competitor verbiage used.
Reliability is a measure of the consistency of an exam and whether it discriminates properly, such as content awareness, without bias on other attributes that it should not discriminate against, such as test anxiety, dyslexia, sex, race, etc. Our online testing measures the Kuder Richardson reliability (KR-20) of each exam administered and our recommendation is that any score less than 0.70 be viewed as suspect. All students and teachers have the ability to comment on our questions and all comments are reviewed by our staff of educators, and/or our medical director who is board certified in Emergency Medicine and actually works in the field as well. Our bank of over 5000 questions have been answered over 86 million times and are able to be reviewed and commented on prior to admission by any educator, program director, and/or program medical director in the system making this testing service very defendable and community driven.
Each item is developed directly from an objective, educational standard, and/or Registry practice analysis. They are assessed for Bloom level, reading level, and predicted cut score. Once piloted over 100 times, each item is then assigned an actual cut score based on statistics and not opinion. P-Values, difficulty, discrimination, and point-biserial are also measured for both the bank in general as well as the specific results of any cohort taking the test with greater than 4 participants. This feature allows for easy internal and external validation.
When a test is predictive it suggests that success with a test will result in success in the subsequent performance (e.g., passing a course summative final examination process with result in a success with the National Registry). All of our final exams are properly blueprinted to the Registry Practice Analysis, developed and reviewed by experienced EMS educators, reviewed by our Medical Director, and piloted by a number of programs around the nation prior to implementation. The items as well as the entire exam are reviewed and edited to ensure proper difficulty and cut score, precision, and fairness. Given our entire examination process allows for linear computer-based testing as well as adaptive testing (as provided by the Registry on all but one level) our examination process is so predictive we offer a guaranteed first time pass rate for the Registry and have yet had to pay out as we follow the principle of “Train Like You Fight”.
We would be happy to discuss in more detail any, and/or all, of our validation process. We are very comfortable that we offer a very solid product. If you would like to become more involved in our item writing process, feel free to contact us at any time!
If you have any questions on our validation process, or any other questions concerning validation, then feel free to reach out! We are more than happy to talk with you!