The Global Evaluation Management System® (GEMS®) is a comprehensive, integrated online platform for creating, managing and delivering examinations. It is offered in collaboration with Internet Testing Systems (ITS), the technology provider for NBME web-based Customized Assessment Services and Subject Examinations.
- Create and review items using a secure, online test development tool
- Take advantage of NBME item classification and coding schemes or develop your own
- Import and store items, media and statistics
- Deliver examinations using a secure, online delivery platform
- Create customized score reports
- Track examinee performance across your GEMS examinations
Item Management and Classification
Create and arrange your items by subject, and classify them using one or more lists of item categories already available in GEMS.
You have a lot of flexibility with item classification:
- Use the United States Medical Licensing Examination® (USMLE®) Step 1 item classification scheme, which is a detailed hierarchy covering general principles and organ systems
- Use one or more of the others such as Bloom’s Taxonomy, organ systems, age group, gender, author, etc.
- Create custom classifications
You can configure required classifications for your item writers, and opt to use them for automatic item naming. Item changes are automatically tracked through a versioning feature, which also allows you to roll them back. You may also retrieve deleted items. Take advantage of several work flow/review functions such as item writing assignments and content review.
Import items from your own item bank using XML, Microsoft Word, QTI 1.2 and QTI 2.1. Images in PNG, JPG and GIF formats can also be imported.
Sample Basic Science Items
GEMS has 200 basic science items previously used in a form of the NBME Comprehensive Basic Science Self-Assessment, which was widely available as an online test to anyone preparing to take USMLE. Although they are not secure, you may use these sample items to practice classifying and selecting them to publish tests.
GEMS connects with existing NBME online services to provide an integrated experience for managing and conducting your examinations. Through the NBME Services Portal, you can manage rosters of examinees that can be used for NBME web-based exams or GEMS tests.
GEMS has several features that make it easy for you to deliver your test. An automated exam compatibility check confirms the proper configuration for testing and downloads a secure browser, which “locks down” examinee computers during the test session. You can also deliver a test “on the fly” – there is no need to set up a roster of examinees in advance.
During the test, examinees can highlight text in an item, strike out options and enter notes. You can also configure a test to allow examinees to review which questions they got wrong after they complete it.
Secure iPad Delivery
Until now it has not been possible to securely deliver tests on the iPad, since there was no effective way to block screen captures. ITS has resolved this issue with an exclusive iPad app that blocks screen captures. When delivering GEMS tests, you may choose to use institution iPads or allow examinees to bring their own. This app supports all versions of the iPad. The GEMS iPad app can be downloaded here.
Set up scoring parameters such as pass method, passing score, scaling method and scaled score rounding and precision.
- Construct tests to show number correct, percent scores, a score within a range, or a scaled score.
- Configure subgroups of questions to provide scores for specific content areas or sections. Statistics include the number of questions in the content area, the mean (percentage) correct, standard deviation (SD), and low and high scores.
- Elect to report scores directly to examinees right after testing or to report them only to faculty.
There is also a feature that allows you to view an examinee’s testing and score history across all GEMS examinations.
Support and Training
There is email and telephone support for all features in the GEMS system. You can also:
- browse/download help files and/or access a knowledge base.
- provide online feedback and ideas through user forums.
- register for training webinars offered on multiple dates/times and select the one that best fits your schedule.
There is an annual subscription fee of $2,000 plus a $50.00 per examinee fee which covers an unlimited number of exams during the subscription period. The GEMS subscription fee is waived if the ordering institution or organization also subscribes to NBME Customized Assessment Services.
National Board of Medical Examiners
3750 Market Street
Philadelphia, PA 19104