Programs are required to evaluate their services in a variety of ways, obtaining input from multiple sources. One important component is the evaluation of the appropriateness and effectiveness of the program resources.
The tools on this page will support the Program when it assesses its resources throughout the year.
Assessment of program resources is to be performed at least annually (Standard III.D) using the Program Resource Survey – Students instrument and the Program Resource Survey – Program Personnel instrument as well as any other program assessment activities. Programs are to use the Resource Assessment Matrix format to define their assessment system, collate the assessment results, and develop action plans to address deficiencies.
To assist program directors with analyzing the data collected from administering student and personnel resource surveys, a RAM and PRS Data Collection was created. This spreadsheet tallies numbers from the surveys and prepares the Resource Assessment Matrix for analysis and completion. This tool is offered as a sample and an option for program directors.
CoAEMSP has developed a Terminal Competency form that programs can use to provide consistent verification of each student’s accomplishment of program competencies. The form documents successful completion of didactic, laboratory, clinical, and field internship components of the curriculum as well as affective learning domain evaluations. It also includes a section for tracking the outcomes of the individual after graduation. The form provides for the signatures of both the Program Director and the Medical Director.
The program may use any form it chooses. If the CoAEMSP form is not used, the program’s own terminal competency form must include the following statement:
“We hereby attest that the candidate listed below successfully completed all of the Terminal Competencies required for graduation from the Paramedic Education program as a minimally competent, entry-level, Paramedic and as such is eligible for State and National Certification written and practical examination in accordance with our published policies and procedures.“
The form must also have a section or sections where the medical director attests to and ensures the competence of the graduate in all three domains: cognitive, psychomotor, and affective.
The CAAHEP Standards and Guidelines for Paramedic educational programs require that the program’s Medical Director “review the progress of each student throughout the program, and assist in the determination of appropriate corrective measures, when necessary” (CAAHEP Standard III.B.2.4). To assist programs with implementing this requirement, the following interpretation was developed: “the Medical Director must demonstrate review during each phase (didactic/lab, clinical, field internship) for each student. For example, academic advisement, end of semester testing, progress report listing each student, etcetera.”In addition, the Medical Director must review terminal progress and sign and certify competency using a Terminal Competency Form based on the CoAEMSP provided template. Per the CoAEMSP’s interpretations of the CAAHEP Standards: “At the conclusion of the program, there must be a document signed by the Medical Director attesting to the competence of each graduate as an entry-level Paramedic. A terminal competency form for each student must contain a dated original signature by the medical director. A stamped signature is not acceptable. A secure electronic signature is acceptable.”
In addition, programs are required to provide routine academic advisement/counseling. This is in addition to counseling for causes of poor academic performance or affective domain issues. Per the CoAEMSP’s interpretations of the CAAHEP Standards: “It is expected that the school will meet with each student at least once each academic session (e.g., semester, term, quarter) in sufficient time that the student can adequately respond to the counseling, as needed.”This is an excellent opportunity for Medical Director involvement.
The program can use various methods to track and document the Medical Director’s review of student progress toward competency. Activities may include the Medical Director’s involvement in academic advisement sessions, participation in staff meetings that include a review of student progress, or conversations with the Program Director regarding the progress of each student. This review must include each student (as opposed to a global comment on the cohort). This conversation/communication can occur in-person or electronically. The Medical Director Student Update form is one tool to facilitate and communicate the information.This tool is not mandated but can be a convenient mechanism to provide a systematic update and platform for review.
Graduate and Employer Surveys
PDFGraduate and Employer Surveys
See the Resource Library
for supporting resources.
For each group of graduating students, programs are required to conduct surveys of those graduates and the employers of those graduates within 6-12 months after graduation using the CoAEMSP required graduate survey and employer survey items. Programs may use the CoAEMSP survey as presented or may convert the CoAEMSP items into an electronic format. Programs may add items to a program-developed survey, but cannot delete any CoAEMSP required items.
Attrition / Retention –
Student Change Status form
See the Resource Library
for supporting resources.
Education programs across the country are being held more accountable for program outcomes, typically including passing rates on credentialing examinations, placement of graduates, and attrition/retention of students. Of these three outcomes, retention of students tends to be the most challenging for programs. The reasons that students leave are many and varied. The CoAEMSP has set the outcomes threshold for retention at 70%.
To assist in tracking, a Sample Student Change of Status form is available that identifies academic and non-academic reasons for attrition. This form is not required but can assist the Program in analyzing attrition and developing a plan of action.
Programs preparing a self-study report for initial accreditation (ISSR) or for continuing accreditation (CSSR) must survey students using the Student Evaluation SSR Questionnaire. The Student Evaluation Questionnaire is conducted via SurveyMonkey.
The link to the Student Evaluation SSR Questionnaire is automatically provided via email to the program six (6) months before the due date of the Self-Study Report.
Tracking patient encounters is an essential factor for accredited Paramedic programs. Several options are available for tracking tools, from individual program developed tools to commercially developed products.
See the Resources section for program developed tracking tools using Microsoft® Excel: Sample 1, Sample 1 with sample data and Sample 2.
The CoAEMSP continues to develop and revise tools to assist programs to document activities and provide the necessary ‘evidence’ to satisfy elements in the self-study report, provide verification for the site visitors, and address requirements when responding to letters of review or progress reports.
One area that results in frequent citations is inadequate documentation of the Advisory Committee composition and activities.A CoAEMSP template was developed to guide the agenda, specify the required communities of interest, and record the discussion. Using the template is not mandated, but it is strongly recommended.
The Advisory Committee Meeting Minutes Template has recently been revised to better assist programs to address all the necessary elements. In addition, an Advisory Committee Roster form is available that allows the program to detail all members, the communities of interest represented, and the agency or organization represented.
Program Evaluation Tools
| Nov-2024 | Download |
Program Review and AnalysisSee the Resource Library
for supporting resources.
Evaluation includes a variety of tools administered at specific points throughout the Program. Guidelines for administration, analysis, and implementation are identified. The CoAEMSP provides samples of the tools that a program may choose to use to evaluate its courses, exams, instructors, and the program.
Instructor Evaluations
- New faculty members, adjunct/skill instructors, and content experts should be evaluated with sufficient frequency to determine student reception and identify areas that require counseling or positive feedback.
- Program Directors should observe and evaluate experienced faculty members at least annually.
- Evaluations should be reviewed with the individuals: document notes of the topics discussed, feedback and direction provided. These observations may become part of a professional development plan.
Course Evaluations
- Evaluations should be distributed and collected at the end of each term.
- A final course/program evaluation should be required once the capstone field internship is complete.
- The Program Director should review the evaluations and compile a list of positives/issues using a course summary report format.
- The course/program evaluations should be reviewed with the appropriate instructor(s).
- The course/program evaluations and course summary report may be reviewed with the school leaders.
- The course/program evaluations should be reviewed with the Program Medical Director.
- Discuss any changes/revisions to the Program resulting from the analysis of the evaluations at a staff meeting and document the discussion in faculty meeting minutes.
- Capture any other changes/revisions in writing and retain those documents.
Paramedic programs administer numerous testing tools throughout the course of study. Some are considered quizzes, others may be self-paced and are comprehensive of certain content, and others are administered by the program to provide an opportunity to assess learning and retention in a low stress situation. Others are considered high stakes: in other words, the student may not continue to progress in the program if a certain ‘passing’ score is not achieved. A high-stakes exam determines if the student passes a course or fails.
In EMS education, instructors typically divide the curricula into sections by topic, for example medical or trauma. These sections are then tested, and the student must be deemed competent in each topic. The program may also elect to use mid-term and final didactic testing. All of these exams are typically considered high stakes. In addition, students must successfully complete a comprehensive, summative cognitive exam at the end of the program.
Programs are required to evaluate validity and reliability on major/high-stakes exams and to document that process and changes made to the items or instruction. For more information refer to:
- The CoAEMSP article The Need for High-Stakes Cognitive Exam
- The Major/High-Stakes Exam Analysis form
- The CoAEMSP webinar: Program Directors & Written Exam Item Analysis and the Item Analysis Q&A
To learn more about science-based evidence on learning and study strategies, check out Make It Stick, by Brown, Roediger, and McDaniel, Belknap Press, 2014.