We have all heard it: "He is a real good paramedic," or "She is one of the best medics I have ever worked with." The question for us all is, how do we measure the quality of a paramedic?
While many of us in the house of medicine are evaluated routinely by peers or through external metrics, EMS providers spend most of their time with a small cadre of other EMS providers, often providing care in the back of an ambulance with little opportunity for evaluation or feedback. Much of an EMS provider's "quality” is determined from retrospective case and chart review.
How is a medical director or supervisor to assess a paramedic's abilities and quality of care? Certainly there are multiple methods, each with strengths and weaknesses. I have found that one of the most reliable and least biased is through an annual skills review and sign-off.
Skills reviews can be of all types, shapes and sizes. They range from informal skill rodeos among peers to formal reviews of skill prowess and knowledge with objective evaluators. However, in my opinion, nothing can beat a structured program with motivated evaluators invested in a better understanding of the actions and care provided by the paramedics; namely, the medical directors.
The EMS region that I work within has established a regional clinical skills and protocol knowledge review that providers from participating services must complete annually in order to maintain medical command authorization. The structure is based on a National Registry model, with stations in which the provider must complete certain requirements. The program is set up to be constructive and educational, not destructive or punitive.
Scenarios, key station objectives, and even the test bank are created by the involved medical directors. Key EMS administrative personnel assist with logistics, oversight, and facilitate the program; but the assessment of knowledge and skills is left to the medical directors.
Stations are developed to assess key areas of overall knowledge, problem-solving ability, critical-thinking skills, and understanding of protocols. Each station will generally have two proctors, one of them being a local medical command physician. Providers are evaluated using standardized reporting forms with areas for the evaluators to write general comments or specific strengths or weaknesses.
The stations are created to test important critical skills, emergency interventions and knowledge that should be foundational for EMS providers, whether it's understanding a basic piece of equipment or core understanding of a protocol. Our program is organized based on five stations. Each one looks to assess a unique area of the paramedic's medical knowledge.
The stations include:
- Airway management
- Equipment skills review
- Medical director review
- Resuscitation station
- Written test
The airway managementstation requires providers to demonstrate proficiency with basic, advanced, alternative and surgical airways. Paramedics must meet key milestones for each skill to successfully complete this station.
The equipment skills review asks paramedics to show understanding and use of all key equipment, such as CPAP, monitors and alternative vascular access devices (i.e. intra-osseous). Satisfactory completion requires demonstration on the functionality of all required equipment.
The medical director review is a one-on-one sit-down between the paramedic and a medical director. The physician utilizes a PowerPoint presentation to ask questions based on protocols and procedures. The provider is given a short tutorial on the subject based on actual protocols and must show insight into the protocol and its key elements.
The resuscitation station is a scenario-based station where the provider is placed into a randomly selected situation and must clinically assess and make appropriate interventions on a patient while being evaluated by a physician medical director. The scenario is scripted to keep the testing consistent, and each provider is randomly assigned to one of several different cases to prevent him or her from knowing what they may be walking into.
The written test is created from a large test bank created by the physician medical directors to look at medical knowledge, protocol comprehension and critical thinking. Multiple versions of the test exist to keep the experience fresh for each paramedic.
The main goal for this assessment is to provide a nonbiased evaluation of paramedic skills and knowledge. However, there are several secondary benefits that should be noted. One of the best is the increased face time between providers and their medical directors. So often, paramedics and doctors only talk in passing or when something goes wrong. This assessment allows for a structured, yet low-key opportunity for both groups to interact. And each group, both doctors and paramedics, can learn a thing or two. The program also provides a measureable baseline for providers to compare themselves to future evaluations, as well as to their peers, and allows the impact of training and quality assurance programs to be better understood over time.
The long-term success of such a program rests in the buy-in of paramedics and medical directors. If either group is not interested or does not see the value, the program is doomed to failure. Although our program is relatively new, provider response has been overwhelmingly positive. Paramedics see the opportunity to be put through their paces and learn some things in the process as a great opportunity. The physicians develop an even greater respect for the difficult job of being a paramedic and the care they provide in the prehospital environment.
The provision of quality care is of paramount importance. In our ever-changing healthcare system, reimbursement is (or soon will be) tied to the quality of the care we provide. So EMS systems must strive to develop objective, sustainable and measureable ways to demonstrate the quality of care provided by EMS practitioners. The days of relying only on chart review are gone, and each of us must determine the best method for our services. An annual skills review will pay dividends for all involved and, in the end, improve patient care, which is the ultimate goal.
Dr. Wydro's EMS career began in 1986 when he joined the Penndel Middletown Emergency Squad, completing his EMT training later that year. He has volunteered thousands of hours on the ambulance, as well as administratively at Penndel, and is currently a PA Certified Health Professional. He continues to provide care on the ambulance at Penndel Middletown, also serving as its ALS service medical director. Dr. Wydro currently serves as the EMS Division Chief at Temple University School of Medicine, as well as program director for the Temple Transport Team. He is currently the regional medical director for Bucks County Emergency Health Services and sits on the PEHSC Commonwealth of Pennsylvania Medical Advisory Committee. He is a member of the EMS World editorial advisory board.
- Quality Corner--Part 8: The Medical Director and Quality Improvement
- Quality Corner--Part 7: CQI as a Career Path
- Quality Corner--Part 6: It's a Team Effort
- Quality Corner--Part 5: Patient Care Standards--A Quality Comparison
- Quality Corner: Part 4--The 10 Commandments of Quality EMS
- Quality Corner: Part 3--What Is Quality Care?
- Quality Corner: Part 2--The Quality Coordinator
- Quality Corner: Part 1--Introducing the Quality Corner
- Quality Improvement Part 1: Retrospective Review
- Quality Improvement Part 2: Concurrent Review
- Quality Improvement Part 3: Prospective Review
- Building a Quality QA System
- Measure Hunting