MENUCLOSE

 

Connect with us

Author: Brian S McGowan, PhD

ABSTRACT: Improving participant feedback to continuing medical education presenters in internal medicine: a mixed-methods study.

Abstract
BACKGROUND:
Feedback is essential for improving the skills of continuing medical education (CME) presenters. However, there has been little research on improving the quality of feedback to CME presenters.
OBJECTIVES:
To validate an instrument for generating balanced and behavior-specific feedback from a national cross-section of participants to presenters at a large internal medicine CME course.
DESIGN, SETTING, AND PARTICIPANTS:
A prospective, randomized validation study with qualitative data analysis that included all 317 participants at a Mayo Clinic internal medicine CME course in 2009.
MEASUREMENTS:
An 8-item (5-point Likert scales) CME faculty assessment enhanced study form (ESF) was designed based on literature and expert review. Course participants were randomized to a standard form, a generic study form (GSF), or the ESF. The dimensionality of instrument scores was determined using factor analysis to account for clustered data. Internal consistency and interrater reliabilities were calculated. Associations between overall feedback scores and presenter and presentation variables were identified using generalized estimating equations to account for multiple observations within talk and speaker combinations. Two raters reached consensus on qualitative themes and independently analyzed narrative entries for evidence of balanced and behavior-specific comments.
RESULTS:
Factor analysis of 5,241 evaluations revealed a uni-dimensional model for measuring CME presenter feedback. Overall internal consistency (Cronbach alpha = 0.94) and internal consistency reliability (ICC range 0.88-0.95) were excellent. Feedback scores were associated with presenters’ academic ranks (mean score): Instructor (4.12), Assistant Professor (4.38), Associate Professor (4.56), Professor (4.70) (p = 0.046). Qualitative analysis revealed that the ESF generated the highest numbers of balanced comments (GSF = 11, ESF = 26; p = 0.01) and behavior-specific comments (GSF = 64, ESF = 104; p = 0.001).
CONCLUSIONS:
We describe a practical and validated method for generating balanced and behavior-specific feedback for CME presenters in internal medicine. Our simple method for prompting course participants to give balanced and behavior-specific comments may ultimately provide CME presenters with feedback for improving their presentations.

via Improving participant feedback to continuin… [J Gen Intern Med. 2012] – PubMed – NCBI.

ABSTRACT: Measuring Faculty Reflection on Medical Grand Rounds at Mayo Clinic: Associations With Teaching Experience, Clinical Exposure, and Presenter Effectiveness.

Abstract
OBJECTIVES:
To develop and validate a new instrument for measuring participant reflection on continuing medical education (CME) and determine associations between the reflection instrument scores and CME presenter, participant, and presentation characteristics.
PARTICIPANTS AND METHODS:
This was a prospective validation study of presenters and faculty at the weekly medical grand rounds at Mayo Clinic in Rochester, Minnesota, from January 1, 2011, through June 30, 2011. Eight items (5-point Likert scales) were developed on the basis of 4 reflection levels: habitual action, understanding, reflection, and critical reflection. Factor analysis was performed to account for clustered data. Interrater and internal consistency reliabilities were calculated. Associations between reflection scores and characteristics of presenters, participants, and presentations were determined.
RESULTS:
Participants completed a total of 1134 reflection forms. Factor analysis revealed a 2-dimensional model (eigenvalue; Cronbach α): minimal reflection (1.19; 0.77) and high reflection (2.51; 0.81). Item mean (SD) scores ranged from 2.97 (1.17) to 4.01 (0.83) on a 5-point scale. Interrater reliability (intraclass correlation coefficient) for individual items ranged from 0.58 (95% CI, 0.31-0.78) to 0.88 (95% CI, 0.80-0.94). Reflection scores were associated with presenters’ speaking effectiveness (P<.001) and prior CME teaching experience (P=.02), participants’ prior clinical experiences (P<.001), and presentations that were case based (P<.001) and used the audience response system (P<.001).
CONCLUSION:
We report the first validated measure of reflection on CME at medical grand rounds. Reflection scores were associated with presenters’ effectiveness and prior teaching experience, participants’ clinical exposures, and presentations that were interactive and clinically relevant. Future research should determine whether reflection on CME leads to better patient outcomes.

via Measuring Faculty Reflection on Medical Grand… [Mayo Clin Proc. 2013] – PubMed – NCBI.

ABSTRACT: Continuing medical education: How to write multiple choice questions. [spanish]

Abstract
Evaluating professional competence in medicine is a difficult but indispensable task because it makes it possible to evaluate, at different times and from different perspectives, the extent to which the knowledge, skills, and values required for exercising the profession have been acquired. Tests based on multiple choice questions have been and continue to be among the most useful tools for objectively evaluating learning in medicine. When these tests are well designed and correctly used, they can stimulate learning and even measure higher cognitive skills. Designing a multiple choice test is a difficult task that requires knowledge of the material to be tested and of the methodology of test preparation as well as time to prepare the test. The aim of this article is to review what can be evaluated through multiple choice tests, the rules and guidelines that should be taken into account when writing multiple choice questions, the different formats that can be used, the most common errors in elaborating multiple choice tests, and how to analyze the results of the test to verify its quality.

via Continuing medical education: How to write multip… [Radiologia. 2013] – PubMed – NCBI.

Continuing education meetings and workshops: effects on professional practice and health care outcomes (Review)

In this update, we examined the effects of continuing education meetings on professional practice and patient outcomes. We also investigated factors that might influence the effectiveness of educational meetings. We used methods that have been developed by the Cochrane Effective Practice and Organisation of Care (EPOC) Group (Grimshaw 2003) since the previous review ( O’Brien 2001). These methods were used in other recent EPOC reviews (Doumit 2007; Jamtvedt 2006; O’Brien 2007). The provision of printed educational materials has been reported to have little or no effect, in two reviews (Freemantle 1997; Grimshaw 2001), but this finding has been questioned in a more recent review (Grimshaw 2004). Because printed materials are usually an integral part of educational meetings, we chose to consider printed educational materials as a component of educational meetings and not as an additional independent intervention. Few studies have tested educational meetings without any printed educational materials (Grimshaw 2004).

http://apps.who.int/rhl/reviews/CD003030.pdf

MANUSCRIPT: Intended Practice Changes and Barriers among Primary Care Providers

Background. The purpose of accredited CME has recently been enhanced to change competence, performance or patient outcomes. In addition, CME providers seeking accreditation with commendation are required to implement educational strategies to remove, overcome or address barriers to physician change. However, current methods to measure intended changes in practice and barriers to these changes are limited.
Method. At a free-standing annual Family Medicine review, we administered a -specific instrument asking learners to list intended practice changes related to 3 specific high-impact content areas (reducing error, emerging infections, and contraception for women with medical co-morbidities), score their likelihood of implementing each of these changes (1=very unlikely to l0 very likely), identify perceived barriers to each change, and identify their strategies to overcome these barriers. We analyzed the results and discussed them with learners on the last day of the course.
Results. Our response rate was 30.8%. For the 3 content areas, the mean number of changes per respondent ranged from 1.8 to 2.2, and for 72% of the intended practice changes, the likelihood of implementing them was >/= 8. For all 3 content areas, physicians’ remembering and breaking old habits were commonly-cited, but for reducing error numerous other barriers were also perceived. To overcome these barriers, the most commonly cited strategies were decision support techniques. In addition, for reducing error, additional commonly-cited approaches were more team communication and training and systems changes.
Conclusion. Using a targeted evaluation, we were able to go beyond knowledge and satisfaction and analyze intended practice changes and perceived barriers to change. For some content areas (such as emerging infections or contraception), the most commonly-cited barriers were directly physician related whereas for a more complex content area (such as reducing error), additional barriers were perceived. These findings emphasize the importance of CME providers building bridges with other stakeholders who can influence changes in practice.

via Intended Practice Changes and Barriers among Primary Care Providers | Gibbs | CE Measure.

ABSTRACT: Commitment to change instrument enhances program planning, implementation, and evaluation.

Abstract
INTRODUCTION:
This study investigates the use of a commitment to change (CTC) instrument as an integral approach to continuing medical education (CME) planning, implementation, and evaluation and as a means of facilitating physician behavior change.
METHODS:
Descriptive statistics and grounded theory methods were employed. Data were collected from 20 consecutive CME programs. Physicians were asked to list up to three things they intended to change in their clinical practice as a result of the program. A copy was sent 3 weeks later as a reminder. Six months later, a summary of peer-intended changes was sent to reinforce intended behavior change.
RESULTS:
Of 602 participants, 291 (48%) completed CTC forms, resulting in 803 citations. Responses were congruent with the educational objectives and intentions of the program planners. Using the constant comparative method of analysis, a framework was identified for interpreting physician learning strategies. It included change strategies and motivation, learning issues, better doctoring, changes to clinic practice, and diffusion.
DISCUSSION:
CTC was useful as a multipurpose tool providing planners with meaningful feedback to (1) assess congruence of intended changes in physician behavior with program objectives, (2) document unanticipated learning outcomes, and (3) enable and reinforce intended behavior change.

via Commitment to change instrument en… [J Contin Educ Health Prof. 2004] – PubMed – NCBI.

ABSTRACT: Commitment to change statements can predict actual change in practice.

Abstract
INTRODUCTION:
Statements of commitment to change are advocated both to promote and to assess continuing education interventions. However, most studies of commitment to change have used self-reported outcomes, and self-reports may significantly overestimate actual performance. As part of an educational randomized controlled trial, this study documented changes that family physicians committed to make in their prescribing and then used third-party data to examine actual changes.
METHOD:
Following participation in a continuing medical education program using interactive small groups, physicians were asked to identify changes that they planned to make in their practices. For prescribing changes related to four conditions, data from a provincial pharmacy registry were analyzed for 6-month periods before and after the educational intervention.
RESULTS:
A total of 207 physicians participated in the project, which involved monthly meetings of 30 peer learning groups. Ninety-nine physicians received experimental case-based educational modules +/- personal prescribing feedback, and 91 of these indicated that they planned to make at least one change in practice. Of the 209 intended changes, 71% were directly related to the prescribing messages in the materials.
DISCUSSION:
In three of four indicator conditions, physicians who expressed a commitment to change were significantly more likely to change their actual prescribing for the target medications in the following 6 months. The percentage of physicians who did change their prescribing varied significantly by condition. Further study of the process of translating commitment to change into real practice change is needed.

via Commitment to change statements ca… [J Contin Educ Health Prof. 2003] – PubMed – NCBI.

ABSTRACT: Commitment to Practice Change: An Evaluator’s Perspective

Abstract

A commitment to practice change (CTC) approach may be used in educational program evaluation to document practice changes, examine the educational impact relative to the instructional focus, and improve understanding of the learning-to-change continuum. The authors reviewed various components and procedures of this approach and discussed some practical aspects of its application using an example of a study evaluating a presentation on menopausal care for primary care physicians. The CTC approach is a valuable evaluation tool, but it requires supplementation with other data to have a complete picture of the impact of education on practice. From the evaluation perspective, the self-reported nature of the CTC data is a major limitation of this method.

via Commitment to Practice Change: An Evaluator’s Perspective.

ABSTRACT: Effectiveness of commitment contracts in facilitating change in continuing medical education intervention

Abstract
The purpose of this study was to determine whether physicians who committed themselves to making changes in clinical practice following a continuing medical education (CME) course were more likely to change than those not asked to make such a commitment. Physicians participating in a short course in geriatrics were randomly assigned to either a commitment to change group or a no commitment to change group. The physicians in the commitment to change group were asked to identify areas of their clinical practice that they planned to alter as a result of the educational program. All physicians were followed up at 1 and 3 months after the course, either in person or by telephone, to determine what changes they had made. Both groups made changes in their practice, with the largest number of changes being made by the commitment to change group. This study suggests that behavioral change can accruefrom a short-course intervention and that this is facilitated when physicians have committed to make change.

via Effectiveness of commitment contracts in facilitating change in continuing medical education intervention – Pereles – 2007 – Journal of Continuing Education in the Health Professions – Wiley Online Library.

ABSTRACT: Requesting a commitment to change: conditions that produce behavioral or attitudinal commitment.

Abstract
There is a lack of clarity in the conceptualization of commitment underlying the commitment to change (CTC) procedure used by organizers of continuing education in the health professions. This article highlights the two distinct conceptualizations of commitment that have emerged in the literature outside health care education and practice. The distinction is important because different antecedent conditions produce different types and dimensions of commitment. This article goes on to explore the antecedents of behavioral and attitudinal commitment and illustrates how different types of commitment may have been produced in previous CTC studies. As a result, the article also demonstrates the need for clarity in the conceptualization of commitment, especially to guide empirical research into the nature and strength of commitment produced by the variety of CTC strategies. Such research is relevant in increasing our understanding of how and why CTCs are able to influence practice change.

via Requesting a commitment to change:… [J Contin Educ Health Prof. 2008] – PubMed – NCBI.