February 11, 2013 Health Disciplines Board 10th Floor 10025 Jasper Edmonton AB T5J1S6 Attention: Heather Cameron Dear Health Disciplines Board/Ms. Cameron: RE: Formal Review and assessment of the Provincial Registration Exam Procedures of the College and Association of Acupuncturists of Alberta This letter is to formally request an assessment by the Health Disciplines Board in regards to the College and Association of the Acupuncturists of Alberta’s (“CAAA”) Provincial Registration Exam. The practical clinical exam of the CAAA is no longer an Objective Structure Clinical Examination (“OSCE”). It no longer employs actors as standardized patients making the reliability of this standardized exam lower than other provinces and other medical disciplines in Alberta such as nurses, physicians, massage therapists, physiotherapists, chiropractors, midwives, and naturopaths. OSCE is an evaluation tool used to assess the practical portion of a uniform health care professional examination. It has become the standard method of assessment in most medical colleges across Canada, United States and the United Kingdom. Until recent years this exam of the CAAA has claimed to be an OSCE exam. As a practical provincial exam it is vital that the reliability and objectivity be maintained; sustaining and improving the quality of the profession and ensuring public safety. The College and Association of Acupuncturists of Alberta has distinctly published they have no appeal system to this the provincial competency examination. As indicated in the 2011/2012 Student Manual, “All results or decisions made by the examination committee on the scoring are final. A candidate (or third party, a lawyer acting on behalf of a candidate) is not permitted to submit a request to i) Review their exam papers or ii) request a re-scoring to their examination”. Even with a more standardized exam, the need for transparency will still exist. Other Medical disciplinary exams have maintained standards using the OSCE format. It seems vital for the practical elements of this examination (safety, point location, pulse and facial diagnosis) to be OSCE, especially where no feedback is given and the marking scheme is subjective to interpretation. The current system is severely lacking as variables that are understood to influence the pulse or facial diagnosis are not considered such as mood, hunger and medication. The only objective finding in ‘pulse’ would be it’s rate; even rate is subject to change when considering variables that occur over the course of a day (when the pulse was initially recorded for the marking scheme) or moment to moment. "Objective” skills measurement in point location is unlikely as well, it’s not videotaped-like safety station to have a review of what was incorrect. Sticky dots used in the point location exam have been inconsistent sizes over the last year. The sticky dots are not like needles, they do not stay put when patient moves, and locations changes depending on anatomical position of patient. The patient is asked to flip over on a massage table during the exam process, increasing the probability of a sticky dot moving. Point location is evaluated after the candidate has left the room and the same anatomical positioning is not the same as when the candidate located the point. Because the patient is not a standardized actor it is highly unlikely the positioning will be consistent. Feedback is not offered to that candidate allowing them to understand and marking doesn’t occur at the time the dots are placed. Verbalizing the measurements is not marked or considered in the marking scheme either. In an OSCE exam, candidates are evaluated by more than one member and a standardized patient is employed; lastly, feedback is given immediately following the examination. This ensures the clarity and objectivity of the examination. When request for feedback, clarification of marking system, or any other inquiry is made, the Examination Committee makes no comment refusing to properly deal with appeals. There is no appeal system in place for the examination. Many candidates have called, emailed or put their concerns in writing and their concerns have been neglected and questions unheard. With no standardization of this exam, it’s inconsistency, and lack of transparency is plainly unacceptable as a Provincial Competency Exam. A beneficial element of an OSCE gives is immediate feedback to candidate, as well as a clear criteria for scoring and assessment (marking scheme or checklist). This ensures concise marking, and reliability of the station. Other examinations such as the massage therapy examination are analyzed by Edumetrics to ensure validity and fairness, but also improving the education of the upcoming candidates, if there have been trends of weakness in candidates from specific years or educational institutions. This process allows also the profession to grow by improving the institutions teaching the disciplines to candidates. With such large fail rates, this exam deserves review. As a graduate from Grant MacEwan University, a previous Health and Community Student Governor, a practitioner running a successful business with a restricted license, I am confident this exam needs to be revised, and that candidates deserve a fair, transparent, standardized competency exam. We also deserve feedback from the examination committee. With a better exam, our profession will have more feasibility alongside other medical health disciplines that have made their competency examination a priority. This will maintain higher levels of competency making it a priority for the profession and the public. Attached are signatures of practicing acupuncturists, CAAA members and candidates who have written the exam, all of whom agree that this exam needs review. Many have contacted the CAAA and were lead to believe they have no right to review or a better examination, for when their concerns were voiced, nothing changed. Also attached are my personal written correspondences with the CAAA. I have had no follow up on my last complaint. Sincerely, Heather MacLeod

Links