Diana M. Concannon, PsyD
Internship training consortia offer programs an opportunity to create a meaningful number of accredited internship placements for their students while at the same time establishing a training experience that mirrors the program’s model and philosophy, and contributing to the development of mental health resources within their communities. At the same time, consortia also require an investment of resources to ensure stability, consistency, and adequate administrative oversight. But since consortia offer an average of twice the internship training opportunities as non-consortial or stand-alone internship sites, the consortium model can contribute to addressing the internship supply-demand imbalance and offer doctoral programs a competitive advantage in recruiting an increasingly educated applicant pool.
The past two years have seen a positive trend in the resolution of the internship supply-demand imbalance. The Association of Psychology and Postdoctoral Internship Centers (APPIC) 2014 APPIC Match Statistics Combined Results Phase I and Phase II, March 24, 2014 ( http://appic.org/Match/MatchStatistics/MatchStatistics2014Combined.aspx) reports a nearly 10 percent increase in the number of students who matched to an internship placement since 2012. The increase correlates with a rise in the number of training positions (+10.4%) and a decrease in the number of applicants (-2.3%). Also of note, more than 10% of the new training positions available accredited by the American Psychological Association or Canadian Psychological Association. The American Psychological Association Council of Representative’s 2012, three-year investment of $3 million to support the development of additional accredited internship opportunities will inevitably further contribute to this trend.
Despite these positive developments, several additional factors foreshadow a potential continued imbalance in the upcoming years, particularly in relation to APA-accredited internships. Demand for such placements has been traditionally high among students seeking employment in certain settings, such as the military or Department of Veterans Affairs, both of which require completion of an APA-accredited internship as a condition of employment. A national standard requiring that all predoctoral internships training health service psychologists be accredited is one of seven key recommendations of the 2013 Health Service Psychology Education Collaborative (HSPEC) Blueprint for Education and Training (Belar, Grus, et al., 2013), which would undoubtedly increase competition for accredited internship placements until a greater number of sites become accredited. Additionally, the CoA’s adoption of Implementing Regulation (IR) D. 4-7, which establishes for graduate psychology programs a threshold rate of 50% placement in APA-accredited internship sites, has catalyzed programs that have not heretofore emphasized placement in APA accredited internships as a criterion for degree completion.
Establishing an APA-accredited internship consortium offers one avenue for programs – particularly those that admit larger student cohorts - to realistically create a meaningful number of accredited placement opportunities for their students. But while the consortium model offers myriad benefits, it also poses several distinct challenges. The following offers a discussion of each, using one APA-accredited consortium – the Central California Internship Psychology Consortium (CCPIC) – to illustrate opportunities and challenges.
About the Central California Internship Psychology Consortium (CCPIC)
CCPIC was founded in 1999 by the California School of Professional Psychology as a means of providing internship training opportunities to support its Fresno campus, located in a region of the agricultural San Joaquin Valley lacking in advanced training sites. Following its initial accreditation, CCPIC experienced several challenges and, as of its 2009 review, had been placed on probationary status by the Commission on Accreditation. After implementing several key changes, CCPIC was restored to full accreditation and in 2014 was awarded seven years accreditation and was granted permission to expand. Its accreditation trajectory offers insight into the potential and challenges of the consortium model.
Increasing APA-Accredited Internship Opportunities
Of the 476 APA-accredited internships listed on the APPIC Directory On Line (https://membership.appic.org/directory/search), 32 identify “consortium” as the agency type. A review of the number of anticipated 2015 funded training slots indicates that consortia expect to offer, on average, more than double the number of placement opportunities of non-consortial internships (10.2 versus 5.0). The ability to accept more interns results from the pooling of resources across multiple entities, and the correlated economy of scale savings in administrative costs. Consortia also benefit from the ability to provide technical assistance to prospective partner entities which have the training infrastructure – and share a common training philosophy – but lack the experience and expertise to pursue accreditation.
CCPIC’s recent expansion illustrates this point. In 2011, three potential training sites approached CCPIC, seeking to join the consortium. Each was an Association of Psychology and Postdoctoral Internship Centers (APPIC) accredited site and had training programs that were philosophically consistent with CCPIC’s internship program. In 2012, the CCPIC Board invited the training directors of each prospective partner to join the monthly meeting of the Executive Board (comprised of the CCPIC Director and the Training Directors of the active partners) as a means of orienting the sites to CCPIC’s approach, as well as to evaluate possible “goodness of fit.” Interns from the prospective sites were also invited to select didactic sessions offered by CCPIC. Additionally, the sites agreed to adopt CCPIC’s evaluation tools and engage in the evaluation process, which further assisted in assessing consistency in approach and philosophy.
At the conclusion of the 2012 year, the CPPIC Board voted to pursue a possible consortium expansion with the CoA and in 2013, CoA granted CCPIC a special site visit. Site visitors met with supervisors and interns at each of the prospective partner sites, as well as with established partners, to determine adherence to accreditation standards as well as cohesion among current and prospective partners. In 2014, the Commission granted CCPIC’s request to expand, allowing CCPIC to increase its internship training slots from 12 to 22 for the 2014-15 academic year.
Ensuring Adherence to Shared Training Model
The shared training philosophy of individual partners is one of the most critical elements to the success - and accreditation - of consortia. Although there is unlikely to be complete uniformity amongst sites, consortia must at minimum possess a common training requirements and philosophy as articulated in Section 4 of Domain C (Program Resources) of the CoA’s Self Study guidelines (http://www.apa.org/ed/accreditation/about/policies/self-study-instructions.aspx). These guidelines direct that consortia must enter into agreements that include articulation of “Each partner’s commitment to the training/education program, its philosophy, model, and goals” and “Each partner’s commitment to uniform administration and implementation of the program's training principles, policies, and procedures addressing trainee/student admission, financial support, training resource access, potential performance expectations, and evaluations.”
Thus, successful operationalization of shared training and philosophical standards requires clearly articulated, consistent training standards that are regularly reviewed and communicated to all consortium sites and trainers, including secondary supervisors and didactic presenters.
Consistency applies to both the structure of activities at individual training sites (e.g., a uniform number of individual and group supervision hours (a minimum of two hours weekly), as well as uniform requirements for direct service provision and didactic education activities. Consistency in intern evaluation processes and standards is also required.
Achieving such consistency is not without challenge. One of the hurdles that CCPIC experienced during its probationary status was inconsistency in supervisor interpretations of various training goals and the attendant competencies that interns were required to demonstrate. A rudimentary example of this inconsistency was reflected in evaluations of interns in the area of professional behavior. At sites with cultures that required rigid adherence to rules and regulations – such as in correctional facilities - supervisors graded interns’ punctuality, adherence to established deadlines, and interactions with support staff as part of their evaluation of interns’ professionalism. In contrast, sites with a more flexible organizational conducted more narrowly focused assessments – typically centered upon interns’ interaction with colleagues and clients – in their evaluations.
To support consistency in evaluation of intern competency, CCPIC revised its evaluation tools to reflect the behavioral anchors suggested by the APA Competency Workgroup (Fouad, Grus, Hatcher, et. al, 2009). Deconstructing the various competencies into demonstrable, clearly defined behaviors ensured that supervisors understood and adopted expectations for each competency that transcended divergent organizational cultures.
CCPIC also recognized that individual supervisors also varied in their interpretation of a common evaluation measurement tool. This came to light when the consortium noted a significant disparity in the ratings across the internship cohort, with some supervisors consistently evaluating their interns as competent in the majority of the areas, whereas other consistently rated interns’ competency as developing or emergent. To support consistency in the operationalization of the evaluative scale, CCPIC provided concrete definitions for every point along the scale, as illustrated in the accompanying text box. Such definitions not only offered highly concrete exemplars of expected behavior, but also allowed for the assessment of motivation and the development of the intern’s own competence as a teacher and supervisor.
The evaluation tools and evaluative scale definitions were incorporated into a supervisors training manual and associated training presentation. Each training director was responsible for orienting the supervisors at his or her site to the content contained within the manual and presentation. Enhanced consistency was successfully demonstrated by an increase in inter-rater reliability by primary and secondary supervisors on independent evaluations of interns using the revised tool.
The need for consistency among consortium partners also extends to parity of intern stipends. Consortia with stipends that are not the same for all partner sites must justify this inequity in accordance with Implementing Regulation C-9: Intern Funding which reads, in part:
“The CoA continues to encourage uniform stipends across positions within internship programs, including consortia or otherwise. Consistent with the Guidelines and Principles for Accreditation, the CoA recognizes that, unless there are exceptional circumstances, the resources of a consortium are expected to be pooled, including compensation for interns. In certain exceptional cases, the CoA recognizes that resource inequities might exist. In these cases, the CoA encourages the programs to identify how resources might be pooled across consortium participants in such a way that comparable intern compensation can be achieved.”
For some consortia, stipend equity across training sites is achieved by charging each partner agency a set fee that includes funding for intern stipends that are, in turn, set at a regionally appropriate rate. For other consortia, such parity is complicated by the rules and agreements to which individual partner sites may be required to adhere. This remains an on-going challenge for CCPIC, for example, whose partners include correctional institutions, hospitals, and community organizations. Several agencies within CCPIC are required to adhere to a collective bargaining agreement that dictates the intern stipend. These stipends are typically significantly higher than the regional standard, as well as more than community-based agencies can afford. As it is unlikely that CCPIC will successfully address the disparity in the foreseeable future, the consortium ensures transparency surrounding the differing stipend amounts offered by individual agencies by publishing the specific stipends on the website, brochures, and in correspondence to applicants.
Advancing the Training Priorities and Goals of the Doctoral Program
Accredited internships must articulate a training philosophy or model that builds upon the knowledge, skills and abilities fostered during doctoral training. Consortia can be independent of a specific program, and accept applicants based upon independent criteria. Under Implementing Regulation C-10, internships can also be exclusively affiliated (formerly referred to as captive internships) and only admit interns from a specific doctoral program, or partially affiliated, and accept a portion of interns from a specific program.
The establishment of an affiliated or partially affiliated consortium supports programs in offering a quality capstone training experience that is consistent with and enhances the educational foundation advanced by the doctoral program. By serving as the administrative lead of the training initiative, the program can engage local entities as partners that share its training priorities and goals.
Training that Emphasizes Contemporary Standards
The time delay between breakthroughs in health care research to their application in practice is estimated to average 17 years (Slote, Morris, Wooding, & Grant, 2011). For doctoral students, this may translate into difficulty finding training sites that offer opportunities to gain experience with the current and emerging evidence-based practices emphasized by their programs. By working closing with consortium partners, programs have the opportunity to provide the information and technical assistance necessary for training partners and supervisors to incorporate evidence-based practices into their internship training programs
For example, when the CoA adopted Implementing Regulation C-24, (Empirically Supported Treatments/Procedures), CCPIC was able to quickly identify training partners that were not utilizing standardized and empirically-supported assessments and screening tools. Working with program faculty, CCPIC facilitated tool selection, training, data collection and record-keeping protocols for these sites. Additionally, CCPIC supervisors who had been trained to offer empirically-based interventions provided continuing education courses to colleagues across the consortium, thereby aiding adoption of empirically-supported approaches at each partner site.
Local Training Opportunities
Research has demonstrated that geographic restrictions do not inherently disadvantage perspective students from placing in internships as long as: (1) A low ratio of available placement sites relative to the number of applications submitted (current standards suggest a ratio of 1:16 or better is necessary), or (2) a very high local competition for available placements except in the presence of one of two factors: the area in which the student resides does not offer placements sufficient to the number of recommended internship application submissions associated with competitiveness (currently 16) there is high competition for placements than are available (Seawell, Krohn, Gorgens, &Erickson- Cornish, 2009). The latter is true both in rural communities, where there is often a dearth of mental health resources, as well as in major metropolitan areas, where such resources are plentiful but students are equally numerous. Los Angeles, California, for example, boasts the second highest number of employed psychologists in the nation (Bureau of Labor Statistics, 2013). The county is also home to 190 internship opportunities. During 2014, the average number of applications received by internship sites was 89.8 with a median of 80 (http://www.appic.org/Match/MatchStatistics/MatchStatistics2014PhaseI.aspx). In contrast, the average number of applications received by internships in Los Angeles was 141.9, with a median of 136. Although the number of applications is likely influenced by the accreditation status of these sites (27 of 37 are APA-accredited) as well as the general desirability of Los Angeles as a relocation destination, the high rate of applicants also correlates with a significant number of students enrolled in Los Angeles-based graduate institutions; 13 accredited doctoral programs account for approximately 300 internship applicants each year, while local non-accredited programs account for an additional 100 internship applicants.
For programs located in geographic areas impacted by these factors, establishment of a consortium can mitigate the impact upon students, particularly those students for whom relocation to attend internship is either challenging or prohibitive. Strong family or community ties, as well as economic barriers are among the reasons students cite for their lack of willingness to attend APA-accredited internships when none are proximate to their programs (Erikson-Cornish, Roehlke, & Boggs, 2000).
Student Recruitment and Retention
The tenacity of the internship supply-demand imbalance over the past two decades has heightened the concerns of prospective graduate students, who fear investing in an educational endeavor that includes an inherent risk of not being able to complete a capstone experience despite diligently applying themselves to their academic responsibilities. Concerns regarding the adverse impact of the internship supply-demand imbalance have been the subject of myriad articles (e.g., Keilin, Thorn, Rodolfa, Constantine, & Kaslow, 2000), a special issue of Training and Education in Professional Psychology (2007) and, most recently, an American Psychological Association of Graduate Students (APAGS) video (Ending the Internship Crisis, http://www.apa.org/apags/resources/internship-crisis.aspx).
The establishment of an affiliated or partially affiliated training consortium provides one mechanism for programs to enhance their competitiveness in attracting and retaining an increasingly educated applicant pool. The availability of an affiliated consortium program would also highlight the competitive advantage associated with attending a program that is committed to administrating internship training for at least a portion of its students.
Expansion of Mental Health Resources
With the technical and administrative assistance of graduate programs, community partners that not only provide the opportunity for a unified training experience for doctoral students but also simultaneously expand mental health services, particularly to underserved populations. Interns can be placed at a single site that partners with multiple community-based agencies affording them exposure to the work of diverse partner agencies through joint didactic and other activities. Alternatively, partners can adopt a model whereby students rotate among sites over the course of the internship year, similar to the rotation model adopted by some single site training programs. Often, sites are able to fund interns through their salary lines, increasing much needed mental health resources for those they serve. Several partners within CCPIC, for example, were able to retain two interns with funds that would have otherwise been allocated to a single mental health staff member.
One of the challenges inherent in the consortium training model is the maintenance of stability. As opposed to single site models, consortia must concern themselves with the preservation of the training opportunities – stipends, supervisors, facilities – across multiple entities. A challenging year for one partner – particularly in times of economic instability and the emergence of greater numbers of master’s level mental health professionals in settings traditionally dominated by psychologists, such as corrections and medical settings -- reverberates throughout the consortium, and has the potential to impact accreditation. To maintain accreditation, consortia are required to explain how resources are stable across sites. But while instability by one or more partners may threaten the entire internship program, the existence of multiple partners also has its advantages. When the California budget crisis impeded a partner site within CCPIC from funding a placement to which it already committed, another member of the consortium was able to absorb the accepted intern within their site, thereby demonstrating the stability of CCPIC as a whole, as well as averting a potentially devastating outcome for the intern.
Consultation and Socialization
Ensuring consistency across training sites typically involves regular meetings of the training directors from each site. Such meetings allow for consultation and discussion of administrative matters associated with the internship, as well as of intern challenges and progress. Often, these meetings serve as a forum for consultation that enhances the efficacy of individual supervisors as well as the success of interns. Supervisors who engage in these meetings have also reported that they serve as an invaluable form of personal support and self-care, and help prevent burn out.
Opportunity to Expand the Scope and Breadth of Didactic Trainings
The pooling of resources across the consortium training model provides interns with access to a broader continuum of expertise than might otherwise be available within a single site or organization. At CCPIC, for example, all interns benefit from training by didactic presenters who have gained expertise working with specific populations that is potentially generalizable to a broader population, such as working with mandated patients, dealing with aggressive patients, working as a member of a behavioral health team, or working within psychologist-psychiatrist partnerships.
The greater number of interns typically served by consortia is associated with an enhanced administrative burden that includes management of a significant number of intern applications, evaluations, and surveys. Larger numbers of interns also results in a greater number of routine requests, such as for verification of internship completion and requests for replacement certificates. Additionally, collection of proximal and distal data associated with quality assurance/improvement efforts as well as compliance with CoA standards is clearly more burdensome for entities that accept more interns.
CCPIC’s experience with this criterion is telling. A lack of adequate administrative support was a factor leading to placement of the consortium on probation. CCPIC was able to successfully address this by funding a full-time Consortium Coordinator charged with maintaining up-to-date files on both interns and supervisors associated with the consortium, a staff addition that no single site could have afforded. Additionally, CCPIC revised its data collection methods (including adopting electronic evaluation tools) that facilitated document tracking across all participating sites.
Establishing an accredited internship consortium requires doctoral programs to invest time, administrative resources, and professional expertise. In turn, this model offers doctoral programs the ability to develop a substantially enhanced number of accredited internship opportunities for its students during a time when the supply of such opportunities remains a concern. It also allows programs to ensure that their students’ training during internship is consistent with the program’s values and priorities and is anchored in contemporary, empirically-based practice. Internship training consortia also offer opportunities for programs to partner with multiple agencies, and to expand the availability of mental health services to often underserved populations within their communities, thereby educating the public as to the unique contributions that psychologists can provide.
Diana M. Concannon, PsyD, is the Director of the APA-accredited Central California Psychology Internship Consortium, and serves as Chair of the Accreditation Committee for the National Council of Schools and Programs for Professional Psychology. She is the former Associate Dean of the California School of Forensic Studies, and former Systemwide Director of Training for the California School of Professional Psychology. Dr. Concannon currently serves as Director for Mental Health and Wellness for the City University of New York.
Roberts M.C. (2013) Professional psychology in health care services: a blueprint for education
and training. American Psychologist, Sept; 68(6), 411-426.
Commission on Accreditation. Implementing regulations Section D: IRs Related to the
Accreditation Operating Procedures. Retrieved September 19, 2014, from
Commission on Accreditation. Implementing regulations Section C: IRs Related to the
Guidelines and Principles. Retrieved September 19, 2014, from
Bureau of Labor Statistics (2013). Occupational Employment and Wages, May 2013, 19-3031
Clinical, Counseling, and School Psychologists. Retrieved September 18, 2014, from
Erikson Cornish, J.A., Roehlke, H.J., & Boggs, K.R. (2000). Half-time doctoral psychology
internship programs in university counseling centers: Advantages and Disadvantages.
Professional Psychology: Research and Practice, 31, 349-350.
Erikson Cornish, J.A., Smith-Acuna, S., & Nadarni, L. (2005). Developing an exclusively
affiliated internship consortium: a novel approach to internship training. Professional
Psychology: Research and Practice, 36, 9-15.
Fouad, N.A., Grus, C.L., Hatcher, R.L, Kaslow, N.J., Hutchings, P.S., Madson, M. B., Collins, F.L., Crossman, R.E. (2009). Competency benchmarks: A model for understanding and measuring competence in professional psychology across training levels. Training and Education in Professional Psychology, 3, No. 4 (Suppl.), S5-S26.
Keilin, W.G., Thorn, B.E., Rodolfa, E.R., Constantine, M.G., & Kaslow, N.J. (2000). Examining
the balance of internship supply and demand; 1999 Association of Psychology Postdoctoral and
Internship Centers’ match implications. Professional Psychology: Research and Practice, 21,
Slote Morris, Z., Wooding, S., Grant, J. (2001). The answer is 17 years, what is the question:
understanding time lags in translational research. Journal of the Royal Society of Medicine, 104,