A Department or Program self-study and the corresponding external review are a holistic department/program quality and effectiveness analysis. Departments and Programs reflect on and assess their contributions to student learning, scholarship, and the College’s mission. They also plan for the future.
The self-study and external review, a comprehensive process, usually occur every ten years. This process begins with initial meetings between the Department Chair or Program Coordinator and the Associate Dean of the College charged with Curriculum and between the Department Chair or Program Coordinator and the College’s Accreditation Liaison Officer (at Pomona College, the Director of Institutional Research). These initial meetings take place approximately one year before three external reviewers visit our campus, during the Spring semester. The selection of outside reviewers also begins as early as one year before the external review. Approximately one month after the external review, the reviewers submit their report to the Dean of the College, who then forwards it to the President and the Department Chair or Program Coordinator and meets with the Department or Program to discuss its contents. One year later, the Department Chair or Program Coordinator writes a brief follow-up report to the Dean, outlining how the Department or Program has responded to the external reviewers’ recommendations and discusses any further actions with the Dean. This meticulous process ensures a thorough and fair assessment of the quality and effectiveness of our departments and programs.
The budget for each self-study and external review is $7,000. Most of this amount will cover the travel expenses and honoraria for the outside reviewers, whose standard fee is $1,000 apiece. The rest can cover other costs, such as photocopying, mailing, meals, or space rental for a retreat. The Dean’s Office will issue the three honoraria directly. Individual departments and programs manage the other expenses as they see fit, including the travel expenses for the reviewers. Departments and programs are also responsible for collecting a Vendor Data Record form from the reviewers, attaching it to a completed Request For Check form, and sending it to the Associate Dean’s office.
Selecting the Outside Reviewers
By the end of May, the Department Chair or Program Coordinator sends a list to the Associate Dean with the names, titles, institutions, and email addresses of six to eight potential outside reviewers, ranked in order of preference and grouped according to their specific fields. Before submission, all continuing Department or Program members must discuss and approve these names. The Associate Dean selects three external reviewers from this list in consultation with the Dean. The Dean and Associate Dean may consider non-listed individuals, but only after consulting with the Department or Program. The three external reviewers are selected based on their expertise in the field, understanding of the program’s mission and goals, and ability to provide constructive feedback. The three external reviewers will typically include at least one person from a comparable liberal arts college and one from a research university offering PhDs in the pertinent discipline. The latter can provide feedback on how well we prepare our students for further studies; the former can appreciate the unique character of liberal arts colleges. One of the three reviewers must be a current or former Department Chair.
The Department or Program will indicate the dates on which it would prefer the outside reviewers to visit campus. The Associate Dean will ascertain the Dean’s availability on the proposed dates and contact the reviewers to determine their willingness to serve. Once accepted, the Department or Program coordinates travel, housing, and on-campus scheduling for the reviewers’ two-day visit.
The Self-Study Document
The Department or Program embarks on a collective endeavor, creating a report it will share with the President, the Dean, the Associate Dean, and the external reviewers no later than two weeks before the external review. The Department Chair or Program Coordinator oversees its overall organization, but the self-study document is not the sole work of one individual. It is a shared enterprise among department or program members, emphasizing the importance of each member’s contribution. Student liaisons may be enlisted to help gather data from students and alums, further highlighting the collaborative nature of the process. By the beginning of the Fall semester, the Department or Program meets with the Director of Institutional Research to obtain a Department or Program self-study data package before the self-study starts. The Department or Program will reserve time in the Fall semester for in-depth discussion of the self-study issues during a retreat or a series of meetings. To facilitate discussion, the Department Chair or Program Coordinator may want to distribute a written set of topics or issues for the self-study to all faculty before the initial meeting.
As part of Pomona’s 2021 reaccreditation process, our accreditor, the Western Association of Schools and Colleges (WASC), issued the requirement that Pomona “Demonstrate achievement of the institution’s liberal arts education goals through regular faculty assessment of learning outcomes for each program.” The Teaching and Learning Committee (TLC) has considered this requirement and recommends to the faculty that assessment of Student Learning Outcomes at the major level be included as a core component in Department or Program self-studies. The TLC also recommends that Departments and Programs planning a self-study consult with the Associate Dean overseeing self-studies and the Office of Institutional Research as early in the process as possible for guidance on conducting and reporting the results of assessments of Student Learning Outcomes to demonstrate our commitment to monitoring and improving student learning.
Core Information in a Department or Program Self-Study Document
- History of the Department or Program, including changes in faculty, facilities, etc.
- A statement of the Department or Program learning objectives.
- An assessment of student learning in the major based on actual student work, not student opinion, considering the Department or Program learning objectives (could be a review of the Department or Program’s recent annual assessments of student learning in the major).
- An explanation of how the Department or Program determines that learning goals are being met with its current curriculum and what evidence is used.
- A discussion of how the Department or Program uses this evidence to evaluate and improve its curriculum.
- Data on enrollments and majors, disaggregated by gender and ethnicity (Institutional Research will supply data for the Department or Program to analyze).
- An assessment of the Department or Program’s efforts to incorporate inclusive pedagogy and support a diverse student body.
- An evaluation of any striking ethnic, racial, or gender disparities among majors or non-majors taking classes in the department/program, including plans to address them.
- Feedback from and profiles of current students and alums obtained from surveys taken early in the self-study process.
- Comparative data from departments or programs at similar colleges.
- Faculty profiles, such as CVs or biographies, including teaching, research, and service materials.
- The Department or Program’s tenure and promotion standards.
- Catalog copy and course syllabi.
- History of recent financial support for faculty and students, including grants for research, travel, and senior projects.
Sample Issues or Questions for Self-Studies
The following questions may spark further discussion among your Department or Program colleagues. They are only suggestions, not required questions to answer.
Teaching and Learning
- How are the learning goals of the Department/Program connected to the College’s institutional learning goals?
- How does the Department or Program determine the learning goals are being met with its current curriculum? What evidence is used?
- How does the Department or Program use this evidence to evaluate and improve its curriculum?
- What active or experiential learning opportunities does the Department or Program offer students?
- What sorts of faculty/student collaboration does the Department or Program support? How effective is this collaboration?
- How appropriate is the senior exercise, given the Department or Program goals? How does it help students integrate the information, concepts, and skills they have learned?
- How well-prepared are majors for graduate study? How successful are majors in getting into graduate school or finding employment? What kinds of post-graduate occupations do the majors pursue?
- How do department/program offerings help all students lead satisfying, productive lives? Does the curriculum offer courses that provide transferable skills and knowledge for students who will take only one or two courses?
- Are there striking ethnic, racial, or gender disparities among majors and non-majors taking courses in the Department or Program? What can be done to address them?
- How coherent are the major and minor requirements?
- How well-prepared are majors for graduate studies? How successful are majors in getting into graduate school? What other post-graduate alternatives do the majors pursue?
- How do Department or Program offerings help all students lead satisfying and productive lives? Are there valuable concentration courses available for students taking only a few courses?
- What courses outside the Department or Program do faculty advisors encourage students to take?
- How fully does the major take advantage of other Claremont College Colleges courses?
Faculty Development
- To what extent do professors find their teaching at Pomona College satisfying? How can the Department or Program help faculty members continue to improve their pedagogy or develop new areas of teaching expertise?
- How appropriate is the faculty staffing given the curricular goals and enrollments? To what degree are students taught by regular and continuing faculty members? Are resources available at the other Claremont Colleges that can help the Department or Program achieve its curricular goals?
- How are the Department or Program goals hampered or facilitated by its physical facilities?
- In what ways does the Department or Program facilitate research productivity among faculty members?
Department or Program Governance
- How is the Department or Program performing? Is decision-making centralized in a minority, or is there a system of shared governance among faculty? Are there written guidelines for Department or Program governance?
- How does the Department or Program distribute or delegate responsibilities to individual faculty members? How is leadership encouraged and developed across the Department or Program?
- How does the Department or Program distribute resources among individual faculty members?
- How are junior faculty members mentored with respect to their teaching, scholarship, and service? Are information and expectations communicated effectively?
- Are there sufficient opportunities for the Department or Program faculty to interact with one another and share experiences (through both formal meetings and informal interactions)? Is there potential for better interdepartmental and intercollegiate cooperation and complementarity?
- How well is the Department or Program meeting its staffing needs?
Resources for Self-Studies
- National studies of general curricular issues for liberal arts colleges in general.
- Studies by national organizations in your discipline or field.
- Self-studies conducted at Pomona by other departments or programs (permission is required from the department or program to view the self-study document).
- The Director of Institutional Research can offer advice on strategies for data collection, provide technical support for web-based surveys (of alums and students), and supply existing data from institutional surveys to inform the review where appropriate. The Senior Survey and the Enrolled Student Survey cover many topics relevant to departmental self-studies.
The External Review
Reviewers will spend at least two days at Pomona College, usually either a Monday-Tuesday or a Thursday-Friday, arriving the evening before the first review day. The review team meets the Dean and Associate Dean at the start of their visit and has an exit interview with the Dean and Associate Dean the following afternoon. After this exit interview, the reviewers should have time to discuss their written report without further contact with Department or Program members. To comprehensively understand a department or program, the reviewers will meet with faculty, support staff, and individuals from related departments or programs at the other Claremont Colleges. Departments and programs may wish to have the reviewers meet with the Associate Dean who oversees research support. The reviewers should meet with students, and they may visit one or more classes and see any significant resources, such as laboratories, libraries, theaters, study rooms, and common areas. Reviewers must have sufficient time to review student work, including senior exercises and class projects. Reviewers and the entire Department or Program often enjoy a reception or dinner on the first evening of the visit. The Department Chair or Program Coordinator compiles the two-day schedule of the reviewers’ visit in consultation with the Associate Dean.
Following Up
After digesting the reviewers’ report, the Department or Program meets with the Dean and Associate Dean to discuss it. The Department Chair or Program Coordinator should contact the Dean’s Office to arrange this meeting, generally within a month of receiving the report.
The Educational Quality Committee of the Board of Trustees hears a summary of the self‐study and external review within a year of the visit. One year after the external review, the Department or Program submits a brief follow-up report to the Dean, with a copy sent to the Associate Dean. The report includes information on implemented and pending recommendations, their rationale, the impact of changes, and the Department or Program’s intended course of action. The report can also provide an opportunity to remind the administration of any steps it should be taking. The Department Chair or Program Coordinator meets with the Dean and Associate Dean to discuss the follow-up report, helping ensure a productive outcome for the self-study and external review. After the President and the Associate Deans of the College and for Academic Affairs have reviewed the report, the Dean of the College makes recommendations for further action.