IB3. Self Evaluation

The standard implies an assessment mechanism exists. The assessment mechanism that was deployed by the college in 1997 was the Institutional Effectiveness committee. This committee was tasked with, among other things, looking at the processes began as Program Health Indicators two years earlier.

The Institutional Effectiveness Committee and its successor the Assessment Committee would have periods of regular meetings punctuated by long periods of dormancy. More fundamentally, the membership of the committee was not the same as the Curriculum Committee. While the Institutional Effectiveness committee included members of the Curriculum Committee, the committee did not duplicate the membership of the Curriculum Committee in full. The result was that analysis of assessment and data based decisions made in the Institutional Effectiveness and later the Assessment Committee were not being fed back into the Curriculum Committee. During dormant periods assessment efforts tended to wane.

Table IB3.1 Log of Institutional Effectiveness Committee meetings pre-1998
CommitteeChairDate Have minutes?Gap (Days)Cause?
Prog Health Ind WorkshopS. James 06/17/95 NoNA
Inst EffM. Abram 03/30/97 Yes
Inst Eff M. Abram 05/12/97 No, implied in previous minutes 43
Inst EffM. Abram 05/26/97 Yes14
Inst EffM. Abram 06/09/97 No, implied in previous minutes 14
Inst EffM. Abram 07/07/97 Yes28
Inst Eff M. Abram 07/28/97 No, implied in previous minutes 21

Records from 1997 and earlier are sketchy at best, only some of the minutes remain extant. Then Director of Research and Planning Marie Abram was not on board during the critical 1995 workshop in which the division chairs first learned about program health indicators.

Table IB3.2 Log of Institutional Effectiveness Committee meetings 1998
CommitteeChairDate Have minutes?Gap (Days)Cause?
Inst EffM. Abram 02/04/98 Yes191 Unknown
Inst EffM. Abram 02/11/98 Yes7
Inst EffM. Abram 02/18/98 No, implied in previous minutes 7
Inst EffM. Abram 02/25/98 Yes7
Inst EffM. Abram 03/04/98 Yes7
Inst EffM. Abram 03/18/98 No, implied in previous minutes 14
Ladera IA conference Habuchmai05/21/98 YesNA

In 1998 Director Abram left the college. This provided an opportunity for the college to send non-administrative faculty to a critical conference on assessment and student learning outcomes led by James Ratcliff, Director National Center Post-Secondary Teaching and Assessment, on Guam during May of that year (Transcriptional Report from the Conference on Institutional Assessment, 1998, http://www.comfsm.fm/comfsm/ladera.html). With turn-over at the Director of Research and Planning, this conference would pass information on assessment to rank and file faculty. This concept, of decentralizing assessment and moving assessment closer to the students, to the division chairs and their faculty is a central theme in this document.

Table IB3.3 Log of Institutional Effectiveness Committee meetings 1999
CommitteeChairDate Have minutes?Gap (Days)Cause?
Inst EffG. Myers 01/21/99 Yes309 Change in DRP
Inst EffG. Myers 02/04/99 Yes14
Inst EffG. Myers 02/18/99 Yes 14
Inst EffG. Myers 03/04/99 Yes14
Inst EffG. Myers 03/18/99 Yes14
Inst Eff G. Myers 04/08/99 No, implied in previous minutes 21
California Assmnt Inst G. Myers04/21/99 NotesNA
Inst Eff G. Myers 04/29/99 Yes21
Inst EffG. Myers 05/06/99 Yes7
Inst EffG. Myers 05/27/99 No, implied in previous minutes 21
Inst EffG. Myers 07/16/99 Yes50 Unknown: Summer?
Inst EffG. Myers 08/26/99 Yes41
Inst EffG. Myers 09/14/99 Yes19
Inst EffG. Myers 09/21/99 Yes7
Inst Eff G. Myers 09/28/99 Yes7
Inst EffG. Myers 10/12/99 Yes14
Inst EffG. Myers 10/19/99 Yes7
Inst EffG. Myers 11/02/99 Yes14

During 1999 Greg Myers would take over as the Director of Research and Planning. Director Myers would come on board without the experience of having taken the institution through a self study and self study report. The new director would face a heavy load of daily tasks as well coming up to speed on the recommendations made in 1998.

Table IB3.4 Log of Assessment Committee meetings 2000
CommitteeChairDate Have minutes?Gap (Days)Cause?
AssessmentG. Myers 11/06/00 Formation of assessment noteNA
Assessment G. Myers 11/14/00 Yes378 Unknown

For reasons this author cannot ascertain, the Institutional Effectiveness committee appears to have gone dormant in the spring of 2000. There exists the possibility that intense work associated with the planning council and development of the mission statement and college goals caused the Institutional Effectiveness committee to fail to convene. Institutionally, there are so many tasks delegated solely to the Director of Research and Planning that the position becomes a critical single point failure point in the college system.

The committee would be reborn in the fall of 2000 as the Assessment Committee. Whether other meetings occurred in 2000 has yet to be documented by this author. As of the writing this report there is no indication there were other meeting during this year.

Table IB3.5 Log of Assessment Committee meetings 2001
CommitteeChairDate Have minutes?Gap (Days)Cause?
AssessmentG. Myers 02/20/01 Agenda only, implies prior minutes 98Unknown
AssessmentG. Myers 06/29/01 Fragment only129 Unknown
AssessmentG. Myers 07/07/01 Yes8
AssessmentG. Myers 07/20/01 Yes13
Assessment R. Churney 07/27/01 Yes7
California Assmnt InstG. Myers 08/31/01 Announcement only
AssessmentG. Myers 09/13/01 Yes48 New schedule
AssessmentG. Myers 10/11/01 Yes28
AssessmentG. Myers 11/15/01 No, implied in schedule 35
AssessmentG. Myers 12/06/01 No, implied in schedule 21

At the 20 February 2001 meeting of the Assessment committee the committee was told to be prepared to discuss the purpose of the committee, with one definition offered, "Gather data and analyze all COM-FSM programs and recommend actions for improvement." The minutes for this minute appear to be missing except for a single last page including an item six. The committee membership included six directors, four of whom have since left the college (Email address list for committee members, 20 Feb 2001).

Table IB3.6 Log of Assessment Committee meetings 2002
CommitteeChairDate Have minutes?Gap (Days)Cause?
AssessmentG. Myers 01/24/02 Yes49
AssessmentG. Myers 02/07/02 Yes14
Plenary presentation: Issues in SLOs & Accred S. Moses 02/22/02
AssessmentG. Myers 02/28/02 Yes21
AssessmentR. Churney 03/14/02 Yes14
Assessment 03/28/02 No, implied in schedule Convened?
Assessment 04/11/02 No, implied in scheduleConvened?
Assessment 04/25/02 No, implied in scheduleConvened?
Assessment 05/09/02 No, implied in scheduleConvened?
Assessment 05/21/02 No, implied in scheduleConvened?
Individual SLO meetings with chairs S. Moses 10/2002
SLO dialog science math begins D. Lee Ling 10/12/02
SLO meetings state directors S. Moses 12/05/02

In 2002 the committee members were all provided with a copy of the recently approved Strategic Plan 2001-2006. The minutes also noted the development of a distance education plan for the college. Attendance, however, was not good. The meeting on 24 January had only 10 of 21 members present, although one absence was an off-island member. Many committees include members from the remote state campuses. Their participation has always been constrained by the complications of distance. In the past attempts were made to use the PeaceSat radio satellite network to include state directors in critical meetings. There was also a brief attempt in the 1999 time frame to use instant messenger services to include remote state campus personnel in a meeting. A transcriber typed the conversation in the meeting in real-time, while in another window the remote state campus personnel could respond or chime in. The transcriber would have to stop to read the input from the remote state campus. Neither system worked sufficiently well that either experiment was continued.

Today the only inclusion of remote state personnel in meetings in via the email traffic prior to a meeting during which issues that will come up in the meeting are discussed. This is not only imperfect, but it excludes the state campus member from the actual decision making vote on an issue.

During 2002 the Director of Research and Planning would depart. Prior to his departure he would form four committees to work on the self study. These committees would meet on three occasions in mid-2002 (Personal conversations with a participant). In August 2002 an Acting Director of Research and Planning would call a college-wide meeting and call for the formation of four new committees to work on the self study. The four new committees were not in any way related to the former four committees. Turn-over in a critical position, possibly unavoidable turn-over, once again impacted an assessment effort of the college.

Attendance is also impacted by the sense of institutional commitment or lack thereof. In an institution where all faculty are on limited term contracts that can be non-renewed with 60 day notice and without cause being given for non-renewal, full-time faculty are in the same position as a part-time faculty at other institutions. Coupled with the remote location of the college, a pay scale half that of equivalent positions in the United States, and the difficulties for many expatriates in living on small island, obtaining a sense of institutional commitment is difficult. This has translated, in part, to poor attendance for some committees. Assessment committee has had its attendance problems.

During 2002 Susan Moses began presentations and one-on-one training sessions with the academic chairs on student learning outcomes. This is cited in the above table in part because it constitutes evidence that work on assessment was continuing amongst the faculty during a time when the assessment committee was dormant. This is further evidence that decentralized assessment can proceed even in the absence of a functioning assessment committee. In section IB6 an alternative structure for assessment that fits with this new decentralized reality will be proposed.

Table IB3.7 Log of Assessment Committee meetings 2003
CommitteeChairDate Have minutes?Gap (Days)Cause?
AssessmentG. Snider 03/10/03 Yes361 Change in DRP
AssessmentG. Snider 03/20/03 Yes10
AssessmentG. Snider 04/10/03 Noted in S. Moses log21
AssessmentG. Snider 05/13/03 Notes54
AssessmentG. Snider 05/27/03 No14
Palikir Elem School SLO presentations S. Moses 07/17/03 No
Half-day faculty SLO workshop S. Moses/J. Thoulag 08/05/03 No

In spring 2003 the Assessment committee reformed. The Acting Director of Research and Planning, however, would depart in early August 2003.

Complicating the ability of these committees to function was the perception by 2001 that the committee work was unfocused and primarily a talking session for the then Director of Research and Planning (Personal observations of a member and author of this section). At the 07 July 2001 meeting of the Assessment Committee only six of seventeen members were present. The following meeting had only five of seventeen members. Even once the term started attendance rose to ten of nineteen appointed members.

Assessment committee meetings in 2001 and 2002 left one participant with the impression that they were addressing the need of the moment for the Director of Research and Planning (Personal observations of a participant). A sense that the committee work would lead to an annual cycle to regular assessment was not evident.

Communication of decisions made within the assessment committee to those affected by the decisions was not accomplished. The 10 March 2003 meeting of the Assessment Committee passed a resolution that the development of standards for program level outcomes would be the responsibility of the Assessment Committee. This pronouncement, however, lacked input from critical non-members who were, at the same time, developing program learning outcomes based on dialog in their divisions and based on the recommendations of national organizations in their field. This isolation and detachment of the assessment committee from planning and deploying committees such as curriculum is addressed in the planning section below and in the planning section for IB6.