Unified Assessment Plan: Flowing assessment up the plan

> ...how soon can we have [a] clear picture of whether our PLO's, CLO's and SLO's are viable or need further assessment...To be honest our PLO's are clear,... - BM

Bear in mind that the "unified assessment plan," or whatever one wishes to call it (someone suggested Fairly Obvious and Objectively Logical plan), takes those "clear PLO's" and provides a framework from which to generate inferential information on achievement of program and institutional level assessments.

In the world of mathematics we cannot prove that the square root of two is irrational. We can only prove that it is not rational. The only other choice left is that the root is irrational.  Inference works in this manner.  I may never be able to show that the communication program learning outcome is directly achieved by measuring learning at the program level. I can, however, show that the program learning outcome can never be achieved if I am able to show that no course in the program has an outline with a communication student learning outcome.  The matrices in that spreadsheet show distributions of institutional effort: where we are putting our emphasis.

We have fine PLOs, no need to alter these.

Obviously the holy grail, that which everyone desires, is to know what a graduate knows, can do, and values as they walk down the graduation aisle.  Like the holy grail, this may prove ethereally elusive. And in the end it will turn out not to be a gold chalice but a simple carpenter's cup. 

In pursuit of knowing what a graduate knows, one's mind is likely to leap to exit tests. Yet if I gave the entrance test to the faculty, my guess is that many would place into developmental mathematics if not MS 090 - the lowest developmental level at this time.

I suspect that ultimately we will look at the achievement of course level student learning outcomes and use those to infer accomplishment of program learning outcomes, which can in turn be used to infer accomplishment of institutional learning outcomes. I have no idea if this is acceptable, but it simplifies things in my small mind. One measures student learning outcomes at the course level. The higher levels are inferred. Faculty would only have to focus on what students are mastering and not mastering in their own courses and then reporting this information. 

Permit me to digress in order to come back to my point.

I was asked today whether we would eventually be able to generate numbers such as "70% of the students communicate effectively through writing, speaking, performing, exhibiting, or other forms of expression." Or to use such a statement as an institutional objective, "70% of the students will communicate effectively through writing, speaking, performing, exhibiting, or other forms of expression."

Before I share my response, think for a moment what one would have to do to be able to say this. One would have to do as one instructor in my division attempted to do this term: to measure every outcome for each and every student individually. It looked achievable in August and we discussed a possible presentation on the system in December. The faculty member has asked to defer the presentation until after the spring term, I gather he has found a need to modify the system.  My understanding is that the data entry load was ferocious. Bear in mind each outcome was to be assessed ten times during the course of the term for each and every student to ensure reliability of the assessment. 

I have long argued that one's ability to handle the data entry and tracking load varies with the sheer number of outcomes and students. If the outcomes are few and the class sizes small, this is a feasible approach. Learning can be proven and documented on a student-by-student basis.

In larger classes with large batteries of outcomes record keeping and management become problematic. Proving learning becomes difficult.

Gödel proved the unprovability of "sufficiently complex systems" once and for all in mathematics - the home court of quantification. Any sufficiently complex system will have theorems that can neither be proven true nor shown to be false. I have long asserted that proving achievement of student learning, especially that which the student knows, can do, and values at graduation, will have Gödelian limitations.

One thing I suspect we can do is the internal inferential model such as that proposed by the so-called unified assessment plan and other inferential models such as those used at Mesa College. At a recent workshop the president was given the Mesa model. I noted that their "program assessment model" was, at its core, a set of three surveys.  The students, the community, and employers. The surveys sought to determine the value of a particular program to each of those stakeholders.  Again, inferential.

My response to the question of whether we would eventually be able to say, "70% of the students communicate effectively through writing, speaking, performing, exhibiting, or other forms of expression," was that I thought this is something the faculty should probably decide. That is, whether or not we would want to attempt to generate such data or set some such goal.