The physical science outline is not in a form
that lends itself to the aggregation approach I have used in MS 100 and
MS 150. The present outline is 108 specific student learning outcomes -
a broad, shallow river of many facts and few deep understandings.
Worse, the course level outcomes, though only five, are oddly specific.
For example, the only one of the five to reference chemistry is, "Give
the importance of organic chemicals in everyday life." Given that I
have already predetermined that the course will not reach organic
chemistry, the outcome is not going to be attained. And even if it were
attained, it is rather tangential to the thrust of that part of the
course (and the present text) which is introductory chemistry.
Thus in terms of test instruments I can only report on the results of
item analysis. With week two lecture focused on linear motion and the
week two laboratory also being focused on linear motion, quiz two
borrowed lab data in its questions. The quiz noted that:
Science is built on experimentation. Science is a field in which a theory is not considered tested until experiments have been unable to prove the theory false. In lecture I present theories and we work homework problems. But the only thing you know for sure based on experimentation is that there is a relationship between distance and time for a ball rolling. And the 8:00 lab section should still have doubts about the nature of the relationship due to some of the data they collected.
The 8:00 science team reinvented the laboratory, allowing a subsequent team of scientists working at 11:00 to collect more data more efficiently. The following data [in this quiz comes from the revised experiment performed by the 11:00 team].
| Src | Q | Description | Corr | Corr% |
| q02 | 1 | calculate speed | 25 | 81% |
| q02 | 1 | units | 21 | 68% |
| q02 | 1 | significant digits correct | 15 | 48% |
| q02 | 2 | calculate speed | 24 | 77% |
| q02 | 2 | units | 23 | 74% |
| q02 | 2 | significant digits correct | 20 | 65% |
| q02 | 3 | draw a conclusion from quantitative data | 19 | 61% |
| q02 | 4 | identify units for slope | 10 | 32% |
| q02 | 5 | identify units for intercept | 1 | 3% |
| q02 | 6 | interpret physical meaning slope | 6 | 19% |
| q02 | 7 | predict d given t | 8 | 26% |
| q02 | 7 | units | 5 | 16% |
| q02 | 7 | significant digits correct | 7 | 23% |
| q02 | 8 | predict t given d | 7 | 23% |
| q02 | 8 | units | 5 | 16% |
| q02 | 8 | significant digits correct | 7 | 23% |
| 1. Question/Problem statement | |
| Given. Not scored. | |
| 2. Hypothesis/Prediction | |
| 2 | Clearly stated with any associations among the variables described, complete |
| 1 | Unclear statement, or variables left unaccounted for, incomplete |
| 0 | Omitted |
| 3. Procedure (not marked in lab 01 due to its being given in the lab) | |
| 2 | Sufficient detail such that lab could be reproduced/duplicated. All activities listed. |
| 1 | Insufficient detail, lab could not easily be reproduced, unclear, missing activities |
| 0 | Omitted |
| 4. Data tables | |
| 3 | Clear and complete, laid out clearly with labels and units |
| 2 | Missing one or more labels or units or other minor issues |
| 1 | Unclear, confusing, poorly done, incomplete |
| 0 | Omitted |
| 5. Data analysis | |
| 2 | Appropriate statistics calculated and/or appropriate data analysis including units |
| 1 | Missing units or other minor issues |
| 0 | Omitted |
| 6. Data display: graphs and charts. None in lab 01 | |
| 2 | Complete |
| 1 | Incomplete |
| 0 | Omitted |
| 7. Conclusions | |
| 4 | Strong reasoning based on the data, logically reasoned, complete and thoughtful |
| 3 | Moderately well-reasoned, but tangential to the data or based on misconceptions |
| 2 | Overly brief or compromised and impaired by grammar and syntax errors |
| 1 | Unclear, not well reasoned, highly incomplete |
| 0 | Omitted |
| Grammar and Syntax | |
| 3 | No errors of grammar or word order. |
| 2 | Some errors of grammar or word order but communication not impaired. |
| 1 | Frequent errors |
| 0 | Errors of grammar or word order so severe as to make comprehension virtually impossible. |
| Vocabulary | |
| 3 | Appropriate terms used consistently, clear command of vocabulary with a focus on correct usage of physical science vocabulary, no misspelled words. |
| 2 | Occasionally uses inappropriate terms or relies on circumlocution; expression of ideas not impaired; or a few misspelled words. |
| 1 | Frequent errors in vocabulary or spelling, ability to communicate limited by vocabulary |
| 0 | Vocabulary limitations so extreme as to make comprehension virtually impossible. |
| Cohesion | |
| 3 | Ideas flow logically. Connector words assist the reader. |
| 2 | Ideas are disconnected and the conclusion reads more like an outline or answers to a list of questions without connector words. There is a choppy and disjoint sense to the writing style. |
| 1 | Communication impaired by a hodgepodge of inappropriate and misused cohesive structures |
| 0 | Incomprehensible collection of disconnected ideas and words. |
[This was subsequently modified such that the tables subsection is also marked on a two point scale. Making this symmetric made using the rubric easier.]
Each individual lab includes the individual
scores for each section of the rubric. Coupled with the rubric, the
hope is that the scoring will help a student understand where their
specific weaknesses are and then address those weaknesses. Explaining
this to the students and helping them understand how to improve will be
a process that will take time during the early part of this term.
Although I could record in a large grid each student's individual
scores in each section, which would lead to an item analysis-like
assessment of student performance on each of the ten sections, the
amount of work that would entail is simply prohibitive. This early in
the term this seems more work than the value to be gained from the
information generated.
Some students are submitting their laboratories by email, providing the
opportunity for me to gather sample laboratories for use in "course
portfolio" assessment. i would note for the benefit of the other
science instructors that this approach does not utilize a "lab
notebook" of any sort in which students collect all their labs. Each
lab report is an individual Microsoft Word document often using
Microsoft Excel data tables and charts.
At present the conclusion "essays" for most students have a ways to go
before they meet the writing expectations I have for the laboratories.
I would also note that I do not know if the above rubric based system
is more work than whatever systems of lab marking other instructors in
the division are using. I was able to mark about 20 labs in a period
of about two to three hours, I may get more efficient in future runs of
the rubric. I also write comments on the labs and note
grammar/vocabulary issues which adds to the marking time. In some sense
I've given myself the load of a writing class instructor for what is
the equivalent of a writing section and a half.
The course has a homepage, quizzes, tests, labs, and other material will be linked to that page. Many of the materials are in OpenOffice.org formats.