# MS 100 College Algebra item analysis

The following table reports an item analysis for MS College Algebra final examinations from three instructors as aggregated by student learning outcomes on the outline. The first column references the outline. "1_" refers to final exam items that met course level outcome 1 but were not easily assigned to a specific learning outcome (1a, 1b, etc.). The fac columns represent averages for a given faculty member, the names were stripped out to keep the data anonymous as per Dr. Mary Allen's advice in her books. The z column is not a true z but rather represents standard errors away from the mean. Cond refers to whether the average might be considered low, high, or ordinary - not unusually high or low. The distribution of averages was uniform, not at all a normal distribution, hence the large number of items that were either high or low. Performance simply varied widely on items.

 Outline fac1 fac2 fac3 Overall z cond Students will be able to: 1_ 0.72 0.38 0.60 3.28 high Graph and solve linear and quadratic equations and inequalities including those with complex roots. 1a 0.43 0.17 0.44 0.33 -4.31 low Sketch the graph of an equation 1b 0.70 0.36 0.30 0.47 -0.33 ordinary Solve linear, quadratic, polynomial, and radical equations. 1c 0.47 0.31 0.38 0.39 -2.69 low Perform operations with complex numbers. 1d 0.76 0.76 7.60 high Solve linear, quadratic, polynomial, and radical inequalities. 2_ 0.54 0.41 0.81 0.65 4.64 high Evaluate and analyze functions and their graphs including combinations and compositions of functions. 2a 0.57 0.46 0.48 0.00 ordinary Find and use slopes of lines to write and graph linear equations in two variables. 2b 0.74 0.39 0.64 4.38 high Evaluate functions and find their domains. 2c 0.59 0.57 0.59 2.87 high Analyze the graphs of functions. 2d 0.31 0.44 0.43 0.40 -2.18 low Find arithmetic combinations and compositions of functions. 2e 0.15 0.30 0.13 0.22 -7.27 low Identify inverse functions graphically and find inverse functions algebraically. 3_ 0.24 0.24 -6.70 low Sketch and analyze graphs of polynomial functions and mathematical models of variation. 3a 0.46 0.26 0.61 0.46 -0.74 ordinary Sketch and analyze graphs of polynomial functions 3b 0.62 0.63 0.62 3.82 high Use long division to divide polynomials 3c 0.36 0.36 -3.30 low Write mathematical models for direct, inverse, and joint variation. 4_ 0.38 0.50 0.44 -1.26 ordinary Determine the domains of rational functions, find asymptotes, and sketch the graphs of rational functions. 4a Find the domains of rational functions. (Not asked on any final). 4b 0.26 0.28 0.27 -5.85 low Find the horizontal and vertical asymptotes for graphs of rational functions. 4c 0.29 0.44 0.41 -1.94 ordinary Recognize graphs of circles, ellipses, parabolas, and hyperbolas. Avg: 0.58 0.37 0.48 0.48 0.00 ordinary

The upshot is that we now know where students tend to be weaker and where they tend to be stronger. We also have a value of 48% that can be reported, on aggregate, for the mathematics program learning outcome.

That 48% reminds me of the following chart, presented at the November 25 mathematics meeting:

The chart shows results for students passing (D or better) or being eligible for the next course (C or better) for all students spring 2004 to fall 2005. Note the pass rate for MS 100 over this period is around 52%. Half of the students (48%) are able to correctly answer final examination questions (on aggregate) and half of the students (52%) have passed the course on a historical basis. Qualified faculty implementing courses based on student learning outcomes produce trusted grades. Grades have meaning. Where qualified instructors implement courses based on student learning outcome, grades can be used as a valid assessment of student learning - on aggregate.

This is not to say we can dispense with formative assessment - the data above provides each of us who teaches MS 100 useful information on where students tend to succeed or fail. This is valuable. On aggregate, however, grades also have a learning meaning.

The following is a table where each row is a question on a final. The table includes the instructor's original description and the outline item I assigned to that item. For the benefit of those who might wish to quibble with my choices, the table below could be copied to a spreadsheet, changes could be made, and the result could be re-pivoted to produce a variation of the above.