Sent: Sunday, April 06, 2003 11:06 PM
Subject: Entrance test 2003: A look at the high schools

Every year I try to report on the relative strengths of our high schools in math and science.  While the state directors want to know how many students were accepted to the national campus, the reality is that understanding the mechanics of who gets accepted means fully comprehending functions such as =IF(OR(AND(OR(AND(F2>=22;G2>=27);L2>=49);P2>1);AND(F2>=16;G2>=20;P2>5));3;IF(AND(F2>=16;G2>=20;P2>0);2;1)).
 
Whatever numbers the admission's board utilizes, they are ultimately numbers chosen by people and are not reflective of the inherent capacities of our high schools.
 
Many things changed on the entrance test this year.  Too many.  As a result I have to re-engineer most of the statistics I report.  I begin with mathematics as I will present a new way of measuring high school performance.  The result will be NO HISTORICAL comparison data this year.  That will come next year. 
 
The new entrance test places students into math classes.  The obvious theoretical goal would be to place 100% of one's students into college level mathematics.  No nation on Earth accomplishes this, so the goal would be to improve one's year-to-year percentage of students placed into college level mathematics.
 
This year the private school that placed the most students into College level mathematics is Xavier, with PICS just barely edging out KHS to take the top public school honors.
 
I will presume all reading know the structure of the entrance test.  A practice version can be examined at: http://shark.comfsm.fm/~dleeling/math/practice.xml
 
The function works column by column to place students:

=IF(AND(H2>4;I2>4;J2>4;K2>5);101;IF(AND(H2>4;I2>4;J2>5);100;IF(AND(H2>4;I2>5);98;IF(H2>5;95;90))))

Where:
H2 contains the number correct out of ten possible for MS 090
I2 contains the number correct out of ten possible for MS 090
J2 contains the number correct out of ten possible for MS 090
K2 contains the number correct out of ten possible for MS 090

For those who have payed real close attention, this function is a slight modification of the original function.  The original function looked for the rightmost column with a six and then placed the student in the next higher course.  I had noted in commitee that there were very few instances of "retrograde" scores.  Until now:

Seq HS 90 95 98 100 MS
170 CHS 1 2 4 6 90
44 CHS 2 0 4 6 90
33 Weno 2 2 4 6 90

I had whole swaths of scores, primarily from the Chuuk public schools, that looked like the above.  The above three students placed into MS 101 under the old function.  This is clearly improbable.  The new function requires that no preceding column to a six be less than five.

The result using the new function is:

 
High School p90 p95 p98 p100 p101 p10x
Xavier 23% 8% 35% 27% 8% 35%
CCA 56% 11% 0% 33% 0% 33%
PSDA 50% 0% 23% 27% 0% 27%
Ohwa 80% 0% 0% 20% 0% 20%
PATS 42% 8% 33% 17% 0% 17%
PICS 53% 11% 21% 14% 1% 15.4%
KHS 66% 11% 8% 13% 2% 15.1%
PPSD 83% 9% 0% 9% 0% 9%
SCA 79% 7% 7% 7% 0% 7%
YHS 81% 5% 9% 4% 2% 6%
CHS 97% 2% 0% 0% 0% 1%
CSDA 92% 0% 8% 0% 0% 0%
Berea 96% 0% 4% 0% 0% 0%
OIHS 86% 11% 4% 0% 0% 0%
NICHS 97% 0% 3% 0% 0% 0%
SNHS 94% 4% 1% 0% 0% 0%
Weno 96% 4% 1% 0% 0% 0%
YSDA 80% 20% 0% 0% 0% 0%
PLHA 95% 5% 0% 0% 0% 0%
Mizpah 95% 5% 0% 0% 0% 0%
NCHS 100% 0% 0% 0% 0% 0%







Overall 77% 6% 9% 7% 1% 8%

The p10x column adds the percentage who placed into MS 100 and MS 101 (bearing in mind that placement into MS 101 makes a student eligible for MS 101, MS 103, or MS 150).

The big picture result is that 77% of the students place into pre-high school level mathematics (MS 090 Conceptual Math).  Only 8% of the high school seniors place into college level math.

Kudos have to go to PICS and KHS for turning in double the percentage of college level math capable students of the number three public school on the list.  Also deserving of praise is the Pohnpei Police department (PPSD) which outperformed the seniors in the public schools in two states.  That is incredible, a police force of which the nation can be proud.

Ohwa appears to have strengthened in math over previous performances.

In my culture we say that if you are on the bottom then there is no where else to go but up. On the bottom of the list Neighboring Islands, Southern Noumeneas, Weno, Yap SDA, Pentecostal Lighthouse, Mizpah, and Nukuno all can only improve. The challenge is there to all of us who work in education.  Over half the nation is being left behind in mathematical literacy.

This is the first year we can report direct placement results, next year I will look at changes in the trends.
~~~
In English the change in the structure of the test created similar problems with historic comparisons.  For now I will report simply a pseudo-z value rank order.  The z-value is the school average on the structure and reading sections minus the averages of the schools (not of all the students) and then that difference is divided by the standard deviation of the schools.  These values then provide the capacity to generate a rank order.  From strongest to weakest:

HS 2001 2002 2003 Rank 01 Rank 02 Rank 03
Xavier 1.53 1.21 2.01 1 2 1
CCA 1.33 2.05 1.87 2 1 2
PSDA 1.2 0.96 1.11 4 3 3
PATS 0.64 0.46 0.8 5 5 4
PICS 0.38 0.18 0.49 7 7 5
SCA 0.26 0.34 0.36 12 6 6
YHS 0.33 0.05 0.34 9 8 7
KHS 0.1 -0.04 0.34 14 9 8
CSDA -0.24
0.3 15
9
OIHS -1.13 -0.16 0.29 20 11 10
OHWA -0.34 -0.33 0.07 16 12 11
Mizpah
-0.08 -0.25
10 12
YSDA 1.28 0.75 -0.4 3 4 13
Woleai (NICHS) -0.84
-0.45 18
14
Berea 0.44
-0.63 6
15
PentLHA -1.58
-0.99 24
16
CHS -1.37 -1.39 -1.12 22 15 17
SNHS -0.58 -1.33 -1.26 17 14 18
Nukuno (NCHS)
-1.22 -1.4
13 19
Weno -1.42 -1.46 -1.49 23 16 20

The numbers are effectively the number of standard deviations away from the mean for the schools.  PICS pulls top honors for a public school while YHS and KHS tie for the second slot among public schools.  OIHS has improved over the last three years. Among the private schools Ohwa should be noted for its steady three-year climb in the ranks. And Xavier and Calvary Christian Academy continue to duke it out for who sits on top.  Statistically the small sample sizes mean Xavier and CCA are not statistically separated.  Their exchanging of positions is random change and not significant.
 
A graph of the z data:
 
Although this year the PICS data is not yet in, in the future I would shift from using TOEFL scores to using an analysis of the essay. The essay is marked under the following rubric:
1 Incompetent
2 Possibly incompetent
3 Developing competence
4 Minimal competence
5 Competent
6 Clearly competent
 
I have participated in the marking and there are details under each category that provide clear guidance.  The result is that any two people marking papers will usually give the same mark. In some instances they might deviate by one, in which case a third reader is brought in.
 
The goal for a high is then to produce students who can write a competent or clearly competent essay.  The results in descending rank order:
 
 
High School 1 2 3 4 5 6
Xavier 0% 0% 0% 0% 19% 81%
CCA 0% 0% 33% 33% 0% 33%
YHS 4% 13% 34% 17% 25% 6%
PSDA 0% 0% 32% 23% 41% 5%
KHS 1% 6% 34% 34% 20% 5%
PATS 4% 13% 33% 29% 17% 4%
SCA 0% 0% 19% 52% 26% 2%
CSDA 0% 15% 31% 23% 31% 0%
OIHS 0% 11% 32% 46% 11% 0%
Ohwa 10% 40% 30% 10% 10% 0%
Mizpah 14% 50% 32% 0% 5% 0%
CHS 48% 30% 13% 6% 2% 0%
YSDA 0% 0% 50% 50% 0% 0%
NICHS 10% 41% 34% 14% 0% 0%
Berea 8% 21% 58% 13% 0% 0%
Weno 49% 26% 15% 9% 0% 0%
NCHS 45% 45% 0% 9% 0% 0%
SNHS 38% 25% 29% 8% 0% 0%
PLHA 21% 58% 16% 5% 0% 0%





Overall: 22% 21% 25% 18% 10% 4%
 
Note that PICS has yet to take the essay section of the entrance test, but there is some data that suggests they will likely come in on par with Yap High School and Kosrae High School. 
 
Producing a six is difficult, schools can take pride in generating solidly competent fives. 
 
With 100% of their students producing fives or sixes, Xavier leads the pack.  The essays exhibited not just good grammar and solid structures, the students exhibited critical thinking in their essays. 
 
Of note among the public schools is the strong performance of Yap High School with a third of their seniors able to produce a competent or clearly competent essay.  Kosrae High succeeds in lifting one in four to this level.
 
Of concern are the large public schools of Chuuk.  I think it helps to read a one from there, transcribed as written:
 
I'am [name deleted]. I'm growup on my Island and my family and my parents told me go to school becouse in my feature¹ I'm very happy and to my family in the years two thousand I came in [public high school in Chuuk] and my parents told me to to school and try you pest in my classroom I'm very lasy.. Becouse my frend told me is very Badly school is very no good and my teacher your late your name is illimerate² in my Booklet I'm very sad becouse I tryout in my class.

¹ Likely to be the word "future."
² Like to be "illiterate"

Here is young student from a family that clearly values education, she wants an education, and we can see she was denied an education. I suspect a really bright young lawyer could successfully sue the state for breach of social contract.  And as the grading team can vouch, this is but one of dozens such essays.  Those are the details under the statistics.  I work with statistics, but I am keenly aware that under every statistic is a young person with a hope for a bright future. 

Weno and SNHS produced not a single student who can write a competent essay (five or six).  None out of 197 students.  17 students, 7 at SNHS and 10 at Weno can produce a minimally competent essay(4).  Chuuk High School produced five competent essays out of 268 students.  16 produced minimally competent essays. 

63% of the SNHS students, 75% of the Weno students, and 78% of the CHS students produced incompetent or possibly incompetent essays.  I've lived in Africa, in a place far poorer than Chuuk, and I have never seen such a disregard for human potential.  Pardon me, but if someone does not get angry at some point then change is unlikely. 

This is the first time we have had an essay question.  The ability to write is a crucial skill.  There is so much room for improvement that any effort will help, no matter how small.