Friday, February 15, 2008

Cheers - Overall Test Scores Are Up

Click Here for the New Jersey Report Card

First, the good news. On the statewide HSPA English exams, Matawan-Aberdeen student scores have gone up across the board and surpassed the state average. A quarter of our pupils scored “Advanced Proficient” in math. Plus, our students’ scores went up while the rest of the state witnessed a decline in scores. The lower grades show even more dramatic improvement.

Now, the bad news. The proportion of students who failed the HSPA Math exam leapt 61%. Our students scored below average in HSPA science. SAT scores in math, verbal, and essay, are below the state average and dropping.

The discrepancy is puzzling. Why would math scores on the high end jump higher while those on the low end plummet? Why would our pupils score above average in all ranges on the HSPA English exam but then score below average on the SAT English portion?

My theory, and this is only a theory, is that we are witnessing the scourge of “teaching to the test”. In talks with school officials, the focus is frequently on test scores. Tests are only supposed to be used as a measure of overall education but they’ve become an end in themselves, as if the ability to answer multiple choice questions had any special value. In turn, teachers have been pressured to “raise test scores”. This explains why HSPA scores would go up while SAT scores go down. Students are becoming more proficient in taking HSPA exams but not necessarily becoming more proficient in math, English, and science.

In high school, my teacher for Calculus BC lost control of the class and we were woefully unprepared for the AP exam. Still, the assistant principal strongly encouraged us to take the exam and we did. Looking over prior exams, I discovered a system in the multiple choice portion. The multiple choices were designed to hide the correct answer by being as similar as possible. This meant the most common elements were in the correct answer. For example, regardless of the question, if given the following choices: (a) 2 (b) X (c) 2X (d) 4X (e) 2X^2, I can tell the answer is (c) 2X. Why? Both 2 and X appear in three of the five choices.

Despite being unable to do the short answer portion of the exam, I still passed, scoring a 3; I must have gotten a near perfect score on the multiple choice questions.

I believe teachers spend a fair amount of class time training their students to excel in the HSPA exam. The result is that test scores go up even though students learn less, as is evident in the SAT exams.

This is a natural and disastrous consequence of putting so much emphasis on the state standardized exams.

Yet, I remain a strong proponent of standardized exams. The trick is to devise a testing method where the easiest way to train for it is by actually mastering the material. One example is the adaptive testing method used by Northwest Evaluation Association. Adaptive tests change the difficulty level of successive questions based upon the student’s ability to answer them. These adaptive tests are far more accurate in measuring a student’s proficiency level than the state exams and it’s less likely a student could excel without mastering the coursework.

This still doesn’t answer why there was a dramatic increase in high scorers on the HSPA Math exam but an even more dramatic increase in failures on the same exam.

My guess, and this is only a guess, is that students at the lower end of the spectrum are being targeted for “special instruction”. Considering our high school has three times the number of special education teachers per student as does the state average, I’m assuming we expend a significant amount of resources on underperforming students. If so, we’re doing something very wrong and I’m certain the administration will be quick to examine what happened.

One year doesn’t make a trend but it may mark a turning point for our school district. The faculty, administration, and student body should be commended for a job well done. There’s more to do but it appears we’re heading in the right direction.
>>> Read more!

12 comments:

Anonymous said...

The school report card contains a lot of information. In it we can see how our town’s students compare to the New Jersey’s State Average and we see how they compare to past graduates.

I would like to respond to the comments about the SAT scores. The SAT scores are tricky to interpret. When we look at the number of students that are planning to attend college, in MARSD, we have 94.5% of the graduating class with plans to attend either a 2 or 4 year college. This district’s students are encouraged to further their education after graduation. I suspect that students are taking SAT’s (in our district) while in other districts similar student with the same academic aptitude would not even consider attending college. We have a higher graduation rate and likewise a lower drop out rate than the state average. So some schools in this state have students dropping out and not taking the SAT while this district gives “students at risk” the help they need to follow their dreams and attend college. While their scores would not make you swoon, they are able to attend college.

Another second point that I would like to make involves an issue that is not evident by reviewing the districts report card but should be mentioned. That issue is the role of county vocational magnate schools and how those schools impact this district scores. High schools such as High Tech High, Allied Health, Communication, and MAST to name a few, receive many of our top students each year. Students must take a test to attend those schools and those schools only take the best. So, if you see terrific SAT scores for those high schools; it is because they siphoned the best students from our high school. Ironically we send many students because of the superb education students receive during their first eight years in our district). Friends, family, and neighbors of mine (in this town) have sent the children to the county vocational schools; all of them received scholarships from the colleges that they attended because of their high SAT scores. I dare say this would not have been possible without the basic knowledge that they received at the K-8 level. So while statistics can give you a picture, you really need to understand the entire situation before making pronouncements about our town’s educational system. Oh, and by the way, speaking of taxes, you pay with your taxes dollars for those students to attend those county high schools and for their transportation to and from those schools. Ironic isn’t it?

Aberdeener said...

Excellent points. Aside from my theory, can you think of any other reasons why HSPA scores would rise while SAT scores decline?

(On a side note, I would have included those students intending to join the military along with the students intending to enter college.)

Truth In Matawan said...

Excellent point as well about the specialized high schools. Our best students continually leaving to better themselves(as they should) hurts our overall scores, obviously.

Anonymous said...

HSPA vs. SAT scores
In regard to your question about the rise in HSPA and decline in SAT scores, the short answer is that the scores are radically different because they are radically different types of tests. For a more lengthy analysis read below.
The New Jersey Education website, (address:http://www.state.nj.us/education/assessment/hs/hspa_guide_english.pdf)
indicates the following about the High School Proficiency Assessment:
The HSPA is a state test given to students in the eleventh grade to measure whether they have gained the knowledge and skills identified in the Core Curriculum Content Standards. …Special education students will be working toward achieving the standards at levels appropriate for them and with any accommodations or modifications they may need. These accommodations are defined in their Individualized Education Programs (IEPs). The accommodations or modifications should be the same as those used by these students in other classroom testing, and may include Braille, extended testing time, or a different testing site.
It should also be noted that my research uncovered that the state of New Jersey ranks second in testing difficulty. In other words, 48 states have less difficult tests then our state and while we expect a New Jersey High School graduates to be able to successfully complete questions in mathematics that involve: Number and Numerical Operations, Geometry and Measurement, Patterns and Algebra, Data Analysis, Probability, Statistics, and Discrete Mathematics; most other states test require less.
Further, students are tested on how well they have learned the material and when tested, if you are a special education student with an IEP, that states you can have more time to finish the test, it is given.

The SAT is a different test completely. The College Board states that the SAT measures critical thinking skills that are needed for academic success in college. They state that the SAT assesses how well the test takers analyze and solve problems—skills they will need in college. The current SAT, I understand takes about 4 hours and there are no adjustments made for IEP’s. It should also be noted that students’ scores throughout the country seem to be on a downward trend. Last years scores were lower than the previous year’s score country wide. This is because in 2005 the new test was designed to be more difficult in response to the number of perfect scores. Hopefully, this has answered your question regarding the disparity between test scores.

Aberdeener said...

Anonymous,
Thank you for the detailed response. My concern is why HSPA scores would rise while SAT scores would continue a downward trend. Best I can tell, you offer three independent reasons that could all be occurring concurrently.

Here is a link to the prior year’s 2005-2006 report card.

The first argument is that scores could be affected by the number of students classified as “learning disabled” taking the exams. HSPA allows special accommodations while the SATs do not. If more students were so classified, HSPA scores could rise from special accommodations while SAT scores would still decline. The problem is that the percentage of students classified as learning disabled dropped from the prior year. Also, a smaller proportion of students took the SAT exams. If students with Individualized Education Programs (IEPs) played a significant factor in the trend, then HSPA scores should have gone down while SAT scores went up.

Another possibility is that our district was part of a national trend of declining SAT scores. The problem here is twofold: First, our district scored below the state average on the SAT but above the state average on the HSPA. Secondly, if you look at the HSPA math section, our district’s scores rose dramatically across the board while the state declined in all ranges. Yet, the state average in the SAT math only dropped by 7 points while our district plummeted 25 points.

The third suggestion, that the exams test different skill sets, sounds very much like my argument that instructors are teaching to the test. After all, as students become more proficient in math and English, they’re scores in HSPA and the SATs should both be rising.

I believe the data most supports the notion that teachers are teaching to the test. I still don’t see any alternative.

Anonymous said...

In this "standards based" age, I am not surprised that teachers are "teaching to the test" because that is how you are evaluating them and the school district. It is what happens when people fail to see and treat children as complex human beings who are in the process of maturing and developing to their full potential. I read a study that evaluated a group of students from the same elementary school 20 years later. The cluster that achieved the most success all had the same kindergarten teacher. It would be interesting to see her teaching methods.

Anonymous said...

Auditors' Management Report has been now put on to the Distict web SiTe. You can see interesteing info in this report

Anonymous said...

There are 4 county vocational schools and MARSD gets two seats in each school, so, even if these kids are bright, I would suggest that they would have little impact on the average when 300 kids take the HESPA and 285 (avg 95% of the senior class) take the SATs. They are not going to move the average much. The second point is, what is overlooked is that the vast majority of students are average and as these scores indicate, they are under served

Anonymous said...

To respond to the last comment, there are five Magnate schools:
High Tech High
Marine Academy of Science and Technology
Communications High School
Academy of Allied Health & Science
Biotechnology High School

Years ago, I also heard that only two are accepted from each town but in my son’s class four students were selected to go to Communications High School. My nephew went to MAST only three students took the test but all three were accepted. They have a limited number of seats which averages to 2 per town but if one town does not send any and we have 3 or 4 high scoring students, it appears that they all are accepted. Perhaps this is because some of the applicants are from Aberdeen and some are from Matawan.

As for the HSPA vs. SAT, I think it is best to use the following analogy: to receive a driver license in the state of New Jersey, first you must pass a written test then you must pass a road test. The two tests are completely different though they applied to the same body of knowledge (i.e. rules of the road). Everyday you have people who pass the written test but later fail the road test. Why? They knew the rules of the road; their written test proved that so, why did they fail the second portion? Because the test requires that they apply what they know to a different situation and some people fail the first time. Just like some students do not do well on the SAT the first time.

Another consideration is that current SAT tests are different. It assumes that knowledge has been assimilated and requires high level reasoning. Currently on the SAT, 19% of the math questions that have no multiple choice answers provided. Instead students must solve the problem and fill in their own answer.
Student-produced response questions will continue to appear on the SAT.
For example:
In isosceles triangle ABC, the measure of angle A is 80°. If another angle of the triangle measures x°, where x 80, what is one possible value of x?
So, this assumes that students would remember the theorem from their geometry class (of 1 or 2 years ago) and can produce an answer. I repeat, students are not given a choice of possible answers provided in the multiple choice grid as mentioned earlier in the blog.
One last note, it should be mentioned that SAT scores are often generational. Many years ago, when I took the SAT, my high school class scored high scores (second highest in the state) and we had a dozen students with perfect scores on both math and verbal. In contrast, the class that graduated the year before us did not do as well; they had the same teachers and facilities and even the same books as we did. It was something that could not be explained and no one cared to try to explain it because everyone was pleased with the scores. The call for explanations only occurs when the reverse happens and the scores go down. Oh and by the way, this would have been difficult to recognize if we had vocational magnate schools because they would have been split up and no one would have realized that so many from our town had perfect scores.)

Anonymous said...

If the MAHS is so great as everyone spouts, why would anyone coinsider going anywhere else.....?

Anonymous said...

There are a few reasons why students would choose the vocational schools over our high school. The first is that they are prestigious. High Tech is the number one school in the state according to APP. Colleges recruit at these high schools to get the best students. The schools are specialized so that students who have a clear idea of what they want to be when they grow up can pursue their interests in better facilities (i.e. engineering labs) than most high schools could afford. The other major attractions are great parental support and like-minded students. There are downsides: no formal art or music instruction, no array of major sports teams and the friends that the students make are often far away. One of my own children went to a specialized high school and two attended Matawan Regional. They are completely different individuals and all three received challenging educations that prepared them for college and beyond.

Truth In Matawan said...

12 perfect 1600s in one class? Wow.