Testing, Testing & More Testing
athom
As the weeks whiz by I am learning more and more about the testing
industry, more and more about this whole subject. I know, or hope,
that on this list I am preaching to the choir, but if there is anyone
out there still sitting on the fence about testing, or rationalizing
that, well, their kid is good at it so who cares, or telling yourself
that if your kid wants to go to college you are going to have to
test, no choice, so why think about it, here are a few things to
think about:
* Most companies in this industry (test scoring, BIG BUSINESS!) only
contract for a 70% agreement rate. That means that when a specific
response is read by two different readers that they must agree on the
score 70% of the time. Which means that they will disagree at least
30% of the time. That means your kid has at least a reasonable
probability of having a 30% fudge rate on any test. Fine if the
fudge goes your way, but not so fine if it goes agin' em.
* And that 30% fudge rate is only average. On some very difficult to
score questions the rate may be closer to a 70% fudge rate. Because
no matter how carefully the readers score the responses, some
questions are so badly written, and some rubrics defined by the
testing institution are so difficult to score, that the rate of
agreement drops like a rock. The odds just keep getting worse.
* Depending on what time of day your kid's test questions are scored
the degree of compliance with the defined rubric will vary
considerably. First thing in the morning some readers are likely to
be half asleep, others over-caffeinized. Don't know which is worse.
At other times cyclical low blood sugar could account for all kinds
of variations in accuracy on reading and scoring responses. And a
score of other factors, from room temperature, to barometric
pressure, to the war in Iraq, to the usual domestic issues in
anyone's life, will all affect the test scorer's ability.
* Depending on when any one child's test is scanned into the system
readers must meet different scoring goals daily. During the early
part of the reading contract for any one test they will have low
daily goals and may only be reading hundreds of responses a day, thus
able to read and score more carefully. During the later days of the
contract goals may rise to thousands a day and scorers will be
pressured to read for volume and not accuracy or agreement. This
pushes that 30%-70% fudge rate right through the basement.
* Handwriting or hand-printing play a big part in scoring. Those who
write infinitesimally are extremely difficult to read. Scorers get
annoyed by these and may not read them as carefully (?) as others.
And very messy handwriting definitely does not read well. Test
readers are reading hundreds to thousands of different documents a
day and as the day wears on the more difficult they are to read the
less likely it is that they will get a fair reading.
* Many test readers are taking all kinds of medication. Many are
retired teachers and school administrators, supporting the AMA and
the American pharmaceutical industry on their retirement incomes with
a sack full of meds they take daily, if not hourly. It is difficult
enough to do this type of job well with a clear unmedicated mind, but
those on medication may have even more difficulty.
* Many test scoring contracts have a huge bonus ($$$$) for tests that
are completed before a certain date, thus rewarding companies for
speedy scoring rather than accuracy, even though accuracy is a myth
to begin with.
Read Lemann's THE BIG TEST for more details on the whole meritrocracy
issue and do whatever you can to not only unschool, but to untest
your kids. When you think about college degrees, therefore, think
about options and schools that do not give grades (Antioch, for one),
and ways of earning credit without testing. And if your state gives
home educators the option of testing or anything else, pick anything
else!
If you really want a test score to submit here is what I suggest:
make up a bunch of questions of your own by picking some readings and
asking questions about those readings. Tack the answers up on the
wall or a tree with numbers around the page. Throw darts at the
pages. Add up the numbers closest to where the darts land. This
should pretty well give you the same validity as any handscored test
your kid could take.
Remember that many of these issues also apply to machine scored tests
due to poorly worded questions and poorly worded, if not totally
wrong, response choices.
So why do people insist on these tests? Money, money, money! And
elitism, meritocracy, as in "my kid is better'n your kid." No other
reason to do this to children, and to waste such huge amounts of
taxpayers' money. Parents (taxpayers) need to revolt against this.
Instead more and more testing is being mandated by the feds and the
state government. More and more kids lives and self-esteem are being
challenged, if not seriously eroded, by all kinds of testing. Who
runs this country, anyway? We need to speak up and let our elected
representatives know how we feel about all this. If anyone knows any
other good books on this subject please post info so that others can
read what has been written and use those "expert" sources to convince
our paid representatives that we aren't going to fund this kind of
rubbish anymore.
The one book I have consistently referred to is THE BIG TEST: THE
SECRET HISTORY OF THE AMERICAN MERITOCRACY by Nicholas Lemann.
Anyone know of any other good ones? The key word is "secrecy." All
of this industry is shrouded in secrecy so that most who subject
their kids and themselves to this kind of testing have no idea what
is really going on. You do, at least you know more than most, so I
hope that some will speak out and let their representatives know what
a huge rip-off all this testing is.
Norma
industry, more and more about this whole subject. I know, or hope,
that on this list I am preaching to the choir, but if there is anyone
out there still sitting on the fence about testing, or rationalizing
that, well, their kid is good at it so who cares, or telling yourself
that if your kid wants to go to college you are going to have to
test, no choice, so why think about it, here are a few things to
think about:
* Most companies in this industry (test scoring, BIG BUSINESS!) only
contract for a 70% agreement rate. That means that when a specific
response is read by two different readers that they must agree on the
score 70% of the time. Which means that they will disagree at least
30% of the time. That means your kid has at least a reasonable
probability of having a 30% fudge rate on any test. Fine if the
fudge goes your way, but not so fine if it goes agin' em.
* And that 30% fudge rate is only average. On some very difficult to
score questions the rate may be closer to a 70% fudge rate. Because
no matter how carefully the readers score the responses, some
questions are so badly written, and some rubrics defined by the
testing institution are so difficult to score, that the rate of
agreement drops like a rock. The odds just keep getting worse.
* Depending on what time of day your kid's test questions are scored
the degree of compliance with the defined rubric will vary
considerably. First thing in the morning some readers are likely to
be half asleep, others over-caffeinized. Don't know which is worse.
At other times cyclical low blood sugar could account for all kinds
of variations in accuracy on reading and scoring responses. And a
score of other factors, from room temperature, to barometric
pressure, to the war in Iraq, to the usual domestic issues in
anyone's life, will all affect the test scorer's ability.
* Depending on when any one child's test is scanned into the system
readers must meet different scoring goals daily. During the early
part of the reading contract for any one test they will have low
daily goals and may only be reading hundreds of responses a day, thus
able to read and score more carefully. During the later days of the
contract goals may rise to thousands a day and scorers will be
pressured to read for volume and not accuracy or agreement. This
pushes that 30%-70% fudge rate right through the basement.
* Handwriting or hand-printing play a big part in scoring. Those who
write infinitesimally are extremely difficult to read. Scorers get
annoyed by these and may not read them as carefully (?) as others.
And very messy handwriting definitely does not read well. Test
readers are reading hundreds to thousands of different documents a
day and as the day wears on the more difficult they are to read the
less likely it is that they will get a fair reading.
* Many test readers are taking all kinds of medication. Many are
retired teachers and school administrators, supporting the AMA and
the American pharmaceutical industry on their retirement incomes with
a sack full of meds they take daily, if not hourly. It is difficult
enough to do this type of job well with a clear unmedicated mind, but
those on medication may have even more difficulty.
* Many test scoring contracts have a huge bonus ($$$$) for tests that
are completed before a certain date, thus rewarding companies for
speedy scoring rather than accuracy, even though accuracy is a myth
to begin with.
Read Lemann's THE BIG TEST for more details on the whole meritrocracy
issue and do whatever you can to not only unschool, but to untest
your kids. When you think about college degrees, therefore, think
about options and schools that do not give grades (Antioch, for one),
and ways of earning credit without testing. And if your state gives
home educators the option of testing or anything else, pick anything
else!
If you really want a test score to submit here is what I suggest:
make up a bunch of questions of your own by picking some readings and
asking questions about those readings. Tack the answers up on the
wall or a tree with numbers around the page. Throw darts at the
pages. Add up the numbers closest to where the darts land. This
should pretty well give you the same validity as any handscored test
your kid could take.
Remember that many of these issues also apply to machine scored tests
due to poorly worded questions and poorly worded, if not totally
wrong, response choices.
So why do people insist on these tests? Money, money, money! And
elitism, meritocracy, as in "my kid is better'n your kid." No other
reason to do this to children, and to waste such huge amounts of
taxpayers' money. Parents (taxpayers) need to revolt against this.
Instead more and more testing is being mandated by the feds and the
state government. More and more kids lives and self-esteem are being
challenged, if not seriously eroded, by all kinds of testing. Who
runs this country, anyway? We need to speak up and let our elected
representatives know how we feel about all this. If anyone knows any
other good books on this subject please post info so that others can
read what has been written and use those "expert" sources to convince
our paid representatives that we aren't going to fund this kind of
rubbish anymore.
The one book I have consistently referred to is THE BIG TEST: THE
SECRET HISTORY OF THE AMERICAN MERITOCRACY by Nicholas Lemann.
Anyone know of any other good ones? The key word is "secrecy." All
of this industry is shrouded in secrecy so that most who subject
their kids and themselves to this kind of testing have no idea what
is really going on. You do, at least you know more than most, so I
hope that some will speak out and let their representatives know what
a huge rip-off all this testing is.
Norma
Sorcha
I read an enlightening article online a few weeks ago about the Bush and
McGraw families, and their vacation homes next to each other, and how
George W. and the President/C.E.O. of McGraw-Hill grew up together and
are close friends. It chronicled the passing of the No Child Left
Behind Act and talked about how much money McGraw-Hill makes from their
standardized tests and their workbooks to prepare kids for standardized
tests, and their curriculums teaching to the tests. You can't walk into
a Barnes and Noble or Borders kids' section without seeing dozens of
McGraw-Hill Test Prep workbooks, prominently displayed. Wish I'd
bookmarked the article. I think it was linked from a site about high
school kids who are protesting standardized tests.
Sorcha
[Non-text portions of this message have been removed]
McGraw families, and their vacation homes next to each other, and how
George W. and the President/C.E.O. of McGraw-Hill grew up together and
are close friends. It chronicled the passing of the No Child Left
Behind Act and talked about how much money McGraw-Hill makes from their
standardized tests and their workbooks to prepare kids for standardized
tests, and their curriculums teaching to the tests. You can't walk into
a Barnes and Noble or Borders kids' section without seeing dozens of
McGraw-Hill Test Prep workbooks, prominently displayed. Wish I'd
bookmarked the article. I think it was linked from a site about high
school kids who are protesting standardized tests.
Sorcha
[Non-text portions of this message have been removed]
[email protected]
In a message dated 4/10/2003 7:32:31 AM Eastern Daylight Time,
sorcha-aisling@... writes:
"education". Our governor just got elected because he wants "tougher"
schools---in every sense of the word, in every way! (Testing, discipline,
teachers, etc.)
I think there is a senator or congressman somewhere out west who said schools
suck, but ecucation is a very common platform. They're ALL into testing.
~Kelly
[Non-text portions of this message have been removed]
sorcha-aisling@... writes:
> I read an enlightening article online a few weeks ago about the Bush andInteresting, but there hasn't been a politician who's not BIG into
> McGraw families, and their vacation homes next to each other, and how
> George W. and the President/C.E.O. of McGraw-Hill grew up together and
> are close friends.
"education". Our governor just got elected because he wants "tougher"
schools---in every sense of the word, in every way! (Testing, discipline,
teachers, etc.)
I think there is a senator or congressman somewhere out west who said schools
suck, but ecucation is a very common platform. They're ALL into testing.
~Kelly
[Non-text portions of this message have been removed]
Mary
From: "athom" <tessimal@...>
<<Many test readers are taking all kinds of medication. Many are
retired teachers and school administrators, supporting the AMA and
the American pharmaceutical industry on their retirement incomes with
a sack full of meds they take daily, if not hourly. It is difficult enough
to do this type of job well with a clear unmedicated mind, but those on
medication may have even more difficulty.>>
Just wondered if the book actually researched this part of it or is it just
making a blanket statement about retired people? Just seems very unfair to
say unless they actually know all the people who are test readers.
Mary B
<<Many test readers are taking all kinds of medication. Many are
retired teachers and school administrators, supporting the AMA and
the American pharmaceutical industry on their retirement incomes with
a sack full of meds they take daily, if not hourly. It is difficult enough
to do this type of job well with a clear unmedicated mind, but those on
medication may have even more difficulty.>>
Just wondered if the book actually researched this part of it or is it just
making a blanket statement about retired people? Just seems very unfair to
say unless they actually know all the people who are test readers.
Mary B
athom
--- In [email protected], "Mary" <mummy124@b...>
wrote:
<<>>Many test readers are taking all kinds of medication. Many are
retired teachers and school administrators, supporting the AMA and
the American pharmaceutical industry on their retirement incomes with
a sack full of meds they take daily, if not hourly. It is difficult
enough to do this type of job well with a clear unmedicated mind, but
those on medication may have even more difficulty.<<>>
<>Just wondered if the book actually researched this part of it or is
it just making a blanket statement about retired people? Just seems
very unfair to say unless they actually know all the people who are
test readers.<>
Mary B.:
This does not come from a book but from a "mole" working in the
business. And, no, not every senior working on a project is on tons
of meds, but plenty of them are. And, yes, it does effect their
ability to think clearly sometimes, as do many of the other things I
mentioned in my post earlier.
Norma
wrote:
<<>>Many test readers are taking all kinds of medication. Many are
retired teachers and school administrators, supporting the AMA and
the American pharmaceutical industry on their retirement incomes with
a sack full of meds they take daily, if not hourly. It is difficult
enough to do this type of job well with a clear unmedicated mind, but
those on medication may have even more difficulty.<<>>
<>Just wondered if the book actually researched this part of it or is
it just making a blanket statement about retired people? Just seems
very unfair to say unless they actually know all the people who are
test readers.<>
Mary B.:
This does not come from a book but from a "mole" working in the
business. And, no, not every senior working on a project is on tons
of meds, but plenty of them are. And, yes, it does effect their
ability to think clearly sometimes, as do many of the other things I
mentioned in my post earlier.
Norma