Sigh. You need both. The reality is that there's a lot of book-learning in the world. You're never going to build a rocket to the moon by starting in your backyard with some sheet metal - your lifespan isn't long enough if you take that approach. You have to read books written by people who have gone before, and learn from their mistakes, which you can do at a far more rapid pace than you can by making all those mistakes yourself. Only by standing on the shoulders of those who have gone before - that is, by reading their books - can you hope to push boundaries.
The writer's complaint is daft. The purpose is to test if you can read a passage and comprehend it. Has nothing to do with microscopes. Has nothing to do with "teaching science", as the writer asserts. Just because he doesn't understand the purpose of the test doesn't mean there isn't one. Reading and comprehending text is one of life's fundamental skills. It is, in fact, useful to know if schools are teaching that well or not.
Before I get downvoted into oblivion I should say I also think schools should do hands-on work (which, of course, they do).
The writer's complaint is daft. The purpose is to test if you can read a passage and comprehend it. Has nothing to do with microscopes.
From my understanding of cognitive science, the question asked is deeply flawed if you want to test comprehension without testing knowledge of microscopes.
One approach to answering questions like this is to build a little model in your head of what is being described. You then answer questions about the model in your head. Experience with microscopes or similar devices is incredibly valuable to building such a model in your head and answering questions about it. This was one of the “big problems” in first generation AI: to answer questions, a computer must be able to construct models, and to construct models, a computer needs a huge amount of real-world experience.
What I’m saying is that answering this question well requires experience with microscopes or similar devices. Being able to answer the question without prior knowledge of what a microscope is, or how light behaves, or lenses period, or refracting telescopes, and so forth, is ridiculously hard.
Now, there may be a good reason why an entire test full of a variety of such questions is effective for accomplishing whatever the school board wants accomplished—getting funding, I imagine—but I stop well short of saying the author’s complaint is “daft.” I think there are good reasons to be curious about the biases inherent in this question, and about where schools end up if they build their entire approach to teaching around scoring well on questions like this.
> One approach to answering questions like this is to build a little model
This test question isn't testing that.
> What I’m saying is that answering this question well requires experience with microscopes or similar devices.
It sure doesn't. It requires you to be able to read and comprehend text. That's it. The subject could have been Martian fizzbozzles instead of microscopes.
> Being able to answer the question without prior knowledge of what a microscope is, or how light behaves, or lenses period, or refracting telescopes, and so forth, is ridiculously hard.
It sure isn't. Anyone who can read and comprehend text can answer it with no prior knowledge of the subject.
My HN friend, my argument was that the writer isn’t daft, and neither is his complaint, and I stand by that. My argument is that the writer raises interesting and useful questions, and I stand by that too.
As to the fact that the question doesn’t test internal modelling, of course it does. If you made it a question about something nobody has ever seen before, then it wouldn’t test knowledge of microscopes. But it does ask about microscopes.
Perhaps, in aggregate, the test covers enough subject areas that the students tested will be exposed to a roughly equal mix of things they have experience with and things they don’t. But as I noted elsewhere, the fact that it’s possible to answer this question from the text alone in no way changes the fact that it’s easier to answer the question with prior experience. And from that, I think it’s perfectly reasonable to ask whether this is a good question.
Yes, if it were truly just about reading comprehension, then the test-writers would be horrifically incompetent, and therefore unfit to test anyone. Anyone who competently designs tests (experimenters, pollsters, etc) knows to address such biasing factors.
But I think the testers aren't total morons, and know that such a question is biased towards children with proper (privileged) backgrounds where microscopes and the respective training are commonplace. They'll be faster at answering the question and not feel ignorant — crucial advantages on tests.
(The testers may not even like this facet of their jobs, like ad industry developers who use ad-blockers, but it sure beats unemployment.)
And the whole question smacks of "scientism", where tools are confused with science. You never see a question like, "Scientists have believed something for 200 years. Is it necessarily true?"
> It requires you to be able to read and comprehend text. That's it. The subject could have been Martian fizzbozzles instead of microscopes.
And therein lies a great fallacy which is still widely believed in our post-Enlightenment world. But the map is not the terrain. A mental model cannot be completely represented with words or pictures, or anything. Words and pictures, even good ones chosen for pedagogical value, are only pale projections of a complex thing onto a flat surface.
>Anyone who can read and comprehend text can answer it with no prior knowledge of the subject.
Without qualification, this statement is strictly false. There may be certain kinds of text that can be comprehended with no prior knowledge of the subject if you assume a certain baseline of internalized knowledge. There are many old parlor games (which have sadly gone out of style) which rely on hacking this qualification. For example, one of them requires that one person follow the instructions of another. One person tries to describe how to draw a particular geometric figure. The person doing the drawing must follow the instruction, but they also are free (in fact encouraged) to ignore the intention of the instruction. It's a game that puts the difference between words and reality up-front and center.
I understand what you're getting at, but that's not the case in this question, since one of the four answers are stated almost word for word in the text. (caution point 4)
The rest of the alternatives are "plausible guesses" which you might think are reasonable if you don't remember the point above. So what they're REALLY testing is if you read the text carefully enough to remember that line or not.
Actually having knowledge of microscopes are probably a bad thing on this particular question since you're likely to skip over the boring warning part, like I did when I quickly read the question just now.
You put this wonderfully. People are very little more than our experience with the world itself. Without external stimuli, we would be trapped within our own minds and only be capable of evaluating a limited set of possibilities. Merely the act of touching an object generates complex cognitive experiences. This is not possible by abstraction alone.
Did you read what raganwald wrote? He's saying that reading comprehension is not typically about memorization, it's often about model building. As a human reads they don't just fill up big bin with data, they build up models. When reading a narrative we will build up ideas of characters. We'll fill in more and more of their details. And when probed about the story we will fall back to our knowledge of those characters, and it may take us a while, if we even can, to figure out exactly which events in the narrative contributed to each aspect of a character.
In this case the object of the narrative is a microscope. And as people read through the text they aren't memorizing sentences or facts they are filling out the model, or character, of the microscope. But people who already have an accurate model of a microscope from personal experience will have an easier job doing so. It's similar to the way standardized tests can be racially biased. If I put forth a "story problem" about the rules of football, being familiar with those rules already gives one an advantage.
> Anyone who can read and comprehend text can answer it with no prior knowledge of the subject.
That's the writer's complaint. In order for this question to be testing reading comprehension rather than laboratory experience, it must be assumed that the test taker has not used a microscope. Otherwise, they could answer the question without even looking at the text.
It is a reading comprehension question. Read the text provided: "To avoid crushing the glass slide when focusing, begin with the lens close to the specimen"
The microscope lens should initially be placed close to the glass slide:
C. To avoid breaking the glass slide when adjusting the focus.
I saw that too. It doesn’t change the fact that understanding of the domain strongly biases the result of answering the question. What you’re saying is that it is possible to answer the question while staring bewildered at a device you have never seen or interacted with. What I’m saying is that it is much easier to answer the question if you have hands-on experience.
And I think what the author is saying is that we should spend more time giving kids microscopes and testing what they see with them.
TANGENT ABOUT BIKE SHED COLOURS:
Questions like this remind me of questions about OOP. First the preamble: “Organize objects into classes to create a hierarchy of abstract types....” Then a question: “Classes are used to (a) provide a constructor for objects, (b) to promote reuse of behaviour, (c) to model a type hierarchy, (d) to facilitate modular software development.”
You ask someone to build a model in their head, and then the most successful strategy for answering the question is to turn the model off and use brute force to scan the exact words of the question, because the more knowledge you have of the domain, the less likely you are to answer the question correctly from your experience.
What you’re saying is that it is possible to answer the question while staring bewildered at a device you have never seen or interacted with.
Based on my life experience so far; this is one of the most important skills to learn.
What I’m saying is that it is much easier to answer the question if you have hands-on experience.
Of course; but that isn't the point of the test.
And I think what the author is saying is that we should spend more time giving kids microscopes and testing what they see with them.
Yes, but again this test is not about that. And if the author is arguing that practical experience should entirely supplant this sort of critical thinking then I entirely disagree.
Critical thinking is taught from a young age, in very subtle ways. The ability to approach a topic you know nothing about, dissemble it, and answer a question (either posed to you, or by you) is fundamental to all of our lives.
I agree with everything you are preaching about critical thinking. Personally, I don’t think this particular type of question is good at testing critical thinking, and worse I think that the incentives in the school system are all about gaming the test rather than teaching critical thinking.
My feeling looking at this is that if I wanted kids to get the highest possible score, I’d teach them how to take tests rather than buy them a bunch of tools and have them figure out how to use the tools from the instruction manuals (reading, critical thinking, problem solving). Whereas if I wanted them to learn critical thinking, we’d be solving problems with tools.
But I accept that this is a very large topic, it’s not like this is a one-question test, and students are exposed to a variety of stimuli. Also, there is always the democracy argument: This might be the worst test ever devised, except for all of the others we’ve tried. Perhaps this is the “least worst” way to test and teach given other factors in play.
> What you’re saying is that it is possible to answer the question while staring bewildered at a device you have never seen or interacted with.
The question is completely invalidated by knowledge of the domain. If anything, they should have constructed an artificial device that would guarantee the student didn't have domain knowledge that would allow them to bypass the necessary critical thinking:
To avoid smaggling the fizdoodle, make sure you begin with the smunderson in the upright position.
Q: Why should the smunderson start in the upright position?
"The microscope lens should initially be placed close to the glass slide: C. To avoid breaking the glass slide when adjusting the focus."
Incidentally that statement only sense if you know which direction to turn the focus knob, and you recognize the fact that the hypothetical microscope is capable of crushing the specimen. I had one when I was a kid that was incapable of doing so.
I answered the question correctly, but it only due to my cynicism, not my reading comprehension.
Blah blah blah blah.
"To avoid crushing the glass slide when focusing, begin with the microscope close to the specimen and then gradually back off to focus."
Blah blah blah blah.
And you know what? I'll tell you why people get this wrong. It's because their brains turn off. They say "omg microscopes scary! big block of text!" and guess at answers instead of actually bothering to try and read the text. That's one reason why they get poor scores at reading comprehension: a lack of reading with intent to comprehend. Heck, if they're pressed for time and the guessing-wrong penalty is low enough, it's an entirely rational response.
The fact that the journalist is that dense is mostly a symptom of "oh hey I've found an argument against standardized testing to fit my preexisting conceptions of the matter! Screw reading it, it looks complicated, I'll just use it to justify things." Same basic problem.
Yes. My mom said "Now what do I do?" when there was a box on her screen that said "Click Next to Continue" and that's all that could be done. I've been trying to teach her to read the boxes that come up before she calls me for help.
On the other hand, when she doesn't call me for help, she ends up printing off a picture from a webpage and scanning it back in to get it into a power point presentation. That's better than giving up on the problem, I guess.
So wait, you're saying the reason for the lens to begin close to the glass is the same reason the lens should begin close to the specimen? How did you come to that conclusion?
When I was taking these sort of tests, I learned to delay processing of non-question text until I've read the question. Reading comprehension, in most of the standardized tests I've taken, was nothing more than ability to quickly pattern match keywords from a question to unstructured data preceding it. This kept me from building models of questions in my head and thusly prevented any preconceived notions or facts divert me from the answer the test considered correct.
Why is it always necessarily to add this? It's like "I apologize in advance but I'm going to insult you".
As an aside I was wondering this morning if people who down vote read an entire comment or downvote on the first negative thing they don't like, disagree with etc. If that is the case you would think that putting your last sentence (sans the bigdio) would suffice as a prophylactic statement?
Maybe I'm just too cynical, but I see that as a form of passive aggressive manipulation. The way I read that kind of comment is "I know the mob will downvote me instead of discussing the topic; you don't want to be part of the mob, do you now?"
Yes, but that is often justified and understandable. HackerNews has one of the better communities out there but I know that getting downvoted without an explanation annoys me. This happens even on HackerNews occassionally.
I guess I phrased the question incorrectly as I know the reason is is to avoid downvotes and the person is aware of what they are saying will get downvoted.
My feeling is that by saying "sorry if this is insulting" or "I apologize in advance" or "before you downvote me" you are rolling over on your belly and appearing vulnerable by acknowledging your vulnerability and trying to show some humility. And in a sense you are setting up the contrast principle because the belly in comparison to the crime just doesn't seem so bad.
"sorry if this is insulting" won't be effective if what you follow it with is truly insulting
we all know thin skinned people though, sometimes the best way to get them to hear a benign (or neutral, or even constructive critism) comment that they might dismiss is to procede it with "sorry if this is insulting"
so there are certain scenarios where I'd say those phrases are ok (this board with their sometimes quick downvoting, seems like an ok use to me)
putting a sentence about how your comment may be downvoted, is just a request not to have that small number reflexsively downvote, because sometimes that happens
I agree with you that the extra noise of the "downvote prophylactic" is annoying, however, I went ahead and downvoted your comment because the meta-complaining adds even more noise. I realize that my comment is meta-meta so feel free to downvote it also.
I think you come from a false premise. The idea that "book learning" can be successfully measured through standardized testing does not have as much backing as you think it does.
One of the big problems of standardized testing and school in general the way we've been doing it is that you can inculcate a great deal of pseudo-knowledge, and you can probe for its existence through these targeted tests, but in many cases you fail at instilling true understanding and knowledge.
Math and science becomes rote memorization of formulas and terms and plug-and-chug. But this isn't learning.
A classic example of the problems involved here is to go ask a bunch of recent college graduates about where the mass of a tree comes from. Even if you only ask people who have studied photosynthesis, you'll still get an abysmally low percentage of correct answers, if any. The same goes for any question on any topic that's covered in college which requires actual understanding and thorough application of the alleged "knowledge" that the students have acquired.
I've seen trees where the largest single component of the mass comes from butterflies. They alit on the branches. Water for most trees' mass mostly comes up out of the ground, though.
On the larger point, testing learning is hard in principle in that you are usually testing theory. Practice is the basis of theory and much harder to test.
But even then, writing a good test to test theory is very hard. Most objective test questions on university tests are terrible and only slightly, if at all, more likely to reward the testee for knowing the theory.
If anyone wants to know the answer to that question, Richard Feynman explains it wonderfully: (He starts talking about trees at 2:00, but his explanation of fire is equally elegant!) http://www.youtube.com/watch?v=N1pIYI5JQLE
My first instinct was to say that a tree's mass comes from dead stars. Imagining how it got from those stars to the air that eventually becomes the tree would keep you busy for a while :D
With regards to reading a passage and comprehend it. That is obvious but completely missing the point.
We don't have a shortage of people with critical skills we have a shortage of people who's knowledge isn't only abstracted from the phenomena they are reading about but a combination.
Testing and experimenting should always run before abstraction and critical thinking exactly because abstraction is based on premises made by experimenting and testing.
Academic discipline is way overrated and the best scientists aren't those who only deal in abstraction (with a few exceptions) but rather those who more more like engineers.
We didn't bring people to the moon by reading about it. Someone had to make those experiments. But it is more and more a lost art and that is sad.
This also breeds a society that is increasingly willing to accept things at face value. Nobody asks if the correct answer on a multiple choice test is really correct. No one is allowed to justify their reasoning--they're wrong. After getting beaten down so many times, people just accept that things are the way they are. This is also the same reason that people can just memorize things for tests and parrot them back without a real understanding of anything.
Book learning is only valuable insofar as there is experience in the real world that it can be applied to. The sole reason I learned algebra and eventually calculus was because I taught myself very basic Java in 6th grade. I had a useful framework to apply the math to.
We need to remember that "if it disagrees with experiment, it is wrong." (Feynman) Not because the test says it's incorrect. It just may be that those two statements coincide, but it is because of the former.
The author, I think, takes a pretty big leap in concluding that the test is being used as a teaching tool, rather than a testing tool. Stating that the students are now stuck learning about microscopes through the test, rather than actually using microscopes seems like quite a stretch to me. Of course kids still use microscopes in school and do lots of hands-on learning. Both types of learning are essential and both are practiced. Does our education system focus too much on testing and not enough on hands-on learning? Maybe, but I think that still varies a lot from school to school, so it's hard and perhaps a bit unfair to make overall assumptions.
However, it does illustrate how the test is a poor evaluation of reading comprehension.
The problem I see is that it's not necessarily a reading comprehension question. It's a question about microscopes. If you don't happen to know anything about microscopes, then the answer is clearly there in the text and, at that point, it's about reading comprehension, but if you do, reading the text is completely unnecessary. So then it remains a question about general knowledge of microscopes, not reading comprehension.
Here's another example... My husband got into IT when he was young and never finished his degree. He's now decided that he wants to go back to school for a related, but different-enough field that he needs that piece of paper and additional training. Because it's been over 20yrs since he's been in school, and he's never attended school in the US, at the age of 40, he found himself taking the ACT a couple of weeks ago.
The "reading comprehension" portion of the test was full of paragraphs about various topics that anyone with an average amount of general knowledge would know and so, being pressed for time, he simply didn't read half the material. He got a perfect score on this section of the test.
That clearly demonstrates a level of intelligence on a variety of topics, and probably indicates some good deductive reasoning as well, but it doesn't show how well his reading comprehension is at all.
Now, he's 40, not 12, but the ACT doesn't look that much different than any other standardized test and it is (along with the SAT) a sort of be all and end all of evaluating a high school student's academic performance.
I think that test item is a good example of the role that testing is playing in the US education system today. The real problem is that, for many students, this test item is the closest they will ever come to using a microscope.
We should be focusing on getting more students in front of a microscope, in classes where they learn effectively from texts as you describe. Instead, we are so caught up in testing that many people lose sight of this goal.
> The purpose is to test if you can read a passage and comprehend it. Has nothing to do with microscopes. Has nothing to do with "teaching science", as the writer asserts.
There are lot of comments here referring to the fact that this exercis used in original post was a reading comprehention test, not 'understanding of the microscope' test. It may be so, but it's still a bad test, and the mistake is on the test authors' side. Why?
Because one of the principles of testing is that you should control for any variable that you do not wish to test. Here, they didn't control for the "I know better than test author how this work, because I have (read a whole book about it || been using it since 8 years old || built one)" factor. And that, for many students, converts the test into a choice - should I write what they expect me to, or should I write how the world really is?
If we teach kids to always choose the first option, do we want to live in this world when they grow up and start running it?
So why not, instead, test reading comprehention on fictional topics (like operating a 'fluxklux' device), thus making sure that no-one can have pre-existing knowledge of it to interfere with reading comprehention?
I think it's necessary to draw a distinction between book-learning, standardized tests as a means to test book learning and the absolute focus on standardized tests that followed the NCLB Act.
Book learning is indeed essential. But standardized tests are far from a necessary component of that. They just happen to be the cheapest/least-offensive devil-we-know for judging whether a student comprehends a passage.
And the alternatives to standardized tests are far from a panacea. For all their desirable traits, they suffer heavily from problems of subjectivity and bias.
But the emphasis on standardized tests, particularly as a follow-on from NCLB is something the country really needs to re-examine and discuss. And this discussion should (ideally) be utterly divorced from whether we need book-learning and even from whether we need standardized tests in some capacity.
Complaints about NCLB and what it's done to our schools and students need not be an assault on testing and book learning. And defenses of testing and book learning need not be defenses of NCLB and its effects.
The emphasis on tests, yes, that is a problem. I agree with you. But of course people see something used poorly and want to get rid of it completely. This happens in many areas, but perhaps nowhere worse than in public education. The trends over the years are a wild backlash from one side to the other over and over, with the system invariably throwing the baby out with the bathwater on each cycle.
Over and over in education debates I see false dichotomies posed without the slightest hesitation, and people passionately argue whether we should do A xor B. But in almost all such debates the correct answer seems to me not to take one side, but to find the proper balance between the two.
I agree: of course there is a lot of value to book learning as well. However, I do think that practical or hands-on learning can be a way to show the relevance of the material taught in books. I remember having trouble with probability and statistics in high school, because it was very abstract. It wasn't until university when things clicked. Mainly, because I all of a sudden saw why it was useful.
In the same way I believe you can make some of the book learning more interesting by relating it back to the hands on learning. Explaining the physics of shooting a rocket up into the air is a lot more interesting if you ran into some of these problems yourself. Of course, this isn't always practical to do for all subjects.
I agree with sentiment of the author, and believe schools have sidelined physical learning to a frightening level. But this test is a comprehension test. If you look at the passage, then the question, you can clearly see what is being examined:
Passage
=======
3. Keep the stage clean and always use a glass slide for specimens.
4. To avoid crushing the glass slide when focusing, begin with the lens close to the specimen and gradually back off to focus.
Question with correct answer
============================
The microscope lens should initially be placed close to the glass slide: C. To avoid breaking the glass slide when adjusting the focus.
Aside from being able to correctly identify which part of the passage relates to the question (showing broad understanding of the passage of the whole) the specifics are:
a) Do you understand that "crushing" is synonymous with "breaking"?
b) Do you understand that you put the specimen on the glass slide?
c) Do you understand that "begin with ..." and "initially ..." are essentially the same construction?
d) Do you understand that "focusing" is the action of "back off to focus"
To me, the surprising thing was how well this question was written.
As an aside, I imagine these questions are purposefully written about a subject that the person being examined is expected to have little knowledge about, so that only the comprehension is being tested.
So are you calling standardized tests "book learning" or "hands-on"??? I say these things devalue "book learning" as much as they devalue building things.
I would agree kids should be reading books, writing essays, writing math proofs and building robot but not doing (and endlessly preparing for) standardized tests. Comprehension isn't going to ever be fully exercised by multiple choice questions.
Seriously, the problems with standardized tests just don't allow them to be glossed over as "book learning" or any other way. The worst problem of standardized tests is they reduce teachers to proctors and game-strategists rather than demanding they have constructive engagement with their students.
The two APICS tests I took sofar are really standardized. The only thing they didn't test was actual knowledge and comprehension of the subject and the thing they didn't deliver was realistic results. What they tested was you ability to read the question in the right way. I take these tests since I assume that they are pretty american-style (atleast I was told so). Doesn't mean we don't have them in good old europe, too.
And in my experience, that's all these test do besides being somtetimes completely random. I agree that it is easier to give marks based on these tests,but they don't really tell you want you want to know.
> The purpose is to test if you can read a passage and comprehend it.
I disagree. The purpose is not to test if you can read a passage. The purpose is to test if you can show someone else you can read a passage. That's not only an incredibly important distinction, because the ability to communicate is an evaluatable skill, but also the skill itself is essential to have.
On the other hand, I do have to admit that a lot of tests are designed to evaluate things that can't really be evaluated... but that's largely because decent testing is hard to scale.
With that said, schools right now often emphasize testing and test prep at the expense of any hands on work. Naturally, we should have the right balance, which seems to be what you are proposing, but if we must overreact, I would much rather overemphasize building, exploring, creating and underemphasize testing than the other way around.
The writer's complaint is daft. The purpose is to test if you can read a passage and comprehend it. Has nothing to do with microscopes. Has nothing to do with "teaching science", as the writer asserts. Just because he doesn't understand the purpose of the test doesn't mean there isn't one. Reading and comprehending text is one of life's fundamental skills. It is, in fact, useful to know if schools are teaching that well or not.
Before I get downvoted into oblivion I should say I also think schools should do hands-on work (which, of course, they do).