I’ve had mixed success typing things verbatim vs hand writing verbatim.
When learning to program, I found typing out code samples had a similar effect to writing them out by hand, and felt more natural to the domain. But in other subjects in school, it felt like my comprehension was lower when I typed lecture notes instead of handwriting them. I was never sure whether that was something inherent to hand writing vs typing or it was just because I had more of a barrier between typing and thinking than I did with handwriting.
It always seemed clear that the value of writing something verbatim was that it forced your brain to internalize it in a way you don’t need to when reading. It takes longer to hand write than to type — maybe that is the value of hand writing vs typing? Or when typing I needed to think more about the act of typing, whether that was thinking about formatting/positioning or just being less natural than handwriting? Maybe programming is just a special case where you always think through typing?
I lean towards thinking it’s somewhat domain specific. Clearly with physics or math if you had to typeset equations with LaTeX that would get in the way, and typing would be a barrier to understanding. With programming, every idea is expressed through typing, and so typing is a natural way to imprint ideas on your brain.
It's actually really interesting that you mentioned physics and math - I used to typeset my lecture notes in LaTeX and I found it more effective to do it for exactly the same reasons you mentioned. The time it took to typeset equations forced me to figure out what those equations were saying.
I suspect it's time to digest the concepts and when it's slower to handwrite versus type that results in better comprehension.
There are studies (I don’t know how good - this is a second hand account of a conference talk) that suggested using difficult to read typefaces or deliberately degrading texts by photocopying multiple times helped retention. Perhaps writing in awkward forms is a similar effect.
Daniel Oppenheimer et al.: Fortune Favors the Bold (and the Italicized): Effects of Disfluency on Educational Outcomes (Proceedings of the Annual Meeting of the Cognitive Science Society, 2010), https://escholarship.org/uc/item/4wd1s7hj
"Study 1 found that harder to read fonts led to increased retention in a controlled laboratory setting. Study 2 extended this finding to real-world classroom environments."
I'm not an expert on this topic, so I don't know whether this result was replicated since. If it indeed works, then kudos to my university's physics department for making use of this effect - they had a habit of handing out physics cheat sheets and various MATLAB code snippets that have been photocopied for 15+ years from one copy to another.
Yeah, interestingly I tend to pause and think rather than taking notes, and I’ve often found that’s more effective for me because I can spend more time thinking, and less time an the mechanics of writing.
I have a similar experience. Programming is best learned by typing, but not good for example, materials science notes, those were better written out by hand.
For some of my undergrad courses we were allowed a 'cheat sheet' which is a single 8.5"x11" sheet of paper that we could write anything we wanted on and could bring in to the exam. It became a skill to carefully put an entire course's worth of information on a single sheet. But for me, interestingly, once I had made this sheet I rarely referred to it in the test. The act of making it was enough for me to remember most of it.
Maybe programming is just a special case where you always think through typing?
For me, yes. It's a question of speed. I type much faster than I can handwrite. For a complex unit of code, it's better for me to dump it as fast as possible so I can hold it in my mental pipeline as a whole.
Long ago, learning touch typing was a huge boost to my programming capacity.
For other stuff, I find it better handwritting so the ideas have the time to sink in. Programming seems a more complex activity.
I do a form of this when I want to learn a new codebase. I force myself to write out on paper every class/datamember/method declaration in the file and give a one line summary of what I think it does, as well as I can from the code itself. I am often completely wrong about what the function does but still get huge benefits in understanding the scope of the whole thing.
When I'm learning a new programming language, I force myself to type in lots of example code. I've found that it helps to drill in new syntax and way of thinking on some sub/unconscious level, maybe even physical - like muscle memory of fingers.
I'm way too lazy to write anything by hand though.
I just find that paper slows my thinking process enough that I am forced to focus on the details. When I have a keyboard in front of me my brain is in production mode and just wants to get stuff done not stop and reflect. It may have to do with growing up without laptops in school though.
There's probably something in all of these methods but everybody is different. I don't know if actual pen and paper is required, perhaps typing is just as good. The benefit of typing is that the data is preserved in a useful format. If I write it out simply to learn it, what do I do with the paper? Crumple it up and throw it out?
My process is to open about 20 browser tabs on a subject and cull them off one by one as I create flashcards from the information (process it, write bite-sized chunks in my own words). I learn via flashcards for about 15 minutes per day for a while. Nothing strenuous. That cements the 10,000ft view into my head before trying trying the hands-on stuff, so that I don't give up out of frustration. From there I try to use the information for real.
I have done this too for familiarizing with large/complex codebases (i.e. for a C++ game engine). Afterwards I usually felt like it was a waste of time (and paper) as I didn't reach back to those notes often. But that's because they had already served their purpose.
I recently ran across a “How to Self-Study Math” video[1] which mentions the same idea, that you can throw out the paper after. I think I’d want to keep them.
Maybe it’s good, maybe it’s bad. A lot of anecdotes about “important” historical figures or institutions that use copy by hand as a method for learning.
I would have preferred citing actual research, not an appeal to historical methods.
Given that some of the conclusions that word for word copy may be less efficient than summarized copy[1], it may be a less efficient and less effective way of learning than going through a reading and summarizing every paragraph.
In my personal anecdata, writing a thing down is an important mechanism to note and recall facts I’ll find important later. I nearly never consult my notes. I’m even a terrible note taker, and always have been. I write things down in random places which normally I discard as they become clutter, just because the act of writing them is an additional point of recall when I might fail to recall otherwise. It doesn’t always serve me well, but it always serves me better than not making the note somewhere.
> I would have preferred citing actual research, not an appeal to historical methods. Given that some of the conclusions that word for word copy may be less efficient than summarized copy.
I can at least provide some links. [1] is my own research on giving students optional typing practice in a CS2 course. [2] is Mickie Chi's overview of the ICAP framework which categorizes learning activities based on their level of engagement (Interactive, Constructive, Active, Passive). Chi's work notes that higher modes of engagement provide more learning gains, or I > C > A > P.
Copying would be considered an Active exercise and theoretically would not give as much learning gains as a Self-Explanation exercise ("summarized copy", Constructive). However, much of the research into self-explanation shows that lower-performing students do not provide good summarizations/self-explanations. Thus, in my [1] work, I make the argument that for these students, completing a lower ICAP mode (typing practice) is a better use of their time. While it does not provide as much learning gain as a Constructive activity, it can still give students some gains that could potentially elevate them to a mental model that can successfully complete Self-Explanations.
> In your research area, is there a significant textbook or summary paper you would recommend that summarises current findings well?
It depends on what you're referring to by "research area". My focus is specific to novel CS exercises like typing exercises, Parsons Problems, coding problems, etc. In that regard I really like Teaching Tech Together [1] as a broad, here's a blanket review of CS education and its exercises. If you mean more generally to just CS Education, then Teaching Tech is still starting point I think, as it provides a nice literature review of the domain as well.
> What would you recommend to a complete amateur orienting themselves?
It'll depend (again) on what you mean here. If you mean learning about CS Education research, then the link above will be great. Then its about going down rabbit holes from the citations to read in more detail about those findings.
If you mean simply learning CS, the biggest recommendation I can make is carve out 1-4 hours a week (depending on your schedule) and commit to learning to code via MOOCs, tutorials, videos etc. Find a CS1 syllabus from a university that has a schedule on it and follow that. A lot of learning can be traced back to "time on task". Ignoring the recent HN post about how years of experience doesn't equal most skilled coders, that article is looking at what research calls "experts" vs. "novices" (beginners). We know spaced repetition works, we know cramming (trying to learn it all immediately) doesn't. Following the syllabus' schedule will space out your learning, force you to recall the information, and produce better learning in the long run.
The idea is to make it a part of your weekly "routine" to the point where if you DON'T do it, you feel weird. For example, I've trained martial arts for 15+ years. Somewhere in that time, I'm so used to going to train that when I take nights off, its weird because I'm just USED to training. Even my body wakes up cause its used to needing adrenaline. That needs to be a part of any learning process.
Oh I've definitely figured out the trick to learning CS, tékhnē.
No worries there.
I'm simply curious about the state of the art of the pedagogy.
CS education is a fascinating situation. You have an interaction of strong mathematical, socioeconomic, and generally academic backgrounds, and often the complete opposite studying it. Plus very little institutional knowledge around pedagogy or anything else relative to almost all other fields of academia. And yet, there are all sorts of interesting factoids around the place, like https://www.youtube.com/watch?v=7J-wCHDJYmo.
I don't pretend get how it all fits together.
Thank you very much, Teach Together looks like the perfect starting point, and I can follow the citation breadcrumgbs from there.
"Way of learning" is a very broad term, and the two ways may be good for different aspects of it.
Verbatim copying is good for bringing your attention to the material, it helps you notice more, and doesn't let you mind wander. While summarizing is good for cementing what you've noticed and learned.
For example, if I try to copy part of text I see the writing techniques used there a lot better. I don't see how summarizing can give me the same effect.
I'm almost ready to try both techniques simultaneously, although it seems like an overkill, so much writing.
Like any of this kind of advice, I think there are 2 ways of doing it: robotically and mindfully. If the latter is done, then I think it may have some use. If the former, then it's similar to the argument of listening to learning material before going to sleep - of dubious effectiveness.
I'm naturally sceptical because I hated being forced to pointlessly take "notes" as a child (I always felt like they were more intended as a way for teachers to secure their own ass). It took me 20 years to start writing my own notes again. (...on an e-ink device. I guess I'll just hate paper forever. This is that school does to people.)
Anyway, I'd love to see some research comparing, for example, the retention gained by writing stuff out by hand, to just reading the material multiple times until an equivalent amount of time has passed. It seems possible to me that what The Suck achieves is forcing you to slow down and take some time to absorb the information, but the method doesn't seem efficient/optimal in principle.
Regarding being forced vs doing it of one's one volition, I hazily recall some research about checklist effectiveness in hospital and industrial work. If they were handed down from on high they were less effective than teams coming up with their own checklists.
That matches my experience. If I'm being forced to do something, my brain is focused on compliance, meaning doing whatever it takes to get the authority figure off my back. But if it comes from intrinsic motivation, I'm focused on achieving a practical result as best I can.
I've had the exact same thought. This is anecdotal, but I found that writing stuff out by hand indeed helps me tremendously to absorb the material, mainly by creating a more focused state of mind than I'm in otherwise. It's almost as if the hand is guiding the mind, and facilitating the right mental state. That said, I've been getting better at achieving that level of focus without writing things down by hand, and find that actually verbally recapping what I wrote to myself helps me internalize what I read even better, although it's more hit and miss—when my monkey mind acts up, I go back to writing things out by hand.
I'd be very curious to see someone doing a more rigorous experiment across a larger group of people and see what conclusion they arrive at.
It'd be interested in seeing a comparison against typing notes.
My prior is, having a ritual to set your brain in learning mode helps activate it. For most of us (at least over 30 or so) the ritual of written notes was taught. But there's definitely been enough time to get some adults who used typed notes to learn.
The most interesting test to me would be a 2x2 comparison. People who learned to <type|write> notes, test them on recall of information <typed|written>. All four combinations so we can really suss out what makes the difference.
Perhaps, but most of us use computers for lots of other things, e.g., doomscrolling facebook. It seems just as plausible that putting fingers on the keyboard would prime us for a de-focused, short attention span mode.
Hunter S. Thompson claimed to have typed out The Great Gatsby several times on a manual typewriter. Definitely makes you pay close attention to the structure of phrases and sentences.
When I was around 17 years old I started preparing for my exams by writing out the material I read.
Say it was very effective from retention/ memorization perspective but very inefficient in terms of how long it took.
I became dependent on this method for my preparation though. Which ...sucked I guess. Often felt like a handicap when I needed to quickly go through some material for review.
Having gone through a similar thing myself at the same age, I’ve since come to conclude that most education on offer just isn’t for me. I want to learn things well, not just well enough for an exam in an efficient time frame. Corollary: I want to learn things that are worth learning well.
Through this lens, much of conventional education feels like a desperate babysitting tactic to keep the young busy.
<rant>
For this reason I’m completely biased toward believing all of the positive stories about homeschooling coming out of the pandemic. The socialising argument against homeschooling has always seemed suspicious, and I completely buy that real socialising happens outside of the confines of school. From my own experience, this always feels true … extracurricular sports, music or out of school social events always seemed to be more meaningful and relevant than the classroom dynamic.
Oh, I agree. School and formal education tend to be defined in a relationship to normal, and this lens alone explains most of how it operates.
The pandemic changes were dramatic precisely because they struck at norms, and that was the question and argument the whole way through: "it's not normal to have a pandemic" vs "the pandemic is the new normal." I remember overhearing a couple in the park in April 2020, one saying to the other, "when things are back to normal," and just thinking, yeah, fat chance. Norms are adopted for your survival, not the other way around.
With school the defined norms are actually quite strict and take up a huge part of the day. Most of the assignments are related in some way to meeting a norm, and this makes a certain kind of person - someone sufficiently willing to compete within and master the unspoken rules, or anxious enough not to fail - a "star student". But take the class outside of the classroom, and it's totally different.
From my perspective, it's not norms themselves being bad, just the particular set we've been herding along institutionally. We need some, some of the time, because the alternative is everyone being a philosopher, which besides creating unworkable arrangements, many people don't have it in them to try.
With schooling/education, I suspect there's a potential trap for any society where the "norms", should they entail sufficiently large institutions, as they do now in the West, can become excessively "sticky" and inertial as they, and, pivotally, their institutions, become monopolistically attached to prestige.
Once this occurs, it may be too difficult to reason about alternatives because competing within the unspoken game, as you put it, is too dominant a social factor and becomes the social air that we're all breathing. The educational institution and culture arguably becomes (and now the unworkable armchair philosopher comes out) a meta-norm or "Grundnorm"[1] from which others such as what makes a worth-while life, who has "prestige" and power (both real and earning) etc.
For a few reasons, I think allowing this to go too far is undesirable for a society. And with the West having transitioned in modernity to this idea that nearly everyone should be getting a tertiary education of some sort I do wonder if some major scaling issues and negative secondary effects come into play.
From my own peek into the workings of academia, one of the more disheartening things I observed was how Professors, the most successful researchers we have, were increasingly being burdened with duties not too dissimilar from those of high school teachers. Seemed to me to be a mismatch between the scale of the research and education sectors. And, I suspect, we're yet to see the knock-on effects.
Hard to say about the efficiency -- what if it just enabled you to do that memorization? Then it isn't inefficient, it is just the work required to get the task done.
Just a personal experience, but the enforcement of this practise at my school had a real detrimental effect on my learning.
I have very bad vision which wasn't picked up for various reasons, so learning to write did not go smoothly (my handwriting is and always has been barely legible). So being forced to write out everything I learned by hand was a real bottleneck for me.
Things went a lot better after I discovered touch typing, and nowadays I learn best via keyboard shortcuts and typed commands.
I have studied giving students typing practice, even wrote a paper on it[1]. Long story short, yes they are beneficial to most students. However, your situation is something I noted in my closing remarks - specifically that not all lower level practice will be beneficial to all students. In that case, to get the same learning experience from the activity, you should have been given a comparable lower level practice activity, like fill in the blank, Parsons Problems, or something else completely.
The idea of "writing it out" is to give the student familiarity without higher level "figuring it out on their own". I also teach martial arts, and so copying solution steps, or drilling, involves showing students how to do a technique and then having them do it. Both worlds are attempting to build muscle memory so if the instructor says to "do a tenkan" or "write a for loop", you have enough exposure to that concept to do it without thought.
The only major downside I see to this is preserving and quick retrieval of knowledge. It's a well known fact that many luminaries over the course of several centuries had systems similar to a Zettlekasten, most notably, Niklas Luhmann, the prolific sociologist apparently had several thousand index cards preserving info.
While it's commendable, it should also be noted that all this was done prior to the information age we're living in. These are physical objects that at the end of the day take up sizeable space in your house, gather dust, need cleaning/dusting etc. And you can't throw a computer at the problem of search and retrieval, at least yet.
I wish there was more research into active learning by typing as well. Typing does force you to think, especially if you don't type something you learn verbatim but actively try to process info and possibly summarize it in your own words.
> I wish there was more research into active learning by typing as well.
Speak of the devil and he shall appear! This is a link to my research of "active learning by typing"! The jist of my results were - students performed better in the course, initially lower performing students benefit from the practice and by sampling having more examples to refer to, regular completers of typing practice submitted less buggy code to GitHub.
I've studied math books this way. Never a whole book, but certainly sections or chapters here and there. I found it gave me a much improved appreciation for the high level structure of the material (or at least of it's presentation).
I wonder if writing by hand is the optimal output speed we can have, given our biological bodies. The information throughput (both in and out) are quite limited, we can only afford so much bandwidth. Anything beyond that and we lose some information. Writing by hand seems to be the optimal bandwidth that engages self-reflection simultaneously. Typing is good too, but I often find myself "think more" about what I'm writing if I do it by hand vs. if I type it.
I would imagine the bottleneck for processing information has more to due with the density of the information content rather than the speed of sensing.
at University I would copy all the notes I took, which were poorly written and confused, into nice hand-written notes with charts, plots and formulas. That was for all the courses, for 7 years.
I think it's a powerful way to learn and remember, but requires some time and effort. Alas, I am not doing it very much nowadays, but that's also because my learning is much less structured and much more "operational" (I need to learn this in order to do that).
Interesting. I used this method mainly when cramming for lectures I didn't care much about. Copy the material by hand a day or two before the exam, pass.
For the interesting stuff where I was paying attention and did the coursework in time I didn't need to.
"These are... [the] originals?? The showed no corrections of any kind. Not one. It's simply written down music that was already finished in his head. As if he were *taking dictation*."
I wrote C code by hand. Mind you, I wasn't typing it nor copying it from anywhere. I was writing it, by hand, from my head. Hear me out...
I was on the collegiate ACM programming team with Russ Cox and Matt Caywood. The format was you were given like 6 puzzles, 4 hours, and one terminal. Russ insisted we follow this process:
1) One person picks the easiest and gets the terminal.
2) Other two teammates choose two other puzzles and each of us would write our code, by hand, pen and paper. ("The New York Times crossword puzzle is a form of meditation. But doing it in pen if pencil is available is grandstanding." - Glenn O'Brien, from _How to be a Man_)
3) When one was finished, we would acquire the lock on the terminal from the other teammate, transcribe our code, and hit run with our fingers crossed. No collaboration. No cross-talk.
4) Repeat until victory is achieved and your competitors are decimated. (We laughed with no small amount of schadenfreude that the MIT team lost in the regionals because they all stood around their sole terminal, imcomprehensibly bickering at each other.)
We got 8th place in the world, so---uhm---thank you Russ for the joy and pain? I wouldn't recommand to any CS student that this slow form of torture is necessary by any means to assert that you are damn fine coder. But, GOD DAMN you feel like a beast when you type C into a terminal, run compile, and it works with no bugs.
You get a special satisfaction of bravura and elan that would reduce Salieri to tears reading the scripts of Mozart and realizing there were zero hand-writing errors. https://www.youtube.com/watch?v=RfJ0QyiI68c (This is historically apocryphal, but makes for great screenwriting.)
Through this rigor, you learn something very valuable, albeit something you might learn in more conventional and less tedious ways: You learn always---at least in those fleeting moments---to write correct code at the optimal level of conciseness. Because you simply don't have the time to fiddle around the way you might if you have a fast wpm typing rate, since you write slower than you think. And learn to think faster than you can write.
I guess this is similar to the form of discipline you would get when you programmed punch cards and sent them via post, except under time pressure it was blitz chess than punch-card postal chess.
And don't get me wrong. I use Jupyter notebooks and putz around a lot these days. I'm too old for some shit, and putzing around is fun.
I did this also. C and C++. Although none of my teammates are now Known Names, as far as I know, and also we did sometimes collaborate a little bit. But not at the "terminal" (not really a terminal), because in the labs used for keying in the code, the A/C couldn't handle more than one contestant per computer, so the contest organizers required you to hold a mutex to the room, not just to the computer.
I believe it was 3 hours, and in my day it was 10 puzzles. I still have my crib sheets, and the printed-out problems sets, which were printed on brightly-coloured sheet of paper. I agree that, once you apply yourself to this, it is not that difficult to learn to code more proficiently on paper than you would with an IDE, in less time. That's not to say we didn't have bugs in our algorithm designs.
Although we did not get eighth place in our region, much less the world, I too think it was pivotal in my development as a programmer.
Speaking of writing out code - in the eighties making games professionally on ye olde 6502 systems - we would go to the local coffee shop with paper notebooks in hand and scrawl out code - Go back to the office and then type it in and press run. A lot of code was hand-written 'offline' back then. I wonder what effect that had on our brains.
This is obviously brain-structure dependent. I for one would take 10x more time to learn python if some teacher decided that the best way to do it is to write out python by hand on paper for instance. (But this probably does work well for some brains, maybe more towards 'word rotatators'.)
You want a combination of learning activities. So, you wouldn't give students ONLY typing practice for the top. Rather, typing practice, some low level multiple choice/code tracing activities, and traditional programming tasks. While it may seem like more work, those low-level practices reduce overall study time when compared to if the activities were all traditional coding.
I also have typing exercises as a practice for students in my courses and part of my research. While I haven't formally analyzed how much time they take, I can say that students take roughly 7-10 minutes to complete 30-50 line typing practice.
Self-absorbed yes, and could be condensed into a few paragraphs - it was clearly written stream of consciousness and not edited. Nevertheless, I was able to sponge up the core idea.
I second this. When you have to actively pay close attention to the proportions and stroke order, distinguishing other characters at a glance becomes much easier.
Though with that said, I also rote memorized a lot of chemical structures, pathways and several rows of properties of the period table. It’s essentially the same as learning a language. You need a certain “vocabulary” before you can reach a higher order of reasoning.
But I would also be careful with suggesting “copy everything verbatim to learn it”, since it helps you to mostly memorize it, but not necessarily to reason about it. Memorizing the first 5000 words of a language is useful. Memorizing the dictionary is not.
I third this. For Chinese, the initial slog is unavoidable - you need to battle your way through thousands of rote strokes to build your foundational vocabulary.
Once you have a handle on it, though, it's like jet fuel for learning. You can synthesize things literally five times faster.
I let out a groan when I got to the word "Zettlekasten". Then a sigh of relief when I got to "private coaching of Neo-intellectuals" because I realized the article was obviously satirical.
This guy needs a copy editor. The meandering writing is off putting when someone is writing on the subject of becoming efficient at a task or process. He is in love with his own writing, maybe considering it entertainment, which he proclaims not to do.
I only found that at the start but I was able to scan (Fast Forward) to the point.
He also did say - which you may have scanned past -
"I aim to have these emails be more along the lines of "infotainment." Meaning, they aim to contain brief bits of information while also being entertaining."
When learning to program, I found typing out code samples had a similar effect to writing them out by hand, and felt more natural to the domain. But in other subjects in school, it felt like my comprehension was lower when I typed lecture notes instead of handwriting them. I was never sure whether that was something inherent to hand writing vs typing or it was just because I had more of a barrier between typing and thinking than I did with handwriting.
It always seemed clear that the value of writing something verbatim was that it forced your brain to internalize it in a way you don’t need to when reading. It takes longer to hand write than to type — maybe that is the value of hand writing vs typing? Or when typing I needed to think more about the act of typing, whether that was thinking about formatting/positioning or just being less natural than handwriting? Maybe programming is just a special case where you always think through typing?
I lean towards thinking it’s somewhat domain specific. Clearly with physics or math if you had to typeset equations with LaTeX that would get in the way, and typing would be a barrier to understanding. With programming, every idea is expressed through typing, and so typing is a natural way to imprint ideas on your brain.