elkorn >>=

Reading Academic Materials

Reading research papers and articles oftentimes tends to be quite a daunting task.

I, for one, tend to have difficulties understanding the argument of a paper. Additionally, it often took me a lot of time to make sense of its contents.

The main issues seem to be:

  • I cannot grasp the process that lead to the presented conclusions,
  • Parts that appear crucial to get the whole picture seem to be missing from the reasoning.

Even if you understand the contents, there is the matter of effort to reward ratio. If you are well versed in the field to which the paper belongs or are some kind of prodigy, you most likely don’t have any issues. Any normal person, though, will likely finish reading a paper with a ton of questions nesting in their head. There might be confusion and little to no answers.

Re-reading the paper may help. But unless it is especially long or dense, there is a good chance that you are going to spend more time and effort on it than required.

Long story short - “just reading” scientific papers is not the most efficient way to get the most of them.

There is some information on that topic in various parts of the net. You need to do quite a bit of link crawling to follow those sources though. I decided that it would be beneficial to stitch together the more or less related materials into a more comprehensive post. There are also some pointers based on my own experience.

Start!

When starting out with paper reading, be aware that authors expect you to have background knowledge. The extent of knowledge deemed adequate can vary drastically from paper to paper.

To assess whether your level of knowledge in the subject area is enough, read the title and the abstract and review what do you know about it. Do this before going any further. You may find yourself missing elements treated as given by the authors. If so, find a review or a textbook and gain the required background. If you find out that you do have enough knowledge to carry on, this type of analysis is beneficial as well. Its value lies in becoming aware of the ways in which upcoming topics may fit in with your current information.

When approaching a paper, make a preliminary list of subtopics you expect or would like to find in your reading. These will give you “a-ha!” moments and help labelling notes.

Peter Klein’s method - the art of skimming

Professor Peter Klein has published this method as a handout to his students in 2010. I find it extremely helpful in dealing with works that leave me stumped otherwise. The original publishing can be found here.

The method consists of two stages. Firstly, the reader needs to focus on “embracing” the paper mentally. Secondly, to cement the understanding of the matter and argument, an individual critique of it should be developed.

Step 1: Skimming

  1. Read the abstract if provided.
  2. Read the introduction.
  3. Read the conclusion.
  4. Skim the middle
    • look at elements that stand out the most: section titles, tables, figures etc.
    • the idea here is to get a feel for the style and flow of the paper
      • is it methodological, empirical, tehoretical, conceptual or something else?
      • is it a theoretical contribution, an empirical application of a theory, a critique or something else?
  5. Read the whole paper quickly.
    • skip the equations, figures, tables, code listings etc.
  6. Go back again and carefully read the whole thing, focusing on the most “meaty” parts.

Step 2: Analysis through critique

  1. Ask if the reasoning makes sense.
    • Is it internally consistent?
    • Is it substantiated by arguments and evidence? What is the quality of those?
    • Why are the conclusions important?
    • Mr Klein states here that this skill takes some time to develop. It’s so true that it almost hurts. :) I’ve had some success using the following means.
      • Try literally talking yourself through the critique process.
      • Write down questions, then read and answer them aloud.
  2. Compare the article to others you have read on the same subject or a related one.
    • If this is your first, find some other ones and skim them, focusing on introductions and conclusions.
    • See how the arguments match up between the papers. Are they consistent or contradictory? Maybe they aren’t even related in any way?
    • See Google Scholar, a citation index, science publisher web pages, discussion groups, blogs etc. Look for places where they quote the article or express opinions about it. What are they saying?
    • Check out a reference work. Something from a well-established journal, handbook or encyclopedia should suffice. It will you perspective on how this article fits in the broader context of its subject area.

Note-taking

(At first, I named this part “supplementary techniques”. But after reading it again I noticed that it’s all about taking notes so… ;) )

I can say from my own experience that note-taking is a crucial part of intentionally reading an academic paper. This rather applies to cases when you have difficulties understanding the presented concepts. If you don’t have any issues with that and just want to focus on optimizing the effort to reward ratio, you may want to skip some of this advice.

Maximize the usefulness of your notes. Taking notes may seem pretty obvious, but it’s important that they be useful enough to serve as a reliable anchor. You need to be able to get back to them after some time and bring the topic back into your head. Two aspects are of utmost importance here:

  1. Leave a lot of space in the notes for your comments. Write down questions, second thoughts, cross-references and so on.
  2. Make it a priority to express the encountered ideas in your own words. This is one of the more efficient ways of internalizing the paper’s contents.
    • Just paraphrasing word by word is a waste of time. By doing only that, you are not exercising your understanding of the subject and thus not getting familiar with it.
    • A good way to do this is to write the gists of the most important ideas down as headings. Then, add a few sentences to explain them the best you can and provide examples.
    • Do not rely on underlining and highlighting! This is a context-less method of attracting attention, which is not what you are aiming for at this point.

When encountering temrs and statements you don’t understand, include them in the notes. Also, append to them your doubts as questions. The questions can be trivial (e.g. what is this?) or more involved (e.g. How would changing stated assumptions affect this or other proofs?). If you have other thoughts or when you find answers to your questions, write the down there as well. It is beneficial to keep a “reading journal” that you can consult anytime in the future.

Colour-code the non-trivial elements of the paper serving as a basis for bigger conclusions. Non-trivial elements are those that warrant explanation (they don’t fit in your head at the time) or those you have to look up. Having a consistent way of colour coding concepts in your notes and makes the flow of the argument more visible. You can even go as far as drawing a “timeline” (more like a git branch graph) of the concepts and how they fit into main conclusions.

To have an even clearer view of the structure, put notes on separate cards. The main point of this practice is to keep your notes as focused as possible. This may supersede the idea of colour-coding with its usefulness. Having the physical ability to move the cards around and group them enables you to see things from another perspective. You may discover connections you haven’t noticed before as well as see how the argument flow may proceed.

Take advantage of the spacing effect. Put the paper and your notes down for some time, even if you don’t understand everything from the get-go. Read the notes again after a few days, following up on the things you forgot during that time. The fresh perspective oftentimes lead s to a breakthrough. You may also have a stroke of understanding under the shower (or doing any other relaxing activity) in the meantime.

I try to maintain a habit of taking my notes by hand and then typing them into a computer (vimwiki or org-mode). This makes me go over them more honestly and tends to further cement the knowledge that the material bears.

Some words of consolation

Don’t beat yourself up too much if you can’t seem to get through a paper and understand it as well as you would like. There is a strong chance that it is just written in a poor way. Think about it - the majority of scientists are not Hemingways. Most of them probably does not enjory writing at all and is happy to get it over with and move on to more interesting activities.

Also, think about moments when you tried to explain an involved topic that was natural to you to somebody who didn’t have a clue about it. You may have spent a significant amount of time and effort on it or it just happens to fit your brain like a glove. Did you have any clarity issues while explaining? Or maybe “too much detail” or sidetracking issues? In the state of mind of having internalized a difficult concept, it tends to be difficult to take a step back and look at it in an objective way. Even more so trying to assume the point of view of a less experienced reader. In academic papers, this leads most often to the following issues:

  • Leaving out logical connections. These may seem of little value to the author, but are often peritent to the reader. They usually bridge the gap between obscurity and clarity.
  • Lack of a clear literary backbone. Less important details are often given too much attention. This upsets the balance between vital and ancillary information that the reader should receive. This leads, of course, to confusion. The reader is more prone to get sidetracked and lose the main point of the argument by following those digressions.
  • Long chain of back-referrals. The core of this issue seems like the one stemming from deep submersion in the topic. The paper treats the foundations constituting its basis as common knowledge. Often, the only explanation is a referral back to source literature. All is well if the source literature is the originator of those ideas (i.e. introductory work). Unfortunately, following back-referrals tends to unravel a chain of questions due to more back-referrals. Back-referrals are usually a necessity due to space limitations e.g. when a paper is to be presented at a conference. In such cases, the healthiest approach is to just get out your shovel and start digging. Just try to look for comprehensive source material.

Resources