Jeff McAllister: Welcome. I'd like to thank you for attending this NIDA
International Program Webinar, "Reading a Scientific Paper: Tips for Cross-
Disciplinary Collaboration". Our presenter today is Mr. Barrett Whitener. For more than 15 years Mr. Whitener has
trained professionals in science, healthcare, and public policy to become effective
communicators. He has led workshops on presentation skills
for postdoctoral fellows, senior researchers, and administrators throughout the National
Institutes of Health; the National Aeronautics and Space
Administration; Cornell University; Gates Millennium Scholars; Compact for
Faculty Diversity; Center for Strategic and International Studies; World Resources
Institute; and many others. He is also coauthor of the book, Speaking
About Science: A Manual for Creating Clear
Presentations. It is my pleasure now to turn it over to our
presenter, Mr. Barrett Whitener. Barrett Whitener: Thank you, Jeff. And welcome everybody. It's great to have you join us today or tonight,
depending on where you are in the world, to learn a little bit about some tips for reading
scientific papers. And the aspect of -- or the rationale -- for
reading scientific papers I want to focus on in this webinar is the role it can play in
helping you with finding collaborators or potential collaborators for your research. I'm sure you don't need help in how to read a
paper, but when you think about reading a
paper with that particular goal in mind, then there
may be some specific steps that I hope you'll find helpful that we'll discuss
today. That's because obviously you're going to often
need to collaborate with other researchers who are outside your specialty, and yet I think it
can be challenging sometimes to decide on who the most promising collaborator is
going to be -- even when you're reading their
studies. So I think if you know some signs to look for in
the published study that you're reading, it can increase the odds of a fruitful
collaboration and one that you are glad you embarked on. So today we're going to examine some of the
most important signs I think you can look at in each part of a study as you're thinking, "Is
this author, lead author, perhaps somebody I'm going to want to think
about collaborating with in the future?" Now, of course what we're going to talk about
today will also help you in writing your papers more effectively, too. And in fact, many of you I'm sure attended last
week's webinar on writing a scientific paper. That's going to be posted on the NIDA IP
website in a couple of weeks. And this webinar will be posted as well. But if you didn't attend the Writing a Scientific
Paper webinar, don't worry because I'm not going to be
referring back to that except once. You don't need to have seen that webinar to get
anything on today's webinar. So the difference about today's is we're taking
the perspective of the reader, of course, as opposed to the writer. But we're also going to go through a paper
section by section and check for key elements for you to look for in each one. Now, a word or two about myself. I'm not a scientist but I am a communications
expert who has worked in the scientific field for about 20 years. Jeff mentioned some of the places that I've
worked and I've helped folks design both
presentations, posters, and also helped with
paper writing for scientists in a variety of fields and basically
how to communicate more effectively in all of those
formats. So let's start with what we think clear writing
means. It means many things, some we discussed
last week but today I want to focus on the idea that clear scientific writing is a great mirror for
clear scientific thinking. And that's to say that good scientific writing
doesn't only describe what happened
accurately and truthfully, it also describes it clearly, that is
at every point the reader -- that's you in this case, for today's webinar the
reader is you -- you want to know what's being described. You want to know how what's being described
fits into the larger story of the writer's project. So when you read a paper, if you repeatedly
find yourself wondering exactly what the writer
means when they describe, say, the goal of the study,
the method, a finding, or what a chart or figure is showing and why, you may begin to
wonder if the writer's scientific thinking is very clear,
too. And that makes sense, I think, because the
words on the page or the screen are really all
we have to go by in those situations; that's all we have
to base our judgment on. But if you do find yourself wondering repeatedly
what's being described in the paper or its significance and the significance is a very
important part, the why factor -- why something is being described, why it plays
this part in the experimentation or the story being told in the paper -- then I
would suggest that you read other papers by the same author and see if it's a recurrent
problem in a number of papers, even in different journals. Because then you'll want to note especially
who the lead author is since he or she can be considered primarily
responsible for the overall continuity and clarity of the paper. And if you see that name associated with
several papers where the reasoning is very
clear, the writing is very clear, then terrific -- that might
be someone you want to consider pursuing as a possible collaborator or at least speaking
with. So today, again, we're going to examine each
element and section of the study for clues that you can search for, clues for the clarity of
the writer's thoughts and the process. So let's start right now with the title. Some things that I think we as readers want to
ask ourselves when we look at the title of the paper are: Is the title related to the
research question of the study? Sometimes it is, sometimes oddly enough it
isn't. It seems to address what in the paper at least
looks like a secondary issue or sometimes even just the order of the title --
what's in the main title, which is the subtitle -- can be a little different in
the priorities it reflects compared to the research question or to material in the
abstract. So look at that. You also want to see if the title reveals what's
novel, what's new about what the research is
uncovering. And of course, you want to make sure that the
title matches the findings of the paper. Now, something I didn't put on the slide but
also applies to the title, I think title should also identify the study
population, people versus rodents for
example. Or say if it's a unique population such as a
specific age group or a group from a particular geographic area. And the title might also mention any unique
measurement methods that the study used -- doesn't have to do all those things obviously. But if any of that material is relevant to the
paper, the title might be a good place to suggest it. Those of you who heard my webinars last year
about presentations may remember that I think that the purpose of a paper title is very different from the purpose of a title for a talk. So in a paper title there's room to get in a fair
amount of this information. And in any event, you want to make sure that
what is featured as is being most important by virtue of being in the title is what also
seems to be most important in the actual
paper. Moving on, let's look at the abstracts of what
are some clues you want to look for here. First of all, you want to make sure that the
abstract represents each section of the paper, from methods through results and discussion, that includes the key finding, example for
example. You also want to make sure that the research
question in the abstract is clear, reasonable, and
focused. And by "reasonable" I mean is it something
that can be answered in the amount of space that the paper takes? Or does it seem to be, for example, a research
question that would take several papers to answer? Or a book even? You also want to make sure that the
conclusions follow from the research question. So let's look at an example of some material
from an abstract and see how well we think it -- this example -- demonstrates these qualities
in action. Because I think it can be a little trickier than it
might as first seem to make sure that an abstract is actually doing all of these
things. So this is one paragraph from an abstract. Read that one, please. And that obviously is the main research
question from the abstract. Now I'm going to put up a second paragraph
that's the main result from the same abstract. So look at those together and see if you think
the second paragraph follows clearly from the first paragraph. And you may be coming to the conclusion that,
well, it does to an extent at least, but maybe it could be a little bit clearer that the result is actually addressing the
research question. It's clearly addressing the general subjects of
the research question. But whether it's addressing the specific
research question that's just been laid out in the abstract in the first paragraph might
not be as clear. And I think that may simply be because some
terminology is different in the two paragraphs. You'll see the ones I've highlighted in red here
in the first and second paragraph actually refer to the same group but they use different
verbiage to describe it. Same with the green highlights in the first and
second paragraph. So it's simply a matter, I think, in this writer's case they could simply use the
same exact terminology as in this second paragraph as they use in the
first paragraph. And then it becomes absolutely clear that the findings follow directly from the
research question. Now, if you see something that looks like the
previous not ideal example like this one, it's not necessarily a danger sign. And I want to make clear that really the
absence of anything I suggest you look for
today in a paper doesn't mean that you dismiss
immediately the possibility of working with that person of course. What it may indicate is simply a wording
choice. But if you, again, see this happening in
multiple papers by the same lead author or if you see several of the issues, several of
the clues we're going to talk about today that aren't present in several papers by the
same author, that might be something where you'd want to, you know, think about
different questions you might ask that person than you would somebody who seems to have
everything in their paper that makes it clear and helpful for the reader and it shows good
scientific logic. Whereas you see, the more you see like this
one where the match is equivalent on both
sides, I think, more confident you can feel that the
clear scientific thinking is there as well as the clear scientific writing. So let's move onto the introduction. The introduction, I think, is good to review for
four key elements. I think just about every introduction ought to
have all of these four. It should give the background -- that's obviously the focus on the paper's
unique contribution. So before we move forward with the others,
let's look at one version of a background
statement. Now, I've colored different phrases and words
in this statement to indicate that it covers many things. In fact, I think it covers too many things for a
single paper. It includes a disease; a risk factor; a therapy; a
technique; patient population. And the reason this may be a red flag for you is
that it's very unlikely, I think, that all of these elements are going to be
reflected in the research -- certainly not -- I mean, if they're all present in the research,
that's fine, but they don't all likely represent the
contribution of the research. So what you want to make sure is that it's clear
from the writing. And this, again, is in the introduction section,
that you know exactly what the main player as it were is in the paper, what's the main
factor that's going to be examined, the main entity and what's secondary. So this writer -- I'm sure with the best of
intentions -- put all this in, indicating everything's that's
going to be addressed in the or maybe the majority of things going to be
addressed in the work itself. But that doesn't necessarily all belong in the
background section. The background section is a place, again, to
identify the unique contribution of the work. So an alternate way of doing this -- there would
be many others, of course -- but one approach here would be to focus on
the technique of MRI. I'd say if that's the focus of the paper, then
that's what should get the highlighting as it does in the second paragraph, the
second sentence. So going back to our list for background, it
should also include -- rather for the introduction in addition to
background -- it should also include previous research. Now, this should follow very logically from the
background because in that situation the writer of the paper is moving from more general to
specific information. Problems with previous research should also
be addressed in the introduction. This section should say or this part should say,
"My previous studies needed to be improved
on." Now, it doesn't have to be that those studies
were wrong; maybe they have simply found conflicting
results. But this is where the author should point out
"Here's why this study was needed. Here was the gap in our knowledge," or
"Here's what's broken or fuzzy about the
previous state of knowledge," even if it's just the previous
papers had come to conflicting conclusions. And then the introduction should end with what
the current study did to fix the problems. I think at the end of the introduction you should
almost always see the major improvements that the research in this paper you're looking at
makes on past results. And I think it also probably should reveal that
the study is a sequel to a prior study from the author's work group, that is, if it
directly follows from a previous study. And if it is a sequel, then it should say how it
expands on that work. For example, is it reanalyzing previously
reported data? Or did the new study enroll new or additional
subjects? Or did it develop a new methodology? Any of those could be addressed there. So all those things, I think -- those four
elements -- you're going to want to look for in an
introduction. If one of them's missing, again, that might go
kind of on your red flag checklist. So let's move onto the next section of the
paper, which is methods. The method section, I think, should have
enough detail that another investigator could replicate the
research. So that you reading the paper could simply
take the information presented in the method
section about the methods used and conduct the
experiments yourself. That's, I think, a very useful litmus test to apply
when thinking, "Is this paper telling me enough about the
method?" It should also include or describe the
inclusion/exclusion criteria for subjects if that's appropriate, such as whether the
subjects are representative of others with similar conditions. The methods section should definitely
describe the sample size, the timing of the
intervention. And something I did not put on the slide is it
should provide primary and secondary outcome measures or the
primary statistical analyses conducted, etc. So
again, the takeaway here for me is that you want to
see all that represented in sufficient detail that anyone -- any other scientist -- could replicate the process that's described in
the paper. Now, the result section should almost always
include descriptive results. And these of course are a description of the
sample such as demographic; characteristics or clinical characteristics; key predictor values, which is what the researchers ask the
subjects; or the outcome variable, which is what the researchers were looking to
see or not see. Those kinds of descriptive results should
definitely be the result section. Clearly you want a main finding. Just as in the research question we saw
earlier about MRI, you want to see one element being the clearly
predominant major element in the research question. You also want to see the same thing when it
comes to the findings. And ideally that main finding will directly
answer the main question in the research question, the main part of the
research question. You want to see that the results are consistent
and any additional results that may be inconsistent and an explanation of
why they're inconsistent. And then if the research deviates from the
study design, then you definitely want to look in the result section to make sure it explains
when and why the research deviates from the study
design. So those are four -- actually more than four, I
think -- good things to look for in the results section,
that you want to make sure are present. And again, if one of them's missing in one
paper by the same author, then that's not necessarily a big deal. But if you find a consistent absence of one or
more of these, that might be something to take notice of. Now in the discussion section, I think really the
focus of this section should be on interpreting results and then reviewing all
the possible implications of the results. You want to make sure that the interpretation
and the implications are both supported by the research that's presented in the paper. Now, let's look at a figure here and see how
this might play out in terms of an actual paper, in terms of the text used to describe a figure
and within the figure itself. So this figure obviously is a scatter plot. And it's showing the effect of individuals'
baseline intestinal permeability on their change in plasma zinc concentration. And it also examines the effect of the
supplementation group on this relationship. So on the x-axis you see we have the urinary
LM recovery ratio, that's lactose and mannitol -- synthetic sugar and a sugar alcohol. We add that to LM recovery ratio and baseline
with a larger LM indicating that there's more altered intestinal
permeability. Now, on the y-axis we have the change in
plasma zinc concentration from baseline to post-supplementation. And children who received the zinc supplement
are represented in red and those who got the placebo are represented in blue. So look at this for a moment. And I'm going to read you one finding from the
accompanying text to this figure. And see if this appears in the figure, the
information described in this finding. The finding is "Supplementation resulted in a significant increase in plasma zinc
concentration." Now, does that appear in the figure? And you can see most definitely it does. As the change in concentration increases -- as
the concentration increases, so does the
amount. As we see here in the legend of zinc
absorption increased as well. Now, another finding described in the text that
accompanies this figure is "There was decreased absorption of either
exogenous or endogenous zinc." Now, does this finding appear in the figure? And of course the answer is no, it does not. So in this case the author had a great
matching with the text and figure in the case of one finding but not in the case of another. And I'm a broken record here, but I'll just say it
one more time: If this appears in one paper, not necessarily to be concerned about, though
it does get your attention; but if it appears in several papers, then that
might be something to really take note of. So the discussion section should also
describe study limitations, both of design and interpretation. And it finally should conclude with summary
thoughts. And you want to make sure the summary
thoughts, again, follow directly and clearly from the findings, that everything said in the
summary is supported within the paper itself. Now, one thing I suggest to you do is to take
the criteria I've gone over so far, plus the criteria from last week's webinar on
writing the scientific paper and as you read a paper, apply the criteria and
grade each section of the paper, grading for clarity and continuity. And I discussed those characteristics in some
depth in the Writing a Scientific Paper webinar. And the reason I suggest that you consider
grading the paper using either these criteria or any other that you think are most appropriate
is that it will help you -- especially when you're looking at several
papers from the same author, lead author -- determine whether the issues that you may be
identifying, the problems you may be
identifying with the paper are stylistic ones, that is, it's just
the way that this author seems to present their research in paper after paper
or their discussion section or their abstract. Or if it's really substantive, that is, if it's referring
to more than just style, but actually to a problem in the content that
maybe the, say, that since clear scientific writing mirrors clear
scientific thinking. If you see these kinds of problems with the
writing occurring again and again, then maybe it's an issue more of substance, in
which case you want to look very carefully at that person as a potential collaborator. Now, moving on from the text to figures in the
paper. I think figures ideally -- obviously they clarify
methods, they present evidence. It's the main function in the paper. And you want to make sure that the figures you
see in the papers that you read are used for overall effects, not for details about the
research because that's better handled either in text form or sometimes in a table, but rarely
in figures. They're great for illustrating overall effects,
though. Figures can also be used to clarify methods or
to present research for evidence that supports the results. And let's look at this example and see with
those criteria in mind, specifically what figures being used for overall
effects, not details, what you think about this figure. I think this one shows a little too much
information for describing one point in the research in the
paper. For example, it shows the interaction of
arginase and IL-10 in three different study groups, plus the role of
cytokines, plus results for the protein MCT1 and MCT1's interaction with CCR2, plus -- and
so much more. I'm just scratching the surface of everything
that this shows. If you can't understand the purpose of a figure,
check the rest of the paper to see how the figure relates to the text,
specifically if the figure shows things that aren't described or things that are
secondary to the main point. Now, I think this figure does that. And just for the record, what was most
significant in the paper at this point was the insulin resistance factor. So you can see how there's a lot more in the
figure at this point than just the insulin resistance factor. And so, so much information given at one point
is not only not really helpful to the reader, it can be, you know, actively confusing or it can
actually impede the reader's or diminish -- take away from -- the reader's understanding
of what's going on at this point in the research. I'll show you another figure. And this one represents the idea that elevated
glucose metabolism through the GLUT1 glucose transporter drives
inflammation and the deletion of GLUT1 blunts
inflammation. Now, I think this one is a clearer graphic than
the previous one because it shows just the essentials. Obviously, neither this nor the previous
example are what the process actually looks
like. And so the designer of this figure clearly took
that and -- that idea -- and really used it as kind of one of the guiding
principles for the figure by pairing it down to the essentials: Only the information
that's needed to represent the contrast in the elevated glucose metabolism and the
deletion versus the deletion of GLUT1. So less is more when it comes to figures. I think that's just a good overall criterion to
apply as you're looking at them. And again, it's not necessarily a huge problem,
but if you see a lot of figures in a paper that look like this one or across several
papers, then it may be that you want to make sure that you double check the rest of
the paper to make sure that the scientific thinking is very clear in the
paper as well; whereas the more figures you see like this
one, I think the more confident you can be -- assuming all else in the paper is good, of
course and the science is solid and accurate -- that may be a good sign. I'm going to show a figure checklist that you
might apply -- oh actually, sorry, I'm getting ahead of myself. We want to first of all, before we get to the
checklist, mention that the appropriate use of flow diagrams and line graphs is also
something that figures can do. And by "appropriate" I mean is the figure
selected to carry out its appropriate function? I'm going to focus on the example of flow
diagrams. And they're useful for explaining complicated
sampling schemes, algorithms, protocols, because they break that down -- as figures
often do -- and make it easy to scan a lot of information
quickly. So here's an example of a flowchart that pretty
efficiently helps you scan the entire process and to compare the two groups, those who
receive the zinc supplement versus those who receive the placebo supplement, and the
numbers in each of those groups. It would take a lot of text to supply this
information. And of course there's going to be some text
that describes this as well. But the use of a flowchart in this kind of
situation I think helps the reader to kind of fix the comparison and the process as a
whole in their mind. So when you see flow diagrams used for this
purpose, that's a good sign. So now let's move onto our figure checklist. These are some criteria I think you can use as
you look at any figure to determine how effectively it's doing what a
figure really is meant to do. First of all, you want to see "Does each figure
in the paper make a clear point?" That is just like the earlier one about the
GLUT1 versus -- deletion of GLUT1 versus presence of GLUT1. It makes a very clear point about the relative
impact of each of those. Are there labels on the figure for axes, lines,
bars, and points? May seem like an obvious thing, but
sometimes they're not always there -- or all of them at least aren't there. You want to make sure the scales are correct
on the figure. That's because, you know, some measures
are not symmetric in linear scales, such as the confidence interval for an odds
ratio. You know, a 95% confidence interval, for
example, for an odds ratio of two could be anywhere
between 1.1 and 3.6. So make sure that the scales are not only
correct but consistent through the figures and that if they aren't consistent, that the writer
is explaining why they aren't consistent and taking that inconsistency into account in
the interpretation of the data. You want to make sure there's a legend for
every figure and that the figure is numbered for easy
reference. And perhaps most importantly, you want to
make sure that the figure is complemented by the text. In other words, do the figure and the text say
the same thing? Remember back to our scatter plot figure a
minute ago, it illustrated one finding in the text but not the
other. And it can be confusing, as we all know, when
pulling together a paper to make sure everything lines up that way when there are a hundred other things to do
with the paper. But by the time a paper reaches final
publication, you want to see that kind of complementarity, that kind of match between
the text and the figure. So enough for figures for the moment. Let's move onto tables. Tables serve a different function from figures. I think tables mainly give essential information
in a meaningful format. What I mean by that is that they allow you, the
reader, to make a side-by-side comparison of key
facts. So I think in these information should be
ordered in I'll call it a meaningful way. And that means a way that's essential to the
point the paper is making at that stage where the table appears. And also, you know, "meaningful" means "not
random." So for example, in this table of common urinary
tract pathogens, these pathogens are listed alphabetically. Now, this order doesn't really make a lot of
sense because the alphabetic order doesn't really
have anything to do with the way the pathogens are discussed in
the paper, obviously not with the way that they're
behaving. So this is an example of a table that's I'd say
organized in a nonmeaningful way; whereas in this example -- redone from the
same paper -- this version is much better because it does list
the pathogens in a meaningful way, in this case by frequency. And it also subcategorizes them by type, which
presumably is relevant for the actual research being discussed. It also gives subheadings and it gives a more
informative title than the previous version. So when you see a table that is meaningfully
organized in this way and provides information that is essential to the paper, to what's being
discussed at that moment in the text, then that's a very good sign. If you see that consistently, it's an excellent
sign that there's clear scientific thinking, as well as table writing going on. Now, let's say the situation is a little more
complex because in addition to tables giving essential information, they can
also be used to compare groups by emphasizing the relevant features in each
group. And I'm thinking here of the kinds of tables that
present basically two types of information: They present first the measurements for each
of the two groups that are being compared and they also present the differences between
those two groups. And when a table does that -- presents those
two types of information -- then the author should organize the table so
that it shows which type of information is more important. And I'll show you what I mean here with an
example. In this table, the characteristics of the 117
subjects enrolled in the vacuum away dust or VAD study by type of pulmonary disease. This table is characterizing -- or rather
comparing the characteristics, I should say -- of patients with asthma with those of chronic
pulmonary disease. The study, by the way, is about the effects of
intense vacuuming on subjects' carpets. Now, in this table the emphasis is on the
characteristics themselves: The age; forced expiratory volume; peak expiratory flow;
Prednisone dose. And kind of help us play out in asthma versus
pulmonary disease, chronic pulmonary
disease. But the problem here is that everybody already
knows that the two types of patients are very different. So the table could simply show that there are
statistically significant differences. But the fact that there are differences, which
seems to the point of this table, is really nothing surprising or new. But because of the way the table has been
organized, that information does seem to be what the authors want us to focus on as
being most important. And that seems highly unlikely. So a more meaningful way, I think, of
organizing this comparison in a table is shown in this version. Now, this table puts the emphasis on the
comparison so that you want to know if there's a statistically significant difference in
the size of that difference, I think you'll find that information here. It shows a measure of the effect size, it shows
an estimate of how precisely the effect size was measured. And the point again, really, is that not every
table should do this -- that's not the point -- but again, that the information is presented in
a meaningful way. So the author has really thought about "What
does the reader most need to know here about this stage of the process?" I presume, for example, that even with this
version of the table that the text would have been similar to what it
is for the improved table because -- we hope so at least -- because the research is
what it is. But in this design of the table, this version of
the table, the table matches more closely
what's in the text and presented in a meaningful way. Now, in a case control or a cohort study, a table
could also clarify key relationships. And this is a different kind of challenge for the
authors in a case control study or a cohort study when you're comparing or
they're comparing who developed the outcome and who didn't. So let's look at a table that is present in this
kind of study and see what we think about it. Look this one over for just a minute. Now, this table, I think, makes the reader work
harder than is really necessary. For example, it looks at first pass as if -- just
picking one at random -- as if, for example, as if 33% of the subjects
with a history of the diabetes developed strokes during the
study. But in fact, it's not the case at all; 33% of those
with strokes actually had diabetes. So not sure why this was designed this way,
but nonetheless it's seeming to imply the opposite of what's the case. Now, it's even harder, I think, to understand the
relationship between stroke and the previous number of myocardial
infarctions. And most importantly, maybe the table does
not show what the reader really wants to know
most, and that's whether the previous number -- or the number of previous myocardial infarction
affects the incidence of stroke. I think the revised version of this table does a
better job. The purpose of this table as it's laid out is to
show the incidence of stroke according to selected characteristics. And again, I think that's what's perhaps most
relevant for the work. So the authors, you'll notice, put the actual
numbers in parenthesis, and that's to keep you from being distracted from that overall
purpose, which is the incidence of stroke by selected characteristics and the
percentages in the different groups who either
had or did not have the characteristic that's being
presented. So the authors also characterized the age of
the subjects. And so again, this table's much easier to
follow in terms of getting that, okay, in those who were at or over 70, 12% had
strokes; those outside that age group, 6% had strokes, etc. Much easier to follow and were meaningfully organized than this
previous version is. So you want to make sure that a table's format
is determined by its purpose like this one and not the other way around. You don't want the kind of the table seeming to
distort the purpose of that point of the research itself. So you can review how well a researcher's
table show what's most relevant, I think, for the process of answering the research
question. Because again, that research question is
what's guiding -- we hope -- every figure, every
table, in every part of the text as well. So let's look at a table checklist. I think you can look at tables to make sure they
have a descriptive title that also is actually accurate, that is reflective
of what happens in the table itself. We're making sure the rows and columns are
lined up clearly so that, for example, you can tell categories from subcategories at a
glance. We want to make sure the rows have
denominators and that the columns have
units. You want to make sure that all data is shown
that is necessary for the story of the paper, that is, that the table shows everything that you
need to know to make sense of that point in the research as it's described in the text. If there's something, for example, a point of
comparison between two groups described in the text that isn't in the table, that might be
something to take note of. And you want to make sure that the table's
cited in the text so that, again, you're not kind of left to wander and to wonder
as you go through the paper about when you should be looking at a table or
a figure when you're reading the text and so forth. So let's look at a few other things to consider,
not just about tables and figures, but about the studies as a whole as you look at
them. You want to know if a study replicates other
studies. You want that definitely to be addressed in the
paper. If, for example, it's just simply trying to resolve
previously conflicting results found by studies, which ones is it replicating and
why? Does it break new ground? Is there something new in a study? Now, when you look at whether a study breaks
new ground, I don't mean that as an absolute. You want to look at the journal factor and at the
institutional factor. And by that I mean that while obviously, let's
say Drug and Alcohol Abuse -- that journal -- and Addiction, those are the top journals in the
field. But others may be just as good in their niche,
even if they're smaller journals. So by "breaking new ground" I don't mean
necessarily breaking, you know, headline-making ground but the breaking
ground for that journal and for that institution or for the specialty audience within the smaller
journals. It doesn't necessarily imply that there's lower
quality research, for example, if new ground isn't being broken in one of the
smaller journals. And also, I think you can when in doubt,
consult with your mentor and others on whether this paper that you're looking or
this lead author's papers in general have the same strengths and
weaknesses. Again, one incidence in isolation of a problem
isn't necessarily a red flag, but a pattern of the same kind of problems or
flaws in a paper may be something that you want to pay attention to. And this is all to say -- when we get down to
brass tacks -- that I think if you read a paper critically with
these criteria in mind that we discussed today or other criteria that you or your mentor or other
places you hear webinars, read books, other criteria they suggest are going to be very
helpful. And it's not that you need to know everything as
a result of reading critically, but if something's unclear, seek more
information either from a peer, a mentor, or another source to validate your thinking
about what's going on in a paper, or a series of papers by the same lead author,
and whether that is the kind of pattern that may be significant. Now, in the Writing a Scientific Paper webinar I
gave several resources. Today I'm just going to give one but it's a great
one: Jean-Luc Lebrun's Scientific Writing 2.0: A
Reader and Writer's Guide. And I'm putting this one in this webinar, of
course, because he is addressing not only writing a
paper, but also reading a paper and what to look for. And he goes into a lot more detail than I have
the time to do today on these issues and on many other issues, as well. That book is periodically updated and revised. And it got a major revision just last year. So that might be one you might want to
consider looking at. So now it's time for me to stop talking for a
minute and to listen to you. I would like to know your questions, both ones
you've typed in previously or would like to type in now. And, Jeff, if you could start reading some of the
first ones, I'd appreciate it -- if we've had any. I see one that says, "What do we mean by legend as a checklist for
figures?" And by "legend," -- let me back up and find one. I just mean the -- it's the list that shows you
what the different variables mean. Go back to our scatter plot -- and in the figure
itself. Let's see. Scatter plot was the one with the best legend. So this is what I mean by the legend. So you want to make sure that a figure of this
nature has that kind of legend so that you know in this case, for example, what the red squares
are indicating versus the blue squares. I'm going to advance now to the checklist in
case -- for the figures and tables -- in case anybody would like to write down
anything that maybe you didn't get before. Here's one for the figures. And, again, your questions are welcome. Jeff McAllister: Okay, we have another question
here: Where would be the best place to put the
objectives? Barrett Whitener: I think the objectives in the
paper itself, the place to put those, the place to look for them since the abstract,
again, is not necessarily going to have space for more than the main research question,
where you want to look for those in the paper is in the introduction. Because that's where you want to get not only
the background, but which is where the author should describe
why they're looking where they're looking, why the questions they're going to ask in this
research are meaningful. That's also a natural place, then, to move
directly into stating those questions. So there may be, in some cases, one primary
research question that has several different components, or there
may be two or three separate research
questions, but all have more or less the same weight. But they should probably appear in the
introduction. Thank you for the question. Jeff McAllister: We have another question here:
Is it better to have limitations at the end of your method section or, as you mentioned,
to be in the discussion section? Barrett Whitener: I think the answer to that
depends on how central the limitations are to the
research. For example, you wouldn't want to get or have
to wait to the end of the discussion section to learn that the method being used has been
shown to be faulty in previous research for this reason, that reason, etc. That's a
limitation you would want to see stated up front, perhaps followed by something like,
"But it's the best method we have because of this reason and that reason and
the other reason" so that the limitations that may be more significant are addressed
earlier in the paper. That's a great question because it's not that
people will necessarily be trying to deceive their readers, it could just be a
matter of them not realizing that in an organizational sense that kind of,
let's say, more significant limitation should be stated
earlier. So that's always something to look for. Thanks for the question. I'm now going to move ahead into the table
checklist. And please send us more questions if you
have any. Jeff McAllister: Okay. We have a comment and
questions from Megan: I always enjoy reading
results in which the question being sought after is
located in the title; what are your thoughts on this? How important is it? Barrett Whitener: Well, Megan, I think that there
are pros and cons both ways. For the purposes of our webinar today talking
about reading papers, I don't think either approach that is typically
taken in a title is superior, that is, whether the title states essentially the domain
finding or whether the title instead states essentially
the research question and just maybe alludes to the main finding or doesn't even allude to it. I think either approach like that is fine when
you're reading. I wouldn't call either of those a red flag
approach, something to arouse your concern. I'll put on my writing hat for a minute here and say that as writers I think it's typically better
-- I mean, it's certainly more common to see the
main finding stated in the title of the paper. And while that's okay, I think it does have some
drawbacks potentially, the main one of which is that a reader who sees the title that gives away
the main finding or includes the main finding -- depending on their level of interest in the
subject -- may or may not go on to read the paper. They may think, "Oh, okay. I understand what that paper's all about" and
continue with their other business. But if the title does not include the main finding
-- and if they're interested enough, then we at least look at the abstract to find out
what the main finding is. And maybe by that point after reading the
abstract they'll get hooked. They'll get a little more interested in reading
more of the paper. So for us as writers I tend to say consider at
least writing the title to include more of the main focus of the research question as
opposed to the main finding. But as readers, either way is just fine. Thanks, Megan. Great question. Jeff McAllister: Okay. It looks like Megan has a
clarification to her comment and I'll read it out
loud: "I just want to clarify I'm alluding to different
sections of the results section, titles of each section, not the main paper." Barrett Whitener: Okay. In that case, really the same holds. But I would say that you would be much more
likely to see and probably to write the findings themselves in that case. An alternative, though, is just to use those titles
as a label, as opposed to saying "Removal of GLUT1 Decreases Inflammation" you might
use that title instead to say something like
"GLUT1 and Level of Inflammation" or "Roving GLUT1:
Effect on Inflammation." Either way I think, again, it's just fine with that. But again, I think for purposes of keeping readers engaged you might
consider more the second approach where you don't give away the finding. Thanks. I will repeat something I said earlier, which is
that this webinar and the Writing a Scientific Paper webinar will
be available at the NIDA IP website. I believe in the Virtual Collaboratory is where
they're stored. Along with webinars I did last year on giving
oral presentations on science, designing research posters, and using
spoken English in professional settings and in everyday
settings. So there are a lot of resources there. And now I'm going to move on and put the
reading resource up on screen again in case anybody didn't get that. Everybody have a great day or great evening,
wherever you are. Thanks a lot.