Transcript Doing Research in Economics

Doing Research in Economics
Professor Charlie Karlsson
Jönköping International Business
School and CESIS
1
Agenda
•
•
•
•
•
•
•
•
•
•
•
What is research?
The research process in economics
Surveying the literature
Writing as a tool for economic research
Writing as a product of economic research
Critical reading
Theorising
Locating data
Manipulating data
Empirical testing
Communication of results
2
1. What is research?
• Research is the creation of new valid
knowledge, i.e. an extension of the research
frontier
• Research is based upon the existing up-to-date
knowledge in the field
• Knowledge is more than data or information – it
is structured information
• Knowledge is created by constructing a line of
arguments
– An argument is an assertion or a claim supported by
reasons or evidence
3
The construction of knowledge
• Scholars create knowledge by constructing
competing arguments using the following tools:
–
–
–
–
Mental Processes – thinking about an argument
Oral Discourse – verbalising the argument
Diagram Techniques – illustrating the argument
Mathematic Techniques – manipulating equations to
test the logic of the argument
– Writing – to present and spread the argument
4
How are arguments evaluated?
•
•
•
•
What are the reasons behind the argument?
Does the argument make sense? Why or why not?
Is the logic flawed?
What are the underlying explicit and implicit
assumptions? Are they flawed?
• How critical are the assumptions, i.e. would different
assumptions lead to different conclusions?
• What is the empirical evidence? Does it support the
conclusion?
• In light of the reasons and evidence provided, is the
argument persuasive? If so, the argument is valid until it
is invalidated by new arguments.
5
The scientific method
• Select a scientific problem or question
• Apply a theory to derive hypotheses about
the problem or question
• Test the hypotheses by comparing its
predictions to evidence from the real world
• If a hypothesis fails the test check the line
of arguments and if it is OK reject it
• If you can’t reject a hypothesis you
complete your line of arguments
6
2. The research process in
economics
•
•
•
•
•
•
•
•
Develop a well defined research question
Survey the literature in the field
Define a clear purpose
Select a theory proper for the research question
Develop hypotheses
Test your hypotheses
Interpret your results and draw conclusions
Communicate your research
7
The objective of a research project
• is to analyse some aspect of a significant
issue or problem
8
Step 1: Defining the scope of the
research
• What is the research topic?
• What is the research question?
• What is the tentative research
hypotheses?
• What is the current knowledge, i.e. where
goes the research frontier?
• What is the purpose?
9
What characterises a good
research question?
•
•
•
•
•
Problem-oriented
Analytical
Interesting and significant
Amendable to economic analysis
Feasible, given the time and resources
available
10
Strategies for selecting research
questions (1)
•
•
Pick a general topic area that interests
you, ideally one in which you have some
background
Start reading the literature, not merely to
see what has been done, but also to
identify what research questions remain
to be answered or what problems remain
to be solved.
11
Strategies for selecting research
questions (2)
• Select a promising from what you have
found in the literature:
– Could an interesting previous study be
applied to a new place or time?
– Are there conflicting findings on some
question, which you might try to reconcile?
– Studies often conclude suggesting questions
for future research.
– The literature survey may reveal gaps in the
current knowledge that you can explore.
12
Step 2: Surveying the literature
• What is currently known?
• What has been discovered to date on a
given topic?
• Objective: to identify and become familiar
with the major studies that have been
published on a topic
• Start with the most recent publications and
work backwards to the roots
13
Step 3. Selecting one theory
• Based upon your literature survey you
must choose one theory, which is relevant
for explaining your research problem
• Motivate your choice
• Never spend time on long presentations of
alternative theories – instead give
references to literature, where competing
theories are presented
14
Step 4: Analysing the problem
• The theoretical analysis of the research problem, is the
process, where theory is applied to shed light on the
problem:
– What are the essential concepts comprising the problem being
analysed?
– How are these essential concepts related?
– What do these relationships imply?
• The result of this analytical process is the research
hypothesis (hypotheses).
• A research hypothesis is the proposed answer to your
research question.
• Hypotheses are derived from economic theory (and
earlier empirical research)
15
Step 5: Testing your analysis (1)
•
•
•
The scientific method in economics is strongly
dependent upon empirical testing
Empirical testing implies comparing the
predictions of theory, i.e. the hypotheses, with
appropriate real-world evidence
You must decide how you will go about to test
your hypotheses, i.e. decide your research
design, which involves
1. Finding a good, large and relevant data set, and
2. Selecting an econometric method
16
Step 5: Testing your analysis (2)
• Key questions:
– How to choose an econometric method, which
produce the best possible and most reliable results?
– How to adequately test the hypotheses?
• Remember that a hypothesis is either rejected or
not rejected. It is never accepted!
• If you can not reject a hypothesis, then the
theoretical proposition is provisionally accepted
until it eventually is rejected
17
Step 5: Testing your analysis (3)
• If a hypothesis is rejected then one must ask the
following questions:
– Is there something wrong with the theory?
– Is there something wrong with the data?
– Is there something wrong with the econometric
method?
• If you find that something is wrong you have to
retake the process, otherwise you continue to
present your results in the thesis
18
Step 6: Interpreting the results and
drawing conclusions (1)
• What are the results of the empirical testing of
the research hypothesis?
• Are they consistent with the predictions of
theory?
• Are they consistent with the results form earlier
empirical research?
• Are there any problems (multicollinearity,
heteroskedasticity, etc.) with the econometric
testing that need to be corrected?
19
Step 6: Interpreting the results and
drawing conclusions (2)
• Are there shortcomings with the econometric
method that limit or weaken the results?
• Given the answers to the above questions: what
can be concluded about the results?
• To what extent are they in line with the
hypotheses?
• What can be concluded about your analysis and
about your research question more broadly?
20
What is good research?
• Good research, is research that follows
the scientific method, wherever the results
lead, even if they reject one or several
hypotheses
• A research projects that rejects a
hypotheses is not failed because it still
advances our knowledge – in this case by
eliminating one hypothesis as an
explanation to the research problem
21
Step 7: Communicating the findings
of the research project
• A written report where the author makes a case
for the validity of her/his results based on the
logic, rigour and empirical evidence of the
research.
• Oral presentation (and defence) at the final
seminar
• Oral presentation at conferences
• Publication as a working paper
• Publication in a scientific journal (or in an edited
book)
22
Writing a research proposal
1.
2.
3.
4.
5.
Statement of the nature of the problem
The research question
Survey of the literature
Research design
References
23
3. Surveying the literature
• Why is a literature survey necessary?
• Where to search: popular literature vs
scholarly literature vs internet sources
• How to search: Developing an effective
search strategy
• Obtaining the resources
24
Why is a literature survey
necessary?
• To advance the state of knowledge, you need to
know what the state of knowledge is
• You must create your own sense for what is
known and what is not known
• The literature provide ideas for your own
research
• The literature helps you to design your own
study by showing how previous approaches
either were or were not successful
25
The quality hierarchy
•
•
•
•
Scientific journals with peer review
Dissertations
Edited books with peer review
Monographs published by scientific
publishers
• Working papers
• Internet sources
26
Where to search?
•
•
•
•
•
•
•
•
•
EconLit
JStore
IDEAS/Repec
Palgrave’s
Handbooks
Collected volumes
Journal of Economic Literature (JEL)
Journal of Economic Perspectives
Course literature
27
How to search?
• To locate information efficiently, one needs
to use a search strategy
• Two general approaches:
– Browsing
– Keyword searching
28
Browsing
• Manual examination of written material for
useful information or references to useful
information
• The American Economic Association and
JEL use a hierarchical system to classify
information in economics:
http://www.aeaweb.org/journal/elclasjn.html
29
Keyword Searching
• Keyword searching use search engines
– on the World Wide Web such as Goggle and
Google Scholar or
– on specialised data bases such as EconLit or
Social Science Citation Index (SSCI)
• Remember to
– start with the most recent years
– Use Boolean searching (AND, OR, NOT) to
make your search more precise
30
A basic search strategy
• Begin by stating your research topic or question
• Identify important concepts related to your topic
• Brainstorm to create a list of keywords that describe
these concepts
• Determine which search features may apply
• Choose the appropriate database
• Read the search instructions for the database
• Create a search expression using the appropriate syntax
• View the results
• Modify the search if necessary
• Try the same search with an other data base
31
Obtaining the material
• More and more journal articles and
working papers are available in full-text
• Some of them you can reach via EconLit
at KTH Library.
• You may also test: http://rfe.org
• Other resources you may test are
www.nber.org http://papers.ssrn.com
http://econwpa.wustl.edu
32
REMEMEBER!
• You need to collect a substantial material
to cover the current knowledge and to get
ideas for your own research
• Think like this: your thesis must contain a
list of relevant references at least two
pages long, i.e. at least 40 references
33
Scholarly references and citation
styles (1)
• We strongly recommend that you use the
parenthetical form for references in the text, for
example Lööf (2005)
• You may consult
– Turubian, K. (1996), Manual for Writers of Term
Papers, Theses and Dissertations, 6th ed., for
general references and citation and
– Harnock, A. & E. Kleppinger, Online: A Reference
Guide to Using Internet Sources available at
http://www.bedfordstmartins.com/online for citation of
online documents
34
Scholarly references and citation
styles (2)
• There are three major schools for
referencing and citations:
– Modern Language Association (MLA)
available at
http://www.wisc.edu/writing/Handbook/DocMLA.html
– American Psychological Association (APA)
available at
http://owl.English.purdue.edu/handouts/research/r_apa.html
– The Chicago style avilable at
http://www.liu.edu/cwis/cwp/library/workshop/citchi.htm
35
4. Using writing as a tool for
economic research
•
•
•
•
•
Writing to learn
Composition as a creative process
The structure of an argument
Examining an argument
Three types of reasoning: deductive,
inductive and warrant-based
• What makes for a persuasive argument?
• An important caveat
36
Writing to learn
• Economist use writing for two purposes:
– Writing as a Product, a form of communication
to disseminate research results
– Writing as a Process for deriving the research
results
• Writing forces you to think concretely, i.e.
to figure out exactly what you mean.
• Writing is a tool of discovery
37
Composition as a creative process
(1)
• The process of writing is called
composition, which includes
– analysis, i.e. taking something apart to
understand it, and
– synthesis, i.e. putting pieces together to make
a whole
• It involves searching for relationships
between facts, theories and ideas that
make up the raw material for your
research
38
Composition as a creative process
(2)
• Composition needs time to develop and to
mature
• Hence, it is important
– to start early
– to write a draft
– to discuss the main ideas with supervisors
and colleagues
– to take time off from your writing to allow your
sub-conscious to do its job
39
Composition as a creative process
(3)
• Refining a thesis or a paper implies
– re-viewing the information
– re-thinking the way it is organised
– questioning the theoretical framework
– questioning the hypotheses
– re-constructing the arguments
– re-viewing the data
– questioning the econometric methods
– looking for new patterns of meaning
40
Composition as a creative process
(4)
• Creative and critical writing almost always
implies that you have to throw away parts of
what you have written
• Always concentrate on the most relevant and
most important aspects
• In Economics, we always disregard aspects of
lesser relevance and importance
• What you shall provide is a structure, which
highlights the most influencing factors in
explaining a phenomena but never try to make a
catalogue of all factors that may influence a
phenomena
41
The structure of the argument (1)
• The purpose of scholarly writing is to make
an argument that is persuasive to experts
in the field
• When completed, scholarly writing follows
a logical, hierarchical structure, in which
the main thesis is supported by a series of
nested arguments that lead logically to the
thesis as a conclusion
42
The structure of the argument (2)
• The thesis is at the top
• It is supported by a number of major
reasons
• Under each major reason there is a
number of supporting arguments
• Points that do not lead to the thesis either
directly or indirectly shall be omitted
43
What does it mean to say that a
conclusion follows from the evidence?
• An inference is a conclusion reached after
reasoning logically about facts and
relationships
• If the evidence leads us logically to the
inference as a conclusion, then we say
that the conclusion ”follows”
• A logic fallacy is an argument that is
flawed because the conclusion does not
actually follow from the reasons stated
44
Logical fallacies (1)
• Straw man – mischaracterizing a position by
omitting its strongest reasoning
• Special pleading – selectively using the
available evidences
• Begging the question – making an assertion in
which the reason given doesn’t really support
the conclusion
• Affirming the consequent – drawing conclusions
based on unexamined premises
45
Logical fallacies (2)
• Ad hominem – refuting the argument by
attacking the person, rather than his/her
arguments
• Appeal to authority – accepting an argument
because an expert endorses it
• Appeal to the people – accepting a position
because many others do, without examining the
argument
• Post hoc, ergo propter hoc – what comes before
was the cause
46
Logical fallacies (3)
• Fallacy of composition – what is true at the
micro level must be true at the macro level
and vice versa
• Appeal to pity – using sympathy for one
issue as justification for another issue
• False analogy – Drawing parallels
between two cases where there are
enough substantive differences to
question the comparison
47
Examining an argument
•
•
•
•
•
•
Identify the major claim
Identify the major evidences
Identify the supporting points
Is there a logical sequence of reasoning?
Are there any logical fallacies?
Does the evidences and the supporting points
lead to the claim?
• Don’t be afraid to criticise reasoning that isn’t
logical and supported by evidence
48
Three types of reasoning (1)
• Deductive reasoning starts from one or
more general principles and derives
specific predictions from them
• The predictions are deductions
• A valid deduction is one which the
conclusion must follow from the premises
• When scholars in economics theorize,
they are typically using deductive
reasoning
49
Three types of reasoning (2)
• An inductive argument is one that reasons in the
opposite direction from deduction
• Given some specific cases, what can be inferred
about the underlying general rule?
• The reasoning process follows the same steps
as in deduction
• The difference is the conclusions: an inductive
argument is not a proof, but rather a probalistic
inference
• When scholars use statistical evidence to test a
hypothesis, they are using inductive logic
50
Three types of reasoning (3)
• Warrant-based reasoning
• Warrants are un-stated or underlying
assumptions on which an argument stands
• Often warrants are higher-order assumptions or
axioms that are not testable, e.g. the assumption
that consumers maximise utility
• The purpose of a warrant is to establish the
relevance of the evidence in supporting some
claim
51
What makes for a persuasive
argument? (1)
• For an argument to be persuasive, the
reasons supporting it must be true, and
the conclusions must follow from the
reasons
• The evidence should be accurate,
authorative, precise, clearly explained,
complete and representative
52
What makes for a persuasive
argument? (2)
• Factual evidence needs to be accurate – always
control that the facts you build your arguments
upon are true
• Evidence should also be authorative, e.g. data
must come from a reliable source
• Evidence needs to be precise – try, for example
to estimate the size of an effect instead of just
claiming that many are affected
• Evidence must be clearly explained – it is not
enough to just present tables with econometric
estimations, they must be explained in writing
53
What makes for a persuasive
argument? (3)
• Evidences need to be complete, i.e. they
should have depth and breadth
• The evidences presented in your
argument should be representative of
thought on an assertion, i.e. you shall not
only report those evidences, which
supports your assertions – with other
words you should be intellectually honest
54
An important caveat
• A conclusion can follow from the evidence,
the evidence can be correct, and the
argument may still be incorrect
• Internal consistency in an argument is a
necessary but a sufficient condition, if
another conclusion explains the evidence
better and more correctly
55
5. Writing as a product of economic
analysis
•
•
•
•
•
•
•
What is economic writing?
Writing steps
Writing the first draft
Revising the paper
Writing style
Writing mechanics
Proof-reading
56
Writing as a product
• Writing as a product is the report from the research
process
• The audience is someone other than the author – in your
case economists at the master level
• Scholarly writing embodies an argument that attempts to
persuade experts in the field
• The writing needs to be explicit and formal
• All important elements needs to be spelled out clearly
and in sufficient detail to get your point across
• Proper standards of punctuation and grammar must be
followed
57
What is economic writing?
• What fundamentally distinguishes
economic from other disciplinary writing
and what all types of economic writing
share is the use of economic analysis
• All economic writing applies economic
theory to derive insights about and explain
answers to a question or a problem
58
Writing steps
1. Pre-writing or exploration – writing the
research proposal (spend relative much time
for this step)
2. Writing the first draft (spend relative less time
for this step)
3. Revising (several times) (spend much time for
this step)
4. Editing (if you apply the writing rules from the
very beginning and are careful with details you
need relative little time for this step)
59
Writing the first draft
• Features of good economic writing
• Getting the ideas down on paper
• Giving credit for intellectual property
60
The first draft
• One of the hardest steps in completing a
paper
• Prepare the first draft by writing throughout
the research process:
– Taking notes as you research the topic
– Writing critiques
– Drafting your own ideas
• As a first step define the audience for your
paper
61
Features of good economic writing
• Good writing should be
– Focused, i.e. have a clearly defined purpose
– Organised, i.e. follow a logical, hierarchical
structure
– Solidly developed, i.e. major points must be
explained in detail and supported by evidence
– Clear, concise and precise
– Free of grammatical errors
62
Getting the ideas down on paper
(1)
• Focus on getting your basic ideas down on
paper
• Don’t worry about details, mechanics or
grammar at this step
• Always take notes about what you have
done and where you are going next before
you end a work session
• Try to get long work sessions, preferably
3-4 hours
63
Getting the ideas down on paper
(2)
• Create an outline of what you are trying to say,
i.e. a disposition and a synopsis, OR
• do brainstorming, writing down your ideas as
they appear and only then create an outline to
organise them
• Try to find out which approach suits you best
• Remember that a research paper should explain
what you found, not what you did, i.e. readers
are not interested of the steps in the process
that you went through writing the paper
64
Giving credit for intellectual
property
• Totally avoid plagiarism, i.e. taking credit for
someone else’s words or ideas, even when it’s
unintentional
• Two types of plagiarism:
– Using someone else’s words as if they were your own
– thus, if you quote use quotation marks and give
proper reference but do not use quotations
excessively
– Using someone’s unique idea without attribution –
thus, be generous with references
65
Revising the paper
• Every first draft can be improved by
revision!
• There must be several revisions between
the first draft and the final draft
• To attain the full potential of the writing
process you need start drafting far enough
before your deadline to have time to do
multiple revisions
66
Is the thesis clear?
• The purpose of revision is to craft your paper so
it better embodies the features of good writing:
– Does your paper has a clear focus, i.e. does it only
contain one idea or theme?
– Is the thesis or principal assertion of the paper clear?
– Can you underline the thesis sentence in the paper’s
introduction?
– Can you underline the corresponding sentence in the
paper’s conclusion?
67
Is the paper well organised? (1)
• Is the organisation logical?
• Each chapter, section and paragraph
should be the explication of a single
thought, idea or theme – if not you should
split them up
• They should all have a thesis sentence,
typically the first sentence
68
Is the paper well organised? (2)
• It must be possible to identify a ”red
thread” throughout the paper, i.e. it must
be easy to follow your line of
argumentation
• You must provide services to the reader by
making the theme of each part explicit but
also by summarising your arguments at
the proper places
69
Are your points supported by
evidence? (1)
• Examine the development of each major
point in your paper
• Each point is itself an assertion that needs
to be supported by evidence
• If the first sentence in a chapter, section or
paragraph spells out the main point, the
reminder should flesh out and support that
main point
70
Are your points supported by
evidence? (2)
• Does the main point need to be explained
in more detail?
• Can you provide examples of what the
main point says?
• What makes you think that the main point
is valid?
• What evidence can you provide to bring
the reader to that conclusion?
71
Writing style
•
•
•
•
•
•
Strive for clarity
Use the active voice
Describe action with a verb
Be precise and concise
Let Microsoft Word help you
Dictionary and thesaurus
72
Strive for clarity
• Compose each sentence so that the
subject is the main actor of the story, and
the verb is the main action
• Whenever possible use strong verbs over
weak ones
• Don’t try to express complex ideas that
you do not fully understand
• Take responsibility for what you write –
don’t be vague
73
Use of active voice
• Passive voice is not incorrect per se, but it
makes it harder for the reader to figure out
what exactly you are saying
• Don’t think that the use of passive voice
make your writing sound more objective
• The reader needs to know who stands
behind the arguments – you should not be
afraid of arguing actively
74
Describe action with a verb
• Academic writers tend to nominalize, i.e.
to turn verbs into nouns
• However, it makes the writing sound more
pretentious and harder to read
• Nominalization tends to add words to
sentences without adding meaning
• To write clearly choose strong verbs to
describe the actions in your sentences
75
Be precise and concise
• Discussing complex issues often requires
nuance, i.e. word choice matters, since
synonyms have slightly different meanings
• Always choose the word that means exactly
what you wish to say to achieve precision
• Also try to make every point as concisely as you
can, i.e. avoid empty words
• The quality of a paper never depends upon its
length but – a shorter paper making the
necessary points is always better than a longer
paper
76
Let Microsoft Word help you
• Go to Tools/Options/Spelling and
Grammar and select ”Grammar & style”
under Writing style
• Under Spelling see to that you don’t ”Hide
spelling errors in this document”
• Under Grammar see to the you don’t ”Hide
grammatical errors in this document”
77
Dictionary and thesaurus
• If English isn’t your mother tongue you a
large, good dictionary
• Everyone needs a large and good
thesaurus
78
Writing mechanics
• Use complete sentences
• Don’t let sentences run on
79
Use complete sentences
• Serious writers employ complete
sentences
• A complete sentence implies a complete
thought, while a sentence fragment implies
a fragmented or incomplete thought
• A sentence fragment is a sentence without
a subject or without a verb or a sentence
that is incomplete in another way
80
Don’t let sentences run on
• A run-on sentence is two or more
independent clauses that are not
separated with the proper punctuation
• In a run-on sentence several complete
thoughts are jammed together, making it
difficult for the reader to determine where
one ends and the next begins
81
Proof-reading
• We naturally assume that every student uses the
spell-checking to avoid spelling errors and
simple grammatical errors
• However, spell-checking is not enough, which
implies that you must always leave yourself time
to proof-read your final draft very closely.
• To perform this task properly requires
considerable time and concentration, because
proofreading quickly turns into superficial skimthrough unless it is done in ‘quality time’ and
with academic breaks.
82
6. Critical reading or how to make
sense of published research
• Making sense of published research
• Taking research notes and writing
abstracts and critical reviews
83
Making sense of published
research
•
•
•
•
•
Understanding format
Evaluating the argument: reading critically
Questions to guide critical reading
Evaluating published research
More questions to guide critical reading
84
Understanding format
• Economics research papers tend to follow
a common format that illustrates the
scientific method
• In economics, there are three types of
scholarly work:
– A survey of the work of others
– A purely theoretical study
– An empirical study
85
The typical empirical research
study
• An introduction
• A theoretical analysis of the problem
• An empirical test of the hypotheses
derived in the theoretical analysis
• A conclusion
86
The introduction
• should
– define the general topic and the specific
research question
– explain the motivation for the research
– review the work of previous researchers on
the topic, especially
• what is lacking in the existing literature, and
• how the current study proposes to address that
shortcoming
87
The theoretical analysis
• is the heart of any economics study
• is the application of theoretical economic
analysis to shed light on the research
question
• develops the theoretical model used by
the study
• derives the testable implications or
hypotheses of the model
88
The empirical analysis
• explains how the proposed analysis from
the theoretical exercise is tested
• states explicitly what results would confirm
the theoretical propositions
• presents the results obtained from the
testing procedures and interprets them
• answers the question: To what extent is
the theoretical propositions rejected or not
89
The concluding sector
• explains the insights learned from the
research
• What answer did economic theory suggest
for the research question?
• Was this answer rejected or not by the
empirical evidence?
• How to interpret the results?
• What research questions should be given
priority in future research?
90
Evaluating the argument: reading
critically
• To understand a research paper you need to
read deeply and critically and to understand it’s
arguments
• This implies that you don’t just read the papers,
you study them to discern and evaluate the
author’s arguments
• To save on time you the first time skim the paper
to see whether it is useful
• If you find it useful you read it again more
carefully
91
Questions to guide critical reading
(1)
• What question is the author asking?
• What answer does the author propose (i.e.
what are the principal assertions of the
study)?
• In what way does the study improve upon
previous research?
• How does the proposed answer compare
with those provided by previous research?
92
Questions to guide critical reading
(2)
• What are the major logical or theoretical reasons
for the author’s argument?
• What empirical evidence does the author
provide?
• What type of econometric techniques do the
author apply?
• Are there any problems with the econometric
results?
• What assumptions are the author making in his
her reasoning?
93
Evaluating published research
• Once you have identified the argument in a
published work, the next step is to evaluate the
argument, i.e. to assess its validity and reliability
• Does the author have an apparent conflict of
interest?
• Is the study published in a refereed journal?
• If not, in what way has the quality of the
argument been checked?
94
More questions to guide critical
reading
• Does the theoretical analysis make sense?
• Are the data used adequate to the task?
• Does the empirical methodology adequately test
the hypotheses?
• Are the assumptions reasonable?
• Is the analysis (theoretical and empirical) clearly
explained?
• Do the conclusions follow from the evidence
presented?
• On balance, is the author’s argument convincing
to you?
95
REMEMBER!
• The questions for critical reading are also
very useful when you shall prepare the
discussion of a paper at a seminar or
conference
96
Taking research notes and writing
abstracts and critical reviews (1)
• When you take notes on a reading you read it
more carefully
• The notes are addressed to yourself and
preferably you save them in a computer file
• Always record the complete bibliographic
information
• Make your own copies of complete articles and
book chapters that are important to your
research project
97
Taking research notes and writing
abstracts and critical reviews (2)
• You can write more formal notes in form of
– an abstract (a summary of the author’s
arguments), or
– a critical review, i.e. an abstract augmented
with a critical evaluation of the work
• You may also construct an annoted
biography, i.e. a list of references that
includes a few sentences summarising
and critiquing each item
98
7. Theorising or conceptualising the
research
• What does it mean to apply theory to a
research topic?
• What is theorising?
• Narrative reasoning
• Mathematical reasoning
• A commonly used shortcut: modifying an
existing model
• What makes a good research hypothesis?
99
The development of a theoretical
framework
• is the most abstract part of the research
process,
• requires both analysis of the research
problem and synthesis of an appropriate
theoretical framework to explain it, and
• requires sufficient knowledge of the
appropriate economic theory on which to
build
100
What does it mean to apply economic
theory to a research topic?
• Survey the potential economic theories of
relevance for your research topic
• Assess which of the potential theories that
is closest related to your research topic
101
What is theorising? (1)
• Theorising is the process of brainstorming
about an issue so as to identify the logical
connections that explain the issue
• The result of the process is a theory that
analyses the research question, and in
particular provides propositions in the form
of research hypotheses
• Theorising involves constructing a
conceptual or theoretical argument
102
What is theorising? (2)
• When you theorise, you ask three basic
questions:
– What are the essential concepts involved in the
problem being researched?
– How are the essential concepts related?
– What implications or predictions can be drawn from
these relationships?
• Notice that there is a distinction between existing
economic theory and the resulting theory that is
developed for a specific research project
103
Narrative reasoning
1.
2.
3.
4.
5.
The creation of a ”primary narrative”, i.e. a document
that gives a detailed description of the research topic
”Concept creation”, i.e. a review of the primary
narrative so as to identify the essential concepts
The creation of a ”higher order narrative”, i.e. a revised
version of the primary narrative that focuses on the
concepts developed in step 2
The examination of the rewritten narrative to identify
possible relationships between concepts
The postulation of hypotheses from the theoretical
relationships
104
Mathematical reasoning
1. Identify the relevant economic
assumptions for the problem
2. Use mathematics to manipulate the
assumptions so as to derive a conclusion
or hypothesis
3. Mathematical models can be
optimisation models or ad hoc models
105
A commonly used shortcut:
modifying an existing model
• Rarely do economists create entirely
original models
• Often researchers take an existing model,
which has already been applied to the
topic they are interested in, and modify it
in some way that seems to be an
improvement over the original
• This can be done both with optimisation
and ad hoc models
106
What makes a good research
hypothesis?
• It should be stated clearly and specifically
in a way that can’t be misinterpreted
• It must be non-trivial
• It must be able to discriminate clearly from
alternative hypotheses
• It must be capable of being proved false
• It should be empirically testable
• It must be derived from the theoretical
analysis
107
8. Locating (and collecting)
economic data
•
•
•
•
•
Data creation
The structure of economic data
Organisations that collect and publish data
Major primary data collections
Major secondary data collections
108
Data and empirical research
• Data collection and manipulation is a key
part of any empirical research project
• Start early on to look for potential data
sources
• Check the data sources used, when you
review the literature
• BUT, don’t let data availability govern,
which variables you include in your
theoretical discussion
109
Data creation
• Don’t view data as facts – the vast majority
of data are constructed rather than
collected
• Data construction
• Sample data
110
Data construction
• Steps involved in the construction of data
series:
– Definition of the concept
– Decision on how the concept should be
measured
– Determination of how to define the sample on
which the data is based
• Illustrating example: how to measure
average family income?
111
Sample data
• Much social science statistics are based on
sample data rather than population data
• Thus, the data that are published are
extrapolated from samples
• Only if the sample is random, and thus truly
representative for the population, will the sample
statistics correctly measure the population
• Therefore, think of much data as estimates
rather than facts
112
The structure of economic data
• It is important to differentiate between those
organisations that collect or produce data and
those who publish it, i.e. between primary and
secondary sources of information
• Existing data are typical the result of a specific
data collection effort or process
• The product of this process is a specific data set
that includes a collection of certain variables
113
Characteristics of data sets
• Time-series data, which are available at
different frequencies
• Cross-section data, which vary in terms of
the unit of analysis
• Longitudinal data, i.e. a cross-section data
set that is followed over time, is an
example of a micro data set
114
Organisations that collect and
publish data
• Central bureaus of statistics, e.g. Statistics
Sweden (SCB)
• Institutes for economic analysis
• Labour market agencies
• Central banks
• Patent agencies
• International organisations, such as UN,
IMF, The World Bank, OECD, EU
(Eurostat)
115
Major primary data collections
•
•
•
•
•
•
•
•
•
•
National accounts
International financial flows and balances of payment
Imports and exports
Population data
Income data
Employment data
Education data
Consumer surveys
Industrial data
Financial company data
116
Major secondary data collections
• World economic outlook database
http://www.imf.org/search97cgi/s97is_eng.dll/search97cg
i/inetsrcheng.ini?action=FilterSearch&filter=spquery.hts&
Query/Text=weodb
• Penn world tables
http://datacentre.chass.utotronto.ca/pwt/index.html
• Joint BIS-IMF-OECD-World Bank statisitcs
http://www.oecd.org/statistics/jointdebt
• Eurostat
http://epp.eurostat.cec.eu.int/portal/page?_pageid=1090,
1137397&_dad=portal%_schema=PORTAL
• OECD main economic indicators
http://www.oecd.org/statsportal/0,2639,en_2825_293564
_1_1_1_1_1,00.html
117
REMEMBER!
• Always consult your supervisor concerning
available data
• Never start collecting your own primary
data without the permission of your
supervisor
118
9. Putting together your data set
• Developing a search strategy for finding
your data
• Data manipulation
119
Developing a search strategy for
finding your data
• Step 1: Before you search
– What are the desired variables?
– How should each variable be defined?
– What data frequency and sample period or what levels of
analysis?
– What are the potential sources for data on each variable?
• Step 2. As you search
– What data are available?
– Are there suitable proxy variables for variables that are
unavailable?
– If not, how can the empirical model be modified to use the data
available but still test the hypotheses?
120
Data manipulation
•
•
•
•
•
•
•
•
•
•
Level of variable
Change in variable
Real versus nominal magnitudes
Index numbers
Quantity indices versus real quantities
Price indices versus implicit price deflators
How inflation distorts nominal values
Rebasing data series
Data smoothing
Constructing a data appendix
121
Different forms of data
•
•
•
•
•
•
•
•
•
•
•
Levels
Per capita
Changes
Rates of change/percentage change (or growth rates)
Annualised growth rates
Proportions
Nominal, i.e. running prices
Real, i.e. fixed prices
Index numbers for prices and quantities
Categorial data
Count data
122
10. A first look at empirical testing:
creating a valid research design
•
•
•
•
•
•
•
Key issues in research design
How does one analyse data?
Random variation in human behaviour
Statistical methods
Simple statistical hypothesis testing
Confounding variables
Casual validity
123
Key issues of research design
• Two general types of empirical methods:
– Experimental methods
– Survey or non-experimental methods (the
traditional method in economics)
• A critical factor in designing an empirical
study is the degree to which the method is
valid
124
Validity
• A study has internal validity if the impact
observed can be attributed to the study variable:
– Instrument validity – does the test instrument
adequately measure what it purports to?
– Relationship validity – how conclusive is the empirical
testing?
– Casual validity – can one be sure that the
hypothesised causal relationship is valid?
• A study has external validly if the results can be
generalised to other situations, applications or
circumstances
125
Empirical testing (1)
• Purpose: to search for evidence in the
data to evaluate the hypotheses
• A good empirical test rejects alternative
hypotheses
• However, the data in the real world may be
consistent with alternative hypotheses
• Thus, you must select an empirical test
that adequately discriminates between
alternative hypotheses
126
Empirical testing (2)
• The power of a test is the probability of correctly
rejection the null hypothesis when it is not true
• When selecting a test method always ask: ”If the
test yields the strongest possible statistical
results, how confident can I be that my
hypothesis is not rejected?”
• If the test does not adequately discriminate
between alternative hypotheses, you should
consider a more powerful test
127
How does one analyse data?
• Thinking about empirical testing
• Casual empiricism
128
Thinking about empirical testing (1)
• Start by asking the following questions:
– What are the implications (or predictions) of
my theoretical analysis?
– If the hypothesis is not rejected, what
evidence should one expect to see?
• The answer to these questions is called
the theoretical prediction of the analysis
129
Thinking about empirical testing (2)
• Once the predictions of the theory are identified,
the researcher next ask:
– Is the evidence of the real world consistent with these
predictions?
– How exactly does one examine the evidence to
answer this question?
• We can here differentiate between three
methods:
– Casual empiricism
– Simple hypothesis testing
– Multiple regression analysis
130
Casual empiricism
• This is the type of analysis you do when
you present your data and it includes
– Data tables
– Data graphs
– Simple descriptive statistics
– ”Visual” examination
131
Descriptive statistics
• Measures of central tendency are averages:
– Mean
– Median
– Mode, i.e. the most common value in the sample
• Measures of dispersion:
– Range, i.e. maximum and minimum values in the data
– Standard deviation
– Variance
• Measures of relationships between variables:
– Covariance
– Correlation
132
Major problems in data analysis
• The effects of random variation in human
behaviour
• The fact that relationships between two
variables can be concealed by the effects
of other variables
• The fact that correlation is not the same as
causation
133
Random variation in human
behaviour
• This poses a problem for relationship validity
because the effects of random variation can
obscure any underlying relationship
• With large random samples the positive and
negative errors tend to cancel each other
• However, truly random samples are rare, which
introduce bias into the results as almost any
data sample will include a non-random selection
of errors due to sampling errors
134
Statistical methods
• The way to deal with random variation in human
behaviour is to incorporate statistical methods to
determine the likelihood of a sampling error
• When employing statistical methods it is critical
to discriminate between two concepts:
• The null hypothesis or the statistical hypothesis
• The maintained hypothesis, i.e. the theoretical
prediction of a model
• Economists generally test the null hypothesis
135
Level of significance (1)
• How certain do you need to be to reject
the null hypothesis?
• The level of significance is the risk that the
researcher is willing to take that the null
hypothesis will be rejected when it is true
or alternatively the probability that the
researcher will not reject the maintained
hypothesis when it should be rejected
136
Level of significance (2)
• Two schools:
– Select one level of significance, normally the
5 % level
– Mark the different levels of significance, i.e. 10
%, 5 %, 1 %
137
Simple statistical hypothesis testing
• Simple statistical hypothesis testing is one
case of what is known more generally as
statistical hypothesis testing
• If you are uncertain consult your textbooks
on statistics and econometrics on t-tests
and the use of p-values
138
Confounding variables
• Confounding of explanatory variables, i.e. the
fact that relationships between two variables can
be concealed by the effects of other variables
• It affects both casual empiricism and simple ttests
• The general solution to this problem is provided
by multiple regression, which offers an
opportunity to control for other variables that
could influence that studied variable
139
Casual validity
• Correlation between two variables does
not necessarily imply causation
• Regression analysis does not prove
causation
• Theory can help you make a case for
causation
• However, be observant for the
endogenously problem, i.e. that causation
may go both ways
140
11. Introduction to regression
analysis
• The 6 steps of regression analysis
141
Steps in regression analysis (1)
• Step 1: State the hypothesis
• Step 2: Choose a proper mathematic
model and motivate your choice
• Step 3: Test the hypothesis
– Choose an estimation method and motivate
your choice
– Estimate the model
142
Steps in regression analysis (2)
• Step 4: Interpret the test results
– To what extent do the parameter estimates conform
to the maintained hypothesis identified in Step 1?
– Are the parameter estimates statistically significant?
– Are they economically significant?
– Are the parameter estimates
• plausible for the real world?
• consistent with economic theory?
• within the range of previous estimates?
– How ”good a fit” is the overall regression model?
143
Steps in regression analysis (3)
• Step 5: Check for and correct common
problems of regression analysis
• Step 6: Evaluate the test results
– To what extent are the results in line with the
hypotheses?
– Can you go on to analyse and present your
results or do you need to retake the process?
144
Step 1: State the hypotheses
• Identify your dependent variable and your
explanatory or independent variable
• State clearly the sign (positive or negative)
of the expected influence of each of the
independent variables on the dependent
variable
145
Step 2: Choose a proper
mathematical model
• There are numerous mathematical models
used in regression analysis
• You have to choose one model dependent
upon your expectations about the
relationship between the independent
variables and the dependent variable and
between the independent variables
• You must motivate your choice
146
Step 3. Test the hypothesis
• Choose an estimation technique – you can not
always rely on OLS!
• Motivate your choice – the validity of each
estimation method depends upon a number of
technical assumptions hold true
• Estimate the relationship
• Sometimes it is necessary to test alternative
model formulations or alternative models or to
use alternative estimation methods
147
Step 4: Interpret the test results (1)
• To what extent do the parameter estimates
conform to the maintained hypothesis?
– Are the parameter estimates statistically
different from zero, i.e. are they statistically
significant?
– Are the signs the expected signs?
– Are the size of the parameter estimates in the
expected range?
148
Step 4: Interpret the test results (2)
• Has the results economic significance, i.e.
are the size of effects of such magnitude
that they matter?
149
Step 4: Interpret the test results (3)
• Are the parameter estimates plausible?
– What are the units of the parameter estimates?
– Given the units, are the parameter estimates
plausible?
• Check the size of the elasticities!
– Are the parameter estimates reasonable stable
between different model formulations?
– Are the results consistent with economic theory?
– How do the estimates compare with previous
research?
150
Step 4: Interpret the test results (4)
• How ”good a fit” is the regression model?
• There are several test statistics:
–
–
–
–
R-square
Adjusted R-square
The F-statistic
Etc.
• Don’t overestimate the importance of a high R-square in
hypothesis testing
• However, a low R-square may be an indication of
missing explanatory variables
• R-square is normally lower for cross-section data than
for time series data
151
Step 5: Check for and if necessary correct
for common problems of regression analysis
• Problem 1: autocorrelation (and spatial
autocorrelation)
• Problem 2: heteroskedasticity
• Problem 3: simultaneous equations bias
• Problem 4: specification error
• Problem 5: multicolliniarity
• Always consult your textbooks in
econometrics at this stage!
152
Step 6: Evaluate the test results
• This is the main point
• What do the results mean with respect to
my hypotheses?
• Interpreting regression results is more an
art that a science
153
Some additions
• Transformation of data – natural
logarithms
• Non-linear relationships – squared terms
• Inclusion of time trends
• Dummy variables
• Interaction terms
• Qualititative or limited dependent variables
– logit models, tobit models, etc.
154
12 Communicating the results of a
research project
• Writing the research report
• Presenting research orally
155
Writing the research report
•
•
•
•
•
•
Introduction
The written literature review
Writing the literature review
Theoretical analysis
Empirical testing of the analysis
Other components of the paper
156
The purpose of a written report
• The purpose of a written report
– is to present the results of your research, but
– more importantly to provide a persuasive argument to
readers of what you have found
• You must think of both format and argument
• Always remember that the purpose of research
is to advance knowledge in a field by providing
convincing arguments supported by logic and
empirical evidence
157
Components of an empirical
research paper in economics
•
•
•
•
•
•
•
•
•
•
Title
Abstract
Table of contents
Acknowledgements (not necessary)
Introduction and literature survey
Theoretical analysis
Empirical testing
Conclusions
List of references
Data and other appendices
158
Introduction (1)
• The purpose of the introduction is to provide a
rationale for the research
• What is the nature of the issue or problem the
research investigates?
• Why is it worthy an investigation?
• What have previous researchers discovered
about the issue or problem?
• What is the purpose with your research, i.e. what
is the contribution that your research will make
to the literature?
159
Introduction (2)
• The introduction is the most critical part of the
research report
• You must design the first part of the introduction
so that it catches the readers interest
• You may present some dramatic statistical
figures or you may start by questioning some
recent research
• Avoid to start the introduction in a ”traditional”
manner
160
Introduction (3)
• Start the introduction by sketching out the
research problem.
• What evidence can you offer to describe
the issue?
• Why is it a problem?
• Is it a public policy issue, a social
problem?
• Is it purely an intellectual puzzle?
• Who would be interested of the problem?
161
Introduction (4)
• Explain to the reader why the topic is interesting,
significant and substantial.
• Is it, for example a hot topic among experts in
the field, among politicians, in industry, etc.
• Show that there are some aspects of the
problem that have yet not been understood
• Next step: summarise what has been done
already to study the problem
162
The written literature review (1)
•
The literature survey is usually included
as a part of the introduction but can in
rare cases be a separate chapter
163
The written literature review (2)
• The literature review shall accomplish three
things:
1. It should identify the major findings on a topic up to
the present with a concentration on the most recent
contributions to the knowledge in the field
2. It should point out the principal deficiencies of these
studies and/or provide a sense of what is lacking in
the literature
3. It should conclude by leading into your research
question, by explaining how your research proposes
to contribute to the literature or address some
shortcomings of earlier studies
164
Writing the literature review (1)
• Novice researchers tend to write
excessively long literature surveys
• The literature survey should be brief but
cover the major contributions
• The trick is to cast your net wide enough
• What number of sources to consult
depends on the number of studies
completed in the field
165
Writing the literature review (2)
• For each study you include you shall
– Give complete citation information –
”Johansson (2002)”
– Indicate the question that the author
examined
– The author’s findings
– Anything about the methodology relevant for
your study
– Sometimes it is enough with one sentence for
one reference
166
Theoretical analysis (1)
• The purpose is to present the theoretical
analysis of the issue or the problem that you are
studying
• This section presents the logical evidence as a
deductive argument
• You must clearly
– describe the theory you are applying to your research
problem
– explain in detail why it is relevant, and then
– sketch out how it diagnoses the problem
167
Theoretical analysis (2)
• As you sketch out the logic of your theoretical
analysis it is helpful to remember that you are
trying to develop a deductive argument that
culminates in the research hypothesis(es), which
are contained in the form of an equation of the
model
• Remember that you need to do a complete job
of explaining your theoretical analysis – the
theoretical analysis is more important than you
think!
168
Empirical testing of the model
• This chapter must contain
– A presentation of the data and where it comes from
– Descriptive statistics
– The empirical model, its motivation and the
hypothesized results
– The econometric testing methodology and its
motivation
– Actual results with the proper statistical indicators
– A description of how you have dealt with common
econometric problems
– Interpretation and analysis of the results
169
Conclusions (1)
• The purpose of this part is to summarise
your findings, i.e. to restate your
arguments and conclude whether your
hypotheses were rejected or not
• In light of the statistical results, what can
you infer about your hypothesis(es)?
• To what extent are the empirical results in
line with the theoretical expectations
170
Conclusions (2)
• If your hypotheses are rejected you must
suggest reasons why
• Something flawed with the theory?
• Something flawed with the data?
• Something flawed with the econometric
techniques?
171
Conclusions (3)
• What can be concluded about the
research question more broadly?
• Are there any policy conclusions that can
be drawn?
• What research questions should be taken
up in future research?
172
Other components of the paper
• Title should be carefully composed
• You must include an abstract (together with 5-8
key-words and relevant JEL-codes)
– What was the research problem?
– How have you dealt with it?
– What is your results?
• Table of contents
• Acknowledgements (optional)
• Reference list (complete and with all necessary
details)
• Appendices (optional)
173
Presenting research orally
• An oral presentation is different from a
written one
• Preparing the presentation
• Using visual aids: handouts,
transparencies, PowerPoint
• Practicing the presentation
• Giving the presentation
• The role of the discussant
174
The seminar
• The purpose of the seminar is to
– present the research work to a larger
audience,
– provide time for criticism of the research work
to make it possible to improve it and for a
scientific dialogue
• All participants are expected to be active
in the discussion at the seminar
175
An oral presentation is different
from a written one
• You will not be able to present all your material –
you must a selection of the most strategic parts,
i.e. the presentation shall summarise the major
points of your research paper
– The research problem in relation to earlier research
– Your purpose
– Your theoretical analysis and in particular your
hypotheses
– Your data
– Your econometric technique
– Your major results
– Your major conclusions
176
Preparing the presentation (1)
• Remember that your presentation shall highlight
what is new with your research
• Prepare presentation ”notes”, i.e. put down on
paper the points you want to make
• NEVER READ FROM YOUR NOTES WHEN
YOU MAKE YOUR PRESENTATION – BUT
DON’T FULLY MEMORISE THEM EITHER!
• You are supposed to ”talk about” your research,
i.e. you need to know the material but you don’t
need to know every world
177
Preparing the presentation (2)
• When you prepare your presentation, you
should explicitly and carefully think about its
introduction.
• How can you engage the interest of the
audience?
• Do a careful time planning and allocate extra
time to discuss the principal contributions of your
research
• Also, think explicitly upon how you shall end the
presentation
178
Use of visual aids
• Making presentations of research reports
needs some visual aids, at least to show
figures, diagrams, tables, equations, etc
• You can use handouts, transparencies or
PowerPoint – each technique has its
advantages and disadvantages
179
Practicing the presentation
• Always practice a few times before you
make the presentation so you know what
time it takes and if there is some flaws with
your presentation structure
• Remember that the presentation takes
much less time when you rehearse on
your own
180
Giving the presentation
• Dress professionally for your presentation
• Bring a few extra copies of your paper
• Arrive in time so that you can make the necessary
technical preparations
• Stand up to get better contact with the audience
• Start by introducing yourself
• Think about your body language
• Avoid filler words like umm, aah, and so on
• Don’t exceed the time limit - it is unprofessional
• End your presentation with a thank you
• Be prepared for questions
181
The role of the discussant (1)
• A two-fold role:
– To offer a well-thought-out, educated reaction to the
paper
– To give the author creative feed-bask on how to
improve the paper
• Always read the paper carefully and prepare
your comments
• Never by rude but be frank about the less
satisfactory aspects of the paper
• Never start with saying that this was a very good
paper – in particular not if it isn’t true
182
The role of the discussant (2)
• Start by briefly summarising the paper and
highlight the contribution it attempts to
make to the literature
• What does the paper try to accomplish?
• How does it go about doing it?
• What results are reported?
• To what extent did the paper succeed in
reaching its goal?
183
The role of the discussant (3)
•
•
•
•
What can you suggest to improve the paper?
Could any parts benefit from more detailed explanation?
Are there parts that you are unable to understand?
Are there any problems with the data that weakened the
paper’s argument?
• Are there any problems with the econometric techniques
and the statistical tests employed?
• Are there any errors in interpreting the results?
• Are there any previous studies that the author might
benefit from reading?
184
The role of the discussant (4)
• Specific suggestions are always more helpful
than general comments
• The content of the paper is more important than
the writing but technical details and language
deficiencies matter to the extent that they
detracts from the argument
• End with summarising what contribution the
paper makes to understanding the issue or
problem it seeks to explore
• Give a copy of your comments to the author and
also the paper if you have annoted it
185