Transcript Institutional Research: What*s It All About? Part II

Using National Data for
Decision Support in US
Colleges and Universities
Karen Webber
Associate Professor,
The Institute of Higher Education, UGA
Presentation for HEISEE, June 6, 2013
[email protected]
•
•
•
•
Institutional Research
Institutional Effectiveness
Quality Assurance
Planning and Research
KWebber HEISEE 2013
Institutional Research
A focus on:
the collection, analysis, and reporting
of information that leads to improved
understanding, planning, and operating
of institutions of higher education
KWebber HEISEE 2013
Data Collection and Use of Data
• Very helpful for institutional planning, quality
assurance
• Can be used for benchmarking
• May start small, informal
• As more institutions get involved, grows larger, may
take on formal procedures
• Critical to have good infrastructure/technology
• Critical to have definitions to ensure consistency
KWebber HEISEE 2013
Institutional or System Database(s)
Student
Data
• Addresses,
enrolment
qualifying scores,
course grades, etc
HR Data
• Address, phone,
faculties group, etc
Central data
repository
or Data
Warehouse
Finance
Facilities
KWebber HEISEE 2013
• Tuition & fees,
• E&G expenditures,
salary & benefits
•# buildings, sq
ft figures,
equipment,
Who Uses National/Regional
Data and Why?
• Many with Inquiring Minds:
–
–
–
–
–
Institutional Research/academic planning officials
Educational researchers
Graduate students
Legislators
Policy Analysts
• Why?
–
–
–
–
–
Academic/administrative planning within an institution
Assessment; evidence of meeting goals, strategic plans
Scholarly inquiry
Policy, budget planning
Generalizable; reliable, good for policy considerations
KWebber HEISEE 2013
Some data systems for higher
education
•
Australia
– Higher Education Statistics Collection
http://www.innovation.gov.au/HigherEducation/HigherEducationStatistics/Pages/
OverviewOfHigherEducationStatisticsCollections.aspx
– HERDC
http://www.innovation.gov.au/Research/ResearchBlockGrants/Pages/HigherEduc
ationResearchDataCollection.aspx
•
South Africa – CHE and HEMIS
– http://www.che.ac.za/about/
•
EUMIDA
– http://datahub.io/dataset/eumida
•
UK – HESA
– http://www.hesa.ac.uk/
– http://www.ukcisa.org.uk/about/statistics_he.php
•
Middle East, Latin America – beginning
KWebber HEISEE 2013
Data Systems for Higher Education
•
EU – OECD Statistics and Data Lab
– http://www.oecd.org/statistics/
•
Eurostats
– http://epp.eurostat.ec.europa.eu/portal/page/portal/education/introduction
•
The World Bank data
– http://data.worldbank.org/data-catalog/ed-stats
•
World Higher Education Database Online
– http://www.whed-online.com/about.aspx
•
Croatia –
– Croatian Bureau of Statistics, IDIZ, IED, CEP, ? Institutions, others
•
US
– IPEDS, NCES, NSF
KWebber HEISEE 2013
U.S. Department of Education
Mission – to ensure equal access to education and
to promote educational excellence throughout
the nation.
Supplements and complements the efforts of
states, local school systems and other
instrumentalities of states, private sector, public
and private nonprofit educational research
institutions, community-based organizations,
parents, and students to improve the quality of
education.
http://ed.gov
KWebber HEISEE 2013
U.S. Department of Education
• Establishes policies on federal financial aid
(distributes and monitors those funds)
• Collects data on America's schools and
distributes research via reports and in some
cases, datasets
• Focuses national attention on key
educational issues
• Prohibits discrimination and ensures equal
access to education
KWebber HEISEE 2013
National Center for Education
Statistics (NCES)
NCES is the primary Federal agency
responsible for the collection, analysis,
and the reporting of data related to
education in the United States
KWebber HEISEE 2013
National Center for Education
Statistics (NCES)
On the web…
• Monitoring Programs
• Publications
• Collecting Data
http://nces.ed.gov
KWebber HEISEE 2013
• Educational Level
– Early childhood
– Primary/secondary
– Postsecondary
• Type of Data
– Cross-sectional
– Longitudinal
• Unit of Analysis
– Individual Student or Faculty member
– School (institution)
KWebber HEISEE 2013
NCES Data Sets
•
IPEDS (Integrated Postsecondary Education Data System)
•
•
•
•
•
•
•
•
Baccalaureate & Beyond (B&B)
Beginning Postsecondary Students Longitudinal Study (BPS)
Career/Technical Education Statistics (CTES)
High School & Beyond (HS&B)
National Longitudinal Study of 1972 (NLS)
National Postsecondary Study of Financial Aid (NPSAS)
National Study of Postsecondary Faculty (NSOPF)
Postsecondary Education Transcript Collection (PETS)
•
•
Interagency Expanded Measures Enrollment & Attainment
(GEMEnA)
Statewide Longitudinal Data Systems Grant Program (SLDS)
•
•
Postsecondary Education Descriptive Analysis Rpts (PEDAR)
Postsecondary Education Quick Information System (PEQIS)
http://nces.ed.gov/surveys/SurveyGroups.asp?group=2
Data Access
• Both NCES and NSF have some
publicly-available data
• And some restricted access files
KWebber HEISEE 2013
Data Tools from NCES
• Executive Peer Tool (ExPT)
• Peer Analysis System (PAS)
• Powerstats and Data Analysis System
(DAS)
KWebber HEISEE 2013
Short Demonstration
KWebber HEISEE 2013
KWebber HEISEE 2013
KWebber HEISEE 2013
National Science Foundation – Nat Center
for Science & Engineering Statistics (NCSES)
•
The responsibilities of NCSES have been broadened from those of the former
Division of Science Resources Statistics. Data collections related to U.S.
competitiveness and STEM education are part of these new responsibilities.
NCSES is responsible for statistical data on the following:
–
–
–
–
•
•
•
•
•
Research and development
The science and engineering workforce
U.S. competitiveness in science, engineering, technology, and R&D
The condition and progress of STEM education in the United States
Core Activities. As one of 13 federal statistical agencies, NCSES designs, supports,
and directs periodic national surveys and performs a variety of other data
collections and research. The America COMPETES Reauthorization Act codifies the
role of NCSES in supporting research using the data that it collects and its role in
research methodologies related to its work. The legislation specifies the
responsibilities of NCSES in supporting the education and training of researchers
who use large-scale data sets, such as the ones NCSES now collects. The following
activities form the core of NCSES work:
The collection, acquisition, analysis, reporting, and dissemination of statistical data
related to the United States and other nations
Support of research that uses NCSES data
Methodological research in areas related to its work
Education and training of researchers in the use of large-scale nationally
representative data sets
http://nsf.gov/statistics/about.cfm
KWebber HEISEE 2013
NSF - NCSES
• Each year produces about 30 publications, which can be
roughly divided into the following categories:
• InfoBriefs highlighting results from recent surveys and
analyses;
• Detailed Statistical Tables (DSTs) containing extensive
tabulated data from a particular survey;
• Periodic "overview" reports, such as Science and
Engineering Indicators and Women, Minorities, and
Persons With Disabilities in Science and Engineering; and
• Special reports, such as Interstate Migration Patterns of
Recent Recipients of Bachelor's and Master's Degrees in
Science and Engineering and Gender Differences in the
Careers of Academic Scientists and Engineers.
KWebber HEISEE 2013
NSF Data Tools
WebCASPAR and SESTAT
• provides easy access to a large body of
statistical data resources for science and
engineering (S&E) at U.S. academic
institutions.
• emphasizes S&E, but its data resources
also provide some information on nonS&E fields and higher education in
general.
• https://webcaspar.nsf.gov/
• http://www.nsf.gov/statistics/sestat/
KWebber HEISEE 2013
These Data Are Useful
• Intra- and inter-institutional data
comparisons
• Monitor institutional progress, help
with decisions
• Help national leaders monitor US
higher education broadly
– Reports and other information to the
public
KWebber HEISEE 2013
Data Used on Campus
• Ad hoc or regular, on-going reports
to senior administrators
(e.g., deans, rectors, vice rectors)
• Institution Fact Book
KWebber HEISEE 2013
Also used for analytic studies
• Almost endless number of good research
questions can be generated
• Examples:
– Student access to higher education, graduate
school
• Student demographics, location, type institution
– Effects of financial aid on student completion
– Where/what do degree recipients go/do
after graduation?
– What is relationship between financial
allocations and institutional outcomes
KWebber HEISEE 2013
Example Research Studies
In US- a current focus
on student financial aid
Study on graduate student
financial aid debt
(Belasco, Trivette, & Webber, 2013)
• NPSAS data 2000 and 2008
• About 49% all graduate students borrowed
for school in 2008; increased to 59% in
2008
• Mean debt $42,000 in 2008
• Higher for doctoral and professional
students, but more master’s
• More debt incurred by Black and Hispanic,
less for Asian than white students
KWebber HEISEE 2013
Also a focus on degree
completion
Study on benefits of the earned
doctorate
(Webber et al, 2013)
Using NSF’s Survey of Doctorate Recipients:
• What individual economic benefits accrue for
doctoral degree recipients from the time of
doctorate graduation (1998-99) to the present
time (2008)?
• Are there differences in the economic benefits
resulting from debt assumed and sources of
financial support during degree enrollment?
• What effect does time to degree completion have
on the economic benefits obtained by doctoral
degree recipients?
KWebber HEISEE 2013
KW study on benefits of the earned doctorate
Average Annual Salary
of Doctorate Recipients by Job Sector
$130,464
2008
$112,799
$104,336
$117,001
2006
$99,309
$90,175
Business &
Industry
$107,503
2003
$90,055
$80,893
Government
Education
$99,846
2001
$81,560
$71,738
$86,999
1999
$72,189
$62,051
$0
$20,000
$40,000
$60,000
$80,000
Source: Survey of Doctorate Recipients, National Science Foundation
$100,000
$120,000
$140,000
KWebber HEISEE 2013
Interest in faculty work and
faculty productivity
Study on Postsecondary Faculty
(Webber, 2012)
Using 2004 National Study of
Postsecondary Faculty:
• What factors contribute to a faculty
member’s research productivity
• Are there differences in productivity for USvs. Foreign-Born faculty/academic staff
KWebber HEISEE 2013
Results from NSOPF study
Recent Scholarly Works by US- V. Foreign-Born Status
Born in US
N=2990*
Mean
SD
Foreign-Born
N=1190
Mean
SD
p
Refereed articles
3.81
4.846
5.56
5.862
**
Non-refereed articles
1.40
2.885
1.86
3.840
**
Book revw, chptrs, creative wrks
1.23
2.180
1.20
2.063
.61
1.525
.66
1.496
**
Presentations
5.84
6.792
7.22
7.177
**
Exhibitions, performances
1.23
6.453
.36
3.067
**
.15
.683
.24
.830
**
7.07
9.491
9.29
9.355
**
Books, textbooks
Patents
Total Recent Written Works
** p <.01
* all Ns are weighted and rounded
KWebber HEISEE 2013
Results from NSOPF:04
Time Allocations and Salary by US- V. Foreign-Born Status
Born in US
N=2990*
Mean
Foreign-Born
N=1190
SD
Mean
SD
p
% Time on Research
32.83
22.205
39.43
21.926
**
% Time on UG Instruction
30.96
26.839
24.47
23.163
**
% Time on Graduate Instruct
22.37
19.834
23.56
17.183
% Time Other Activities
13.85
12.880
12.55
12.437
**
227.76
269.761
184.70
206.012
**
Base salary
$78,277
$34,453
$79,936
$33,677
Total annual income
$95,726
$46,423
$94,540
$43,154
Student Credit Hours Generated
Annual Salary
KWebber HEISEE 2013
Growing Discussions About
Postdoctoral Researchers
KWebber HEISEE 2013
Study on postdoctoral researchers
(Yang & Webber, 2013)
• Does the choice to take a
postdoctorate research experience
affect career path?
• Does taking a postodoctorate
research experience affect
productivity or salary a decade later?
KWebber HEISEE 2013
Results on postdoc study
• Taking one or more postdoc appointments
significantly increased likelihood of going to
education sector and earning a tenure-track
faculty appointment
• Take one postdoc increases productivity, but
two or more does not add more
• Taking postdoc does not affect salary 10
years later
KWebber HEISEE 2013
Questions, Discussion
• Questions on IPEDS?
• What are the most pressing questions
that you have that would be better
answered with data similar to IPEDS?
• What are the obstacles to interinstitution data collection in Croatia?
KWebber HEISEE 2013
Critical Elements
• Good data infrastructure
• Data definitions to ensure consistency
KWebber HEISEE 2013
This may be of interest
Principles and Practices for a
Federal Statistical Agency
Constance F. Citro and Miron L. Straf, Editors
Committee on National Statistics; Division on
Behavioral and Social Sciences and Education;
National Research Council , 2013.
KWebber HEISEE 2013
Principles for a federal statistical
agency (Citro & Straf, Eds., 2013)
•
•
•
•
Relevance to policy issues
Credibility among data users
Trust among data providers
Independence from political and other
undue external influence
KWebber HEISEE 2013