Exploring Evidence-Based Outcomes Through Self Reflection on Teaching Experiences

Presenters

Dr. Margaret M. McMenamin

Executive Vice President Educational Services Brookdale Community College [email protected]

Maris A. Lown

Dean Science & Health Science [email protected]

Avis McMillon

Manager Public Relations Chair of Governance [email protected]

Brookdale Community College

• • • • • Dr. Peter F. Burnham, President The County College of Monmouth Located along the Jersey Shore Collective bargaining environment Two-campus system • • >15,000 Credit students 260 Full-time faculty

Governance Structure

• • • • • Governance Forum Steering Committee Standing Committees Permanent College-wide Committees Governance Gazette

Governance Authority

• • • • Governance does not make final decisions Aids in policy formulation & dissemination Recommends policies to President Board of Trustees approves policies

Steering Committee

• • • • Leadership body of governance system Chair elected from among the constituent groups (administration, faculty, staff) Monitors committee membership Develops agenda for Forum meetings

Steering Committee, cont.

• • • • Develops charges for standing committees Routes issues to appropriate committees Manages a year round agenda/meetings Conducts year end retreat

Voting Constituencies

• • • • Full-time faculty Full-time and part-time staff Full-time and part-time administrators 14 students who serve on standing committees

Standing Committees

• • Academic Council • • Functions as the College curriculum committee Initiates/revises programs and courses Academic Standards • Addresses College wide academic regulations

Standing Committees, cont.

• Institutional Planning & Effectiveness • • • Assesses strategic planning process Guides formulation of strategic planning

Matrix

Monitors

Major Indicators of Institutional Effectiveness

Standing Committees, cont.

• • Professional Development • • Guides College-wide PD initiatives Coordinates new employee orientation with HR Student Development • • Addresses student behavior issues Addresses student extra-curricular concerns

Standing Committees, cont.

• • College Life • • Addresses campus environment Addresses non-academic services Diversity Council • Advises on strategies to foster an open, diverse, & inclusive environment • Plans & implements programs to achieve College-wide commitment to diversity

Reflective Practice

• Evaluation of experiences, beliefs and knowledge • Data used in conjunction with reflection to develop and verify insights • Practitioner understands and appreciates self within the context of learning • Reflection used to improve practice

Reflective Practice

• Pre-reflective – knowledge is certain • Quasi reflective – knowledge is uncertain but personal and interpreted by individuals • Reflective – knowledge is constructed from inquiry leading to evidence based reasonable solutions

Reflective Practice in Governance • • Creates time and space for inquiry • Inculcates a culture of using research and information • Educates faculty and staff regarding the larger issues through committee charg.

• Encourages faculty and staff to apply the research and information from governance to parochial issue

Reflective Practice in Governance • General education courses • Grading system • Credit to contact hour ratios

Governance & Assessment

• • • Middle States Mandate Governance formed ad hoc committee to develop College Core Competencies • Link with Program & Course Learning Outcomes • Link with General Education

Institutionalize Assessment

Core Competency Committee

• • Formed as ad hoc Governance Committee Developed the Core Competencies

Academic Council

• • Incorporated Student Learning Outcomes into course syllabi Implemented three levels of Student Learning Outcomes • • • Core Competencies Program Learning Outcomes Course Learning Outcomes

General Education Committee

• • Linked Gen Ed requirements to Core Competencies Created rubrics for assessment of these competencies

Institutional Planning & Effectiveness • Reviewed and revised

Major Indicators of Institutional Effectiveness

• • Created a dashboard of critical indicators Led college-wide planning & assessment efforts through

Matrix

development

Academic Leadership

• • Student Learning Outcomes Plan • • Initiated by Academic Dean Developed by Division Chairs Results • • • • Initial resistance from everyone Streamlined process Increased acceptance Improved quality of the outcomes

Assessment Buddies

• • • Fostered faculty buy-in In-serviced on how to write SLOs Taught how to set up assessment projects

Educational Services Master Plan • • Developed collegially Includes an entire Chapter devoted to supporting assessment of SLO

Forum Presentations

• • • • • Committee reports Assessment Buddies reports SLOP dissemination by Chairs Assessment project presentations ESMP/FMP/ITSP/

Matrix

presentations

Reflective Practice in Governance & Assessment • Reflection before action • Reflection in action • Reflection after action

Reflective Practice in Governance & Assessment • The Buddies • SLOs • The SLOP

Levels of Student Learning Outcomes

Core Competencies Program Learning Outcomes Course Learning Outcomes

Examples of Student Learning Outcomes Core Competency :

Communication

PLO:

Communicate the Economic Events in the form of General Purpose Financial Statements including Income Statements, Net Earnings Statements, Balance Sheets and Cash Flow Statements

CLO:

Analyze, measure and communicate the business transactions of a business entity to decision makers

Curriculum Maps

• Core Competency by Program • Core Competency by Course • Program Learning Outcome by Course

Foundation Assessment Questions

Level 1

: What do you want the students to

learn

?

Level 2

: What

tools

are you using to assess achievement of learning outcomes?

Level 3

: What

data

have you gathered regarding achievement of learning outcomes?

Level 4

: What

changes

have you made to your instruction/curriculum based on assessment data?

Level 5

: What

evidence

do you have that changes you made improved student learning?

Core Competency Data Sources

• • • • Graduate Surveys Employer Surveys Advisory Committee Surveys Threaded assessment of CCs in the Career and General Education Courses

PLO Data Sources

• • • • • • Employer Surveys Advisory Committee Surveys Student Program Evaluations Threaded assessment through CLO Assessment tools in capstone courses Embedded questions in comprehensive examinations

CLO Data Sources

• • • Internally developed rubrics Link instructional strategies to CLOs • • Thread assessment though CCC & PLO assessments “Blueprint” exams to reflect CLOs Standardize exams across the department

Artifacts

• Course Products – Exam blueprints – Rubrics – Research papers – Data • Memorialized in Division Offices and in the office of the Dean of Academic Affairs

Reflective Practice in SLO Assessment • Action Research – planning, action, reflection • Creates an inquiry culture • Integrates theory with practice • Researching while teaching

Reflective Practice in SLO Assessment • Speech – deconstruction • Human Services – social inquiry • Nursing – qualitative dialogue

Speech Assessment Project

• • •

Level 1: CLO

• Deliver demonstrative, informative, persuasive speeches

Level 2: Assessment Strategies/Tools

• Rubrics for the three types of speeches

Level 3: Data

• Data showed that grades on each speech decreased from the first speech to the third speech •

Level 4: Curriculum Change

• Speech sequence changed to informative, persuasive, demonstrative

Speech

• Does changing the speech sequence result in better grades?

• Will students take the assignment more seriously?

• Will students confidence improve?

Speech

Level 5:

Improved student ability to deliver demonstrative, informative, persuasive speeches Changing the sequence resulted in • Improved student performance in each type of speech • • • Grades improved with each attempt Confidence levels of students increased Students took the speeches more seriously

Human Services Assessment Project • • • •

Level 1

: CLO • Conduct an effective client interview

Level 2

: Assessment Tools • Simulated interview/Rubric

Level 3

: Data • Students scored lower on “Exploratory Questions”

Level 4

: Curriculum Change • Increased instructional time devoted to question development

Human Services

• • Are students effectively conducting simulated initial Client/Counselor sessions?

Will the use of Bloom’s taxonomy foster the use of exploratory questions?

Human Services

Level 5:

Awaiting fall 2008 results of student interview assessments to determine impact of increased instructional time devoted to question development

Nursing Assessment Project

• • • • •

Level 1: CLO

• Collaborate with clients, significant others, and members of the healthcare team to evaluate the achievement of client outcomes

Level 2: Assessment Strategies/Tools

• Focus groups

Level 3: Data

• Students will evaluate information for patterns and themes

Level 4: Curriculum Change Level 5: Did it work

Nursing

• Do individuals who work with entry level practitioners perceive that graduates have met program learning outcomes?

• Will students perceive that this learning experience strengthened their leadership skills?

• Will students confidence improve?

Future Projections on Assessment

• Engagement of Students • Narrative Inquiry • Qualitative Strategies • Triangulation • Dashboard/Top Box Model • Standardized Testing and Mandated Metrics

Deliverables: Shared Governance System

• • • • Results in demonstrable outcomes Improves documentation of student learning & institutional effectiveness Produces better assessment process Ensures buy-in from critical constituents

Deliverables: Shared Governance System

• Develops practitioners whose “knowledge of ill-structured problems is constructed from inquiry which leads to reasonable solutions based upon evidence currently available.” (King & Kitchener) • Builds a learning organization. (Senge)