Assessment/accreditation of under graduate Engineering Programs Prepared by Prof. Syed Ahmed M. Said (ME)

Presented by Prof. Ibnelwaleed A. Hussein (CHE)

Accreditation Bodies

 National Commission of Academic Accreditation and Assessment (NCAAA) KSA – Accredits Institutions and Programs  Accreditation Board of Engineering and Technology (ABET)-USA – Accredits Engineering Programs, Computing Programs, and Applied Science Programs. 2

Accreditation Cycle

   NCAAA – every seven years ABET – every six years KFUPM – every five years 3

Accreditation Criteria

NCAAA has 11 Criteria: 1.

2.

3.

4.

5.

6.

7.

8.

Mission and Objectives Governance and Administration Management of Quality Assurance and Improvement Learning and Teaching Student Administration and Support Services Learning Resources Facilities and Equipment Financial Planning and Management 9.

Faculty and Staff Employment Processes 10.

Research 11.

Institutional Relationships With the Community 4

1.Mission and Objectives

      Appropriateness of the Mission; Quality of the Mission; Processes of Development and Review of the Mission; Use of the Mission Statement; Relationship Between Mission and Goals; Relationship Between Mission and Objectives.

5

2.Governance and Administration

      Governing Body/Board of Trustees; Senior Management; Planning Processes; Internal Policies and Regulations; Organizational Climate; Associated Companies and Controlled Entities.

6

3.Management of Quality Assurance and Improvement

     Institutional Commitment to Quality Improvement; Scope of Quality Assurance Processes; Administration of Quality Assurance Processes; Use of Indicators and Benchmarks; Independent Verification of Standards.

7

4. Learning and Teaching

        Student Learning Outcomes; Educational Assistance for Students; Quality of Teaching; Support for improvements in Quality of Teaching; Qualifications and Experience of Faculty; Program Development and Review Processes; Field Experience Activities; Partnership Arrangements with Other Institutions.

8

5.Student Administration and Support Services

      Student Admissions; Student Records; Student Management; Planning and Evaluation Student Services; Medical and Counselling Services; Extra Curricular Activities for Students.

9

6.Learning Resources

    Planning and Evaluation; Organization; Support for Users; Resources and Facilities.

10

7.Facilities and Equipment

      Policy and Planning; Quality and Adequacy of Facilities; Management and Administration; Research Equipment; Information Technology; Student Residences.

11

8.Financial Planning and Management

   Financial Planning and Budgeting; Financial Management; Auditing and Risk Assessment.

12

9.Faculty and Staff Employment Processes

    Policy and Administration; Recruitment; Personal and Career Development; Discipline, Complaints and Dispute Resolution.

13

10.Research

    Institutional Research Policies; Faculty and Student Involvement; Commercialisation of Research; Facilities and Equipment.

14

11.Institutional Relationships With the Community

   Institutional Policies on Community Relationships; Interactions With the Community; Institutional Reputation.

15

Accreditation Criteria

(Cont ’d) ABET has 9 Criteria: 1.

2.

3.

4.

5.

6.

7.

8.

9.

Students Program Educational Objectives Program Outcomes Continuous Improvement Curriculum Faculty Facilities Support Program Criteria 16

Accreditation Criteria

(Cont ’d)

2. Participants Standards: 1. Student Admission 2. Student Retention 3. Staff Sufficiency - Student Support 4. Faculty Sufficiency 5. Faculty Qualifications 6. Faculty Management and Support 7. Aggregate Faculty and Staff Education Responsibility 8. Individual Faculty Educational Responsibility 9. Student Educational Responsibility

17

Accreditation Criteria

(Cont ’d) 3. Assurance of Learning Standards

1. Management of Curricula 2. Undergraduate Learning Goals 3. Undergraduate Educational Level 4. Master ’s Level General Management Learning Goals 5. Specialized Master ’s Degree Learning Goals 6. Master ’s Educational Level 7. Doctoral Learning Goals

18

Key Definitions - ABET 2008-2009

Program Educational Objectives

(PEO) PEO are broad statements that describe

the career and professional accomplishments

that the program is

preparing graduates

to achieve.

Another definition - PEO are defined as

knowledge, skills, values, attitudes

early in their careers.

, or behavior that graduates must demonstrate

Key Definitions - ABET 2008-2009

Program outcomes:

 Are narrower statements that describe

what students are expected to know and be able to do by the time of graduation

.

 These relate to the skills, knowledge, and behaviors that students acquire in their matriculation through the program .

20

Key Definitions - ABET 2008-2009

Performance criteria (indicators

):  Are defined as the

specific, measurable statements

identifying the specific knowledge, skills, attitudes and/or behavior students must demonstrate

as indicators of achieving the outcomes

.

 Simply put, performance criteria are those statements that

define the learning outcomes and enable faculty to measure student competency

.

21

Key Definitions - ABET 2008-2009

Assessment:

Assessment is

one or more processes that identify, collect, and prepare data to evaluate outcomes

the achievement of program and program educational

objectives.

22

Key Definitions - ABET 2008-2009



Evaluation:

Evaluation is one or more

processes for interpreting the data

and evidence accumulated through assessment practices.

 Evaluation

determines

program

outcomes

the extent to which or program educational

objectives are being achieved.

 It results in

decisions and actions

the program.

to improve 23

Continuous Quality Improvement (CQI) :

 CQI is defined as

a systematic pursuit of excellence to satisfy the need of constituents in a dynamic and competitive environment.

 CQI is based upon

assessment of outcomes

and establishes the capacity/capability of the program that improves institutional effectiveness, learning, and accountability. 24

Stakeholders/constituents

The following group should be considered

 Faculty  Administration  Students  Alumni  Advisory Councils  Other Graduate Schools  Recruiters  Employers/Industrial Members 25

Assessment Tools/Methods

 Locally developed instruments  Alumni and employer surveys  Nationally standardized exams  Students portfolios and senior capstone course projects  External reviews  Student exit interviews  Focus groups  Students course evaluations  Graduate schools 26

Assessment Tools/Methods



Recruiters



Rubrics

using performance indicators to measure level of students performance  Performance on

certified exams

 Special students achievements such as

awards in various competition

, student publications and presentations at various meetings  Students internship evaluations  Writing proficiency exams 27

Use of Grades as an Assessment Tools

Most faculty members ask questions such as “We are already grading students, why do it again in a different form ” and “

why can ’t we use grades as assessment tool

, ” even though grades play an important role in this process. Because:  For a given course,

grade represents the faculty member ’s expectations from a student

for that given course. Successful students will receive high grades.

28

Use of Grades as an Assessment Tools

 Grades

do not demonstrate the information on student ability to perform at an acceptable level in each of the learning outcomes

.



Course contents may vary from section to section

if a number of faculty members are teaching the same course. Course contents are based on the topics each faculty member considers important, the amount of time available for faculty member to cover each topic, faculty member expertise interest, number and type of test, attendance policy and grade structure 29

Use of Grades as an Assessment Tools

 Grading policy for a course is defined by individual faculty member.

Some faculty members grade on a curve while others may have fixed standard

. Each faculty member grade exams differently, some may give partial credit for a problem while others may not give any credit.

30

Use of Grades as an Assessment Tools



Grades usually represent a student ’s relative standing within a class with respect to other students

.  Someone looking from outside will be unable to judge what the student knows or what topic a student understands or does not understand.

31

Guidelines for developing PEO

      All

constituents

should be involved in identifying PEO Numbers of PEO should be manageable PEO should be aligned with mission of university PEO should be measurable PEO should be assessed periodically using constituents PEO should be evaluated periodically to continuously improve the program 32

PEO common Problem areas

 Listing PEO

same as Program Outcomes

 PEO

not published

constituents or readily accessible to the 

Non measurable

list of PEO  PEO

not linking with Program Outcomes

 Limited or

no input from constituents

developing PEO in 

Lack of faculty support

33

Program Educational Objectives

Examples of PEO :

   Prepare graduates to

meet or exceed the expectations

of employers Prepare graduates to engage in lifelong learning to

maintain professional competency

Prepare graduates who will be prepared to in engineering and other professional fields

pursue and obtain professional licenses and advanced degrees

34

Program Outcomes

a.

b.

c.

Engineering programs must demonstrate that their students attain the following outcomes

: an ability to apply knowledge of mathematics, science, and engineering an ability to design and conduct experiments, as well as to analyze and interpret data an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, etc 35

Program Outcomes

d.

e.

f.

g.

h.

an ability to function on multidisciplinary teams an ability to identify, formulate, and solve engineering problems an understanding of professional and ethical responsibility an ability to communicate effectively the broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context 36

Program Outcomes

i.

j.

k.

a recognition of the need for, and an ability to engage in life-long learning a knowledge of contemporary issues an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice.

Plus any additional outcomes that may be articulated by the program 37

Program Outcomes

Performance Criteria/Indicators Performance criteria are those

statements that define the learning outcomes and enable faculty to measure student competency

.

 It is

not possible to cover every topic

in a course and it may not be possible to focus on every concept in a topic so faculty

should focus only on important concepts when creating performance indicators.

38

Program Outcomes

Performance Criteria/Indicators When using performance indicators, be careful

not to calculate average of all indicators for a given topic

. For example if communication skills is used as a topic, with presentation skills, sentence structure, grammar, organization as performance indicators,

taking an average and calculating one value for the entire topic will fail to point out the weakest area

that needs improvement.

39

Program Outcomes

Performance Indicators for Outcome (a): 

Understand given engineering problem



Identify the appropriate physics laws applicable to the problem



Apply appropriate knowledge of science, math, and engineering to correctly solve engineering problem



Draw appropriate conclusions

40

Findings

 NCAAA criteria are

very general and comprehensive

 NCAAA almost

covers, directly or indirectly, all the criteria of AACSB and ABET.

 All Higher Education institutions in Saudi Arabia are required to

adopt NCAAA criteria

institutional and program accreditations to get  Need to carry out

self-study that satisfies the accrediting

bodies .

41

Thank You

42