Transcript Stylish - Educational Innovation
Soft Skills Systems / Teaching / Assessment
Kostas Kechagias
Presentation in 2 nd MASS meeting
Work Package 4, Needs Analysis / Piloting June 2010
Contents
• • • • • Skills and Skills Frameworks Teaching Skills and Competences Skills Assessment Experimental Methodology Conclusions/Homework
Skills and Skills Frameworks (1/13)
• • • …towards definitions (DeSeCo) great interest for identification of common skills must be connected to success in life the definition should address how they are used • Definition:
skills or abilities individuals need in order to achieve success in life, within the context of their socio-cultural milieu, through adaptation to, shaping of, and selection of environments
Often they are used interchangeably or in a vague sense
Skills and Skills Frameworks (2/13)
• • • • a proliferation of efforts to develop lists of skills, knowledge, and competencies necessary for success in the workplace and society “basic skills” are not sufficient for success in the workplace …“employability skills” …“enabling skills,” “generic skills,” “core skills,” “key competencies,” “ essential skills,” and “necessary skills.” • methodology: start from a broad definition and then survey or observes workers, supervisors, and experts to determine what skills are common
• •
Skills and Skills Frameworks (3/13)
SCANS 1991, US president and Education Secretary, America 2000 seven skill groups • • Influence: Organizational Effectiveness and leadership • • • Group effectiveness: Interpersonal skills, negotiation and teamwork • Personal management: Self-esteem, goal setting/motivation, and personal and career development Adaptability: Creative thinking and problem solving Communication: Listening and oral communication Competence: Reading, writing and computation • The foundation: Learning to learn .
• • • •
Skills and Skills Frameworks (4/13)
Michigan
Center for Career and Technical Education, Michigan State University human relations skills • • • • • • • • Positive work attitude, Teamwork Listening Honesty and Idengrity Demonstrate initiative Handling pressure & tension Communication … conceptual skills technical skills
Skills and Skills Frameworks (5/13)
• • • • • Scottish Qualifications Authority, 2008 Problem Solving Communication Numeracy Information and Communication Technology Working with Others •
Component: Working Co-operatively with Others
• identifying, with support, your own role and the roles of other people, and helping to achieve a shared goal • to: analyzing the roles and behavior of others and adapting your own behavior to deal with the complexity of changing and challenging dynamics
• •
Skills and Skills Frameworks (6/13)
DeSeCo (Definition and Selection of Competencies) OECD, 1997 A competency is more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilising psychosocial resources (including skills and attitudes) in a particular context.
Skills and Skills Frameworks (7/13)
• • • Using Tools Interactively • • • use language, symbols and text interactively use knowledge and information interactively use technology interactively Interacting in Heterogeneous Groups • • • relate well to others cooperate manage and resolve conflicts Acting Autonomously • • • act within the big picture form and conduct life plans and personal projects assert rights, interests, limits and needs
• • • • • •
Skills and Skills Frameworks (8/13)
ALL (Adult Literacy and Life skills survey project) OECD, 2005 employability skills derived from job and task analysis and psychological theories of intelligence scientifically convincing theory and an established literature accompanying tradition of measurement could be learnt and, therefore, taught had a direct impact on the social, health, educational or economic life chances of individuals
Skills and Skills Frameworks (9/13)
•
European Qualification Framework
“Education and Training 2010” (2004) • • • • • • • • communication in the mother tongue communication in foreign languages mathematical competence and basic competences in science and technology digital competence learning to learn social and civic competences sense of initiative and entrepreneurship cultural awareness and expression
Skills and Skills Frameworks (10/13)
• MASS Framework we don’t have much information Manners Ownership of tasks Attendance Motivation Professionalism Work output Conduct in workplace Timekeeping Verbal Communication Organisation/ planning Team-working/ Respect Helping others Conscientiousness Ability to ask for help Adaptability/ Flexibility • don’t seem to fit in any studied framework
• •
Skills and Skills Frameworks (11/13)
No grouping of skills is provided Some skills may be fit in more than one categories in other skill frameworks, e.g.
• Motivation • Taking on new challenges, working hard to achieve goals, thinking of new ways to do things • Under SCANS contains elements from categories • • • Personal Qualities” (taking on new challenges) “Thinking Skills” (thinking of new ways to do things) Seems to be closely related to “Proffesionalism” • Working to a high standard, being consistent in attitude (not allowing emotions or personalities to influence you)
Skills and Skills Frameworks (12/13)
• • Most frameworks are concerning higher education graduates • One may argue that the target group is different, so the skills to be considered are different • however, this should be established in the grounds of empirical research of literature • The skills should be defined in such a way as to minimize overlap Failure to do so may have impact to teaching, but mainly to assessment.
Skills and Skills Frameworks (13/13)
• • • • • Questions: What is the framework on which the MASS skills are based?
What justifies the selection of the particular skills as the most important ones?
If there is not a given one, referred from the literature, what are the empirical studies that support the selection?
Are the skills chosen in such a way as the interdependencies are minimized? Is the selection justified from the view point of the assessment?
Teaching Skills and Competences (1/9)
• • • • •
Gonczi, 2003 (DeSeCo)
The best examples: active independent learning, especially in ways that simulated contexts they might experience in later life those that integrated the key competencies into the teaching of other material, rather than attempting to teach them as stand-alone entities seemed to be more success in student outcomes when the key competencies were made explicit in many instances, it was not possible to separate out the key competencies in the process of teaching and learning.
very few of the projects in vocational education specifically assessed the acquisition of key competencies directly. The most comprehensive attempt to assess them was via a portfolio of evidence which students collected for themselves.
Teaching Skills and Competences (2/9)
• •
Welsh & Canning, 2003 (SQA)
The prospect of the isolation of Core Skills, as a separately taught and assessed subject, would appear to be unpopular with candidates. From the literature, it can be seen that this is also unpopular with employers.
Completely separate delivery and assessment often not popular with candidates as it’s ‘tantamount to a return to school’
Teaching Skills and Competences (3/9)
• Shuman, et. all., 2005 (ABET) The choice of team task depends on the educational objectives • if the objective is to have students experience a specific aspect of teaming such as brainstorming, specific group exercises new ideas can be used • if the objective is for students to experience a broad range of team processes and behaviors, then more complex activities are suitable
Teaching Skills and Competences (4/9)
• Shuman, et. all., 2005 (ABET) Two guiding principles should be followed in choosing activities: fidelity and complexity • Fidelity: similarity of the training situation to the students’ present and future working conditions. • The higher the fidelity, the more superior the transfer of learning to the workplace • Complexity is defined by two subfactors: task interdependence and cognitive effort. The more complex the activity, the more team skills are required by the participant.
Teaching Skills and Competences (5/9)
• Shuman, et. all., 2005 (ABET) advantages and disadvantages: • • fidelity/high complexity activities: • most resemble real workplace” conditions, • typically are more difficult for the instructor to manage, • • resource intensive, time consuming for the student. lower in fidelity/ low complexity activities • typically more structured and easier to administer, • may be perceived as less relevant by the student, • • the experience having less of a learning impact. Finally, may not challenge the team nor provide the environment necessary for intense interaction among team members
Teaching Skills and Competences (6/9)
• • • • • • • • Teaching methods (Szul, 2002) Coaching Project Work (e.g. Newsletter) Portfolio Job-shadowing experiences Videotapes/Observations Interviews Case studies Role playing
Teaching Skills and Competences (7/9)
• • MASS Teaching seems to be standalone course • • justification needed Interdisciplinary 18 weeks of approximately 4 hours/week • partners should be consulted for the feasibility
Teaching Skills and Competences (8/9)
• • • The activities that the 1 st learning byte contains belong to the following categories: • • • • • • • Presentation:
4
times Group discussion:
2
times Directed Discussion (Class Discussion):
6
times Quiz:
2
times Observation: 1 time Individual work: 1 time Personal discussion with the teacher: 1 time Mainly teacher centered designed “Hopefully, your grades will be improved”
Teaching Skills and Competences (9/9)
• the design of the lessons seems a lot like the designs of the vocational training • • Question about their efficiency Excellent quality of materials, within the concept • • instruction based not project based question whether we train on the knowledge of soft skills rather than soft skills themselves • In some places, as in formation of student groups, more elaboration is needed
Assessment (1/14)
• • indicators and outcomes indicators are the means by which we can measure whether the outcomes have been achieved • achievements which may ‘indicate’ acquisition or progress towards an outcome Not all indicators will be suitable for all target groups, and some will be target group specific.
Assessment (2/14)
• Methods for collection of information on soft outcomes • • • • • • • Individual action planning, personal action planning and goal setting Reviews between trainers/assessors and clients to record soft outcomes Daily diary or personal journal In-depth reflection during or after the course Recorded observations of group or individual activities Presentation of material in a portfolio Tests
•
Assessment (3/14)
• requirements from the tools • • reliability Validity Requirements from the definitions • two ends • so general that it becomes totally abstract and refers more to a system of values than to abilities • less comprehensive in one’s ambitions and is content with assessing a few specific competencies in a necessarily limited context the
Assessment (4/14)
• Competencies do not exist in themselves; they can only be demonstrated when an individual is acting in a specific context.
• pieces of research have shown that the subject’s performance is largely dependent on the contents of the task, on any previous training the subject may have had, on the type of strategy resorted to, and on the formal characteristics of the task • the same tasks undertaken in different situations —such as work situations or job interview situations —can yield very different results
Assessment (5/14)
• • Teamwork (Baker, et al., 2005) (ALL Framework) Focus towards identifying the specific competency requirements of team members “competencies” a cluster of related knowledge, skills, and attitudes that affects a major part of one’s job • • • is correlated with performance on the job can be measured against well-accepted standards can be improved through training and development.
•
Assessment (6/14)
three types of competencies that are central for effective teamwork Cannon-Bowers et al. (1995) • • team knowledge competencies, the principles and concepts that underlie a team’s effective task performance • • • team skill competencies, learned capacity to interact with other team members at some minimal proficiency level adaptability, situation awareness, performance monitoring/feedback, leadership, interpersonal relations, coordination, communication, and decision making • • • team attitude competencies an internal state that influences a team member’s choices or decisions to act in a particular way Positive attitudes toward teamwork and mutual trust among team members
Assessment (7/14)
• • • ALL questionnaire for teamwork Evaluation of teamwork requires observation Target group: International, adults Examines • • • knowledge about team skills, attitudes toward teamwork historical experience with teamwork
Assessment (8/14)
•
Assessment (9/14)
Zhuang, et. all., & Roberts, 2008 (ETS) Questionnaire • • • • target at high school students (mean age 16.10) undertaking Ford PASS courses consists of reliable and qualitatively distinct factors involves a variety of methods has demonstrable validity evidence
Assessment (10/14)
• Contains • Self-report ratings (57 items) • • cooperation with others (15 items), advocate and influence (12 items), • resolve • conflict/negotiate (17 items), • guiding others (13 items).
• Situational judgment tests (SJTs) • eight scenarios describing various situations and required participants to evaluate the effectiveness of four reactions to each situation • Teacher ratings
Assessment (11/14)
• MASS assessment one might form the impression of an underlying assumption that each skill, can be assessed separately • • • • • Is it possible? Interrelated skills?
Some skills, as “motivation” is difficult to assess outside a particular framework Need for clarification of definitions?
Development of indicators?
Framework?
Assessment (12/14)
• the main assessment tool proposed is as self reference questionnaire • students mark their “grades” against each skill, every week • • They can see the grades they put to themselves in the previous weeks “aim being that your scores will have improved” biased toward to a false improvement of grades?
• What about the students that they do not (or they thing they do not) improve? – motivation for more or drop out?
•
Assessment (13/14)
three entities involved in the course, the students, the teacher(s) and the materials • • Student self evaluation Student evaluation from the students (peer evaluation) • • • • Student evaluation from the teacher(s) Teacher self evaluation Teacher evaluation from the students Material and course quality evaluation from the students • Material and course quality evaluation from the teachers.
•
Assessment (14/14)
questionnaires are convenient, but others, like portfolio or in depth interviews may be considered • • • the partners should decide about the aims and the methodology of the evaluation If we decide to confine ourselves in a given framework, the uses of readymade and tested tools are also possible other aspects of influence • self-esteem and self-image
Experimental Methodology (1/1)
• The proposed methodology is the classical experimental one • • • • Two groups, the experimental and the control one Initial Assessment Intervention Final Assessment • What about long term results?
•
Conclusions/Homework (1/3)
• Does MASS framework needs clarification and justification?
Is the proposed schedule 18 weeks X 4 hours suitable for the partners?
• • • Each partner should report on this Suggest modifications Two alternatives • All the same schedule – Comparable results • Different schedules – Test more than one variations of program to see which works best.
• • • • • • • •
Conclusions/Homework (2/3)
Teaching organization How many students How many classes What age What educational background Suggested period of teaching …what else?
Decision about assessment • • • • • Target of assessment Ages of students Assessment procedure: questionnaires only or something more?
Each partner studies and comments proposed questionnaires and maybe contributes with more …what else?
Conclusions/Homework (3/3)
Deadline?
What about end of June?