A critical study about the faculty members' evaluation models

Akram Gazerani¹, Atefeh Dehnoalian², Ali Gharahzadeh³, Farin Tatari⁴*‎

¹Instructor, Department of Operating Room and Anesthesia, Neyshabur University of Medical Sciences, Neyshabur, Iran. ² Instructor, Department of Nursing, Neyshabur University of Medical Sciences, Neyshabur, Iran. ³Instructor, Department of Operating Room and Medical Emergencies, North Khorasan University of Medical Sciences, North Khorasan, Iran‎.^4*Assistant Professor, Department of Public Health, Neyshabur University of Medical Sciences, Neyshabur, Iran.

Correspondence: Farin Tatari; Assistant Professor, Department of Public Health, Neyshabur University of Medical Sciences, Neyshabur, Iran‎.

E-mail: tatarif2 @ nums.ac.ir

Introduction  

Higher education organization and universities are of institutions and systems where human resources are of a particular importance ^[1], and faculty members are the most important element of academic institutions that have the duty of training of specialized staff, and are responsible for providing scientific, and research services ^[2].

The growth and development of higher education institutions depend on the desirable efficiency and effectiveness of faculty members, and the poor performance lead to a crisis in the evolution and development of the academic institutions and finally deleting them from the arena of competition ^[3]. Thus, the evaluation of the faculty performance is one of the basic activities of managers in universities, and has an important role in correct planning, successful implementation of training programs, and improvement of the quality of university performance ^[4].

The performance of faculty members, especially in recent years increasingly has been considered because universities are experiencing the clear change in paradigms. The indication of this change in paradigm includes the reduction of the organizations’ ability to obtain resources, and increment attention to the return on investment is done at the universities and institutions of higher education ^[5]. Thus, the design and implementation of the evaluation process and ensuring the desirable quality of the faculties’  performance in order to avoid wasting human and material resources and the ability to compete in the future where quality is the most important component for continuing  the life of any organization has the undeniable necessity ^[6]. There have been different definitions of evaluation in higher education systems, faculty members’ evaluation is an organized and continuous process for describing, directing and ensuring the quality of their activities. In other words, through evaluation the degree of individual success can be assessed in achieving the goals and mission of the organization ^[7]. One of the most important goals of performance evaluation is decision making about the future job of individuals. Evaluation can be transparent career path and identify the staff training needs. It can also detect inefficient people and guide on how to manage them ^[7]. According to the experts’ opinions, doing research on the performance evaluation of faculty members is important for the following reasons: collect information to understand and improve the performance of faculty members and judge on their performance quality ^[8]; encourage good performance; response to inform people about the quality of their performance ;educational needs assessment ^[9]; quantitative measurement and evaluation of faculty members’ educational, research and clinical activities; decision on the extension of contract, faculty members’ promotion or tenureship, accountability and merit-based payment systems. Faculty members' performance evaluation is one of the most complex types of evaluation ^[10], especially the faculty members of medical sciences’ evaluation system due to the widespread clinical activities in academic medical centers and their role in the health system is more complex compared to the other sciences ^[11]. Higher education system requires strategic model in the field of evaluation management, which can evaluate all faculty members' activities, provide a clear picture of their performance and lead universities to achieve their goals and missions. In medical science institutes, the evaluation is focused on teaching quality assessment that is one aspect and of course the key aspect of faculty members' activities, but in these institutions, not only research but also clinical services should also be considered. It is noteworthy in medical sciences universities in the world; effective models for faculty evaluation have considered all faculty members' activities in evaluation ^[12]. Due to the importance of evaluation, the researchers decided to conduct a critical review about the faculty members' evaluation models.

Materials and Methods

Preliminary review showed that a number of titles are non-research documents, and for such cases, reviewing methods are usually used. Critical review approach was used to achieve a degree of innovation and conceptual synthesis through which the literature was first extensively searched, and then their quality was critically evaluated. The evaluation of the articles was beyond the mere description and the analysis of texts was done, and involved a degree of conceptual innovation. After a brief survey, the articles were ranked based on the conceptual and contextual richness in relation to the subject of the research, and the deep and rich study of the articles began ^[13]. According to the study topic, the related vocabulary and structured texts were selected, and extensive search was conducted, and 262 texts and articles were achieved at this stage. Then, according to purposive sampling, screening was conducted in three stages:

(A) Initial or title screening, based on the titles, which were relevant to the management of faculty members' evaluation. (B) Secondary screening, which was performed according to the study of abstract and the introduction of articles. (C) Tertiary screening: At first, the articles were briefly studied, and the related articles were prioritized based on the conceptual and textual richness on the subject of research, and the irrelevant articles were excluded from the study. Then, a full and deep study was begun from the richest articles. 41 articles were analyzed.

Results

For faculty members' evaluation many goals were enumerated which helped them to improve and revise educational methods and activities; helped managers make more reasonable decisions about the teachers’ hiring, promotion and other matters related to them. There have been different methods and information sources for faculty members' evaluation including peer evaluation, self-assessment, evaluation by students, peer observations of teaching, evaluation of portfolios, evaluation of exam questions and students’ scores. Also, there have been various methods for data gathering from sources such as observation, interview and questionnaire ^[14]. Among the mentioned methods, the evaluation by students has been the most common method for faculty members' evaluation. Nowadays, more than 85% of faculty members' evaluation systems commonly use this method. However, there have been many disagreements on the use of students' opinion for faculty evaluation. Some people believed that students' judgment are subjective and have no credibility; and personality traits, environmental characteristics and factors such as students expected grade, time of class and time of evaluation can distort the validity of evaluation ^[15-17]. Researches have shown if the evaluation forms used which are made on the basis of professional psychometric process and the necessary points to be considered in the evaluation process, students can present reliable and valid judgment about educational efficiency and effectiveness. Evaluation by students should not be more than 30-50% of total teaching evaluation ^[18].

In the literature world associated with the faculty members' evaluation, the issue of individuals' resistance against the evaluation has had a central aspect. One of the major factors of this resistance is the lack of participation of faculty members in designing the evaluation process ^[10].

In universities throughout the world, different performance evaluation systems have been applied some of which include the following:

In Sam Houston State University, faculty members' evaluation was implemented with a disciplined and comprehensive approach. The system was designed to increase the objectivity and reduce the bias. There have been criteria in this approach including the effectiveness of teaching, research and professional effort, professional growth and non-educational activities. Each of these criteria has been weighted ^[19]. In University of North Carolina, the underlying philosophy of faculty evaluation model was that the evaluation is a complex process and model that should promote the success and satisfaction of faculty members and the mission of the university. All activities of faculty members in three areas of education, research and services would be evaluated. Because of the importance of education, 50 to 70% by weight belonging in this area, and weight of research and services were 10% to 40 % ^[20].

In the University of Georgia, faculty members' evaluation was done annually with the philosophy that the evaluation should be more than mere evaluation. Evaluation goals included flexibility; encourage professional development and efforts to improve the university standards. Faculty members' performance evaluation was done based on two main criteria; evaluation of educational, research and services activities, and evaluation of developmental activities. To maintain flexibility, there were 5 sets of job descriptions (traditional, educational, research, services and management) with objective goals and expectations. Faculty members could choose their preferred set on the basis of interest, maintaining a proper balance of work in the department, and mission of the university. Quantitative and qualitative evaluation of the effectiveness of faculty members’ performance in the area of education was based on the existing documents, results of self-assessment, peers evaluation, and evaluation by students ^[21]. In this university, maximum weight (5o %-85%) was allocated to the role of education, and the weight of research and services was 35% and 5% -25%. The weight of each component had an impact in the definition, its general role was important. The weighting as a Composite Role Rating (CRR) was extracted from the information obtained from various sources such as students, colleagues, dean and graduates. By calculating the Overall Composite Rating (OCR) for each faculty member an index of personal success was achieved that was used in decisions related to the promotion, tenure, continuing contract, merit pay, and post recruitment review ^[5].

At the University of Florida, each faculty member should have been evaluated at least once a year for the duties and responsibilities as a member of the academic community. Areas of performance that should have been evaluated included teaching, research, service (such as participation in university, colleges and departments management committees) and other activities (such as academic consultations). Information sources for the evaluation of faculty members, included department managers, faculty members, students, other university authorities and school staff ^[22].

At the University of Minnesota, merit review system for the faculty members' evaluation was implemented. The goals of evaluation in this system included:

1. Continuous quality improvement and productivity of faculty members through follow-up, evaluation and performance feedback
2. Optimal coordination of faculty members' activities to achieve departmental and individual goals
3. Providing the basis for determining salary increases

In this system, seven central areas were considered for the faculty members' activities that included:

1. Education: activities including teaching assistants, teaching in the university ordinary courses, counseling students, teaching or participating in continuing education courses, collaboration with other faculty members to develop courses and decisions about the educational programs.
2. Research: activities including conducting research, publications, attracting research funding and financial support, inventions and discoveries, edits and review articles.
3. Patient care: including the direct provision of health services to patients, and investigating the cases.
4. Organizational management: activities including the participation or leadership in managing the unit, department, college or university; planning; individual and group management and selection.
5. Professional development: activities including participation in professional communities and scientific associations.
6. Personal development: activities including learning personal skills such as updating scientific, teaching or clinical knowledge; participating in continuing education courses or achieving higher degrees; preparation for obtaining degree or its extension.
7. Citizenship: activities including the support of the general objectives of the institution, communication, willingness to teamwork, and timely and orderly attendance.

The annual performance portfolio consisted of four main components:

1. Next year goals and cooperation report: This part began with the goals which faculty had determined for the next 5 to 10 years. The goals helped in the responsibilities and positions that would be given to him and planed for the consistency with the determined goals. In the next section, the faculty member imported major activities that he had done in any area last year and had spent more than 5% of his time. This section contained the main points, and the percentage allocated to each area of activity was also determined. These activities shall be coordinated with the goals determined for him last year. In the next part, the faculty member was assessed according to the determined goals for the next year what portion of his activities would allocate for the area of activity.
2. Standardized reporting: had a special framework that according to its information, the faculty member recorded the components of his activities.
3. Summary of evaluation and performance data: In this section, the faculty member provided a summary of the results of his activities’ evaluation. These results included the results of evaluations carried out by students, residents, patients and managers.
4. Documents about the activity in each area: In this part, all of the documentation that would prove how much of his time and effort has been spent doing the activity in each area, was attached.

Evaluations’ criteria of quality of work in each areas of performance, based on the evidence presented, had 1 to 10 points. The scoring was entirely criterion-referenced. The quality of the products and individual performance was judged based on the standard already agreed on. At the beginning of each year, the portfolio performance was presented to the review committee. The committee was composed of six faculty members in each department, that 2 people were selected by the head of department, and 4 people were determined by the group members. The committee focused on the quality of information provided as the documentation on the seven areas, and scoring was applied to all areas both the scores and notable points. The head of the department reviewed the scores and notable points with the faculty members’ standard report and also the performance scores in the area of the care of patients, organizational management and the principles of citizenship that were provided by the superior authority. Then, he determined the final score for each area. In a meeting held at the end of every year, the head of the department shared his own scoring, as well as the committee scores with the faculty members. The goals and time assigned to each area of activity for the year ahead were discussed and agreed. The scores in this system were used for various aims such as the decision to increase the base or salary, the decision for granting scholarships or study travel costs and changes in faculty post and responsibilities ^[23].

Medicine schools of East Carolina University; West Virginia University; University of California, San Diego; University of Galveston, Texas; Creighton University; University of Maryland and University of Wisconsin use Relative Value Unit (RVU) to measure faculty members' activities that is a mission-based management program.

Association of American Medical Colleges (AAMC) emphasized on the use of evaluation based on the mission. The association has defined the mission-based management as a process for organizational decision-making that was taken from the mission, guaranteed the internal accountability distributed the resources in line with organizational goals, and was based on accurate, transparent and timely information. An important aspect of the mission-based management was the ability to measure the activity of faculties and departments that participated in the mission of the schools. The mission-based reporting system measured the quantitative and qualitative faculty activities in the four missions of clinical practice, research, teaching and administrative activities. System information was designed in three parts, a part of activities for faculty member to enter the annual activity, an evaluation part for the qualitative evaluation of the performance, and an automatic report generator. Each of the three parts according to the four missions of clinical services, research and creative work, education and administrative services were divided into smaller parts. This system was a self-report system whereby each faculty member entered his information (quantitative and qualitative) based on the mission, and immediately observed the relative value of activities.

Faculty members entered the information review and later, were confirmed by the head of department during the faculty members' annual professional evaluation meeting. Mission-based reporting system calculated an estimate of the time spent on each activity based on the entered data by the faculty member in the activity section by using RVU codes that have been placed in the program. The activity scores for each mission were summed. Then, the points were collected in each mission, and its percentage of the total points earned was transferred to the final report. In the final report, expected activities or "target", that was signed before was compared in each mission with actual activities entered by the each faculty member ^[24].

Discussion and Conclusion:

In the new paradigm, being a faculty member is beyond the professional activity. It means this profession requires an individual who plays the variety of roles that need professional competencies and skills in one or more fields. This new meaning has caused universities to revise their old evaluation system to evaluate these different roles, and create or modify a new evaluation system. It is necessary for a comprehensive evaluation system performance of faculty members in all roles to be evaluated, which requires the different roles of faculty members and each role components to be defined and the appropriate weight for them to be determined. In this case, the researchers were able to evaluate all faculty members' activities. The researchers agreed with Areola and believed that the universities should design and implement the evaluation system in the following steps:

1. Determining the role of faculty members
2. Determining the value and weight of each role based on the importance of roles in the university
3. Defining roles
4. Determining the weight for each of the constituent elements of roles (activities)
5. Reach consensus about the impact of the declared appropriate weight according to the source
6. Determining the source of information
7. Designing the data collection forms
8. Implementation evaluation and attention to the validity and reliability of information ^[5].

It seemed that universities have been trying to increase the objectivity of evaluation, consider the effectiveness of criteria, and reduce the bias. Most universities have been using various sources for evaluation. It should be acknowledged that the evaluation performance of the faculty members is one of the most complex types of evaluation; thus, a combination of information sources and evaluation methods should be used for obtaining a clear picture of the performance and final judgment. Sources of information should be accurate, and first-hand information about faculty members' performance should be available.

One of the successful models in evaluation of faculty members has been Minnesota university model (merit review system). The strengths of this system included reviewing all faculty members' activities, considering the value of each activity, and using the evaluation results in appointment, promotion, and payment decisions.

This system led to optimal coordination of faculty members' activities for achieving departmental and individual goals. In addition, the evaluation system used to determine salary increases created a fair compensation system and facilitated the achievement of departmental goals. In the other words, the strengths of this evaluation system had a strategic vision, and linked the individual merits to promotion and compensation system.

This system considered all the central areas of faculty members' activities, and accelerated   achieving the individual and departmental goals by providing feedback. Annual performance portfolio helps responsibility that is given to faculty member which is consistent with the determined goals. This system uses the performance data from various and diverse sources for evaluation, and provides a clear and complete picture of the faculty member’s performance. One advantage of this system is that scoring is not based on opinions and personal preferences of evaluation committee or the comparison of individuals with each other; rather it is a criterion referenced and makes more fair judgment possible. In this system, evaluation is quantitative and qualitative that is complementing each other.

Mission based evaluation is also one of the evaluation models of medical universities' faculty members that is RVU based. This model tries to integrate information related to various aspects of faculty members' mission and measures quantitative and qualitative efforts of faculty members from the aspects of education, research, clinical, and management services. The aim of this system has been providing a reporting tool for use in evaluation of the faculty member and departmental performance. The main motivation of mission-based management thinking has been the need for optimized information for better strategic decisions in the whole institution.

Mission-based management as a way to understand the costs and revenues related to the multiple missions of medical school; aligning expectations of educational groups; make them accountable and provide clear data.

This system creates an appropriate organizational structure to collect information of activities and help in dynamic interaction of senior management with faculty members by identifying the cost of each mission and measuring the activity of faculty members in each mission as well creates accountability of faculty members and departments in connection with the university's mission, transparency and confidence ^[25].

Traditional organizations only use relative value unit based system for measuring the faculty member’s activities, and the faculty member performs academic activities each having the effective equivalent.

In this system, the faculty member was expected to do a certain number of units considered for him. The system was able to measure the volume of faculty member’s activity, but the fundamental problem in RVU system was not completely aligned with the institutional mission and this has led to merit based system. In mission based evaluation system, it was possible that the faculty member was focused on a specific activity, therefore a number of activities were not performed, and the basic institutional mission did not come true. But in mission based organizations such as universities that are dominant merit based systems, tasks and activities of faculty members are regulated align with the mission and goals of the organization as well as the faculty member’s individual interests in the form of academic contracts, and the faculty member should be accountable to the department against commitments undertaken at the beginning of the academic year. Merit based system is not in violation of RVU based system. RVU is necessary for the equivalent of faculty members’ activities but encouragement, punishment and value judgments about faculty members should not be made solely on the basis of RVU ^[24].

According to what was expressed about faculty members' evaluation, in this study, the researchers emphasized and believed that faculty members' evaluation system should be dynamic and efficient, and respect the following issues:

1. Philosophy and goals of the model, should aligned each other and also align with the organization's mission and general goals, and should be developed realistically, achievably and scientifically.
2. The different roles of faculty members and activities in each role should be defined, and the appropriate weight for them should be determined. In this case, all faculty members' activities can be evaluated.
3. Both quantitative and qualitative evaluation should be carried out by various methods and from different sources.
4. Evaluators and faculty members should be fluent in all stages of the evaluation which can be created by methods such as justification courses for newcomers; justification courses for time appointment of managers about evaluation interviews; and induction of faculty members considering  performance evaluation system, needs assessing and counseling.
5. Cooperation and active participation of all evaluation parties should be considered.
6. The strategic committees of faculty members' performance evaluation should be established with the authority in staff affairs such as appointments, encouragement, punishment, payments, etc.
7. Meta-evaluation system should be created because only after meta-evaluation and ensuring the accuracy of the individuals' evaluation results, the results can be entered into a reward or promotion system.
8. Merit based system is a way that can enable universities provide more comprehensive, fair and complete evaluation.

References

1. ‎‎‎Ghofrani M. University autonomy and dynamism.  The first conference of Third Plan of ‎Country Development 2002.‎
2. ‎Ahmady S, Tatari F, Yazdani Sh, Hosseini SA. A Comprehensive Approach in ‎Recruitment and Employment Policies for Faculty Members: A Critical Review. International ‎Journal of Medical Research & Health Sciences. 2016; 5(12):356-64.‎
3. ‎Ahmady S, Tatari F, Yazdani Sh, Hosseini SA. Human Resources Management models ‎for Recruitment of Faculty Members: A Critical Review. Int J Med Res Health Sci. ‎‎2016;5(12):356-64.‎
4. ‎Ahmady S, Faculty development in medical education: A comprehensive approach: ‎Karolinska, Institutet University of Stockholm; 2009.‎
5. ‎Chambers D, Boyarsky H, Peltier B. Development of a mission-focused faculty ‎evaluation system. Dental Education. 2003; 67(1):10-22.‎
6. ‎Areola R.A. Developing a comprehensive faculty evaluation system. Third ed. ed: ‎Anker; 2007. ‎
7. ‎Bazargan A. Educational evaluation: concepts, models and operational process. Tehran ‎SAMT; 2009.‎
8. ‎Beran T, Rokosh J. Instructors' Perspectives on the Utility of Student Ratings of ‎Instruction. Instructional Science: An International Journal of the Learning Sciences. 2009; ‎‎37(2):171-84.‎
9. ‎Rifkin T. The status and scope of faculty evaluation: ERIC Clearinghouse for ‎Community Colleges Los Angeles CA; 1995 [Available from: from: ‎http://www.ericdigests.org/1996-1/status.htm.‎
10. ‎Peterman F. Designing Performance Assessment Systems for Urban Teacher ‎Preparation. New Jersey Associates: Lawrence Erlbaum 2005.‎
11. ‎Tootoonchi M, Changiz T, Alipour L, Yamani N. Faculty Members’ Viewpoints towards ‎Teacher Evaluation Process in Isfahan University of Medical Science. Iranian Journal of ‎Medical Education. 2006;6(1):23-30.‎
12. ‎Kirkpatrick J, Richardson C, Schmeiser D, Schafer K, Valley J, Yehle K. Building a ‎case for promotion of clinical faculty. Nurse Educator. 2001;26(14):178-81.‎
13. ‎Yazdani S. Comprehensive system of evaluation of faculty and academic governance ‎Tehran: Shahid Beheshti University of Medical Sciences; 2014 [Available from: ‎www.iautmu.ac.ir/education/Integrated.pdf.‎
14. ‎J.Grant M, Bootht A. Atypology of reviews:an analysis of 14  review types and ‎associated methodologies. Health Information and Libraries Journal. 2009;26(2):91-108.‎
15. ‎Fischer JD. Implications of Recent Research on Student Evaluations of Teaching Social ‎Science Research Network (SSRN) 2006 [Available from: http://www.ssrn.com/en/. ‎
16. ‎Rampichini C, Grilli R, Petrucci A. Analysis of university course evaluations: from ‎descriptive measuresto multilevel models. Statistical Methods & Applications. 2004; ‎‎13(3):357-73.‎
17. ‎Toyoda H, Nakamura K. Reliability of students' evaluations of university teaching: an ‎analysis of four-facet data by a generalized model and structural equation modeling. Shinrigaku ‎Kenkyu. 2004; 75(2):109-‎
18. ‎Fitzpatrick JL, Sanders JR, Worthen BR. Program evaluation: alternative approaches and ‎practical guidelines. 3rd ed. ed. Boston: Allyn & Bacon; 2003.‎
19. ‎Tatari F. A survey on validity and effective factors of faculty evaluation by studens. The ‎First National Congress of the Challenges and strategies for developing student participation in ‎educational system; 2014; Iran, Mashhad.‎
20. ‎University of Sam Houston State. Faculty Handbook USA 2013 [Available from: ‎http://www.shsu.edu/dept/academic-affairs/faculty-handbook ‎
21. ‎University of North Carolina. Faculty Handbook North Carolina2013 [Available from: ‎https://www.ncsu.edu/faculty.‎
22. ‎University of Georgia. Faculty Handbook 2014 [Available from: ‎http://www2.gsu.edu/~wwwfhb/fhb.html.‎
23. ‎University of Florida. Faculty handbook 2012 [Available from: www.ufl.edu/faculty.‎
24. ‎University of Minnesota. UMC Faculty & Staff Online Handbook Minnesota2009 ‎‎[Available from: http://www3.crk.umn.edu/facultystaff/facstaffhandbook.htm.‎
25. ‎Mohammadi A, Mojtahedzade R, Souri M. Mission based management theory to ‎practice. Tehran ,Tehran University Medical Science(TUMS) 2009‎.