Recommendations
on employment
problems

2.1 Co-created performance indicators

Guideline for employment problems / MAPPING FACTS: 2.1 Co-created performance indicators

Interventions and evidence explained

Most plausible interventions explained

Performance appraisal refers to a formal process, which occurs infrequently, by which employees are evaluated by some judge (typically a supervisor) who assesses the employee’s performance along a given set of dimensions, assigns a score to that assessment, and then usually informs the employee of his or her formal rating. Organizations typically base a variety of decisions concerning the employee partially on this rating (DeNisi and Murphy, p. 421).

Applying a system of performance indicators co-created by employer and employee

The Productivity Measurement and Enhancement System (ProMES) is a human resource intervention that provides feedback to members of an organizational unit via a measurement system constructed by the unit personnel. The unit constructs the system by defining their objectives, identifying productivity indicators for each objective, and developing utility curves or contingencies for each indicator, specifying the overall and relative value to the organization of different performance levels on each indicator (David, p. iii).

ProMES works according to a 4-step model:

Identify Organisational Objectives. In ProMES, these objectives are called products. A design team composed of supervisors, representative job incumbents and facilitators working together.
Developing Indicators To Measure These Products. The team is asked to think of measures that they would use to show how well they were generating their products.
Establishing Contingencies. A contingency can be defined as the relationship between the amount of the indicator and the effectiveness of that amount. This is important because it allows the organization to see how much of a contribution differing amounts of an indicator make it the overall product.
Creating a Formal Feedback Report. Effectiveness scores can be determined based on the contingencies of each indicator.

ProMES is an intervention that relies on feedback to let personnel know their levels of performance; this knowledge then serves as a tool that leads to more efficient and effective ways of performing tasks (Pritchard, 1990). The system is developed and agreed upon by both employees and management, and provides an overall index of productivity (David, p. 5).

Classic top-down performance appraisal

Traditional top-down performance appraisal systems are formal meetings between employer and employee (DeNisi and Pritchard, p. 271). Performance reviews in the classic way work with rating scales. [A common] intervention here has been rater-training. This includes a strategy of telling raters about so-called rating errors (e.g., leniency, range restriction, halo) and urging them not to commit these errors (Adler et. al.).

Selected interventions for comparison (defined as a PICO question)

For employees and employers looking to find out what led to a dispute related to being sacked or made redundant (fact-finding), is applying a system with performance indicators they have co-created or classic top-down formal performance appraisal more effective for well-being?

Search strategy

The databases used are: HeinOnline, Westlaw, Wiley Online Library, JSTOR, Taylor & Francis, and ResearchGate.

For this PICO question, keywords used in the search strategy are: Performance, review, management, appraisal, improvement, productivity, goal-based, relationships, dispute prevention, ProMES

Assessment and grading of evidence

The main sources of evidence used for this particular subject are:

Angelo S. DeNisi and Robert D. Pritchard: Performance Appraisal, Performance Management and Improving Individual Performance: A Motivational Framework (2006)
Robert D. Pritchard, Lawrence G. Weiss, Amie Hedley Goode and Laurie A. Jensen: Measuring Organizational Productivity with ProMES (1990)
Jose Hernan David: Identifying the Factors that Contribute to the Effectiveness of the Productivity Measurement and Enhancement System (ProMES) (2003)
Seymour Adler, Michael Campion, Alan Colquitt, Amy Grubb, Kevin Murphy, Rob Ollander-Krane and Elaine D. Pulakos: Getting Rid of Performance Ratings: Genius or a Folly? A Debate (2016)
Daniel Schmerling and Anne Scudato: Use the Best; Leave the Rest: The Productivity and Enhancement System (ProMES) for Performance Ratings (2016)
Rob B. Briner and Denise M. Rousseau; Evidence-Based I-O Psychology: Not There Yet (2011)
Dev Kumar; Performance Appraisal: The Importance of Rater Training (2005)

Quality of evidence and research gap

The study by DeNisi and Pritchard is a literature review about designing and implementing performance appraisal systems. The authors review trends and their genesis, and propose a motivational framework as a means of integrating what we have learned and generating proposals for future research that focus on employee’s performance improvement. This literature review is mostly based on expert observational studies. The strength of this evidence can therefore be classified as low.
The article by Pritchard, Weiss, Goode and Jensen is a literature review on productivity improvement strategies. The study is mostly based on observational studies and expert opinions. The strength of this article is classified as very low.
The article Adler et al. is based on expert opinions. It reiterates the pros and cons of retaining performance ratings. The study by David identifies the effectiveness of the ProMES system and is also largely based on expert opinions. According to the Recommendations Methodology, the strength of these articles are classified as very low.
The article by Schmerling and Scudato is a response to the article of Adler et al. They discuss the shortcomings described in performance ratings and performance management, as well as challenge the field to further consider this well-researched and established evidence based solution as a viable alternative. This study is based on observational and empirical studies, and according to the Recommendations methodology the strength of evidence is classified as low.
There is a significant gap in the research on fact-finding in an employment dispute setting. Performance management and performance appraisal are important topics within employment dispute resolution research and can provide an important role in fact-finding when a dispute arises on the workfloor. However, research on performance appraisal is very limited. More interventions need to be studied, which would in turn improve this PICO question and employment research as a whole.

According to the HiiL Actionable Recommendations document and the GRADE approach, overall evidence is graded as very low.

Comparing the two interventions

Desirable outcomes of the interventions

ProMES	Formal appraisal
ProMES allows the employees to see whether they are productive versus those aspects that need improving. “The single index [part of the ProMES] allows the effectiveness of a complex [team of people] to be summed into one easily communicable number” (Pritchard et al., p. 265).	Existing formal appraisal systems can be effective for people. “[is] … performance appraisal system perceived to be effectively serving the organisation’s current needs? Do employees view the process as one that is performance enhancing? Are the managers / supervisors effective in their role as performance raters? The answers to these questions are critical and require attention by those responsible for the appraisal system.” (Kumar, p. 13)
There is a bigger chance that ProMES is accepted by organizational personnel and increases employee buy-in. “A productivity measurement system cannot work unless organizational personnel accept it. Because of [personnel] involvement in building the system [including products, indicators and contingencies] they have to defend their work to higher management. This causes them to develop a sense of ownership of the system.” (Pritchard et al., p. 266).
ProMES enables employees to see the results of their efforts. “Most jobs are structured so that doing a better job does not show up in any measurable way. With ProMES, the connection between the actions of the unit and its level of productivity is improved by the frequency of the feedback. When an indicator does improve, there seems to be considerable satisfaction. This allows employees to see how their efforts have increased productivity.” (Pritchard et al., p. 266)
Employees get role clarification. “Through the process of developing, refining and getting approval for the products, indicators and contingencies, employees are helped to more clearly understand their roles. Through the discussions to develop the system, personnel determine what their objectives should be.” (Pritchard et al., p. 267)
ProMES provides information about the work to be done. “The system helps guide personnel by indicating which activities employees should be doing and their importance”. (Pritchard et al., p. 267)
During a test of the ProMES system, measures of both job satisfaction and morale among unit personnel showed significant improvements after the system was implemented. “Employees were clearer about their roles, objectives, and how their efforts would be evaluated by management”/ (Pritchard et al., p. 269) This can in turn make fact-finding easier.

Undesirable outcomes of the intervention

ProMES	Formal appraisal
Employers do not always act on information that derives from performance measurement systems such as ProMES “… organizations are reluctant to accept or act on this knowledge” [performance evaluation knowledge, such as the information following from the ProMES system] (Briner and Rousseau, p. 14).	Most employees don’t find performance appraisals to be valuable or motivating. Instead, they find performance appraisals and performance management systems frustrating, too bureaucratic, and often not relevant to their jobs (Adler et al. p. 221). Formal performance reviews are not only perceived to be of little value, they also can be highly demotivating to even the highest performing employees (Adler et al., p. 220).
	“The key reason traditional performance management approaches have failed to live up to their promise of enabling performance is that the mechanics of formal systems—how ratings are done, the documentation required, how goals are set—are inconsistent with the goal of providing frequent, credible, and useful feedback about performance.” (Adler et al., p. 220). Without ProMES, many people within the organization may not receive objective feedback on all their activities. More frequent, specific and accurate feedback enhances performance. Such feedback is possible with the use of the ProMES formal feedback system [not with regular performance appraisal] (Pritchard et al., p. 265).
	Formal performance reviews do not have added value according to managers. Managers spend a lot of time on formal performance management activities that they likewise believe add little value (Adler et al. p. 221).
	Formal appraisal procedures omit to inform people whether they are performing well. Many employees have reported that before ProMES, they often received feedback only when problems arose and almost never told when they were doing a good job (Pritchard et al., p. 265).

Balance of Outcomes

Taking into account the balance of outcomes, it is clear that employees benefit from a performance appraisal system that they have co-created themselves, such as the ProMES system. The outcomes of systems such as ProMES clearly outweigh those of top-down performance appraisal systems.

First and foremost, their roles and responsibilities will be more clear, which can contribute to better fact-finding. Secondly, a system such as ProMES enables employees to see the results of their efforts. Top-down, formal systems on the other hand often omit to show certain objective results to employees, which can be detrimental to fact-finding. Furthermore, formal, top-down performance appraisal systems without input from employees are considered to have limited value by managers.

Recommendation

Taking into account the balance of outcomes and the quality and consistency of the evidence, we make the following recommendation: For employees and employers looking to find out what led to a dispute related to being sacked or made redundant (fact-finding), applying a system with performance indicators they have co-created is more effective than classic top-down formal performance appraisal.