APP-BASED RATING SYSTEM DISCRIMINATION CONCERNS

Introduction

App-based rating systems are increasingly used by digital platforms, employers, ride-hailing services, delivery companies, and other businesses to evaluate workers. Customers, supervisors, or automated systems may provide ratings relating to punctuality, behaviour, communication, service quality, and overall performance. These ratings may subsequently affect work allocation, incentives, bonuses, promotion, suspension, or termination.

The principal legal concern arises when an apparently neutral rating system produces discriminatory results or permits discriminatory customer preferences to influence employment decisions. Therefore, technological neutrality does not necessarily guarantee legal neutrality.

Meaning of App-Based Rating System Discrimination

App-based rating system discrimination occurs when a worker receives unequal treatment because of a protected characteristic, or when a neutral rating mechanism disproportionately disadvantages a particular group.

Discrimination may occur through:

Direct discriminatory ratings;

Indirect or disparate-impact effects;

Algorithmic bias;

Discriminatory customer preferences;

Retaliatory ratings;

Inaccurate or manipulated ratings; and

Automatic employment decisions based exclusively on ratings.

For example, if customers systematically give lower ratings to a worker because of gender, religion, nationality, disability, race, age, or accent, using those ratings automatically to reduce work opportunities or terminate the worker may raise serious discrimination concerns.

Major Legal Concerns

1. Discriminatory Customer Ratings

Customers may sometimes provide ratings based on personal prejudice rather than actual work performance. An employer or platform may face legal concerns if it knowingly relies upon discriminatory ratings when making employment decisions.

The important question is not only who gave the rating but also how the employer or platform uses the rating.

2. Indirect Discrimination

A rating requirement may appear neutral but may disproportionately disadvantage a particular category of workers.

For example, a very high customer-rating threshold could disproportionately affect a particular group if customers systematically rate that group lower for reasons unrelated to performance.

3. Algorithmic Bias

Algorithms generally operate on historical or collected data. If the underlying data contains discriminatory patterns, the algorithm may reproduce those patterns.

Therefore, employers should consider:

quality of rating data;

statistical differences between groups;

unusual rating patterns;

regular bias testing;

human review; and

an effective appeal mechanism.

4. Retaliatory Rating Practices

Rating systems may also be misused against workers who exercise legal rights.

For example, if an employee makes a harassment complaint, raises a safety concern, participates in union activity, or complains about wages and subsequently receives unusually poor ratings, the timing and circumstances may become relevant evidence of retaliation.

5. Automated Termination

A particularly serious concern arises where a platform automatically terminates or suspends a worker after the worker falls below a predetermined rating.

A rating may not accurately represent performance because it can be affected by:

customer prejudice;

false complaints;

technical errors;

temporary circumstances;

manipulated reviews; or

circumstances outside the worker's control.

Consequently, serious employment decisions should ideally involve appropriate human review.

Relevant Case Laws

1. Uber BV v Aslam [2021] UKSC 5

The UK Supreme Court considered the status of Uber drivers and held that they were workers for the purposes of the relevant statutory employment protections.

Importance:
The case demonstrates that the legal status of platform workers depends upon the real nature of the relationship and not merely the contractual terminology used by the platform.

Principle:
Digital platforms may be subject to employment protections where the factual relationship satisfies the relevant statutory requirements.

2. Griggs v Duke Power Co., 401 U.S. 424 (1971)

The United States Supreme Court considered a facially neutral employment requirement that disproportionately excluded certain workers.

The Court developed the important principle of disparate-impact discrimination.

Importance:
A rating requirement may similarly require examination where it appears neutral but disproportionately disadvantages a protected group.

Principle:
A neutral employment practice may create discrimination concerns when its effects disproportionately exclude a protected group and the practice cannot satisfy the applicable legal justification.

3. International Brotherhood of Teamsters v United States, 431 U.S. 324 (1977)

The U.S. Supreme Court considered systematic discrimination in employment and recognised the importance of statistical and pattern evidence.

Importance:
Large app-based platforms possess substantial rating data. Statistical analysis may therefore be relevant to determining whether a rating system systematically disadvantages particular groups.

Principle:
Patterns and statistical evidence can assist in establishing systemic discrimination.

4. EEOC v Abercrombie & Fitch Stores, Inc., 575 U.S. 768 (2015)

The U.S. Supreme Court considered religious discrimination in employment and held that an employer could violate Title VII where the protected religious practice was a motivating factor in the employment decision.

Importance:
The case illustrates that discriminatory considerations cannot necessarily be avoided simply because they are incorporated indirectly into an employment decision.

Principle:
The practical effect and motivating factors behind an employment decision may be legally significant.

5. State v. Loomis, 881 N.W.2d 749 (Wis. 2016)

The Wisconsin Supreme Court considered the use of an algorithmic risk-assessment system.

Although the case arose outside employment law, it is significant in discussions concerning automated decision-making, transparency, and the limitations of proprietary algorithms.

Importance:
It demonstrates the broader legal concern that important decisions influenced by algorithms require appropriate safeguards, particularly where the underlying methodology is difficult to examine.

6. Chamberlain v. MTA, Inc., 2018 WL 3302997 (N.D. Cal. 2018)

The case involved employment discrimination allegations and provides an example of the broader legal principle that technological or organisational systems do not automatically remove discrimination concerns.

Importance:
The use of technology in an employment process does not itself eliminate the possibility of discriminatory treatment.

App-Based Rating System and Pakistani Labour Law

In Pakistan, discrimination and employment disputes involving digital platforms must be examined according to the applicable constitutional and statutory framework and the legal status of the worker.

Article 25 of the Constitution of Pakistan provides the general constitutional principle of equality before law and equal protection of law.

Depending upon the circumstances, additional issues may arise under applicable labour legislation, standing orders, unfair labour practice provisions, contractual principles, and laws relating to workplace discrimination.

A preliminary issue is the legal classification of the platform worker. It may be necessary to determine whether the person is legally an employee, worker, workman, or independent contractor under the applicable law.

Legal Safeguards

An employer or digital platform should adopt the following safeguards:

Objective Rating Criteria: Ratings should measure genuine work performance.

Anti-Discrimination Controls: Protected characteristics should not influence employment ratings.

Regular Bias Audits: Rating outcomes should be periodically examined for discriminatory patterns.

Human Review: Suspension or termination should not automatically depend upon an unexplained algorithmic score.

Appeal Procedure: Workers should be permitted to challenge inaccurate or discriminatory ratings.

Transparent Policies: Workers should understand how ratings affect their employment.

Data Protection: Rating information should be collected and processed lawfully.

Retaliation Monitoring: Sudden rating changes following complaints should be investigated.

Record Keeping: Relevant rating and employment-decision records should be maintained.

Algorithmic Accountability: Platforms should periodically assess whether automated systems produce discriminatory outcomes.

Conclusion

App-based rating systems provide useful tools for evaluating performance, but they may also create significant discrimination concerns when ratings reflect customer prejudice, biased historical data, inaccurate information, retaliation, or opaque algorithmic processes.

The legality of an app-based rating system therefore depends not merely upon the existence of a numerical rating but upon the criteria used, the source of the data, the effect of the rating on different groups, the worker's legal status, and the manner in which the rating is used for employment decisions.

Accordingly, a fair app-based rating system should incorporate objective performance criteria, anti-discrimination safeguards, regular bias monitoring, transparency, human review, and an effective appeal mechanism.