Considerations and Risks of AI in Employment Decisions
This article is part of our Artificial Intelligence Insights Series, written by McCarthy Tétrault’s multidisciplinary Cyber/Data team. This series brings you practical and integrative perspectives on the ways in which AI is transforming industries, and how you can stay ahead of the curve.
View other blog posts in the series here.
The tech world trend of artificial intelligence (“AI”) is becoming increasingly prevalent in employment matters. For example, employers are using AI to attract candidates, evaluate job performance and automate certain positions; however, there are controversies over the use of AI, including it resulting in the continuation of patterns of bias and discrimination in employment-related decision making and the potential of compromising privacy rights.
The area that has received much of the attention in recent years is the use of AI during the recruitment and hiring process. In a typical recruit cycle, one may encounter AI at multiple stages:
- Creating Job Descriptions: Augmented writing platforms can analyze databanks of job postings and internal demographics to suggest alternative language, with the goal of appealing to a greater proportion of the target audience of the hiring employer.
- Searching for Candidates: AI drives targeted advertising and can be effective in attracting suitable candidates to job postings they may otherwise have overlooked. For active job seekers, previous online activity will lead algorithms to push similar content. AI on platforms like LinkedIn can even target passive job seekers – individuals who are not consciously looking for a new position but would be open to opportunities – by analyzing their profile, recommending suitable job postings and encouraging certain connections. There are also new platforms using AI to make the job search more inclusive; for example, the Ontario Disability Employment Network’s new division, Jobs Ability Canada, assists individuals with disabilities to create profiles that are then intelligently matched to job postings harvested directly from the websites of companies subscribed to the platform.
- Screening: AI may dramatically reduce the time needed to sort through resumes and applications. Many companies use keyword-matching programs to pare down the candidate pool, while others use more complex software to compare applicants based on a variety of factors, including factors that the program itself has determined make successful applicants.
- Communicating with Applicants: Chatbots can be utilized throughout the process for tasks such as asking and answering basic applicant questions, scheduling interviews and sending reminders and feedback. This automation enables the recruiters to spend more time connecting with qualified candidates, while still providing a sufficiently engaging experience for all applicants.
- Assessing and Evaluating: There are various tools at the interviewing stage that purport to assess candidate competency or “fit”. As virtual interviews are becoming more common, vendors of video interview software have included vocal and facial analysis components to provide additional data to the final decision makers. Some companies have also turned to game-based assessments whereby AI takes the raw results of a “game” to make conclusions about a candidate’s cognitive skills and personality traits.
Risks of AI in Employment Decisions
While AI has undoubtedly led to increased efficiency, there are attendant risks when the technology is used to make employment decisions. The greatest concern in this context is discrimination. Despite hopes that AI could eliminate human failings, evidence has shown that bias still creeps into AI systems resulting in harmful, discriminatory impacts. For example, a recruitment tool may end up ranking one gender or ethnicity consistently higher than the other due to the sample of resumes that were provided to the system for training at the outset being overwhelmingly from one gender or ethnicity. While the sample resumes would be intended to identify successful candidates based on prior successful candidates, the tool would ultimately lead to systemic bias for being reflective of one specific gender or ethnicity.
AI programs can also be based on fundamentally flawed premises, an issue distinct from biased source data. For example, the idea that a person’s facial expressions are a reliable indicator of their abilities or likelihood of success in a role is questionable. Expression and tone of voice are both products of culture, health and context. Tools that analyze these features are really measuring how “culturally normal” a person is, not necessarily how suitable they may be for a job, resulting in discrimination against candidates who may deviate from the “typical” successful candidate.
Lack of understanding and transparency may result due to the use of AI. Where a recruiter does not adequately explain to candidates how a decision was made, unsuccessful candidates may not understand the basis of the decision. Candidates may, for instance, believe their human rights were offended due to a misapprehended understanding of why they were not successful.
There is also a general hesitancy around AI due to questions about how it collects, processes and distributes personal information. The level of risk or liability exposure will depend on how and where the AI is being used. Employers must therefore be mindful of the privacy legislation in their jurisdiction and anywhere else the AI may be operating (i.e. if it were used for a potential candidate in a different country, or if the specific tool stores personal information across borders).
The rise of AI in employment decisions and the corresponding risk of discrimination have been acknowledged across several jurisdictions. In the United States, the Equal Employment Opportunity Commission examined how AI and other technology is transforming the hiring process and offers guidance on how to ensure these tools are compliant with federal civil rights laws. Individual states and cities have also taken action.
The European Commission has proposed a comprehensive, risk-based framework to regulate the development, marketing and use of AI in the EU. Under the proposal, AI in the employment context is considered “high-risk”; that is, the draft regulation specifically notes that AI systems have the power to significantly impact individuals’ career prospects and livelihoods, and there is the risk that historical discrimination may be perpetuated by those systems. With the “high-risk” designation, employers using AI in recruitment would be subject to the requirements set out in Title III of the regulation, including data governance, technical documentation, record-keeping, transparency, human oversight and accuracy. We have taken a closer look at the proposed regulation in an earlier McCarthy Tétrault blog post.
Canada’s own response to AI has been primarily in the form of privacy law reform. Quebec’s Bill 64 will require companies that make decisions using personal information that are based exclusively on AI to inform the affected individuals. The federal Bill C-11 would apply more broadly, imposing transparency and explainability obligations on private organizations that use technology to replace or merely assist human decision-makers. Bill 64 received royal assent on September 22, 2021, while Bill C-11 died on the order paper. See McCarthy Tétrault’s Bill 64 Blog Series for further insight on the impact Bill 64 may have on your business and previous blog posts for additional details on Bill C-11.
Though innovation and artificial intelligence will likely continue to transform the way employment decisions are made, there are certain lessons to be learned from its use and attempted regulation thus far. First, transparency and explainability are key to using AI. To gain both internal and external confidence, employers must be able to communicate their use of AI to those who will be impacted by its decisions. Second, AI is not infallible. Even with a general understanding of the process behind AI-powered decisions, employers should undertake regular audits and impact assessments of their systems to ensure they continue to work as expected. Third and lastly, there is still a crucial role to be played by humans in the hiring process and overall decision making throughout the employment relationship up to and including termination. Beyond acting as a check of the AI and a final decision maker, a recruiter, manager or other supervisory employee can create personal connections that, as of now, are still out of reach for machines.
If you have any questions regarding use of AI in your workplace, please do not hesitate to contact a member of our Labour & Employment Law Group or a member of our Cyber/Data Group.
 Illinois’ Artificial Intelligence Video Interview Act has been in force since January 1, 2020, pre-dating the COVID-19 pandemic, and applies to AI that analyzes recorded interviews. Employers that utilize this technology must notify candidates, explain how the AI works and which characteristics it considers, and receive the candidate’s consent for the use of the technology;
New York City Council recently passed a bill, effective January 2, 2023, that will require employers to disclose to candidates their use of any “automated employment decision tools” in the hiring process. All automated employment decision tools will also be subject to a bias audit within a year of their use, the results of which must be made publically available. Violations under this bill will carry fines of up to $1,500 per instance;
The California Fair Employment & Housing Council recently proposed amendments to the state’s anti-discrimination employment laws that would impose obligations and liabilities on both employers that use, and vendors that sell or distribute, employment-screening tools or services that automate decision-making. The use of these automated-decision systems will be unlawful if it is intentionally discriminatory, but also if it is facially neutral in the event that it nevertheless has an adverse impact. Among other record-keeping requirements, the proposal would obligate employers and participating third-party entities to retain all data used to develop or apply algorithms in automated-decision systems.
To learn more about how our Cyber/Data Group can help you navigate the privacy and data landscape, please contact national co-leaders Charles Morgan and Daniel Glover.
- AI and its Role in the Anne Frank Cold Case
- AI in the Canadian Financial Services Industry
- An AI a Day Keeps the Doctor Away?: Regulating Artificial Intelligence in Healthcare
- Could AI get you sued? Artificial intelligence and litigation risk
- AI and Patent Law: Can AI Be an “Inventor”?
- Arbitration and AI – friends or foes?