Interaction between human judgment and AI

Image

In both professional and private contexts, artificial intelligence (AI) helps many of us with decision-making. For example, we use robo-advisors to manage our savings, and AI tools handle parts of the recruitment process. The goal is often to bypass the so-called “human factor” and produce decisions, or recommendations, that are not influenced by human biases or personal opinions. This technology is referred to as Automated Decision-making Systems, or ADS.

Dina Koutsikouri, an associate professor of informatics at the University of Gothenburg and co-author of the article Human Judgment in the Age of Automated Decision-Making Systems, has studied the role of ADS in organisations, emphasising the importance of understanding the differences between human and machine decision-making.

“My colleagues and I argue that ADS lack certain human abilities, or what we might call a distinct type of human intelligence. It would be simple if we could fully rely on a machine that stores all the necessary data to make decisions, but human judgment is essential to complement and enhance the quality of AI-based decisions,” she explains.

Unlike the ADS systems we have today, humans do not base their decisions solely on rational logic and pattern recognition in data, but also on factors like emotions, experience, and ethical considerations. According to the researchers, these human abilities are valuable, even necessary, to factor into AI-supported decision-making. There may be important aspects that AI fails to grasp and fully understand.

“At this point, it’s highly questionable whether machines will ever be able to develop qualities like human judgment, consideration, and empathy,” says Koutsikouri.

The key questions

One area where human capabilities are particularly crucial is Koutsikouri’s own field—research. AI relies solely on raw data and doesn’t pose the same critical questions as human researchers, such as “What am I missing?”, “What do we not know?”, and “What does my ethical sense tell me about this?”. There are many other professions, Koutsikouri argues, that cannot yet be fully automated—if ever.

“I would say that most professions involving interpersonal contact require human judgment. In an emergency room, healthcare professionals must quickly assess a patient’s medical history as well as current symptoms to determine the appropriate treatment. Picking up on specific details about the patient and the situation can make all the difference in choosing the right course of action.”

In her teaching, Dina Koutsikouri also sees how vital these human qualities are.

“It’s not enough to create engaging content for lectures and seminars. You also need to adapt to the dynamics of the room and sense when to slow down or pick up the pace. AI and ADS can serve as valuable support, but they cannot independently handle the unpredictability of real-life situations.”

A continuously evolving source of knowledge

What do we know today about how human judgment works? Koutsikouri describes human judgment as a continuously evolving source of knowledge that guides us in how we act and make decisions. Humans have the ability to take in an entire situation, from sensory inputs like smell and sound to cultural and geographical factors. She emphasises the importance of exploring human judgment to better understand what sets it apart.

“One thing I’ve observed in my research on human judgment is that we often believe we understand human decision-making better than we understand algorithms. We tend to put more effort into explaining the workings of algorithms than we do in explaining human judgment. The human mind often remains an unexplainable ‘black box’,” she says.