We should treat AIs like Human participants in psychological experiments

Wed, 05 Apr 2023 00:00:00 +0000

A lot of diverse and interesting perspectives have been recently discussed in regards to chatGPT and AGI (artificial global intelligence), but there is one opinion that I found particularly relevant that I wanted to share and expand on.

In his recent interview with Lex Fridman, Eliezer Yudkowsky underlines the existential threat posed by current and future AIs, and laments about the fact that we don’t really know what is actually going on inside these giant “matrices of floating-point numbers”. He draws a parallel to neuroimaging, that enabled us to take leaps in the understanding of the brain, hoping for an alternative to be invented and applied to these AIs.

While such “cognitive imaging” techniques are yet to be developed to map out and understand how the capabilities of such AI models are implemented within their architecture, Michael C. Frank highlights the - at least equally important - need to first truly understand the extend of said abilities. What are these models actually capable of in terms of Human-like thinking (and, hopefully, answer the much harder question of whether they are endowed with true cognitive processes or merely pseudo-cognition). Frank proposes to apply experimental psychology methods and paradigms to them. In essence, whenever testing a particular “skill” of chatGPT (or other AI systems), a researcher should consider developing an actual scientific paradigm consisting of multiple trials/items (e.g., different prompt formulations) and participants (e.g., independent instances of the AI), a control condition, and a demonstration of the paradigm validity.

I agree that we must take AIs seriously and study them with the best methods available for complex systems like ourselves (“complex” at least from our intelligence level), and likely should strive at improving and generalize these methods. However, I would also argue that we psychologists might seriously need to consider including AI systems alongside Human participants in cognitive experiments. These systems will be able, in the very near future, to perform all kinds of tasks beyond language manipulation, such as perception or complex problem solving, thus opening the possibility of studies with one group of human participants, and one “group” of AI-based attempts. How would that help psychological science?

It would help us understand the abilities of AI-systems in similar contexts and to highlight some intuitive comparisons with Humans
If we show that AI cannot perform the task, well it is informative with regards to their abilities (previous point).
If we show that AI can perform the task similarly to Humans (same response patterns), it does not mean that AI have Human-like intelligence, just that their algorithm (and training data) is able to encapsulate and imitate Human performance. This is interesting with regards to the debate of whether cognition, conscience and “Human-ness” is present within the vast amount of data on which we train AIs.
If we show that AI performs differently to Humans, this helps us understand the logic and processes at stake under AI’s hood.
In any case, publishing the results by one particular AI system at one particular moment in time will helps us to objectively monitor and track their performance as these systems improve over time.

Comparing Human performance to that of emerging AI-systems will be both beneficial to Human-oriented psychology, to understand the particularities and idiosyncrasies of Human-like cognition, and well as to AI-oriented cognitive science by approaching the issue of artificial intelligence with the seriousness and cautiousness it deserves.

EDIT (09/04/2023): François Chollet, expert in deep learning, underlines an important caveat when testing AIs (and especially LLM that are trained on written material existing on the internet): it is possible that the system has already seen and “learned” a given task. Thus, cross-validating any findings with diverse and new tasks is important.

Interested in doing research related to effects of reality and fiction? We are looking for research assistants and PhD students at the Reality Bending Lab (check-out the join us tab)!

When fiction is better than reality: Cypher's Complex

Tue, 07 Feb 2023 00:00:00 +0000

Did you ever feel empty after finishing a good book? Like (your) reality was dull and boring as compared to the fictional world you were immersed in? Yearning to stay in longer, and at the same time knowing well that it had to come to an end? You might have experienced what we can call Cypher’s Complex.

In the movie The Matrix, Cypher is a “redpill”, i.e., an individual that has been awaken from the matrix (a virtual world). However, he becomes disappointed and unhappy with the true nature of reality, and actively seeks to return to the illusory world of the matrix. Interestingly, he also explicitly desires to forget everything about the true reality, as if keeping the awareness of living in an illusion could prevent him from fully enjoying it.

"You know... I know this steak doesn't exist. I know that when I put it in my mouth; the Matrix is telling my brain that it is juicy, and delicious. After nine years... you know what I realize? Ignorance is bliss."

From a scientific perspective, the latter part can find some echo in the down-regulatory effect of fictional reappraisal. In a few studies, we showed that believing that a stimulus is “fictional” (not real) dampens our emotional state. Tucciarelli et al. (2023) also showed that the simple knowledge that a set of images of faces contains AI-generated images decreased the perceived trustworthiness of all the faces. These results suggest that being aware that the causes of our experience (the events and stimuli) are fictional can be a barrier to enjoyment and engagement. And yet, the desire to supplant reality with a fictional world can be found in real life.

Cypher’s Complex is common in mild forms. Examples can be found in the feelings of emptiness, disconnection and dullness (itself a transient and mild form of depersonalization/derealization) that follows the return from an engaging fictional world (be it in a novel, a movie or a video-game). For instance, many reported feeling blue after watching the Avatar (2009) movie, to the extent where it has been coined the “post-Avatar depression syndrome”. Most of the time, the negative affects passes, and the dissonance gets resolved either through closure (acceptance of the fictional or impermanent nature of the alternative reality), or a compromise that allows the fictional world to take a delimited space in one’s reality. For example, people might engage in activities (e.g., role playing games) or create content (writing a book or doing fan art) to integrate the fictional world into their reality.

However, Cypher’s Complex can also give rise to more severe issues with conscious or unconscious attempts at forgetting or ignoring reality (delusions, denial, …), which can lead to dire outcomes.

Interested in doing research related to effects of reality and fiction? We are looking for research assistants and PhD students at the Reality Bending Lab (check-out the join us tab)!

Fiction | Reality Bending Lab

We should treat AIs like Human participants in psychological experiments

When fiction is better than reality: Cypher's Complex