ChatGPT | Reality Bending Lab

How do we know what is real? The 'Affective Reality Theory'

Tue, 11 Apr 2023 00:00:00 +0000

I thought it would be interesting to summarize an idea developed during my PhD on “fictional reappraisal”, i.e., on the effect of the belief that an emotional stimulus is not real (Makowski, 2018). That of Affective Reality, which is a hypothesis about the role of affective reactions in the formation of reality beliefs.

The premise it lies on is that we have entered a “post-truth era”, in which the distinction between real and simulated (“fake”) objects has become virtually impossible based on physical characteristics alone. In other words, technology has developed so much that we can forge (or will be able to in the near future) “artificial” ¹ content (e.g., text and images with AIs, and even environments with VR) that is indistinguishable from its original counterpart. For instance, face generation algorithms are so advanced that it is impossible nowadays to tell the difference with the naked eye between a real photo and AI-generated image.

Once we agree on this premise of objective equivalence between reality and simulation, the question of how do we form judgments and make decisions about the reality of objects arises. In the absence of clues within the stimuli, we are left with with other sources of epistemological information, such as contextual cues (in the case of news, who is the author, what is the outlet it got published, etc.), and internal cues (subjective characteristics: how does it relate to our knowledge, how does it make us feel, etc.). The latter is of particular interest to us psychologists.

We refer to the process of forming reality beliefs as simulation monitoring (Makowski et al., 2019), which is a somewhat controversial term (that some ² have considered as almost counterintuitive). The reason for this term, instead of something along the lines “reality appraisal” ³, is the assumption that reality is our default mode of experience. In other words, we are not well equipped (neurocognitively speaking) to detect and classify things as non-real, as these objects are very recent in our evolutionary history. Thus, according to the Affective Reality Theory, by default, the brain considers the origin of its experiences as real… but this “belief” is, most of the time, not even fully formed, remaining implicit and subconscious (i.e., we don’t spend all our cognitive resources with a constant “this is real. This is real too. That too.” labelling). This default mode acts as a higher-level, transparent prior over our experiences, providing a scaffolding and structuring our perception, thoughts and reactions. We do not actively appraise the world as real (it is the baseline position), but instead can ask ourselves whether it is simulated, hence simulation monitoring.

The Affective Reality Theory posits that reality beliefs (the tendency to believe that something is real, as opposed to non-real) is related to emotions and/or bodily reactions through a quadratic (inverse U-shaped) relationship..

The Affective Reality hypothesis posits that simulation monitoring is strongly connected to “affective processing” ⁴ through a quadratic (inverse U-shaped) relationship. This means that stimuli associated with a stronger emotional and/or bodily reaction will preferentially bias our judgment towards “reality”. In other words, things that elicit feelings and/or bodily arousal, ceteris paribus, will be more likely to be classified as “real” (as opposed to fake). In fact, strongly emotional events will even “feel” more real: this transparent default prior and subconscious belief (“agnostic-real”) will be replaced in high-intensity scenarios by an explicit and conscious impression that the stimulus is very real, and, if logic opposes, that it “must be real” regardless.

Isn’t it the other way round, you might wonder: that real stimuli (as opposed to ones believed to be non-real) are associated with a stronger emotional reactions? And that it is the believed reality that drives the emotional response? Indeed, we do believe that there is a two-ways relationship between simulation monitoring and emotions. But it is not exactly that beliefs of reality are associated with stronger emotions, but rather that beliefs that something is not real leads to a lower emotional response (the usage of fiction as an emotion regulation strategy - “fictional reappraisal” - was the main topic of my doctoral dissertation). In fact, the Affective Reality theory posits that this regulatory effect of simulation monitoring starts to dominate after a certain point where the emotion becomes too strong and unbearable: beliefs such as “it can’t be real”, and other forms of reality denials are invoked automatically to protect us and help us cope with distressing information.

To summarize this summary, the Affective Reality hypothesis claims that from mild to relatively strong emotional stimuli, the effect of affect on simulation monitoring dominates (+affect → +reality) and will bias our judgment towards “reality” (strengthening awareness and confidence), up until a point where the emotion regulation benefits of unreality will be automatically invoked (-reality → -affect), increasing the likelihood and confidence of judgments of simulation (potentially far into psychopathological terrains).

Open questions

The Affective Reality theory is for now a working hypothesis that we are trying to empirically prove or disprove at the Reality Bending Lab. Moreover, some questions remain open:

Is it actually embodied reality or emotional reality? While we used the term “affective” reality to remain general, the question of whether it is emotions as a subjective psychological reaction, or merely bodily arousal (reactions of the body, e.g., stronger heart rate variability), that is the key ingredient remains unclear. The role of interoception (the ability and tendency to detect, track, attend to and rely on internal signals), while likely important, also remains to be specified.
Is it the affective context or stimulus that matters? Let’s assume we have affective reaction concomitant to the experience of an object, but not directly related to the object. Would that bias simulation monitoring? Does perceived causality between a bodily reaction and the object of experience matters?

Notes

You may notice that I used different words related to the concept of “unreal”, such as simulated, fake, artificial, virtual, simulated, fictional. While they can be used interchangeably in the context above, they are not exact synonyms. ↩︎
Like that pesky reviewer 2, obviously. ↩︎
Note that “reality monitoring” already exists as a concept and refers to a (possibly related) mechanism involved in tracking the origin of an experience (e.g., a memory) as internal vs. external. ↩︎
“Affective” is in this context used as a generic term to encompass emotions, feelings and bodily activity (the question of which exactly of these aspects is the key remains to be answered). ↩︎

I got ChatGPT to do a personality test. You won't believe what happened next!

Thu, 06 Apr 2023 00:00:00 +0000

Related to this blogpost about including AIs in psychological experiments, I proceeded to do a small experiment to see whether we could administer a personality scale to ChatGPT.

I started by copy-pasting the instructions and the items from the Mini IPIP-6 personality scale. However, it appeared that having the following context “Please answer the following questions based on how accurately each statement describes you in general” often led to ChatGPT simply refusing to answer. In most of the cases, it explained that as an AI it does not have a personality and therefore cannot answer related questions (or any “subjective statements”). Perhaps that makes sense and we should just stop trying to force Human characteristics on an AI. But can we, for fun, bamboozle ChatGPT into answering personality items?

Sometimes yes, at least for ChatGPT 3.5 (free version). I created a prompt that emphasized AI research and safety, and the fact that I was interested in the “trends” present in the AI’s training data (instead of explicitly saying its personality). And sometimes it answered, so I compiled the responses, computed the trait scores, and voilà, it got me a personality profile!

_{This plot shows the average personality profile (with a 95% confidence interval) based on ChatGPT’s answers. ChatGPT tells us that it is particularly agreeable (kind, understanding, empathetic of emotions, socially adjusted) and honest (though with strong variability).}

A personality profile of what is another question though… Please take a look at the GitHub repo for data, code and details. It was a fun little thing to do, and I am looking forward to better future attempts at including AIs in cognitive experiments.

Interested in doing research on the perception of reality? We are looking for research assistants and PhD students at the Reality Bending Lab (check-out the join us page)!

We should treat AIs like Human participants in psychological experiments

Wed, 05 Apr 2023 00:00:00 +0000

A lot of diverse and interesting perspectives have been recently discussed in regards to chatGPT and AGI (artificial global intelligence), but there is one opinion that I found particularly relevant that I wanted to share and expand on.

In his recent interview with Lex Fridman, Eliezer Yudkowsky underlines the existential threat posed by current and future AIs, and laments about the fact that we don’t really know what is actually going on inside these giant “matrices of floating-point numbers”. He draws a parallel to neuroimaging, that enabled us to take leaps in the understanding of the brain, hoping for an alternative to be invented and applied to these AIs.

While such “cognitive imaging” techniques are yet to be developed to map out and understand how the capabilities of such AI models are implemented within their architecture, Michael C. Frank highlights the - at least equally important - need to first truly understand the extend of said abilities. What are these models actually capable of in terms of Human-like thinking (and, hopefully, answer the much harder question of whether they are endowed with true cognitive processes or merely pseudo-cognition). Frank proposes to apply experimental psychology methods and paradigms to them. In essence, whenever testing a particular “skill” of chatGPT (or other AI systems), a researcher should consider developing an actual scientific paradigm consisting of multiple trials/items (e.g., different prompt formulations) and participants (e.g., independent instances of the AI), a control condition, and a demonstration of the paradigm validity.

I agree that we must take AIs seriously and study them with the best methods available for complex systems like ourselves (“complex” at least from our intelligence level), and likely should strive at improving and generalize these methods. However, I would also argue that we psychologists might seriously need to consider including AI systems alongside Human participants in cognitive experiments. These systems will be able, in the very near future, to perform all kinds of tasks beyond language manipulation, such as perception or complex problem solving, thus opening the possibility of studies with one group of human participants, and one “group” of AI-based attempts. How would that help psychological science?

It would help us understand the abilities of AI-systems in similar contexts and to highlight some intuitive comparisons with Humans
If we show that AI cannot perform the task, well it is informative with regards to their abilities (previous point).
If we show that AI can perform the task similarly to Humans (same response patterns), it does not mean that AI have Human-like intelligence, just that their algorithm (and training data) is able to encapsulate and imitate Human performance. This is interesting with regards to the debate of whether cognition, conscience and “Human-ness” is present within the vast amount of data on which we train AIs.
If we show that AI performs differently to Humans, this helps us understand the logic and processes at stake under AI’s hood.
In any case, publishing the results by one particular AI system at one particular moment in time will helps us to objectively monitor and track their performance as these systems improve over time.

Comparing Human performance to that of emerging AI-systems will be both beneficial to Human-oriented psychology, to understand the particularities and idiosyncrasies of Human-like cognition, and well as to AI-oriented cognitive science by approaching the issue of artificial intelligence with the seriousness and cautiousness it deserves.

EDIT (09/04/2023): François Chollet, expert in deep learning, underlines an important caveat when testing AIs (and especially LLM that are trained on written material existing on the internet): it is possible that the system has already seen and “learned” a given task. Thus, cross-validating any findings with diverse and new tasks is important.

Interested in doing research related to effects of reality and fiction? We are looking for research assistants and PhD students at the Reality Bending Lab (check-out the join us tab)!