Implementation and Assessment of a Multipurpose Appraisal-Driven Emotion Awareness Tool Based on Self-Report

Mattia A. Fritz
TECFA, Université de Genève

PhD thesis in psychology presented on the 28th of June 2023
Jury: M. Bétrancourt, T. Brosch, P. Dillenbourg, S. Lajoie

Learning Cognitive Social Affective

Baker et al., 2013; Brackett, 2019; Pekrun & Linnenbrink-Garcia, 2014
Lack of para-verbal/contextual cues
Information and Communication Technology
Endow computer-mediated learning environments with affect-related information beyond face-to-face
Cernea & Kerren, 2015; D’Mello & Graesser, 2015; Lajoie et al., 2020

Emotion Awareness Tools

Original figure: abstract model of emotional awareness derived by the literature review
  1. Self-Report
    Boehner et al., 2007; Fontaine et al., 2013; Lavoué et al., 2020
  2. Moment-to-Moment
    Graesser et al., 2014; Molinari, Chanel, et al., 2013; Miller & Hadwin, 2015
  3. Appraisal theory of emotion
    Brosch et al., 2013; Pekrun, 2006; Scherer, 2005; Scherer & Fontaine, 2019
Presentation map

Overview of the tool

Chapters 4, 5 and 6

Experimental factor in CSCL

Chapter 7

Study rationale

Emotional awareness contributes to mutual-modeling
Dillenbourg et al. 2016; Eligio et al., 2012; Molinari et al., 2013

One main limitation of this [design] is the difficulty in disentangling the effect of reflecting upon one's own emotions from the effect of sharing one's emotions.

— Molinari, Chanel, et al., 2013

The tool as experimental factor


Does a different use of and access to emotional information determine the use of the tool?

Mutual > Partner > Self in expressing and perceiving emotions

Simulated problem-solving task

N = 48 participants (M = 37.3, SD = 10.0) reduced to 35 for technical problems

Expressing seems unrelated to interface type

M = 13.8 (5.7) expressed emotions
pairwise estimate p.value
Self - Partner -0.17 [-6.28, 5.95] 1.00
Self - Mutual -3.14 [-8.60, 2.31] 0.34
Partner - Mutual -2.98 [-8.90, 2.95] 0.44

Perceiving seems simply socially-oriented

M = 51.4 (32.5) seconds looking at perceiving AOI
pairwise estimate p.value
Self - Partner -40.20 [-71.03, -9.37] 0.01
Self - Mutual -31.81 [-59.31, -4.31] 0.02
Partner - Mutual 8.39 [-21.47, 38.26] 0.77

Support in distance learning

Chapter 8

Study rationale

Emotional awareness projects learners socio-affectively
Jézégou, 2010; Kirschner et al., 2015; Lavoué et al., 2020; Lowenthal & Snelson, 2017

Tentative scale: Emotional Awareness Usefulness
Crossing literature, existing scales and abstract model

# Dimension Item
1 Frequency I used the tool frequently (e.g. every time I worked for the course)
2 Affordance The tool prompted me to share my emotions
3 Social Presence The tool allowed me to feel less lonely during remote learning periods
4 Self-Understanding The tool allowed me to better understand my emotions
5 Understanding Others The tool allowed me to better understand the emotions of my colleagues
6 Self-Other Comparison The tool allowed me to compare my emotions with those of my colleagues
7 Self-Regulation The tool allowed me to regulate my emotions

What is the perception of usefulness of the tool and does it change over time?

Support in distance learning

33 students (M = 32.96, SD = 7.78) in two successive cohorts in TECFA's Master

Perception of usefulness drops after (non) use

Post-hoc multilevel-model contrast based on N = 798 item ratings from 30 students
Evolution over time
Only a handful of students used the tool

Psychometric properties

\(\omega_h\) \(\alpha\) \(\omega_{tot}\) Uni
0.6 0.86 0.9 0.86
Factor Analysis

Overall Assessment

Chapters 9 and 10

Data-driven assessment

System Usability Scale 73.5(12.8)
Below 80 target (Lewis & Sauro, 2018), above 67.8 (NA) for tool by Feidakis et al. (2014) [N = 40]

9 out of 10 emotion labels belong to EATMINT circumplex
Lexicalized emotions seem useful for the purpose [N = 1097]

3 out of 4 expressions accepted suggested labels
Interest to compare with a random generation of labels to test base rate acceptance [N = 1097]

So far, so good?

Main limitation of the model

Valence and Control not independent \(M_\rho\) = 0.47 (0.43) See also Erbas et al., 2015; Molinari et al. (2016); Scherer & Fontaine (2019)

N-Dimensional Spaces

Number and kind of appraisals and feelings should be based upon research/pedagogical objectives

Multipurpose toolbox

Researchers and instructional designers can create and share instances of the tool and collect data
Admin Area

Thank you for your attention !

Mattia A. Fritz
TECFA, Université de Genève

Presentation created with Reveal.js.
Licence Creative Commons