Implementation and Assessment of a Multipurpose Appraisal-Driven Emotion Awareness Tool Based on Self-Report.

Implementation and Assessment of a Multipurpose Appraisal-Driven Emotion Awareness Tool Based on Self-Report

Mattia A. Fritz
TECFA, Université de Genève

PhD thesis in psychology presented on the 28th of June 2023
Jury: M. Bétrancourt, T. Brosch, P. Dillenbourg, S. Lajoie

Learning Cognitive Social Affective

Baker et al., 2013; Brackett, 2019; Pekrun & Linnenbrink-Garcia, 2014

Lack of para-verbal/contextual cues

Information and Communication Technology

Endow computer-mediated learning environments with affect-related information beyond face-to-face

Cernea & Kerren, 2015; D’Mello & Graesser, 2015; Lajoie et al., 2020

Emotion Awareness Tools

Original figure: abstract model of emotional awareness derived by the literature review

Self-Report
Boehner et al., 2007; Fontaine et al., 2013; Lavoué et al., 2020
Moment-to-Moment
Graesser et al., 2014; Molinari, Chanel, et al., 2013; Miller & Hadwin, 2015
Appraisal theory of emotion
Brosch et al., 2013; Pekrun, 2006; Scherer, 2005; Scherer & Fontaine, 2019

Overview of the tool

Chapters 4, 5 and 6

Experimental factor in CSCL

Chapter 7

Study rationale

Emotional awareness contributes to mutual-modeling
Dillenbourg et al. 2016; Eligio et al., 2012; Molinari et al., 2013

One main limitation of this [design] is the difficulty in disentangling the effect of reflecting upon one's own emotions from the effect of sharing one's emotions.

— Molinari, Chanel, et al., 2013

The tool as experimental factor

Does a different use of and access to emotional information determine the use of the tool?

Mutual > Partner > Self in expressing and perceiving emotions

Simulated problem-solving task

N = 48 participants (M = 37.3, SD = 10.0) reduced to 35 for technical problems

Expressing seems unrelated to interface type

M = 13.8 (5.7) expressed emotions

pairwise	estimate	p.value
Self - Partner	-0.17 [-6.28, 5.95]	1.00
Self - Mutual	-3.14 [-8.60, 2.31]	0.34
Partner - Mutual	-2.98 [-8.90, 2.95]	0.44

Perceiving seems simply socially-oriented

M = 51.4 (32.5) seconds looking at perceiving AOI

pairwise	estimate	p.value
Self - Partner	-40.20 [-71.03, -9.37]	0.01
Self - Mutual	-31.81 [-59.31, -4.31]	0.02
Partner - Mutual	8.39 [-21.47, 38.26]	0.77

Support in distance learning

Chapter 8

Study rationale

Emotional awareness projects learners socio-affectively
Jézégou, 2010; Kirschner et al., 2015; Lavoué et al., 2020; Lowenthal & Snelson, 2017

Tentative scale: Emotional Awareness Usefulness
Crossing literature, existing scales and abstract model

#	Dimension	Item
1	Frequency	I used the tool frequently (e.g. every time I worked for the course)
2	Affordance	The tool prompted me to share my emotions
3	Social Presence	The tool allowed me to feel less lonely during remote learning periods
4	Self-Understanding	The tool allowed me to better understand my emotions
5	Understanding Others	The tool allowed me to better understand the emotions of my colleagues
6	Self-Other Comparison	The tool allowed me to compare my emotions with those of my colleagues
7	Self-Regulation	The tool allowed me to regulate my emotions

What is the perception of usefulness of the tool and does it change over time?

Support in distance learning

33 students (M = 32.96, SD = 7.78) in two successive cohorts in TECFA's Master

Perception of usefulness drops after (non) use

Post-hoc multilevel-model contrast based on N = 798 item ratings from 30 students

Only a handful of students used the tool

Psychometric properties

\(\omega_h\)	\(\alpha\)	\(\omega_{tot}\)	Uni
0.6	0.86	0.9	0.86

Overall Assessment

Chapters 9 and 10

Data-driven assessment

System Usability Scale 73.5(12.8)
Below 80 target (Lewis & Sauro, 2018), above 67.8 (NA) for tool by Feidakis et al. (2014) [N = 40]

9 out of 10 emotion labels belong to EATMINT circumplex
Lexicalized emotions seem useful for the purpose [N = 1097]

3 out of 4 expressions accepted suggested labels
Interest to compare with a random generation of labels to test base rate acceptance [N = 1097]

So far, so good?

Main limitation of the model

Valence and Control not independent \(M_\rho\) = 0.47 (0.43) See also Erbas et al., 2015; Molinari et al. (2016); Scherer & Fontaine (2019)

N-Dimensional Spaces

Number and kind of appraisals and feelings should be based upon research/pedagogical objectives

Multipurpose toolbox

Researchers and instructional designers can create and share instances of the tool and collect data

toolbox.dynamicemotionwheel.ch

Thank you for your attention !

Mattia A. Fritz
TECFA, Université de Genève

Presentation created with Reveal.js.