Differentiating neural sensitivity and bias during face-emotion processing in youth: A computational approach

Теги: mathematics mathematical physics higher mathematics differentiating neural sensitivity

Похожие

Vector calculus, linear algebra and differential forms a unified approach

Introduction to linear optimization [Athena Scientific series in optimization and neural computation]

Discourse and Argumentation in Archaeology: Conceptual and Computational Approaches

Computer science. An Interdisciplinary Approach

Текст

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

Differentiating neural sensitivity and bias during face-emotion processing in youth: A
computational approach

Simone P Haller1, Joel Stoddard2, Sofia I Cardenas1, Kelly Dombek, Caroline MacGillivray1,
Christian Zapp1, Hong Bui1, Caitlin M Stavish1, Katharina Kircanski1, Matt Jones3, Melissa A
Brotman1
1Emotion

and Development Branch, National Institute of Mental Health, National Institutes of
Health, 9000 Rockville Pike, Bethesda, MD, 20892
2Pediatric

3Department

er
r

Mental Health Institute, Children’s Hospital Colorado, Department of Psychiatry &
Neuroscience Program, University of Colorado, Anschutz Medical Campus, 13123 E 16th Ave,
Box A036\B130, Aurora, CO, 80045
of Psychology and Neuroscience, University of Colorado Boulder, CO, 80309

tCorrespondence

rin

to:
Simone P. Haller, DPhil
Neuroscience and Novel Therapeutics
Emotion and Development Branch
National Institute of Mental Health
National Institutes of Health
9000 Rockville Pike
Bethesda, MD, 20892
Phone: 301 451 8861
Fax: (301) 480-4683
Email: simone.haller@nih.gov

1
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

Abstract

Background: The ability to interpret face-emotion displays is critical for adaptive social
interactions. Using a novel variant of a computational model and fMRI data, we examined

behavioral and neural associations between two metrics of face-emotion labeling (sensitivity and
bias) and age and irritability and anxiety in youth.

Methods: In two studies, healthy controls (Study 1: n=44, M age=20.02, SD=7.44, 8-36) and
patients (Study 2: n=84, M age=13.76, SD=2.6, 8-19) with disruptive mood dysregulation disorder

er
r

(DMDD), attention-deficit/hyperactivity disorder (ADHD) or anxiety disorder completed an
explicit face-emotion labeling fMRI task including happy to angry morphed face-emotions. A drift
diffusion model was applied to choice and reaction time distributions to examine sensitivity and

bias in interpreting face-emotions. Model fit and reliability of parameters were assessed on separate
adult data (n=38). Linear and quadratic slopes modeled brain activity associated with dimensions
of face-emotion valence and ambiguity during interpretation.

Results: Behaviorally, age associated with sensitivity in controls. Neither irritability nor anxiety
was associated with bias; irritability associated with longer nondecision time. Age related to more

pronounced neural responses to ambiguity in the bilateral inferior frontal gyri. Associations
between the computational metrics and activation patterns indicated that systems encoding face-

rin

emotion valence and ambiguity both contribute to the ability to discriminate face-emotions.
Conclusions: The current study provides evidence for age-related differences and clinical

associations with distinct metrics of face-emotion processing in youth.

face-emotion

labeling;

computational

modeling;

development;

affective

Keywords:

psychopathology; cognitive bias; fMRI

2
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

rin

er
r

iew
ed

Running head: FACE-EMOTION PROCESSING IN YOUTH

3
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

Introduction

Accurately detecting face-emotions of others is critical for healthy social relationships;

difficulties in adequately identifying face-emotion displays have been linked to several psychiatric
diagnoses and behavioral problems including mood and anxiety disorders and aggression [1]. Here,

we describe a new variant of a classic computational model, a drift diffusion model [2], to parse

two distinct and important aspects of face-emotion recognition: emotion sensitivity, which describes
the ability to accurately perceive subtle differences in the intensity in specific emotions (i.e.

er
r

discrimination ability), and perceptual bias, the tendency to categorize ambiguous face-emotions
as hostile. We apply this model to examine i) normative developmental change in both metrics
during a clinical risk period (i.e., adolescence to adulthood) in typically developing individuals, and

psychopathology across adolescence.

ii) potential alterations in these processes in youth characterized by anxiety- and irritability-related

Sensitivity and bias are distinct constructs in signal detection theory and psychophysics [3];

sensitivity describes the ability to discriminate while bias refers to the response criterion of the
observer. Measurement issues in standard emotion recognition paradigms means that these two

constructs are often confounded in canonical emotion recognition tasks. Specifically, canonical
emotion recognition tasks often cannot distinguish whether performance accuracy is due to

rin

participants’ ability to discriminate emotions or due to a bias in responding with a specific emotion
category. Additionally, very few tasks report on the reliability of behavioral performance on

specific face-emotions.

Studies examining individual differences face-emotion processing have largely focused on

the ability to categorize face-emotions generally. Early behavioral work showed that, despite an
equidistant physical difference, face-emotions within a category are distinguished more poorly than

4
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

those that cross category boundaries, indicating that face-emotions are perceived categorically [4].

Similarly, work on the neural mechanisms of face-emotion processing suggests that neural systems
for categorizing face-emotion may be distinct from those coding emotion intensity [5, 6]. In as far
as overlapping systems have been reported, studies suggest that these are involved differently in
categorization and sensitivity perception. For instance, recent work suggests that populations of

neurons within the amygdala separately encode face-emotion intensity and ambiguity [6]. Despite

a wealth of literature on emotion recognition, experimental work examining associations with

er
r

individual differences in terms of age and clinical status have largely relied on paradigms
confounding these two dimensions and thus an important avenue of work is to understand the
chronometry of development of specific aspects of emotion perception, especially during clinical

risk periods.

Several behavioral studies have provided evidence that emotional category boundaries are
experientially malleable [7] and the ability to discriminate between emotion categories efficiently

continues to develop into young adulthood [8-11]. Specifically, recent work highlights just how
prolonged these trajectories are with development for some emotions not plateauing until the mid-

thirties [12]. To date, only two studies have examined ‘normative’ behavioral and neural
developmental differences in categorical labeling of face-emotions between adolescents and adults

rin

and these have explored age categorically. Wiggins and colleagues[13] asked adolescents (ages 918) and adults (ages 19-47) to label angry, fearful or happy faces at varying intensities of emotional

valence. In the absence of behavioral differences, the authors found that age groups differed in
emotion intensity-related neural responses in fronto-parietal and temporal areas. A second recent

study [14] found behavioral and neural evidence that adolescents (12-15) were less sensitive to
subtle facial expressions than adults (ages 19-54). Adolescents labeled subtly angry faces as more

5
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

neutral than adults and more similarly represented ambiguous facial expressions on a neural level

compared to adults in pre-selected face-selective voxels in the fusiform face area and occipital
regions.

Because of the interpersonal difficulties associated with problems in face-emotions
processing, a large body of work has examined the role of face-emotion processing in symptom

maintenance and origin in clinical populations. For instance, prominent theories of social
processing in aggression posit that response biases in interpreting ambiguous face-emotion to

er
r

influence subsequent cognitive processes to promote an aggressive threat response [15-18].
Similarly, anxiety has been linked to a tendency to interpret rapidly presented ambiguous facial
expressions as angry [19, 20] – albeit it is unclear whether it is increased sensitivity or a response

bias driving these interpretations. Aggression, and irritability more broadly, as well as anxiety are
among the most common clinical phenomena in youth, which, in their severe and chronic form
often first emerging in adolescence. Severe and chronic irritability and anxiety are impairing,

evident in childhood, often co-occurring, and predict poor academic and socioeconomic outcomes
[21, 22]. However, largely because of a siloed research approach on individual diagnoses (despite

their overlap), it is unclear whether biased interpretations that have been reported across these
diagnoses are driven by anxiety or irritability or both. Here, we adopt a dimensional approach to

rin

parse behavioral and neural deficits along these correlated but distinct symptom domains.
Sequential-sampling models are a class of mathematical models that have significantly

impacted neuroscience[23, 24]. We apply a new variant of a classic drift diffusion model (DDM)
that provides a validated ‘meeting ground’ between the processes involved in perception that give

rise to decision making, here interpretations of face-emotions[25]. With the application of the
DDM, we investigate a specific cognitive mechanism by which differences in behavioral

6
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

performance in face-emotion discrimination could arise with age. Two parameters are of particular
relevance: the model’s drift rate parameter[2, 25, 26] [2, 25, 26], which represents the quality of

the stimulus information entering the decision process and quantifies the efficiency in processing
subtle changes in face-emotion valuence, i.e. sensitivity to changes in valence. A second parameter

of interest is the model’s perceptual bias metric. an index that quantifies the point along the face-

emotion continuum where there is maximum ambiguity during the process of interpretation. This

parameter serves as an index of bias in how ambiguous morphs are perceived as in terms of happy

er
r

relative to angry. The model’s perceptual bias metric resides in decisional post-processing, rather
than in the efficiency of raw perceptual sampling. Specifically, the metric reflects the calibration of
the evidence accumulation relative to what is assumed to be the evidence rate for the most

ambiguous stimulus.

Parsing of developmental and clinical differences in these two performance metrics
(sensitivity and bias) is particularly important when examining neural substrates. Encoding of the

two relevant stimulus properties (i.e., valence and ambiguity) appear to be carried out by separate
neural populations[6]. It is thus conceivable that computations particularly reliant on one or the

other stimulus dimension have different developmental trajectories or are differentially affected by
in youth with particular emotional difficulties.

rin

In summary, we examine whether and how age and important dimensions of pediatric
psychopathology modulate the behavioral and neural bases of face-emotion labeling using a

computational approach. We hypothesize that with older age, participants will be better able to
discriminate differences in face-emotions along the spectrum of valence, as indicated by the

computational sensitivity parameter. We do not expect age to moderate processes related to

7
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

with perceptual bias towards threatening, angry faces.

Methods and Materials
Participants

iew
ed

ambiguity [27]. In contrast, we expect anxiety and irritability, assessed dimensionally, to associate

This NIMH Institutional Review Board approved study included a total of 180 participants.

For Study 1 examining age associations across a wide age range, a final sample of 44 participants
(M age=20.02, SD=7.44, range 8-36, 36% female) were included. Youth (ages 8-18) were assessed

er
r

to be free of current psychiatric diagnoses using parent- and child-report on the Kiddie-Schedule
for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version
(KSADS-PL, [28]); adult participants (ages 19-36) were assessed using the Structured Clinical

Interview for DSM-IV-TR Axis I Disorders (SCID, [29]). Adults returned for a second visit ~3
weeks later (range 2-6 weeks, M=22.39 days, SD=8.49). Data from the second visit was used to

assess parameter recovery and test-retest reliability of computational model parameters.
For study 2, to obtain full, distributed ranges of anxiety and irritability symptom we

recruited youth across several diagnostic categories. A final sample of 84 participants (M
age=13.76, SD=2.6, 8-19, 53% female) consisted of patients with primary diagnoses of Disruptive

rin

Mood Dysregulation Disorder (DMDD), an anxiety disorder (AD; generalized, social, and/or
separation anxiety disorder), or Attention-Deficit/Hyperactivity Disorder (ADHD), which are

diagnosis known to be characterized by different levels of anxiety and irritability. Diagnoses were
established using the K-SADS-PL [28], including a DMDD module [30]. Typically developing
youth <20 years of age from Study 1 (HC, n=22) were included to fill out the lower end of the

spectrum of clinical scores. Participants were dimensionally assessed on levels of irritability and
anxiety using the Affective Reactivity Index (ARI[31]) and the Screen for Child Anxiety Related

8
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

report were averaged.

iew
ed

Emotional Disorders (SCARED[32]) parent-report and youth-report forms. Parent and youth-

For both studies, parents and youth provided informed consent and assent, respectively;

adults provided written informed consent. Sample characteristics are detailed in Table 1 for both
samples. Recruitment, compensation, exclusion criteria for patients and details on excluded

participants are included in the Supplementary Materials.

er
r

fMRI Face-Emotion Labeling Task

The face-emotion labeling task required participants to make binary judgments of 15
different facial expressions, ranging from prototypically happy to prototypically angry. A

composite happy and angry male face from the Karolinska Directed Emotional Faces [33] database
was linearly morphed at equal intervals from unambiguously happy to unambiguously angry facial
expressions [34]. Stimuli were generated by [35] using established procedures designed to maintain

relevant features of facial expression [36].
During each trial, participants first viewed a face stimulus (150ms), followed by a white

noise mask (200ms) and a fixation response screen. Participants were asked to respond with ‘happy’
or ‘angry’ as quickly as possible, indicated by button press (see Figure 2A). The intertrial interval

rin

was a jittered fixation cross (min. 500ms, ITI distribution followed an exponential decay curve).
Stimulus presentation and jitter orders were optimized and pseudo-randomized via AFNI’s

make_random_timing.py program. Each morph was presented 30 times at random during the task
in addition to 90 fixation trials, for a total of 540 trials completed in 4 runs. Runs were ~421 seconds

long and included a 10s fixation period beginning and ending each run.

9
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

Computational Model

The computational model is a variant of the DDM [26, 37] applied to a two choice (happy-

angry) face-emotion labeling task; see Supplementary Materials for a formal specification of the
model. The DDM separates different decision-making components and their contribution to

behavioral performance (choice and RT). Binary choice options are represented by two decision

boundaries or criteria; the model parameter ‘𝑎’ denotes the distance between boundaries. The rate
of accumulation of noisy evidence from a starting point z (0 < 𝑧 < 𝑎) toward one of the two

er
r

boundaries (happy or angry judgement) for any given face morph (stimulus ‘𝑠1 ― 𝑠15’) is
represented by the drift rate ‘𝑣’, often the main variable of interest representing the average rate of
evidence accumulation toward the upper response. A response is initiated when evidence

accumulation reaches one of the two choice boundaries. The total time of a decision is the duration
of this process added to non-decision processes such as early encoding/sensory processes and
responses execution, represented by the parameter ‘𝑡0’. Hence, drift rate, decision criteria, and

nondecision processes are the proposed cognitive components of the decision process.
Following the psychometric function relating stimulus value (‘𝑠’) to drift rate (‘𝑣’) reported

by Ratcliff across a range of discrimination tasks[2], drift rate was assumed to vary linearly across

rin

the range of stimuli. Thus, for any given face morph:

𝑣(𝑠) = 𝑣int + 𝑠 ⋅ 𝑣slope #(1)

To ensure robustness of model-based analyses, a second version of the model was fit that

allowed a more flexible nonlinear relationship between and stimulus value and drift rate (see

Supplementary Materials). However, parameter estimates from the nonlinear model were no longer
statistically reliable because of the greater number of parameters needed to be estimated.

10
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

This extension of the DDM yields the two measures of interest, both derived from the drift
rate parameters in equation (1). This first measure of interest is sensitivity, represented by 𝑣slope.

The second main measure represents the point along the morph continuum that reflects maximum

ambiguity during decision processing between happy and angry choices. This is the point at which

𝑣int
#(2)
𝑣slope

er
r

𝑠indiff = ―

the drift rate, 𝑣(𝑠), is zero:

Stimuli on either side of this estimate have negative (happy choice) or positive drift rates
(angry choice), respectively. If participants are not biased during their processing of face-emotions,

(morph 8 in this experiment).

the drift rate should be zero at the 50/50 angry/happy linear morph on the face-emotion continuum

In addition to sensitivity and decisional bias, three other parameters were estimated across

trials for each subject: the starting point of the evidence accumulation process (response bias
𝑧𝑟 = 𝑧/𝑎), boundary criterion (𝑎) and non-decision time (𝑡0). Deviation of 𝑧𝑟 from ½ represents

anything that may bias a decision prior to viewing the stimulus, such as the subject’s prior belief in
the prevalence of happy and angry emotions in other people (as well as motor contributions such

rin

as handedness). Hence, there are two bias parameters contained in the model: ‘𝑧𝑟′ representing
response bias, and 𝑠indiff representing perceptual bias.

Parameter estimation used a genetic algorithm implemented via the Optimization Toolbox
in Matlab 2016b, minimizing a modified KS statistic that combined both response types into a

single RT distribution by recoding RTs for Happy judgments as negative. Parameter values were
allowed to range 𝑧𝑟 ∈ [.1,.9], 𝑎 ∈ [.01,1], 𝑡0 ∈ [.1,1] (all time units in seconds). The 𝑣slope and 𝑣int

11
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

parameters were constrained such that 𝑣(1) ∈ [ ―2,2],𝑣(15) ∈ [ ― 2,2]. The diffusion rate was
fixed at 0.1. To determine parameter bounds for this experiment, we fit the model to a separate
dataset from 38 adults. Comparisons between traditional and computational metrics, as well as a

detailed discussion of the linear drift rate assumption are included in the Supplementary Materials.

Imaging procedures and statistical analyses

For details on data acquisition and preprocessing see Supplementary Materials.

er
r

Study 1: DDM model fit and cross-sectional associations with age

Reliability was estimated through parameter recovery simulations. Model fit to data is
displayed by quantile probability plots. An intraclass correlation coefficient (ICC[2,1] [38]) was

calculated to assess test-retest reliability of the parameters of interest, sensitivity (𝑣(𝑠)) and
perceptual bias (𝑠indiff) in adult participants who returned for a second visit. Supplementary
Materials present comparisons between computational parameters and traditional metrics.

We examined associations between the computational parameters and age with Pearson
correlations. For the imaging data, we applied a linear mixed-effects model using 3dLME in AFNI

[39]. Face-morph spectrum was modeled on the group level as within-subjects factors with a linear
regressor representing valence (happy-angry changes, i.e., a positive slope indicates increased

rin

activity to angry face-morphs) and a quadratic regressor representing ambiguity (overt-ambiguous
changes, i.e., a negative quadratic coefficient indicates relative increased activity to ambiguous

compared to overt morphs face-morphs) centered on the middle morph. In three separate models,
we examined interactions between age, each computational metric and the two morph terms.

Participant was included as a random effect. Sex and mean-centered Motion (% censored TRs) were
included as covariates in all models. All analyses were conducted across a whole-brain, gray matter

12
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

mask of voxels with signal common to at least 90% of participants. The voxel-wise P value

threshold was .005, with multiple testing correction to α = .05 via Monte Carlo cluster-size

simulation with a Gaussian plus exponential spatial autocorrelation function to estimate smoothness
(AFNI’s 3dClustSim -acf, NN=1, two-sided). This resulted in a criterion of cluster size greater than
k=50. Average activity was extracted from each cluster for post-hoc analyses and visualization.

Post-hoc analyses applied the same model formulations as the fMRI mixed model using lme4 [40]

in R [41] version 4.0; linear and quadratic coefficients were extracted from the neural data for each

er
r

participant and correlated with individual difference metrics (parameters, age) using Pearson
correlations. See Supplementary Materials for additional region-of-interest (ROI) analyses of the

bilateral amygdalae.

Study 2: Associations with irritability and anxiety

Effects of psychopathology were examined in terms of continuous levels of irritability and

anxiety across diagnoses. Associations between the computational parameters and anxiety and
irritability were tested with Pearson correlations.

For the imaging data, we applied the same linear mixed-effects model used to examine ageassociations with two regressors representing the face-morph spectrum in linear and quadratic

rin

terms. Sex, mean-centered Motion (% censored TRs) and Age were included as covariates. We
assessed irritability and anxiety-related differences in neural activations to the face-morph terms.

All analyses were conducted across a whole-brain gray mask of voxels with signal common to at
least 90% of participants. The voxel-wise P value threshold was .005, with multiple testing

correction to α = .05 with a criterion of cluster size greater than k=51. Post-hoc analyses were
performed identically to Study 1.

13
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Results
Study 1: Model fit and cross-sectional associations with age
1. Model fit and parameter reliability

iew
ed

Running head: FACE-EMOTION PROCESSING IN YOUTH

Quantile probability plots visually comparing predicted and empirical reaction time
distributions by accuracy per choice response (happy-angry) and face morph showed that the model

represented behavior reasonably well (Figure 1). See Supplementary Materials Figure S2 for

correlations among parameters and parameter recovery simulations showing excellent recovery of

er
r

all model parameters. Intraclass correlations across two time points suggested good reliability of
computationally derived sensitivity (ICC[2,1]=.71), with poorer reliability for the perceptual bias

metric (ICC[2,1]=.40).

2. Behavioral Data

A positive correlation emerged between age and sensitivity (r=.51, p<.001, Figure 2B),

suggesting that sensitivity to subtle changes in face-emotions continues to develop through
adolescence and young adulthood. No association was found between age and perceptual bias,

response bias (𝑧𝑟), or boundary criterion (𝑎) (all rs<-.27, p>.08).

rin

3. Imaging Data

Table 2 details all significant results for activation. See Supplementary Materials for whole

brain activation maps for the two main regressors of interest, valence (linear regressor, increasing

anger intensity) and ambiguity (quadratic regressor).

a. Associations with age

14
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

The right and left anterior insula/inferior frontal gyrus showed an interaction between age
and the quadratic morph term. Post-hoc decomposition of these two interactions showed that, with

increasing age, neural activation to ambiguous relative to overt morphs increased (right anterior

insular: F(1,612)=29.93, p<.001, r(42)=-.63; left anterior insular: F(1,612)=23.64, p<.001, r(42)=.51; Figure 2).

b. Associations with computational parameters

A significant interaction between DDM-derived sensitivity and the linear morph term (i.e.,

er
r

valence levels) emerged within some areas that also showed a main effect for the linear morph term:
the left postcentral gyrus, the bilateral supplementary motor area (SMA) and cerebellum.
Additionally, clusters in the left supramarginal gyrus also exhibited an interaction. Post-hoc

extraction of the linear coefficient revealed that steeper valence slopes were associated with
increased sensitivity (left SMA: F(1,612)=30.71, p<.001, r(42)=.62, p<.001), right SMA:
F(1,612)=24.97, p<.001, r(42)=.57, p<.001; left postcentral gyrus: F(1,612=29.60, p<.001,
right cerebellum: F(1,612)=20.80, p<.001, r(42)=.47, p=.001; left

r(42)=.51, p<.001;

supramarginal gyrus F(1,612)=25.79, p<.001, r(42)=.64, p<.001; superior parietal lobule

(F(1,612)=19.77, p<.001, r(42)=.58, p<.001).

DDM-derived sensitivity also associated with the quadratic morph term (i.e., ambiguity

rin

levels) in several clusters, including two cluster in the superior frontal gyrus, the left middle frontal
gyrus, right anterior insula and left precuneus. In all regions except the anterior insula, increased

behavioral sensitivity was related to increased activation to overt relative to ambiguous face
morphs; for the anterior insula, increased sensitivity was related to increased activation to ambiguity

(right superior frontal gyrus: F(1,612)=38.12, p<.001, r(42)=.66, p<.001; right superior frontal
gyrus F(1,612)=25.30, p<.001, r(42)=.60, p<.001, left middle frontal gyrus: F(1,612)=31.14,

15
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

p<.001, r(42)=.63, p<.001; right anterior insula F(1,612)=22.22, p<.001, r(42)=-.56, p<.001 and
left precuneus: F(1,612)=22.94, p<.001, r(42)=.60, p<.001).

For the DDM-derived perceptual bias parameter, a significant interaction with the quadratic
morph term emerged only in the right precentral gyrus, with a bias toward threat associated with

decreased activation to ambiguous relative to overt face-morphs (F(1,612)=20.17, p<.001, r=-.66,

p<.001). Significant interactions between the perceptual bias parameter and the linear morph term
emerged in the left posterior cingulate cortex and insula/inferior frontal gyrus. Increased bias

er
r

towards threat was associated with increased neural responses to faces at the overtly happy end of
the face-morph spectrum (left posterior cingulate cortex: F(1,612)=19.56, p<.001, r(42)=.57;
insula/inferior frontal gyrus: F(1,612)=18.61, p<.001, r(42)=.57).

In summary, we found evidence that activity patterns reflecting both stimulus dimensions
were related to individual differences in behavioral sensitivity and perceptual bias metrics derived

from the DDM.

Study 2: Cross-sectional associations with psychopathology

1. Behavioral Data

No significant correlations emerged with either sensitivity or perceptual bias and continuous

rin

clinical measures of anxiety and irritability (rs<.08, ps>.47). Irritability was associated with the t0
parameter (r(82)=.32, p<.005). No other associations emerged between clinical measures and

parameters.

2. Imaging data

Four significant clusters emerged for the interaction between irritability and the linear morph

term, in the bilateral postcentral gyrus, right middle orbital/frontal gyrus and left cerebellum. Post-

16
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

hoc decompositions revealed that a more pronounced neural response (i.e., a steeper slope) was

associated with increased irritability (left postcentral gyrus: F(1,612)=28.15, p<.001, r(42)=; right
postcentral gyrus: F(1,612)=20.26, p<.001, r(42)=; right middle orbital/frontal gyrus:
F(1,612)=25.52, p<.001, r(42)=; and left cerebellum: F(1,612)=30.52, p<.001, r(42)=. (r
postcentral gyrus r=.27, p=.01, left postcentral gyrus r=-.2, p=.06) and

No additional significant interactions between morph terms and either anxiety or irritability

er
r

or their interaction emerged.

Discussion

We used a happy–angry morphed face-emotion paradigm and a novel computational

approach to test two main hypotheses: i) sensitivity in labeling subtle changes in emotion valence
associates with age across adolescence and early adulthood in healthy volunteers, ii) biases in
labeling ambiguous face-emotions link with anxiety- and irritability-related psychopathology in

adolescents. We expected associations between behavior and individual differences in age and

ambiguity.

psychopathology to manifest in neural representations of stimuli dimension of emotion valence and

Age was associated with the ability to efficiently distinguish subtle changes in happy-angry

rin

face-emotion valence (i.e., the DDM-derived sensitivity metric). Neural activity in the bilateral
inferior frontal gyrus showed that older individuals had a more pronounced and defined quadratic

slopes compared to youth. The study provides evidence for significant development in labeling
subtle or ambiguous face-emotions across adolescence and early adulthood [13, 14]. Divergent

from previous work [14], the current report finds that age-related differences are not situated within
the ventral stream or face-selective areas but rather within attention- and salience networks.

17
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

Only irritability showed significant associations with DDM parameters, specifically
nondecision time, capturing time spent on sensory encoding and motor output. Parallel imaging

results revealed associations between irritability and a more pronounced neural response in motor
regions, likely indicating more forceful motor responses to overt morphs. Additionally, more

pronounced differential responding to overt happy vs. angry morphs, with increased responses to

overt happy morphs in the cerebellum and inferior frontal gyrus may reflect increased salience of
non-threat social cues in the context of threatening and ambiguous face-emotions.

er
r

We expected to find an bias towards angry face interpretations based on prior work in
anxiety and irritability [42, 43] and the seminal work of [44]. Some reports, while finding diagnostic
associations, have found no associations with individual difference measures [19], suggesting that

perceptual bias may be a marker of psychopathology or severity more broadly. Additionally, other
work suggests that non-emotional features of faces may impact whether biases are detected [19,
45], with only one composite face, we cannot explore the impact of face identity on bias. Further,

Study 1 demonstrates that sensitivity significantly changes across adolescence, future work should
test interactions between age, activation patterns to ambiguity and valence and clinical variables.

The current report more broadly replicates previous findings in adults of brain networks
sensitive to face-emotion valence and ambiguity, including the amygdalae [6]. Activation patterns

rin

reflecting both stimuli dimensions were related to individual differences in behavioral sensitivity
and perceptual bias metrics derived from the DDM: while separate neural systems may encode

these two dimensions, both systems are recruited in interpreting face-emotions. Both slopes
contribute to index the degree of separation of neural representations of different faces, i.e., the

capacity for discrimination.

18
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

We applied an augmented version of a classic mathematical model of binary choice

decision-making with face-emotion inputs varying in ambiguity and valence. The model capitalizes
on unique information contributed by distributions of RT and choice behavior [25] conveying
different information about stimulus processing. For instance, while accuracy asymptotes on the

extreme ends (e.g., few happy decisions on most overt angry morphs), RTs still show a graded

decrease at the most overt morphs. Compared to using raw RT and choice measures, model-based

parameters are more sensitive and specific to the process of interest: a process isolated from other

er
r

decision components (see plots of raw choice and RT data and comparison between computational
and traditional metrics in Supplementary Materials). Age could conceivably affect multiple
decision components (for instance, drift rates and boundary separation). Thus, effects on multiple

components must be accounted for, which the computational approach allows for. While we used
the model to investigate developmental differences in choice decision making, on a much simpler
level, the model includes a process description for how the binary decisions are computed. The

model, however, is agnostic as to whether the diffusion process represents low level perceptual or
higher-level inferential processes. Future work could more directly use imaging data to ground the

abstract theoretical mechanism of the DDM in neurobiology.
This study has several limitations. First, with a small sample and a wide age range in a cross-

rin

sectional design, Study 1 had limited datapoints available per decade. This constrained our ability
to test for trends other than linear developmental effects. Developmental effects can be most

comprehensively addressed in a longitudinal design. Second, reliability and validity was tested in
an adult population and our perceptual bias parameter only showed adequate reliability, which may

have contributed to the lack of associations between the clinical data and the metric. Third, because
it was not ethical to withhold treatment given the severity of illness, we cannot exclude the

19
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

possibility that medication use affected the results in the clinical sample. Forth, across both studies,
we used a compound computer-generated face stimulus from a well-validated stimulus set [33] to
be able to create highly controlled, fine grained representations of face-emotions. Previous work
suggests that i) idiosyncratic features of faces significantly affect how face-emotions are interpreted

[46] and ii) peer-related social cues may have particular motivational salience for youths [47, 48].

a minimum, include and model ‘actor’ as a random effect.

Future work should explore the impact of stimulus age and identity on face-emotion labeling or, at

er
r

In summary, the current study provides evidence for continuous development of subtle faceemotion processing across adolescence and young adulthood. The computational approach provides
precision in processes and mechanism; such precision is important for progress in understanding

developmentally- and clinically relevant features and changes in social-emotional functioning to
ultimately delineate periods where environmental input is particularly impactful for early

rin

intervention and prevention.

20
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

rin

er
r

Characteristics of 44 participants in Study 1
Age M(SD), range
19.99 (7.34), 8-36
Sex N, % female
18, 35
IQ M(SD)
114 (9.85)
Race
White
25
Black or African American
8
Asian
4
Multiple Races
5
Other/Unknown
2
Ethnicity N
Latino or Hispanic
3
Not Latino or Hispanic
41
d
Household's gross income median range $60,000 - 89,999
Characteristics of 84 participants in Study 2a
Age M(SD), range
14.07 (2.9), 8-19
Sex N, % female
48, 57
IQ M(SD)
114 (11.01)
Race N
White
65
Black or African American
9
Asian
1
Multiple Races
7
American Indian or Alaskan Native
1
Other/Unknown
1
Ethnicity N
Latino or Hispanic
11
Not Latino or Hispanic
73
d
Household's gross income median range $90,000 - $179,999
Medications N
Antidepressant
14
Antiepileptic
5
Antipsychotic
4
Stimulant
29
Clinical measures M(SD)
Anxiety (SCARED)
15.44 (11.68)
Irritability (ARI)
3.22 (2.88)
Accuracy
89% (.06)

Table 1. Sample Characteristics for Study 1 and Study 2

iew
ed

Running head: FACE-EMOTION PROCESSING IN YOUTH

Abbreviations: IQ, Intelligence Quotient

21
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

rin

er
r

iew
ed

Note. aIQ was measured by the Wechsler Abbreviated Scale of Intelligence[49]. IQ data were not
available for adult participants in Study 1.
a Twenty-two healthy volunteer participants were included in both Study 1 and Study 2.

22
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

L middle frontal gyrus
R superior frontal gyrus
L Precuneus
R superior frontal gyrus
R insula lobe

-25
-6

56
-55

-25

-5

rin

-41
37

Sensitivity by Quadratic Slope (Ambiguity)
-25
22
48
273
20
37
47
135
-5
-60
29
104
25
21
56
91
41
21
4
59
Perceptual Bias by Linear Slope (Valence)
-33
20
0
73
0
-34 28
69
Perceptual Bias by Quadratic Slope (Ambiguity)
42
-20
54
187
Irritability by Linear Slope (Valence)

er
r

iew
ed

Table 2. Significant Associations of Whole-Brain BOLD Activity and Individual Differences
Center of Mass
Size
F/tPContrast
(k)
Locationa
valueb
valueb
Age by Quadratic Slope (Ambiguity)
x
y
z
37
13
7
183 R anterior insula
-5.39 <.001
L anterior insula/Inferior
-40
28
2
65 frontal
-5.31 <.001
Sensitivity by Linear Slope (Valence)
L supplemental motor
-5
-4
54
192 area
-40
-21
48
109 L postcentral gyrus
-57
-35
29
64 L supramarginal gyrus
5.4 <.001
24
-55
-23
63 R cerebellum lobule VI
5.23 <.001
L superior parietal
-22
-54
64
54 lobule
4.95 <.001
R supplemental motor
15
-5
67
50 area

40
40

Anterior cingulate gyrus
R insula
R postcentral gyrus

162 L postcentral gyrus
103 L cerebellum lobule VI
69

R postcentral gyrus
R middle orbital/frontal
66 gyrus

5.93
5.32
5.21
-4.67
4.53

<.001
<.001
<.001
<.001
<.001

Post-hoc

r=-.64
r=-.54

r=.50
r=.49
r=.58

r=.64
r=.60
r=.61
r=-.49
r=.58

5.25 <.001
5.56 <.001

r=.59
r=.64

-3.94 <.001

r=-.61

10.62 <.001

AD<DMDD, ADHD,
CTL CTL >DMDD
AD, ADHD< CTL
DMDD>AD, ADHD,
CTL

6.65 <.001

ADHD, DMDD< CTL

14.21 <.001
11.52 <.001

Abbreviations: BOLD, blood oxygenation level-dependent
aLocation represents anatomical overlap of cluster with regions.
bValue from post-hoc linear-mixed effects model of mean BOLD signal for extracted cluster.

23
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

iew
ed

Disclosures:
Dr. Joel S. Stoddard has received grant or research support from the National Institute of Mental
Health (NIMH). He has served as a DSMB committee member: Threat Interpretation Bias as
Cognitive Marker and Treatment Target in Pediatric Anxiety (R61 Phase). Dr. Haller received grant
support from the BBRF. Dr. Jones has received grant or research support by the NSF and the NIMH.
Dr. Brotman has served as a principal investigator on a Bench-to-Bedside grant from NIH. Ms.
Cardenas, Ms. Dombek, Ms. MacGillivray, Ms. Bui, Ms. Stavish, Mr. Zapp and Dr. Kircanski have
reported no biomedical financial interests or potential conflicts of interest.

rin

er
r

Acknowledgements:
The authors thank the patients and families that contributed to this research. This research was
supported by the Intramural Research Program of the NIMH, National Institutes of Health (NIH;
ZIAMH002786), and was conducted under NIH Clinical Study Protocols 00-M-0198 and 02-M0021 (ClinicalTrials.gov ID: NCT00006177 and NCT00025935). JS was supported by a grant from
the National Institutes of Health, National Institute of Mental Health, K23MH113731 and the
Pediatric Mental Health Institute at Children’s Hospital Colorado and the Division of Child and
Adolescent Psychiatry, Department of Psychiatry, University of Colorado School of Medicine. The
funding source was not involved in study design; the collection, analysis and interpretation of data;
writing of the report; and the decision to submit the article for publication.

24
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

rin

er
r

iew
ed

References
1.
Collin, L., et al., Facial emotion recognition in child psychiatry: a systematic review.
Research in developmental disabilities, 2013. 34(5): p. 1505-1520.
2.
Ratcliff, R., Measuring psychometric functions with the diffusion model. Journal of
Experimental Psychology: Human Perception and Performance, 2014. 40(2): p. 870.
3.
Macmillan, N. and C. Creelman, Detection Theory: A User’s guide, 2nd Edn New Jersey.
2004, Lawrence Erlbaum Associates Publishers.[Google Scholar].
4.
Etcoff, N.L. and J.J. Magee, Categorical perception of facial expressions. Cognition, 1992.
44(3): p. 227-240.
5.
Morris, J.S., et al., A neuromodulatory role for the human amygdala in processing
emotional facial expressions. Brain: a journal of neurology, 1998. 121(1): p. 47-57.
6.
Wang, S., et al., The human amygdala parametrically encodes the intensity of specific facial
emotions and their categorical ambiguity. Nature communications, 2017. 8(1): p. 1-13.
7.
Pollak, S.D. and D.J. Kistler, Early experience is associated with the development of
categorical representations for facial expressions of emotion. Proceedings of the National
Academy of Sciences, 2002. 99(13): p. 9072-9076.
8.
Vetter, N.C., et al., Adolescent basic facial emotion recognition is not influenced by puberty
or own-age bias. Frontiers in psychology, 2018. 9: p. 956.
9.
Horning, S.M., R.E. Cornwell, and H.P. Davis, The recognition of facial expressions: an
investigation of the influence of age and cognition. Aging, Neuropsychology, and
Cognition, 2012. 19(6): p. 657-676.
10.
Thomas, L.A., et al., Development of emotional facial recognition in late childhood and
adolescence. Developmental science, 2007. 10(5): p. 547-558.
11.
Picci, G. and K.S. Scherf, From caregivers to peers: Puberty shapes human face perception.
Psychological science, 2016. 27(11): p. 1461-1473.
12.
Rutter, L.A., et al., Emotion sensitivity across the lifespan: Mapping clinical risk periods to
sensitivity to facial emotion intensity. Journal of experimental psychology: general, 2019.
148(11): p. 1993.
13.
Wiggins, J.L., et al., Developmental differences in the neural mechanisms of facial emotion
labeling. Social cognitive and affective neuroscience, 2016. 11(1): p. 172-181.
14.
Lee, T.-H., et al., Perceiving facial affective ambiguity: A behavioral and neural
comparison of adolescents and adults. Emotion, 2019.
15.
Wilkowski, B.M. and M.D. Robinson, The anatomy of anger: An integrative cognitive
model of trait anger and reactive aggression. Journal of personality, 2010. 78(1): p. 9-38.
16.
Dodge, K.A., Translational science in action: Hostile attributional style and the
development of aggressive behavior problems. Development and psychopathology, 2006.
18(3): p. 791.
17.
Crick, N.R. and K.A. Dodge, Social information‐processing mechanisms in reactive and
proactive aggression. Child development, 1996. 67(3): p. 993-1002.
18.
Freeman, J.B. and K.L. Johnson, More than meets the eye: Split-second social perception.
Trends in cognitive sciences, 2016. 20(5): p. 362-374.
19.
Stoddard, J., et al., An Open Pilot Study of Training Hostile Interpretation Bias to Treat
Disruptive Mood Dysregulation Disorder. J Child Adolesc Psychopharmacol, 2016. 26(1):
p. 49-57.
20.
Maoz, K., et al., Attention and interpretation processes and trait anger experience,
expression, and control. Cognition and Emotion, 2017. 31(7): p. 1453-1464.

25
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

28.

29.
30.
31.
32.
33.
34.

iew
ed

35.

26.
27.

er
r

25.

24.

23.

22.

Merikangas, K.R., et al., Lifetime prevalence of mental disorders in US adolescents: results
from the National Comorbidity Survey Replication–Adolescent Supplement (NCS-A).
Journal of the American Academy of Child & Adolescent Psychiatry, 2010. 49(10): p. 980989.
Wakschlag, L.S., et al., The neurodevelopmental basis of early childhood disruptive
behavior: Irritable and callous phenotypes as exemplars. American journal of psychiatry,
2018. 175(2): p. 114-130.
Ratcliff, R. and P.L. Smith, A comparison of sequential sampling models for two-choice
reaction time. Psychological review, 2004. 111(2): p. 333.
Forstmann, B.U., R. Ratcliff, and E.-J. Wagenmakers, Sequential sampling models in
cognitive neuroscience: Advantages, applications, and extensions. Annual review of
psychology, 2016. 67.
Ratcliff, R., P.L. Smith, and G. McKoon, Modeling regularities in response time and
accuracy data with the diffusion model. Current directions in psychological science, 2015.
24(6): p. 458-470.
Ratcliff, R., A theory of memory retrieval. Psychological review, 1978. 85(2): p. 59.
Motta‐Mena, N.V. and K.S. Scherf, Pubertal development shapes perception of complex
facial expressions. Developmental science, 2017. 20(4): p. e12451.
Kaufman, J., et al., Schedule for affective disorders and schizophrenia for school-age
children-present and lifetime version (K-SADS-PL): initial reliability and validity data.
Journal of the American Academy of Child & Adolescent Psychiatry, 1997. 36(7): p. 980988.
Spitzer, R.L., et al., The structured clinical interview for DSM-III-R (SCID): I: history,
rationale, and description. Archives of general psychiatry, 1992. 49(8): p. 624-629.
Wiggins, J.L., et al., Neural Correlates of Irritability in Disruptive Mood Dysregulation and
Bipolar Disorders. Am J Psychiatry, 2016. 173(7): p. 722-30.
Stringaris, A., et al., The Affective Reactivity Index: a concise irritability scale for clinical
and research settings. Journal of Child Psychology and Psychiatry, 2012. 53(11): p. 11091117.
Birmaher, B., et al., The screen for child anxiety related emotional disorders (SCARED):
Scale construction and psychometric characteristics. Journal of the American Academy of
Child & Adolescent Psychiatry, 1997. 36(4): p. 545-553.
Lundqvist, D. and J. Litton, The averaged Karolinska directed emotional faces. Stockholm
Q12, 1998.
Lundqvist, D., A. Flykt, and A. Öhman, The Karolinska directed emotional faces (KDEF).
CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska
Institutet, 1998. 91(630): p. 2-2.
Penton-Voak, I.S., et al., Increasing recognition of happiness in ambiguous facial
expressions reduces anger and aggressive behavior. Psychological science, 2013. 24(5): p.
688-697.
Tiddeman, B., M. Burt, and D. Perrett, Prototyping and transforming facial textures for
perception research. IEEE computer graphics and applications, 2001. 21(5): p. 42-50.
Ratcliff, R. and G. McKoon, The diffusion decision model: theory and data for two-choice
decision tasks. Neural computation, 2008. 20(4): p. 873-922.
Shrout, P.E. and J.L. Fleiss, Intraclass correlations: uses in assessing rater reliability.
Psychological bulletin, 1979. 86(2): p. 420.

rin

21.

36.

37.
38.

26
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631

Running head: FACE-EMOTION PROCESSING IN YOUTH

43.
44.
45.
46.
47.
48.

rin

49.

iew
ed

42.

41.

er
r

40.

Chen, G., et al., Linear mixed-effects modeling approach to FMRI group analysis.
Neuroimage, 2013. 73: p. 176-190.
Bates, D., et al., Fitting linear mixed-effects models using lme4. arXiv preprint
arXiv:1406.5823, 2014.
Team, R.C., R: a language and environment for statistical computing. R Foundation for
Statistical Computing. 2014. 2015.
Hommer, R.E., et al., Attention bias to threat faces in severe mood dysregulation. Depress
Anxiety, 2014. 31(7): p. 559-65.
Haller, S.P., et al., Measuring online interpretations and attributions of social situations:
Links with adolescent social anxiety. J Behav Ther Exp Psychiatry, 2016. 50: p. 250-6.
Crick, N.R. and K.A. Dodge, A review and reformulation of social information-processing
mechanisms in children's social adjustment. Psychological bulletin, 1994. 115(1): p. 74.
Marsh, A.A., N. Ambady, and R.E. Kleck, The effects of fear and anger facial expressions
on approach-and avoidance-related behaviors. Emotion, 2005. 5(1): p. 119.
Stoddard, J., et al., An open pilot study of training hostile interpretation bias to treat
disruptive mood dysregulation disorder. Journal of child and adolescent
psychopharmacology, 2016. 26(1): p. 49-57.
Haller, S.P., et al., Measuring online interpretations and attributions of social situations:
Links with adolescent social anxiety. Journal of behavior therapy and experimental
psychiatry, 2016. 50: p. 250-256.
Haller, S.P., et al., Attention allocation and social worries predict interpretations of peerrelated social cues in adolescents. Developmental cognitive neuroscience, 2017. 25: p. 105112.
Wechsler, D., Manual for the Wechsler abbreviated intelligence scale (WASI). San Antonio,
TX: The Psychological Corporation, 1999.

39.

27
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4442631