Self-Calibrating BCIs: Ranking and Recovery of Mental Targets Without Labels

Jonathan Grizou (GrizAI & University of Glasgow) · Carlos De la Torre-Ortiz (University of Helsinki) · Tuukka Ruotsalo (University of Copenhagen, Denmark and LUT University, Finland)

algorithm developmentbrain response datacursoreeggenerative modelimage similarity scoresmental targetnaturalistic imagesperceptual judgmentsranking stimuliself-calibrationstimulus generationunknown mental targetunsupervised learning user study

We consider the problem of recovering a mental target (e.g., an image of a face) that a participant has in mind from paired EEG (i.e., brain responses) and image (i.e., perceived faces) data collected during interactive sessions without access to labeled information. The problem has been previously explored with labeled data but not via self-calibration, where labeled data is unavailable. Here, we present the first framework and an algorithm, CURSOR, that learns to recover unknown mental targets without access to labeled data or pre-trained decoders. Our experiments on naturalistic images of faces demonstrate that CURSOR can (1) predict image similarity scores that correlate with human perceptual judgments without any label information, (2) use these scores to rank stimuli against an unknown mental target, and (3) generate new stimuli indistinguishable from the unknown mental target (validated via a user study, N=53). We release the brain response data set (N=29), associated face images used as stimuli data, and a codebase to initiate further research on this novel task.