Unveiling Hidden Psychological States: Synergistic Fusion of Amplified Color and Motion from Video

The remote, non-contact assessment of human psychological states is a significant challenge in affective computing, as current video-based methods often focus on single, specific physiological channels. While remote photoplethysmography (rPPG) captures cardiac activity from color changes and motion magnification reveals subtle movements, both suffer from limitations in dynamic scenarios and lack physiological specificity, respectively. This paper proposes a synergistic approach, demonstrating that cardiac-related color and vestibular-related motion can be extracted and fused from a single video stream to create a richer psychological representation. We introduce dual-mag, a unified framework that jointly amplifies and fuses these complementary channels. Our core contributions include: 1) The first supervised learning paradigm for video-based head micro-tremor (HMT) amplification using high-fidelity inertial measurement unit (IMU) data as ground truth, establishing HMT as a quantifiable biomarker; and 2) a novel dual-branch architecture with a physio-dynamic magnification block (PDM-block) and a cross-attention fusion module to learn the deep interplay between signals. Extensive experiments on the large-scale ReMAP dataset show that dual-mag significantly advances the state-of-the-art (SOTA), improving perception accuracy over leading unimodal methods by 7.2% for discrete emotions, 5.6% for continuous emotions, and 8.3% for personality traits. Our work establishes that deep, intra-sensor fusion can unlock unprecedented performance from ubiquitous cameras for remote psychological sensing.