The SALSA dataset

Xavier Alameda-Pineda, Jacopo Staiano, Ramanathan Subramanian, Ligia Batrinca, Elisa Ricci, Bruno Lepri, Oswald Lanz and Nicu Sebe

Keywords: Multimodal group behavior analysis, Free-standing conversational groups, multimodal social data sets, Tracking, Head and body pose estimation, Personality traits.

Synergetic sociAL Scene Analysis (SALSA) contains uninterrupted recordings of an indoor social event involving 18 subjects over 60 minutes. It serves as a rich and extensive repository for the behavioral analysis and social signal processing communities. In addition to the raw multimodal data, SALSA also contains position, pose and F-formation annotations over the entire event duration for evaluation purposes, as well as information regarding participants’ personality traits. You can download it from here. Cite [1] if you use it!

  • Scenario and roles. SALSA was recorded in a regular indoor space and the captured social event involved 18 participants and consisted of two parts of roughly equal duration. The first part consisted of a poster presentation session, where four research studies were presented by graduate students. A fifth person chaired the poster session. In the second half, all participants were allowed to freely interact over food and beverages during a cocktail party.
  • Sensors. The data were captured by a camera network and wearable badges worn by targets. The camera network comprised four synchronized static RGB cameras (1024×768 resolution) operating at 15 frames per second (fps). Each participant wore a sociometric badge during the recordings which is a 9×6×0.5 cm box equipped with four sensors, namely, a microphone, an infrared (IR) beam and detector, a Bluetooth detector and an accelerometer.
  • Annotations. Using a dedicated multi-view scene annotation tool, the position, head and body orientation of each target was annotated every 45 frames (3 seconds). Annotated positions and head/body orientations were used for deducing F-formations. Prior to data collection, all participants filled the Big Five personality questionnaire. The Big Five questionnaire owes its name to the five traits it assumes as constitutive of personality: Extraversion; Agreeableness; Conscientiousness; Emotional StabilityCreativity.


  1. X. Alameda-Pineda, J. Staiano, R. Subramanian, L. M. Batrinca, E. Ricci, B. Lepri, O. Lanz, and N. Sebe, “SALSA: A Novel Dataset for Multimodal Group Behavior Analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, iss. 8, pp. 1707-1720, 2016. [ bib pdf data arxiv ]
      author    = {Xavier Alameda-Pineda and
                   Jacopo Staiano and
                   Ramanathan Subramanian and
                   Ligia Maria Batrinca and
                   Elisa Ricci and
                   Bruno Lepri and
                   Oswald Lanz and
                   Nicu Sebe},
      title     = {{SALSA:} {A} Novel Dataset for Multimodal Group Behavior Analysis},
      journal   = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
      year      = {2016},
      data    = {},
      arxiv = {},
      doi  = {10.1109/TPAMI.2015.2496269},

Leave a Reply

Your email address will not be published. Required fields are marked *