NF_EmotionTrackingVR

NF_EmotionTrackingVR is a Unity application that can recognize seven facial expressions (Happiness, Anger, Surprise, Sadness, Fear, Contempt, Disgust), using a VR visor with facial tracking sensors. After recognizing the chosen expression, a dictionary containing the pairs formed by (expression, associated numeric value) is saved in a json file. The present expressions where choosen accordingly to the Paul Ekman's Facial Action Coding System (FACS).

Repository content

Unity folder
   test_VR_NF: the unity project folder
   NF_EmotionTrackingVR_1.0.0.unitypackage: the unity package
   TODO inserire file json: a json sample file of facial tracking data extracted during experimentation

Requirements and installation for the Unity project

• Unity game engine, preferably 2022.3.2f1.
  

• Oculus desktop app, follow steps in this page.
  

• Download Meta Quest mobile app
  

• Meta Quest Developer App
  

• Create a Meta account and a Meta Oculus Developer account
  

• Activate developer mode both in mobile app and VR visor
  

• Follow steps in this page, then in Build Settings > Android > Texture Compression, set ASTC
  

• Enable URP by following steps in this page

• Make sure that you activated face and hands tracking in VR visor settings

• Download the unitypackage from this repository and import it in the project you created in previous step

• Start you experiments!

Installation for standalone apk

• Make sure that you activated face and hands tracking in VR visor settings

• Just load the apk file in the VR visor and run it

Virtual environment interaction

To start the interaction, choose an expression to imitate, by pressing a button you see right in front of you. Use the palm of the hand to press the button. Then, try to imitate the expression. If you do it correctly, you will see a new environment. When the expression is replicated correctly, the system will save all your micro-expression on a file. If you are using the apk version, this file is located to /storage/emulated//Android/data/test_VR_NF/files/data.json. If you are using Unity version, it's located to C:/Users/user/AppData/LocalLow/DefaultCompany/Tesidata.json