Eit MR Chatbot

An XR multiplayer chatbot assistant project developed in Unity. The project combines Meta XR functionality, Unity multiplayer services, and an AI-powered RAG chatbot system.

Requirements

• Unity 2022.3.62f3
• Meta Quest device (for XR testing)

This project uses Unity 2022.3.62f3, which was specifically selected due to compatibility requirements of the packages used. Updating Unity may require package changes and is done at your own risk.

---

Initial Setup

Meta XR Setup

Open:

Meta → Tools → Meta Account Setup Guide → Open Platform Settings

Fill in the Application ID in both required fields.

Add a Meta test user:

• Test user email
• Test user password

A separate test user should be configured for each main project and ParrelSync clone to allow testing of SSA functionality.

---

Unity Services Setup

Configure Unity Gaming Services:

• Unity Cloud Project
• Authentication
• Lobby
• Relay

These services are required for multiplayer functionality.

---

Testing Multiplayer from PC

The project includes ParrelSync for testing multiplayer functionality without requiring multiple XR devices.

To create a second client:

1. Open:

ParrelSync → Clones Manager → Create new clone

2. Launch the cloned project.

The original project and the clone can then be used as separate multiplayer clients.

---

AI Chatbot Setup

Each scene using AI functionality requires an LLMManager component.

Configure the following fields:

• Initial Prompt

  • Defines the AI behavior and role.

• API Key

  • API key used for communication with the language model service.

• Embedding URL

  • Endpoint used to generate embeddings for the RAG system.

• Chat API URL

  • Endpoint used for communication with the main LLM model.

Additionally, include the generated RAG data file:

Assets/StreamingAssets/embeddings.json

This file contains embedded text used by the RAG system.

---

Creating RAG Embeddings

The project includes tools for generating embeddings from documents.

1. Navigate to:

Assets/Tools/

2. Add documents to:

Assets/Tools/data/

3. Create:

apikey.txt

and add your embedding API key.

4. Create:

apiurl.txt

and add your embedding API endpoint.

5. Configure the embedding model in:

fileEmbeddings.py

6. Run:

fileEmbeddings.py

This generates:

embeddings.json

Move the generated file to:

Assets/StreamingAssets/

---

Main Packages and Tools Used

The project uses the following external packages and tools:

• Meta XR SDK
• ParrelSync
• uLipSync
• eSpeak
• piper-unity (modified to work with Unity 2022)
• Whisper

3D character models were generated using the MakeHuman Blender plugin.

---

License

Licensed under the GNU General Public License v3.0. Third-party dependencies are distributed under their respective licenses.

---

Repository Notes

Some files required for local configuration (such as API keys) are excluded from the repository. These files need to be created locally when setting up the project.