Feature request: dtype conversions and optimizations

[bergr7]

🚀 Feature Request

Summary

Optimize data type (dtype) for certain operations to improve performance and memory efficiency. For example, use int for tensor operations with masks.

Also, allow for dtype settings in config files.

Motivation

• Improve performance
• Give user more control over the dtype of the resulting model

[Optional] Implementation

Additional context