How to move a robot by script

[moreuser24]

Hello,

first of all, I want to express my admiration for the magnificent work you've done so far.

I'm considering using your package for simulating some movements of a robotic arm, probably a cobot doing a simple pick and place task. I'm really convinced your package has a great potential because it seems backed by a solid engineering approach and allows you to set up everything directly inside the Unity scene, but so far I was only able to make a robot move by using the TargetFollower.cs script and creating an animation with keyframes for each new pose of the tcp.

I could neither find any additional documentation, aside from the README on this repo, which explain how to "program" the movements, e.g. from a script, nor was able to reproduce what shown in your videos on Youtube (the Task.cs class trivially doesn't exist anymore).

So, which other approach do you suggest if I wanted to program some movements?

Thank you

[Preliy]

Thank you for your feedback.

I'm currently working on this project in its early stages, and I'm still validating some core functionalities.
At this point, I don’t plan to implement motion planning in Flange package because I want to keep it lightweight and suitable for a broad range of use cases where motion planning isn’t required.

The Flange Planner is available as an add-on to the Flange Package for motion planning.

Please note that the Flange Planner is still under development, so I recommend using the development branch to explore these features. I truly welcome any contributions or suggestions you might have.

[moreuser24]

Hello,

thank you for your answer.
I tried to add the Planner package from the development branch as you suggested to the project with Flange but I get a lot of errors and can't use the project.

What are the use cases you think for the Flange package without the planner?

[Preliy]

Actually, this is a development branch, is compliable version. You’ll need to install some dependencies first. If you encounter any console errors, feel free to share them—I’ll try to provide suggestions to help you out.

From my perspective, the Planner functionality is quite complex, with certain operations being performance- and memory-intensive. It was intentionally designed as a separate module to decouple Flange and Flange Planner, helping to avoid testing complications and reduce overall system complexity. Flange is intended for lightweight, general-purpose use, while Planner targets more advanced and demanding scenarios. This separation also makes it easier to define clear testing scopes and focus on delivering stable release versions for each component independently.

For many use cases, there’s no need for built-in planning or trajectory generation:

• Simple animations
• Sales configurators
• VR/AR showrooms
• Projects using a custom planner or vendor-specific virtual controllers
• etc

The idea is to keep Flange dependency-free and compatible with multiple platforms, such as WebGL and Android.

[moreuser24]

Ok, I'll try to stick with the Flange package only for the moment, but I have a few more questions:

1. As you mentioned the vendor-specific virtual controllers, if I generate a trajectory for a cobot, with e.g. ABB Robot Studio, can I export it in Unity and make the robot follow the exact same trajectory? Or is it only for realtime streaming of data?
2. For setting up new robots kinematic chains, is this video sufficient? I mean, if I have the correct DH parameters for each joint, can I model whatever robot I want and then just add the meshes (.obj files) for visualization?
3. Does the kinematic solver work for every robot falling in one the types you defined (e.g. 6R Spherical Wrist, 6R Offset Writs...)? Somewhere you said you worked out the inverse kinematic solution with geometric-algebraic approach, but I guess it generalizes to every kinematic chain defined using your workflow, is it right?

Thank you in advance for your time

[Preliy]

Hello, sorry for the delayed response!

1. There are many possible use cases. In my virtual commissioning project, for example, I connect to virtual robot controllers like RobotStudio to read and write joint values and I/Os. Other use cases really depend on your time and imagination. Importing and exporting data in C#—and using it in Unity—is generally straightforward. The actual implementation depends on your specific needs. In some cases, simply importing a trajectory from an XML or CSV file might be all you need. Look at Open Commissioning Video.
2. A key feature of this package is the flexible setup and configuration of the kinematic chain. With DH parameters and joint configuration, you can build any open robot chain. The meshes are purely for visual representation and don’t affect the underlying kinematics.
3. The next important aspect is the solver. You’re absolutely right—each kinematic chain usually requires a specific solver. That’s why this package includes various solvers designed to specific kinematic types. It would definitely be valuable to implement a more general solver using an algebraic Jacobian approach. Contributions in this area are very welcome—honestly, I just haven’t had the time to dive into that yet.

[moreuser24]

Thank you for your answer and for pointing me to the Open Commissioning tutorial!

For a test, I defined the kinematic diagram and DH parameters for an ABB IRB 1600 and manually moved the arm through the TargetFollower.cs script and it seems to work like in the demos you provided with the package. I attached the Robot6RSphericalWrist.cs script to the outer parent gameobject.

The problem now is I want to do the same thing with an ABB GoFa CRB 15000, but this robot has a particular kinematic structure, with joint axis 6 not intersecting joint axis 5, and I guess the four types of classes available derived from Robot.cs aren't suitable. Do you think a numerical iterative solver must be used to compute the IK solutions?

I could opt for Open Commissioning to have the joint angles computed by RobotStudio, but can I somehow save the sequence of joint angles for each trajectory and later retrieve them to make the arm follow the trajectory at any time, even after disconnecting Unity from RobotStudio?
The reason is that the app should be deployed to HoloLens and work "standalone".

[Preliy]

Yes, the ABB GoFa CRB 15000 uses a different kinematic chain than the ABB IRB series. While a numerical approach can also be applied to this robot, the current package lacks an algebraic solver for the GoFa. It would be valuable to implement one, and I will investigate this the next time.

I’m not certain whether RobotStudio allows exporting complete motion trajectories. If it does, building a trajectory player in Unity should be quite straightforward.

If you plan to use this as a standalone application, I’d like to understand the specific use case. However, if you need to integrate directly with RobotStudio, my focus will be on establishing a connection with RobotStudio and use native motion planning and trajectory generation. In that scenario, I can also look into implementation of virtual pendant panel as UI in Application for user interactions. If RobotStudio integration isn’t required, then I’d prefer to implement an internal motion planner directly within the app.

[moreuser24]

Hi,

I have some data of joint angles for each of the six axis of a robot at constant time steps for a total of 6 seconds. I can't tell you how because I didn't do that personally but, as far as I know, those data were exported from RobotStudio after generating a motion trajectory.

When I said standalone application I mean I wanted to leverage the robot kinematics functionalities of your package to eventually preview the movements of a manipulator trajectory during an AR experience using a wearable device.

After discovering the problem with the non standard IK of the GoFa, I think playing the trajectory in Unity using the data imported from RobotStudio is the most straightforward option, without having to implement a numerical IK solver.

Does it make sense?
Can you suggest me a way to build this trajectory player in Unity that you referred to in the previous answer?

Thank you

[Preliy]

This is my preferred approach to implementing the functionality:
1. Data Import: Implement a loader that reads your trajectory sequence—e.g. from CSV or XML—and deserializes it into Unity-friendly classes. A common pattern is to define a serializable Step type and store the entire trajectory as a List.

[Serializable]
public class Step
{
    public float Time;       // seconds
    public float[] Joints;   // six joint angles
}

[Serializable]
public class Trajectory
{
    public List<Step> Steps = new List<Step>();
}

public static class TrajectoryImporter
{
    public static Trajectory LoadFromCSV(string path) { /*…*/ }    
}

2. Trajectory Execution: Implement an asynchronous playback method using UniTask to step through your Trajectory and apply joint angles.

public class TrajectoryPlayer : MonoBehaviour
{
    [SerializeField] 
    private Controller _controller; 
    [SerializeField] 
    private Trajectory _trajectory; 

    private bool _busy;
    private CancellationTokenSource cts;    
        
    [Button]
    public void Execute()
    {
     if (!Application.isPlaying) return;
           Cycle().Forget();
    }

    [Button]
    public void CancelExecution()
    {
        _busy = false;
        _cancellationTokenSource.Cancel();
        Debug.LogWarning("Cancel execution", this);
    }

    private async UniTask Cycle()
    {
        _cancellationTokenSource = new CancellationTokenSource();
        _busy = true;

        foreach (var step in _trajectory.Steps)
        {
            _controller.MechanicalGroup.SetJoints(step.Joints, true);
            await UniTask.Yield();
            token.ThrowIfCancellationRequested();
        }

        Debug.Log("Execution finished", this);
        _busy = false;
    }
}

[moreuser24]

Thank you so much, I'll give you some feedbacks as soon as I have the scene working properly!