Unable to generate model_tsfm and model_tsfm_semantic files for a synthetic dataset.

[SharmaVibhor]

Hi,

I am trying to run the method on a dataset that I create from blender. I have attached a rendered image for reference.
I suspect it has to do with lack of iris in the rendered image. Is there any way to convert the bust object I have already to model_tsfm or any other suggestions how I could make it work?

Thanks in advance

[0mil]

@s58vshar
Hi,

What exactly does it mean to run the method on data created in Blender?
As far as I understand, if you want to run this program using blender model image instead of real world videos, I believe that it might be impossible.
What I mean is that the essential mechanism of this program called neural radiance fields fundamentally depends on real world light information.

Best regards,

[KeyuWu-CS]

@s58vshar If you use blender, I think you can get correct camera pose with multi-view images. In such case, you don't need to run COLMAP and directly use the correct camera parameters run Instant-NGP. For model_tsfm, you can slightly modify the code and not use iris to optimization. Another possible method is to directly use your head model(include scalp) and set an identity matrix to save it in model_tsdf.

[SharmaVibhor]

Hi @KeyuWu-CS ,

Yes I have the camera parameters without colmap. My issue was with bust_fitting. I have been trying to use my own head model and to extract the scalp but I havent succeeded yet.
Thanks for your response nonetheless 👍

[SharmaVibhor]

@KeyuWu-CS

I want to generate my own scalp.obj using my head object. But the method
generate_flame_scalp(Bust_root, 'assets/data/Wings_head.obj','assets/data/Wings_head.obj','assets/data/scalp_mask.png')

returns a scalp object which contains half of the scalp plus some of the bust. I think it is due to the scalp_mask.png which was provided in the cdoe.

Can you tell me how did you generate it for your model so I can do it for my object? And also how did you decide on the size of 1024x1024?

Thanks in advance!

[KeyuWu-CS]

This is because the UV mapping of the head model used is different (we use the flame template). If you want to use your own head model, you need to manually determine the scalp area in blender or maya and generate a corresponding scalp_mask.png for it. Its resolution is not unique, 512x512 is also OK.

[SharmaVibhor]

@KeyuWu-CS thanks for the insight. I had one last question about the parameter bust_to_origin: [0.006, -1.644, 0.010].
I am unable to figure out how did you calculate it. I seem to have problem with finding num_root variable in HairGrow.py , it is always 0. I think its somehow related to the position of scalp and the hair strands and the variable bust_to_origin.

It would be great help if you could share some insights for this as well!

[Reus59]

@KeyuWu-CS thanks for the insight. I had one last question about the parameter bust_to_origin: [0.006, -1.644, 0.010]. I am unable to figure out how did you calculate it. I seem to have problem with finding num_root variable in HairGrow.py , it is always 0. I think its somehow related to the position of scalp and the hair strands and the variable bust_to_origin.

It would be great help if you could share some insights for this as well!

I have encountered the same problem. Have you found a solution? Looking forward to your reply

[SharmaVibhor]

@Reus59 are you using your own data or are you using the provided data?
I was using my own dataset so it was a bit challenging for to get the parameters right.
I tried visualizing the point cloud and scalp to see if there is overlap or not. In my case they were not overlapping so i had to adjust bust_to_origin parameter.

[Reus59]

@Reus59 are you using your own data or are you using the provided data? I was using my own dataset so it was a bit challenging for to get the parameters right. I tried visualizing the point cloud and scalp to see if there is overlap or not. In my case they were not overlapping so i had to adjust bust_to_origin parameter.

I use my own data. But in the HairGrow program, connect_strands.hair cannot be generated. I want the hair to grow on my own head model. The input data includes Bust (bust_long.obj, flame_bust.obj, scalp.obj) folder, colmap folder, model_tsfm.dat and model_tsfm_semantic.dat. I am trying to adjust the model_tsfm and model_tsfm_semantic parameters. How do you adjust these two parameters? Thank you very much, looking forward to your reply