Android Port

[Duecki1]

Description

This is a really basic port of RapidRaw for Android. This pr builds successfully and can be installed, it doesn't let you edit on Android yet. I thought it would be better to first integrate the android platform as pr and then add features for touch control and more afterwards, to make it more readable.

Type of Change

• New feature

Changes Made

• Moved logic from main.rs into lib.rs to comply with Tauri’s mobile entry point requirements
• Switched from native-tls to rustls to make it work with android as well as improve Linux compatibility
• Added conditions for desktop only crates (trash, tauri-plugin-single-instance)
• Added libonnxruntime.so for aarch64-linux-android (This might need to be changed later to auto download the files)
• Updated build.rs as well to stop it from downloading non android binaries
• Used "npx tauri android init" to initialize android files
• More Conditions for Desktop specific functions in code

Screenshots/Videos

Testing

• I have tested these changes locally and confirmed that they work as expected without issues

Checklist

• My code follows the project's code style
• I haven't added unnecessary AI-generated code comments
• My changes generate no new warnings or errors

Additional Notes

The "Open Folder" workflow should probably change to "Import Pictures" on android, just like lightroom mobile.

To build the debug apk run:

export ORT_LIB_LOCATION=$PWD/src-tauri/libs/arm64-v8a

and then

npx tauri android build --debug --target aarch64

(The 5000 line code changes are due to the switch from main.rs to lib.rs and many generated tauri android files)

What to do next:

If this pr gets approved, the following has to be implemented to make the android port functionally:

• Implement Androids Filesystem (Like importing raw files, exporting)
• Add proper touch control to sliders and other interactions (masking, curves)
• Make the UI mobile compatible (Should be doable with a little bit of css for smaller screens)
• Lens correction needs work
• AI Masking needs work too
• Add Github Workflow for Android with Signed APK
• (maybe even add S-Pen Support/Android Stylus support)

I will gladly implement all of those things, I just wanted to firstly get this basic port approved.

AI Disclaimer:

Please state the involvement of AI in this PR:

• This PR is AI-generated but guided by a human
• This PR was handwritten with AI assistance (spell check, logic suggestions, error resolving)

[CyberTimon]

Will take a look at this later. Quick question: is it not possible to leave the main code in main? Do we really have to migrate to lib.rs? Thanks!

[Duecki1]

Will take a look at this later. Quick question: is it not possible to leave the main code in main? Do we really have to migrate to lib.rs? Thanks!

Thanks, as far as I understood this is the correct way of doing it and I couldn't find any other.

https://v2.tauri.app/start/project-structure/

[CyberTimon]

Thanks for the response. Looks good to me :)

Where do you have arm64-v8a/libonnxruntime.so from? And is it also version 1.22.0, just like in https://huggingface.co/CyberTimon/RapidRAW-Models/tree/main/onnxruntimes-v1.22.0?

I can easily add them there (arm64-v8a and armeabi-v7a (if this is a thing?)), so you can smoothly integrate them into the auto download and verify pipeline.

[Duecki1]

Thanks for the response. Looks good to me :)

Where do you have arm64-v8a/libonnxruntime.so from? And is it also version 1.22.0, just like in https://huggingface.co/CyberTimon/RapidRAW-Models/tree/main/onnxruntimes-v1.22.0?

I can easily add them there (arm64-v8a and armeabi-v7a (if this is a thing?)), so you can smoothly integrate them into the auto download and verify pipeline.

Thank you very much. I got the files from here. (The aar file)

Version 1.22.0 should be correct.

Armeabi-v7a as far as I understand doesn't make sense, since it's for older device support (RapidRaw is too heavy for them anyway).

arm64-v8a is important and x86_64 would be good for emulator testing.

I will integrate the files from hugging face to auto downlaod/pipeline.

[CyberTimon]

Perfect. I will upload them soon. In the meantime just add arm64-v8a to the auto download pipeline and add a sample hash (you can use the huggingface hash when I uploaded it later).

[Duecki1]

Perfect. I will upload them soon. In the meantime just add arm64-v8a to the auto download pipeline and add a sample hash (you can use the huggingface hash when I uploaded it later).

Thanks, will do that as soon as I'm home. :D

[CyberTimon]

No rush, thanks for your hard work!

[CyberTimon]

Can I just rename the .aar to .so and upload it then?

[Duecki1]

Can I just rename the .aar to .so and upload it then?

The aar file should be like a zip, if you open it you will find the runtime.so files.

[CyberTimon]

Thanks. I wasn't aware of this :=)

It's uploaded now: https://huggingface.co/CyberTimon/RapidRAW-Models/blob/main/onnxruntimes-v1.22.0/libonnxruntime-android-arm64-v8a.so

SHA256: 999ecfdb5b5a13e4097487773b6d71ce8a075408a237daab072e8f5e817bd78e

[Duecki1]

Thanks. I wasn't aware of this :=)

It's uploaded now: https://huggingface.co/CyberTimon/RapidRAW-Models/blob/main/onnxruntimes-v1.22.0/libonnxruntime-android-arm64-v8a.so

SHA256: 999ecfdb5b5a13e4097487773b6d71ce8a075408a237daab072e8f5e817bd78e

Thanks, i will implement it soon :D

[Duecki1]

@CyberTimon

Hi,

I think I need a little bit of help with the workflow file for android. Where should I implement one? Create a new android.yml?

And if possible, you would have to create keystone secrets and add them to your GitHub secrets to be able to build a signed APK.

Thanks,
Linus

[CyberTimon]

@Duecki1

Hey Linux

Don't create a separate workflow. Instead, integrate Android as a new matrix entry in ci.yml / ci-pr.yml / release.yml, and add any Android specific steps conditionally in build.yml. Skip signing for now, let's just get the build green first.

Also, please double-check and remove any AI generated code comments.

Thanks!

[Duecki1]

@CyberTimon like this? I'm not sure how to test it tho.

[CyberTimon]

Thanks for the update!

The current approach of injecting android into the args string won't work probably... it'll produce a broken double build command (tauri build android build ...). The tauri-action has a dedicated mobile input for this.

Could you take a look at https://github.com/tauri-apps/tauri-action? Specifically this issue has a fully working Android workflow example.

If reworking the workflows turns out to be too complicated, we can also track that in a separate PR and keep this one focused on the app level Android changes.

[Duecki1]

I'm really sorry, I don't know how to test this.

We can create a new pr if you want, but I don't think I can test it there either right?

[SimonIT]

I think what Timon means you add here and in pr-ci and release something like this:

build-args: '--apk true'

And by looking at the mentioned issue target should be just aarch64, but I'm no expert

[Duecki1]

You are probably right with the aarch64, i didn't notice. The command i use to build the apk is npx tauri android build --debug --target aarch64.

[SimonIT]

I think it needs to be like this, but I can't promise anything

[Duecki1]

Thank you very much.

@CyberTimon does this work? (I will clean these commits if it works)

[CyberTimon]

Thanks @Duecki1 for experimenting with this. Lets get the basics right first and then track the workflows in a seperate PR, because we can't easily test them and I don't think they will work (tauri-actions talks about a mobile: '' parameter, which we don't even use here).

To keep this PR focused, I've already moved the main codebase to lib.rs and did the changes to cargo.toml. Can you please rebase your PR so the changes are more easily verifyable? Thanks so much!

[Duecki1]

Thanks @Duecki1 for experimenting with this. Lets get the basics right first and then track the workflows in a seperate PR, because we can't easily test them and I don't think they will work (tauri-actions talks about a mobile: '' parameter, which we don't even use here).

To keep this PR focused, I've already moved the main codebase to lib.rs and did the changes to cargo.toml. Can you please rebase your PR so the changes are more easily verifyable? Thanks so much!

Sure will do, thank you :D

[CyberTimon]

Looks good to me, will merge now. Thanks again! 💯

[Duecki1]

Looks good to me, will merge now. Thanks again! 💯

Awesome, thanks🫡. I will now work on the ui and filesystem :).

[Duecki1]

@CyberTimon may I ask how you like these UI changes? I implemented them to not only show on android, but in general on smaller devices.

It's not perfect yet, some icons have the wrong size and on android the directory explorer needs to be removed completely.

(The top and bottom in the screenshots are no cutted, its just how the screenshot was made)

[waxm134]

@Duecki1 Please keep in mind that some people would probably like to use it on tablets as well. On those devices it would be great if there are not too many deviations from the desktop look in my opinion. The current UI already works quite well on tablets. I can offer testing on a tablet if that helps.

[Duecki1]

@Duecki1 Please keep in mind that some people would probably like to use it on tablets as well. On those devices it would be great if there are not too many deviations from the desktop look in my opinion. The current UI already works quite well on tablets. I can offer testing on a tablet if that helps.

@waxm134 100%. These are just styles added if the devices screen is smaller than a certain value. On bigger devices (like samsung dex on a screen or android tablet) it will look like this:

[CyberTimon]

@Duecki1
How big are the UI changes here? I want to experiment and implement some mobile UI changes myself - just to make sure they won't conflict. It would be great if you could focus on the backend / gpu side of things, which are harder for me to test. Thanks so much :)

[Duecki1]

By backed / gpu side do you mean optimizing the rendering for android?

I can open a pr soon, if you want to check out the UI changes to mobile (you dont have to merge it tho).

[CyberTimon]

No not neccessarily optimizing, just what's needed to actually drag a slider on android (so the image updates correctly). As I currently have no android device to test, I can't tell if it already works. I'm talking about basic image processing and the wgpu shader. Also, feel free to open the PR for mobile UI ;) Thanks!

[Duecki1]

@Duecki1 Please keep in mind that some people would probably like to use it on tablets as well. On those devices it would be great if there are not too many deviations from the desktop look in my opinion. The current UI already works quite well on tablets. I can offer testing on a tablet if that helps.

@waxm134 may I ask which tablet you have?

[waxm134]

@Duecki1 Please keep in mind that some people would probably like to use it on tablets as well. On those devices it would be great if there are not too many deviations from the desktop look in my opinion. The current UI already works quite well on tablets. I can offer testing on a tablet if that helps.

@waxm134 may I ask which tablet you have?

I have a Xiaomi Pad 7 running a custom ROM. It's a 11" device with 8 GB RAM and a snapdragon 7+ gen 3 processor. I have the corresponding pen as well, in case there are some pen related things to test.

[Duecki1]

@Duecki1 Please keep in mind that some people would probably like to use it on tablets as well. On those devices it would be great if there are not too many deviations from the desktop look in my opinion. The current UI already works quite well on tablets. I can offer testing on a tablet if that helps.

@waxm134 may I ask which tablet you have?

I have a Xiaomi Pad 7 running a custom ROM. It's a 11" device with 8 GB RAM and a snapdragon 7+ gen 3 processor. I have the corresponding pen as well, in case there are some pen related things to test.

Here's a test apk, if you want to try it out: https://drive.google.com/file/d/1C3ZvyFCByqYhsRGYQVfgUbmD1557ZzCk/view?usp=drivesdk

Ai masking, lens correction and some other stuff doesn't work.

I'm not sure if it will run on your device tho.

[mmadesignerunknown]

https://github.com/user-attachments/assets/504e74dd-5e06-4513-a1c6-b2f9182f03e7

Hi @Duecki1 ,
Thanks alot for all this,

It's a UI improvement I want to suggest that the settings icon should be on the right side of the export icon, making it more noticeable. It's just my opinion though and I am open to your suggestions about it.

Regards,

[Duecki1]

https://github.com/user-attachments/assets/504e74dd-5e06-4513-a1c6-b2f9182f03e7

Hi @Duecki1 ,
Thanks alot for all this,

It's a UI improvement I want to suggest that the settings icon should be on the right side of the export icon, making it more noticeable. It's just my opinion though and I am open to your suggestions about it.

Regards,

If you mean the settings icon in the bottom, it's the same settings as on PC. It's for copying and pasting adjustments/edits. Imo it makes sense to be next to he copy and paste buttons.

[mmadesignerunknown]

Ohhh sorry, I thought it was for the UI And accessibility settings (the one we get on the start screen on PC), my bad. Apologies.

[SimonIT]

I'm also curious to know how it would run on a Chromebook

[Duecki1]

I'm also curious to know how it would run on a Chromebook

Probably not the best on cheap chromebooks. As long as it has a good cpu it should run fine. Sadly I only have android phones and tablet emulator to test the apk.

[waxm134]

@Duecki1 Please keep in mind that some people would probably like to use it on tablets as well. On those devices it would be great if there are not too many deviations from the desktop look in my opinion. The current UI already works quite well on tablets. I can offer testing on a tablet if that helps.

@waxm134 may I ask which tablet you have?

I have a Xiaomi Pad 7 running a custom ROM. It's a 11" device with 8 GB RAM and a snapdragon 7+ gen 3 processor. I have the corresponding pen as well, in case there are some pen related things to test.

Here's a test apk, if you want to try it out: https://drive.google.com/file/d/1C3ZvyFCByqYhsRGYQVfgUbmD1557ZzCk/view?usp=drivesdk

Ai masking, lens correction and some other stuff doesn't work.

I'm not sure if it will run on your device tho.

Just did a quick install on my phone. Will try with the tablet later. Where is the library folder located? Seems that it didn't ask for any storage permissions. Will also setup a new fork of the repo when I have time so that I can probably contribute some of my backend changes later with proper commits and pull requests.

[Duecki1]

@Duecki1 Please keep in mind that some people would probably like to use it on tablets as well. On those devices it would be great if there are not too many deviations from the desktop look in my opinion. The current UI already works quite well on tablets. I can offer testing on a tablet if that helps.

@waxm134 may I ask which tablet you have?

I have a Xiaomi Pad 7 running a custom ROM. It's a 11" device with 8 GB RAM and a snapdragon 7+ gen 3 processor. I have the corresponding pen as well, in case there are some pen related things to test.

Here's a test apk, if you want to try it out: https://drive.google.com/file/d/1C3ZvyFCByqYhsRGYQVfgUbmD1557ZzCk/view?usp=drivesdk
Ai masking, lens correction and some other stuff doesn't work.
I'm not sure if it will run on your device tho.

Just did a quick install on my phone. Will try with the tablet later. Where is the library folder located? Seems that it didn't ask for any storage permissions. Will also setup a new fork of the repo when I have time so that I can probably contribute some of my backend changes later with proper commits and pull requests.

I forgot to tell you, sorry. You will have to hold (to right click) on the space where the preview are gonna be. Then click on "Import".

[waxm134]

@Duecki1 on the tablet I can unfortunately not import my CR3 RAW files. Further, when trying to import a jpeg as alternative, the app crashes. Logcat is attached. One remark already to the UI. I think it would make sense to add a padding at the bottom and top so that there is no overlap with the statusbar or the gesture controls on Android. I attached an example how this could look (from my build). Do you plan to add a normal folder browser dialog instead of the import functionality?
logcat.txt

Padding example:

[Duecki1]

@Duecki1 on the tablet I can unfortunately not import my CR3 RAW files. Further, when trying to import a jpeg as alternative, the app crashes. Logcat is attached. One remark already to the UI. I think it would make sense to add a padding at the bottom and top so that there is no overlap with the statusbar or the gesture controls on Android. I attached an example how this could look (from my build). Do you plan to add a normal folder browser dialog instead of the import functionality? logcat.txt Padding example:

The bottom spacing is a good idea, I've not noticed it. (Hopefully tauri has a variable to just sit on top of the navbar) The importing would be the same, but instead of long holding and importing it would just be a + button to add new raws probably.

You are (probably) experiencing filesystem issues. I will take a look at it.

How is the editing performance?

[waxm134]

@Duecki1 on the tablet I can unfortunately not import my CR3 RAW files. Further, when trying to import a jpeg as alternative, the app crashes. Logcat is attached. One remark already to the UI. I think it would make sense to add a padding at the bottom and top so that there is no overlap with the statusbar or the gesture controls on Android. I attached an example how this could look (from my build). Do you plan to add a normal folder browser dialog instead of the import functionality? logcat.txt Padding example:

The bottom spacing is a good idea, I've not noticed it. (Hopefully tauri has a variable to just sit on top of the navbar) The importing would be the same, but instead of long holding and importing it would just be a + button to add new raws probably.

You are (probably) experiencing filesystem issues. I will take a look at it.

How is the editing performance?

Probably not filesystem issues. The "file is locked" entries in the log are normal AFAIK due to how android handles file access, they occur in my build as well. It crashes on the tablet before I can even enter editing mode. This is likely due to shader incompatibility with the adreno driver. Maybe you can take a look at the shader working on this device: https://github.com/waxm134/RapidRAW/blob/main/src-tauri/src/shaders/shader_android.wgsl. Editing performance on my phone with your build was very good, though.
Why do you want to go for an explicit import and not use a folder browser dialog to directly select the folder like on desktop?
Like this:

[Duecki1]

@Duecki1 on the tablet I can unfortunately not import my CR3 RAW files. Further, when trying to import a jpeg as alternative, the app crashes. Logcat is attached. One remark already to the UI. I think it would make sense to add a padding at the bottom and top so that there is no overlap with the statusbar or the gesture controls on Android. I attached an example how this could look (from my build). Do you plan to add a normal folder browser dialog instead of the import functionality? logcat.txt Padding example:

The bottom spacing is a good idea, I've not noticed it. (Hopefully tauri has a variable to just sit on top of the navbar) The importing would be the same, but instead of long holding and importing it would just be a + button to add new raws probably.
You are (probably) experiencing filesystem issues. I will take a look at it.
How is the editing performance?

Probably not filesystem issues. The "file is locked" entries in the log are normal AFAIK due to how android handles file access, they occur in my build as well. It crashes on the tablet before I can even enter editing mode. This is likely due to shader incompatibility with the adreno driver. Maybe you can take a look at the shader working on this device: https://github.com/waxm134/RapidRAW/blob/main/src-tauri/src/shaders/shader_android.wgsl. Editing performance on my phone with your build was very good, though. Why do you want to go for an explicit import and not use a folder browser dialog to directly select the folder like on desktop? Like this:

I haven't had a look at the log yet sorry.

Personally I think on mobile a file import workflow is better than a folder pick workflow. (This is all based on my opinion, workflow). I can't really imagine using the Files App to always move files to a specific folder. On PC it's normal, but on my phone I think a file specific import would be better.

I see you would want the folder workflow just like on pc? I tried this once but had some problems with permissons, maybe I will take another look at it soon.

[CyberTimon]

Personally I think on mobile a file import workflow is better than a folder pick workflow. (This is all based on my opinion, workflow). I can't really imagine using the Files App to always move files to a specific folder. On PC it's normal, but on my phone I think a file specific import would be better.

+1

[waxm134]

Personally I think on mobile a file import workflow is better than a folder pick workflow. (This is all based on my opinion, workflow). I can't really imagine using the Files App to always move files to a specific folder. On PC it's normal, but on my phone I think a file specific import would be better.

I see you would want the folder workflow just like on pc? I tried this once but had some problems with permissons, maybe I will take another look at it soon.

I totally agree with you when talking about usage on phones. I think there are probably slightly different use-cases depending on whether you use it on tablet or on a phone. In my case I would probably just do smaller edits on my phone from time to time and then definitely a dedicated import is better. On tablet my current workflow is: Taking pictures, connect the camera or the SD-card to the tablet and copy over all my images. Then open the folder in RapidRAW and use the inbuilt filtering options to see just the files I'm interested in. Basically use it like I would use it on a laptop.

Regarding the permission topic: those must be added manually to the AndroidManifest.xml AFAIK and for folder browser dialogs with the tauri-android-fs plugin I think you need to request full storage access.

[Duecki1]

Personally I think on mobile a file import workflow is better than a folder pick workflow. (This is all based on my opinion, workflow). I can't really imagine using the Files App to always move files to a specific folder. On PC it's normal, but on my phone I think a file specific import would be better.

I see you would want the folder workflow just like on pc? I tried this once but had some problems with permissons, maybe I will take another look at it soon.

I totally agree with you when talking about usage on phones. I think there are probably slightly different use-cases depending on whether you use it on tablet or on a phone. In my case I would probably just do smaller edits on my phone from time to time and then definitely a dedicated import is better. On tablet my current workflow is: Taking pictures, connect the camera or the SD-card to the tablet and copy over all my images. Then open the folder in RapidRAW and use the inbuilt filtering options to see just the files I'm interested in. Basically use it like I would use it on a laptop.

Regarding the permission topic: those must be added manually to the AndroidManifest.xml AFAIK and for folder browser dialogs with the tauri-android-fs plugin I think you need to request full storage access.

I would like to leave it like that for now and focus on the ui and other more important stuff.

Afterwards we can try out different workflows maybe?

[BHOsteras]

Here is my two cents on workflow between android and computer:

When I shoot using a DSLR I often import the images from camera to my android tablet and edit both on the tablet and on the desktop.

As I import many images to the tablet from my camera I would prefer the ability to import folders into rapidraw (perhaps a choice between file and folder based import would be possible?).
My hope was that I could use an app such as synology drive to sync both the raw images and the rrdata files between the android device and my desktop, thereby syncing both the photos and the edits between android and the desktop seamlessly.
Just a suggestion, thanks for the fantastic work on this app.

[waxm134]

Here is my two cents on workflow between android and computer:

When I shoot using a DSLR I often import the images from camera to my android tablet and edit both on the tablet and on the desktop.

As I import many images to the tablet from my camera I would prefer the ability to import folders into rapidraw (perhaps a choice between file and folder based import would be possible?). My hope was that I could use an app such as synology drive to sync both the raw images and the rrdata files between the android device and my desktop, thereby syncing both the photos and the edits between android and the desktop seamlessly. Just a suggestion, thanks for the fantastic work on this app.

Hi @BHOsteras,
if we allow folder import we would need to request elevated storage permissions for the app which is probably not wanted by some users and could make it difficult to deploy the app to Playstore if that is desired at some point. However, I think we found a good middle way that lets you manage your folders using external file managers (and sync them). Please take a look at #1097.