some remarks and questions

[wszak]

1. Could you please add the option to use launch_persistent_context, to be able to run the browser with some addons (e.g. captcha solvers or ublock). I could mount the directories with my addon and user data to your container using docker -v and also pass them as arguments to create_browser.

2. VNC - on the first request it's on port 6080, next time 6081 etc... It really makes it difficult to follow and watch/debug the browser.
   Can I/you somehow fix the port to one value?

3. sometimes there is an error when creating the browser. When looking into docker logs i see something like
   XIO: fatal IO error 4 (Interrupted system call) on X server ":99"
   I'm sorry I can't provide more details because as a workaround I just restart browser's container every some time to avoid this error,
   so I haven't seen it for a while.

4. If i remember correctly it wasn't possible to ignore https errors. Normally it's possible in playwright.

Apart from that - great product! Thanks.

[neilemeigh]

Thanks @wszak

1. we're working on this, but it may be a couple more weeks before we can get it live.
2. This will be updated in a new version coming out soon (a Major release with updates to anti-fingerprinting as well)
3. Thanks, please report it with details next time it happens after our upcoming release, it may resolve itself there
4. Can you explain this more? Examples / use cases for this?

[neilemeigh]

Hi @wszak

1. This is now live. You can use keepAlive and maxLifetime to accomplish this goal: https://docs.rayobrowse.com/local/connect-endpoint/
2. This now uses port 6080 each time, and will use a unique token for access. The examples in the README show how to output your VNC URL so you can grab it programmatically.

[wszak]

Hello @neilemeigh

1. Aren't you referring to some other question? How does keepAlive and maxLifetime help with running a browser with an extension? AFAIK I need to pass user_data_dir to do that and this parameter is only available in launch_persistent_context.

2. Yes, it now uses the same port, but the URL still changes :-) Previously I meant that it was cumbersome to copy&paste the VNC URL every time I restarted the browser. Now it's even a bit worse because I can't just change a digit in port number, I need to copy the whole URL.

3. One more feature request: right now all services in rayobrowse (e.g. vnc, python) listen on 0.0.0.0. Can you please add a parameter to set the listening interface? At least 127.0.0.1 should be supported. The reason is that I need to run rayobrowse container with network_mode:host setting to enable it to use my proxies which listen on 127.0.0.1 of the host server.
   The server obviously has a firewall, but still I always feel more secure if services listen on localhost, not on all external ips.

And lastly - can you explain how sessions work? Do they maintain just the same browser fingerprint or also browser context like cookies, local storage etc?

[neilemeigh]

1. sorry, I read your "persistent" too literally. Yes, we solved another issue relating to persistence with keepAlive/maxLifetime, but I see yours is different. Noted, we'll get it in queue.
2. Unfortunately the way noVnc works is that you'd always have a unique value when spinning up more than 1 browser. So, even if we removed the token requirement, you'd still be stuck on multi-ports.
3. Yes, makes sense for this to be configurable, we'll add to TODO

[neilemeigh]

Your point 2 was implemented in the latest commit. The other items are still Planned.

[wszak]

Thanks @neilemeigh , the project is making great progress!

I was wondering about another feature - there is sessionId which can be used to reuse an existing session. But what if I have cookies&localstorage and would like to recreate a new session, but with the same fingerprint as the previous session when my cookies&localstorage were created? You randomize your fingerprint whenever you create a new session - so maybe a possibility to save/load fingerprint? Or even simpler - a possibility to provide a random seed value, so that your randomization would be deterministic?
Exaggerated example: created a valid Akamai cookie when the fingerprint said "Chrome 148 Windows". Later want to reuse the cookie but the created browser is "Chrome 150 Mac".

And one more question - on your website it says that the free version supports only "single session". Is that really the case? I can't find anything about this limitation in the docs.

[neilemeigh]

Hi @wszak - yes your suggestion makes sense, we'll keep it in the planned queue with this full Issue.

Regarding single session: please refer to our LICENSE.md which highlights the limitations of usage for the open-source project. If you're not a scraping API company (like we are), then you're free to use it as much as you want. If you're a scraping company similar to Rayobyte, please contact sales@rayobyte.com for commercial licensing.

[wszak]

Thanks @neilemeigh. I was thinking about that random seed a bit more - I think no new parametr is needed actually. That random seed could be based on sessionId. So that when users asks for a specific sessionId but it no longer exists, it would be guaranteed that it's recreated with the same characteristics as previously. Something like random_seed = hash(sessionId). So, no new parameter needed, no api changes.

[wszak]

Also, a minor bug - there is ignore_https_errors parameter, but it doesn't work with proxies like mitmproxy, because the endpoints which determine proxy's ip don't ignore https errors.

[neilemeigh]

@wszak - sorry can you explain, are you saying you don't need: #4 (comment) anymore?

[wszak]

@neilemeigh I'd need it. I was just clarifying that it might not necessarily need a new parameter. sessionId could be the random seed.
But I might be wrong - you need to decide how it can be implemented.