Audio / Video feed?

[acseven]

I've just found your mod for Wibox and was hoping to get it running on my apartment; I couldn't make out for sure though if you have a video and audio feed available, is there or do you have plans for it?

Cheers

[segator]

I'm trying to figure out how to extract video and audio but I'm not an expert on linux, as following @duhow tips I trying to using strace to understand how Sofia (the original app) is working with the video audio devices

• /dev/ai_dev /dev/ao_dev (audio)
• /dev/gk_video (video)

Any help would be welcome

[meidlinga]

A tiny bit of information: The devices have the same names, as in eWeLink FGHGF VG-IC-01 IP Cameras. Maybe this could help with reverse engineering.

https://web.archive.org/web/20221005193807/https://ramgattie.com/posts/2020/002-reverse_engineering_device_update_functionality-ewelink_camera/post.html

[danijelt]

@segator Did you ever find something useful with strace?
I tried to cat those device files but I'm not getting anything while the channel is open in the mobile app.

I guess that the only way to access AV is to use Goke's SDK through a custom app. My research went into the direction of trying to boot OpenIPC firmware to get a more sane working environment. Getting kernel to run was surprisingly easy and it even gets to the point where Sofia is run. The only problem is that the watchdog driver fails to load, so Sofia reboots the device after a short time.

However, the good news is that the rest of the modules load fine, including bit1628a.ko which I believe is the key to getting audio and video. The driver is small, without too much code outside of device init and passing I2C communication, but somewhat messy, mentioning three separate devices: SGM6502 (video switch matrix), BIT1628A (video decoder) and NVP6134C (video decoder and audio encoder/decoder). NVP6134C's datasheet mentions I2S interface. At a quick glance, MC34118G voice interface appears to be wired to BIT1628A, which falls in line with findings in the driver. But then, it appears to a Frankenstein of a driver (and/or the IC).

Finally, there's N76E003AT20 MCU, which is probably connected to ttySGK1 and used for VDS communication. This is actually very good, as it takes care of all the signaling on L line which would otherwise have to be reimplemented outside Sofia.

Alternative approach is to reverse engineer the cloud API, write a local server and replace the URL in the firmware. On that note, UDP port 8300 mentioned in Sofia looks suspiciously like AV channel (though it could be encrypted, I don't know).

I don't know what's worse, though - reversing the device driver and the rest of communication with unknown chance of success, or trying to make sense of possibly very over-engineered and over-encrypted cloud API.

@duhow Did you try to investigate the API communication?

[Alfred99-HA]

I am interested in Audio/Video as well. Following above hint "eWeLink FGHGF VG-IC-0" I found the follwing Link
https://ewelink.cc/ewelink-works-with-home-assistant/
As I do not own the device and due to lacking knowledge I cannot verify whether this irformation helps.

[danijelt]

Follow up on this issue: I was able to play and record audio on the device with adidemo sample app from Goke SDK. I've traced the MC34118 audio pins and when I connect the microphone and speaker, I can record and play PCM audio. This means that the audio feed is doable in theory with a custom app that would utilize the SDK.

GPIO 18 and 19 control MC34118's disable and mute pins.

[segator]

@danijelt can you provide me an example how did you was able to run that sample app? would be cool to write a tiny app that then we can use standard protocol like ONVIF so easy integration in home assistant

[danijelt]

@segator For experimenting, you can upload compiled adidemo from the SDK (GK710X_LinuxSDK_v2.0.0/applications/adidemo/demo/build/arm11-gcc-uClibc-linux-GK710XS/bin/adidemo) to /tmp.

To test playback, you have to prepare an audio file as a raw (no WAV header) signed 16-bit PCM file, little endian, 1 channel, 8k sample rate. Upload it to the device next to adidemo.
Then run the app with adidemo -d. That starts the shell with adi@goke# prompt. Play the prepared audio file with audio -P filename in ADI shell. That will start playback in a loop. Type audio -S to stop.

Recording is done with audio -R filename which will record audio to the file with the parameters listed above. It can be imported as raw data in Audacity.

Here's the photo with audio pins marked. It's been a while since I traced them, but IIRC this is be as close to the SoC as possible, while keeping the filtering caps to avoid noise. You can connect a small speaker (headphone, for example), between the output pin and any ground and you should hear the audio that you play with adidemo.

(It's possible that the input is mislabeled, I'll have to retest it to confirm.)

I have also identified the GPIO pins. The next step would be to start the playback and try dialing from the main panel to confirm that the following commands really unmute the audio interface and test if the audio can be heard:

source /usr/bin/gpio.sh
set_gpio 18 1 # MC34188 chip disable (inverted)
set_gpio 19 0 # MC34188 mute

But it's not practical for me because this system is in a large building so I'd have to run up and down while testing and I just didn't get to it yet.

As a side note, there is an unpopulated header with TXP, TXN, RXP, RXN pins marked which is a functional 100Mbps Ethernet interface that can be started with:

ifconfig eth0 up
ifconfig eth0 192.168.x.x

[Conclusio]

I created a solution for audio on the Wi-Box: https://github.com/Conclusio/wibox-audio
This runs stable in my setup for a few months now.
Video I cannot offer since I have no ways of testing it.

[segator]

Amazing it works!!! @Conclusio

I also installed a asterisk server in the middle and used HACS addon on home assistant to act as a SIP client, works prety decent. I have videocaller how I can help you to debug implement video feed? I'm a developer but this is too high level for me I guess

[segator]

Hey I working on extracting video but I need to debug sofia on boot to see GADI usage on boot, any idea how I could get that info? I trying to flash a new cramfs with the run.sh modified so I run sofia with strace first but I dont know why I always get corrupted FS. incompatible cramfs versions?

[segator]

Guys i make it! well GLM5.2 actually...

I got video. it was crazy complex due the library you use @Conclusio for audio is not working for video because is an older version that the one compiled by sofia, so after a lot of tracing in the original software I got all the instructions needed to extract the video.

I forked your project here https://github.com/segator/wibox-media/tree/main as soon as I figure out how to pass this via SIP I will merge both projects and create an firmware with everything configured

[segator]

I have a fully version working with video audio https://github.com/segator/wibox-media I changed and cleaned a lot of things I considered not necesary on my fork. you will find in the release a .img ready to flash!

it took 4y but we make it!

[josecarre]

I have a fully version working with video audio https://github.com/segator/wibox-media I changed and cleaned a lot of things I considered not necesary on my fork. you will find in the release a .img ready to flash!

it took 4y but we make it!

wow man, really good work, many thanks! :)