This is a sample implementation of a gzip and a bzip decoder plugin for gstreamer-1.0.
These dependencies are needed for you to compile and run the plugin. If you'd rather use Docker, jump to the Docker section below.
- GStreamer
apt install libgstreamer1.0-dev gstreamer1.0-tools - Meson
apt install meson - Zlib
apt install zlib1g - Bzip2
apt install libbz2
Compile as usual with meson. Set GST_PLUGIN_PATH to use the plugin without a system-wide install.
meson build
ninja -C build
export GST_PLUGIN_PATH="$PWD/build"
Note that bzip2 does not play well with build tools such as meson and may require you to specify library and header paths specific for your linux distro in meson.build.
Uncompress files with gzdec and bzdec as shown below. Find sample compressed files in their respective directories under assets/.
# gzip
gst-inspect gzdec
gst-launch-1.0 filesrc location=gst-gzdec/assets/sentence.txt.gz ! gzdec ! filesink location=out_gzip.txt
# bzip
gst-inspect bzdec
gst-launch-1.0 filesrc location=gst-bzdec/assets/sentence.txt.bz ! bzdec ! filesink location=out_bzip.txt
For quickly testing how the plugins output compares to gunzip or bzip2 cli tools, you may use the ./test.sh tool in each of the plugin subdirectories. Spoiler alert: output should be the same.
cd gst-gzdec && ./test.sh assets/paragraph.txt.gz # optional argument
cd gst-bzdec && ./test.sh assets/paragraph.txt.bz
Observe warnings and decompression information per-buffer via GST_DEBUG="gzdec:5" in any of the above scenarios.
To run the plugin via Docker, run these commands to build and log into the container. This image imports and compiles a static copy of the repo.
docker build . -t gst-gzdec
docker run --rm -it gst-gzdec
- GTK Gstreamer types https://api.gtkd.org/gstreamer.c.types.html
- GNOME GstBuffer docs https://developer-old.gnome.org/gstreamer/stable/gstreamer-GstBuffer.html
- RidgeRun plugin guide https://developer.ridgerun.com/wiki/index.php/Creating_a_New_GStreamer_Element_or_Application_Using_Templates
- Zlib's zlib.h header with docs https://github.com/madler/zlib/blob/develop/zlib.h
- Zlib practical usage https://www.zlib.net/zlib_how.html
- Gst reminder of GST_DEBUG https://gstreamer.freedesktop.org/documentation/tutorials/basic/debugging-tools.html?gi-language=c
- Bzip2 manual https://sourceware.org/bzip2/manual/manual.html
This plugin was implemented from the gst-template repository taking GstBaseTransform as base class. It reads the input buffer in the _prepare_buffer method, then inflates it in chunks that are fed downstream as output buffers in the same method. Inflating is done via zlib inflating class, which is instanced at the plugin's _start method and cleaned at the _stop.
- The good: It works! No matter the input blocksize or zlib chunk size.
- The bad: It poorly protects against weird input files and errors.
- The ugly: It trusts the user entirely and doesn't ensure a sane decoding EOF via zlib's retcodes, just some warnings.
Here's a little log to show my thought process.
- Research how to gzip/unzip and how a gzipped file looks
- Fetch sample code on gzipping from C
- Retrieve my old Gstreamer examples
- Figure how to inject and read bytes from/into a buffer
- Take a look around the internet on how to create a plugin
- Read the already known plugin guide and download gst-template
- Download gst-plugins-bad for the templating tools that the plugin guide recommends after telling you to use gst-template
- Compare gst-plugins-bad and gst-template, still can't make any of them work
- Figure how to create a reasonably-sized plugin boilerplate, decide on gst-template
- Figure how to edit the boilerplate and add functionality on the GstFlowReturn method
- Figure how to compile with meson in the two-layered gst-template
- Move the boilerplate into a samples/ dir to experiment more
- For now i was using GstBaseTransform with gst_plug_transform_ip, but I'd need gst_plug_transform and can't seem to simply add it to the gst-template
- A peek on the internet told me that I can also use GstElement base class, though I'm a little more lost in there
- Managed to use GstBaseTransform, I needed a prepare_buffer method too to allocate an output in
- Imported gst template structure onto my repo and modified the source to my sample
- Input data is transformed and fed downstream from the prepare_buffer method
- Read documentation on zlib inflating operation. The stream-like workflow plays well with gstreamer
- Poortly implement zlib inflation on the very prepare_buffer method, should only work for very small input files
- A proper implementation should compose the inflator class into the plugin's, and take advantage of its init and cleanup functions
- Use the start/stop methods of GstBaseTransform to deal with zlib object
- Attempted to keep the zstrm object as an attribute, but couldn't figure it out, neither as property or as g_object_data
- Continue using zstrm as a global variable
- Made the thing more stream-like: upstream blocksize and bad chunksize could break the previous implementation because junk characters would pass through
- Properly aggregate the inflated chunks on a string buffer other than the inflator out buffer
- Use GST_ prints for more reasonable debugging going forward
- Started thinking about an automated validation pipeline, implement test.sh to compare gzdec and gunzip
- Dockerize to deal with gstreamer versions restriction on ubuntu 20, whilst developed in 22
- Fill in FIXME gaps and other aesthetics
- Properly allocate z_stream so it can be stored and loaded as attribute without a global variable
- Assess the code with valgrind and try to clean up as good as possible
- Implement bzip decompression from the zlib plugin
- Add dependencies to docker and update docs and repo accordingly