diff --git a/README.md b/README.md index fe8ddbe..4863412 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ # Capybara +**An Integrated Python Package for Image Processing and Deep Learning.** +

@@ -28,113 +30,66 @@ For more detailed information on installation and usage, please refer to the [** The document provides a detailed explanation of this project and answers to frequently asked questions. -## Installation +## Prerequisites -Before starting the installation of Capybara, ensure that your system meets the following requirements: +Before the installation of Capybara, ensure that your system meets the following requirements: ### Python Version -- Python 3.10 or later is required. +3.10+ ### Dependency Packages Please install the necessary system packages according to your operating system: -- **Ubuntu** - - ```bash - sudo apt install libturbojpeg exiftool ffmpeg libheif-dev - ``` - -- **MacOS** - - ```bash - brew install jpeg-turbo exiftool ffmpeg - ``` - - - **Special Notes**: After testing, there are some known issues when using libheif on macOS, including: - - 1. **Generated HEIC files cannot be opened**: On macOS, HEIC files generated by libheif may not open with certain applications. This may be related to image dimensions, particularly when the image width or height is odd, causing compatibility issues. - - 2. **Compilation errors**: When compiling libheif on macOS, you may encounter undefined symbol errors related to ffmpeg decoders. This could be caused by incorrect compilation options or dependency settings. - - 3. **Example programs do not run**: On macOS Sonoma, the example programs of libheif might fail with dynamic link errors, indicating that `libheif.1.dylib` is missing. This might be related to dynamic library path settings. - - Due to these issues, we currently only run libheif on Ubuntu, and macOS support will be addressed in future versions. - -### pdf2image Dependency - -pdf2image is a Python module used to convert PDF documents to images. Make sure the following tools are installed on your system: - -- MacOS: Install poppler - - ```bash - brew install poppler - ``` - -- Linux: Most distributions already include `pdftoppm` and `pdftocairo`. If not, install them using: - - ```bash - sudo apt install poppler-utils - ``` - -### ONNXRuntime GPU Dependencies - -To use ONNXRuntime for GPU-accelerated inference, ensure that you have an appropriate version of CUDA installed. Here's an example: +#### Ubuntu ```bash -sudo apt install cuda-12-4 -# Add to .bashrc -echo 'export PATH=/usr/local/cuda-12.4/bin${PATH:+:${PATH}}' >> ~/.bashrc -echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc +sudo apt install libturbojpeg exiftool ffmpeg libheif-dev poppler-utils ``` -### Installation via PyPI +##### GPU Dependencies -1. Install the package from PyPI: +To use ONNX Runtime with GPU acceleration, ensure that you install a compatible version, which can be found on the official ONNX Runtime CUDA Execution Provider requirements page. - ```bash - pip install capybara-docsaid - ``` - -2. Verify the installation: - - ```bash - python -c "import capybara; print(capybara.__version__)" - ``` +Here's an example to install cuda-12.8: -3. If the version number is displayed, the installation was successful. - -### Installation via Git Clone +```bash +wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb +sudo dpkg -i cuda-keyring_1.1-1_all.deb +sudo apt-get update +sudo apt-get -y install cuda-toolkit-12-8 +# Post installation, add cuda path to .bashrc or .zshrc +export shellrc="~/.zshrc" +echo 'export PATH=/usr/local/cuda-12.8/bin${PATH:+:${PATH}}' >> $shellrc +echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> $shellrc +``` -1. Clone this repository: +For more details, please see [Nvidia CUDA](https://developer.nvidia.com/cuda-toolkit). - ```bash - git clone https://github.com/DocsaidLab/Capybara.git - ``` +#### MacOS -2. Install the wheel package: +```bash +brew install jpeg-turbo exiftool ffmpeg libheif poppler +``` - ```bash - pip install wheel - ``` +## Installation -3. Build the wheel file: +### PyPI - ```bash - cd Capybara - python setup.py bdist_wheel - ``` +```bash +pip install capybara_docsaid +``` -4. Install the built wheel file: +### Git - ```bash - pip install dist/capybara_docsaid-*-py3-none-any.whl - ``` +```bash +pip install git+https://github.com/DocsaidLab/Capybara.git +``` -### Installation via Docker +## Docker for Deployment -To avoid environment conflicts during deployment or collaborative development, it's recommended to use Docker. Here's a brief guide: +We provide a Docker script for convenient deployment, ensuring a consistent environment. Below are the steps to build the image with Capybara installed. 1. Clone this repository: @@ -149,17 +104,17 @@ To avoid environment conflicts during deployment or collaborative development, i bash docker/build.bash ``` - This will build an image using the [**Dockerfile**](https://github.com/DocsaidLab/Capybara/blob/main/docker/Dockerfile) in the project. The image is based on `nvcr.io/nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04` by default, providing the CUDA environment required for ONNXRuntime inference. + This will build an image using the [**Dockerfile**](docker/Dockerfile) in the project. The image is based on `nvidia/cuda:12.8.1-cudnn-runtime-ubuntu24.04` by default, providing the CUDA environment required for ONNXRuntime inference. 3. After the build is complete, mount the working directory and run the program: ```bash - docker run -v ${PWD}:/code -it capybara_infer_image your_scripts.py + docker run --gpus all -it --rm capybara_docsaid:latest bash ``` - To enable GPU acceleration, add `--gpus all` when running the command. +**PS: If you want to compile cuda or cudnn for developing, please change the base image to `nvidia/cuda:12.8.1-cudnn-devel-ubuntu24.04`.** -#### gosu Permissions Issues +### gosu Permissions Issues If you encounter issues with file ownership as root when running scripts inside the container, causing permission problems, you can use `gosu` to switch users in the Dockerfile. Specify `USER_ID` and `GROUP_ID` when starting the container to avoid frequent permission adjustments in collaborative development. @@ -210,3 +165,16 @@ python -m pytest -vv tests Once completed, you can check if all modules are functioning properly. If any issues arise, first check the environment settings and package versions. If the problem persists, please report it in the Issue section. + +## Citation + +```bibtex +@misc{lin2025capybara, + author = {Kun-Hsiang Lin*, Ze Yuan*}, + title = {Capybara: An Integrated Python Package for Image Processing and Deep Learning.}, + year = {2025}, + publisher = {GitHub}, + howpublished = {\url{https://github.com/DocsaidLab/Capybara}}, + note = {* equal contribution} +} +``` diff --git a/capybara/onnxengine/engine.py b/capybara/onnxengine/engine.py index 5d7065c..46af592 100644 --- a/capybara/onnxengine/engine.py +++ b/capybara/onnxengine/engine.py @@ -14,6 +14,7 @@ class Backend(EnumCheckMixin, Enum): cpu = 0 cuda = 1 + coreml = 2 class ONNXEngine: @@ -45,19 +46,14 @@ def __init__( self.device_id = 0 if backend.name == "cpu" else gpu_id # setting provider options - providers, provider_options = self._get_provider_info(backend, provider_option) + providers = self._get_providers(backend, provider_option) # setting session options sess_options = self._get_session_info(session_option) # setting onnxruntime session model_path = str(model_path) if isinstance(model_path, Path) else model_path - self.sess = ort.InferenceSession( - model_path, - sess_options=sess_options, - providers=providers, - provider_options=provider_options, - ) + self.sess = ort.InferenceSession(model_path, sess_options=sess_options, providers=providers) # setting onnxruntime session info self.model_path = model_path @@ -74,10 +70,7 @@ def __call__(self, **xs) -> Dict[str, np.ndarray]: outs = {k: v for k, v in zip(output_names, outs)} return outs - def _get_session_info( - self, - session_option: Dict[str, Any] = {}, - ) -> ort.SessionOptions: + def _get_session_info(self, session_option: Dict[str, Any] = {}) -> ort.SessionOptions: """ Ref: https://onnxruntime.ai/docs/api/python/api_summary.html#sessionoptions """ @@ -91,30 +84,40 @@ def _get_session_info( setattr(sess_opt, k, v) return sess_opt - def _get_provider_info( - self, - backend: Union[str, int, Backend], - provider_option: Dict[str, Any] = {}, - ) -> Backend: + def _get_providers(self, backend: Union[str, int, Backend], provider_option: Dict[str, Any] = {}) -> Backend: """ Ref: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#configuration-options """ if backend == Backend.cuda: - providers = ["CUDAExecutionProvider"] - provider_option = [ - { - "device_id": self.device_id, - "cudnn_conv_use_max_workspace": "1", - **provider_option, - } + providers = [ + ( + "CUDAExecutionProvider", + { + "device_id": self.device_id, + "cudnn_conv_use_max_workspace": "1", + **provider_option, + }, + ) + ] + elif backend == Backend.coreml: + providers = [ + ( + "CoreMLExecutionProvider", + { + "ModelFormat": "MLProgram", + "MLComputeUnits": "ALL", + "RequireStaticInputShapes": "1", + **provider_option, + }, + ) ] elif backend == Backend.cpu: - providers = ["CPUExecutionProvider"] + providers = [("CPUExecutionProvider", {})] # "CPUExecutionProvider" is different from everything else. - provider_option = None + # provider_option = None else: raise ValueError(f"backend={backend} is not supported.") - return providers, provider_option + return providers def __repr__(self) -> str: import re @@ -132,11 +135,7 @@ def format_nested_dict(dict_data, indent=0): if isinstance(value, dict): info.append(f"{prefix}{key}:") info.append(format_nested_dict(value, indent + 1)) - elif ( - isinstance(value, str) - and value.startswith("{") - and value.endswith("}") - ): + elif isinstance(value, str) and value.startswith("{") and value.endswith("}"): try: nested_dict = eval(value) if isinstance(nested_dict, dict): @@ -153,9 +152,7 @@ def format_nested_dict(dict_data, indent=0): title = "DOCSAID X ONNXRUNTIME" divider_length = 50 divider = f"+{'-' * divider_length}+" - styled_title = colored.stylize( - title, [colored.fg("blue"), colored.attr("bold")] - ) + styled_title = colored.stylize(title, [colored.fg("blue"), colored.attr("bold")]) def center_text(text, width): """Center text within a fixed width, handling ANSI escape codes.""" diff --git a/capybara/onnxengine/tools.py b/capybara/onnxengine/tools.py index aab00f9..b0d71d7 100644 --- a/capybara/onnxengine/tools.py +++ b/capybara/onnxengine/tools.py @@ -3,12 +3,12 @@ import onnx import onnxsim -from onnx.helper import (make_graph, make_model, make_opsetid, - tensor_dtype_to_np_dtype) +from onnx.helper import make_graph, make_model, make_opsetid, tensor_dtype_to_np_dtype __all__ = [ - 'get_onnx_input_infos', - 'get_onnx_output_infos', + "get_onnx_input_infos", + "get_onnx_output_infos", + "make_onnx_dynamic_axes", ] @@ -17,8 +17,8 @@ def get_onnx_input_infos(model: Union[str, Path, onnx.ModelProto]) -> Dict[str, model = onnx.load(model) return { x.name: { - 'shape': [d.dim_value if d.dim_value != 0 else -1 for d in x.type.tensor_type.shape.dim], - 'dtype': tensor_dtype_to_np_dtype(x.type.tensor_type.elem_type) + "shape": [d.dim_value if d.dim_value != 0 else -1 for d in x.type.tensor_type.shape.dim], + "dtype": tensor_dtype_to_np_dtype(x.type.tensor_type.elem_type), } for x in model.graph.input } @@ -29,8 +29,8 @@ def get_onnx_output_infos(model: Union[str, Path, onnx.ModelProto]) -> Dict[str, model = onnx.load(model) return { x.name: { - 'shape': [d.dim_value if d.dim_value != 0 else -1 for d in x.type.tensor_type.shape.dim], - 'dtype': tensor_dtype_to_np_dtype(x.type.tensor_type.elem_type) + "shape": [d.dim_value if d.dim_value != 0 else -1 for d in x.type.tensor_type.shape.dim], + "dtype": tensor_dtype_to_np_dtype(x.type.tensor_type.elem_type), } for x in model.graph.output } @@ -54,10 +54,10 @@ def make_onnx_dynamic_axes( value_info=None, ) - if not any(opset.domain == '' for opset in onnx_model.opset_import): + if not any(opset.domain == "" for opset in onnx_model.opset_import): onnx_model.opset_import.append(make_opsetid(domain="", version=opset_version)) - new_model = make_model(new_graph, opset_imports=onnx_model.opset_import) + new_model = make_model(new_graph, opset_imports=onnx_model.opset_import, ir_version=onnx_model.ir_version) for x in new_model.graph.input: for name, v in input_dims.items(): @@ -72,7 +72,7 @@ def make_onnx_dynamic_axes( x.type.tensor_type.shape.dim[k].dim_param = d for x in new_model.graph.node: - if x.op_type == 'Reshape': + if x.op_type == "Reshape": raise ValueError("Reshape cannot be trasformed to dynamic axes") new_model, _ = onnxsim.simplify(new_model) diff --git a/capybara/vision/improc.py b/capybara/vision/improc.py index 8a042a3..39e59db 100644 --- a/capybara/vision/improc.py +++ b/capybara/vision/improc.py @@ -1,4 +1,3 @@ -import platform import warnings from pathlib import Path from typing import Any, List, Union @@ -6,6 +5,7 @@ import cv2 import numpy as np import piexif +import pillow_heif import pybase64 from pdf2image import convert_from_bytes, convert_from_path from turbojpeg import TurboJPEG @@ -14,16 +14,28 @@ from .functionals import imcvtcolor from .geometric import imrotate90 -if platform.system() == 'Linux': - import pyheif -else: - print('Image file with `.heif` or `heic` are only available on Linux system.') - __all__ = [ - 'imread', 'imwrite', 'imencode', 'imdecode', 'img_to_b64', 'img_to_b64str', - 'b64_to_img', 'b64str_to_img', 'b64_to_npy', 'b64str_to_npy', 'npy_to_b64', - 'npy_to_b64str', 'npyread', 'pdf2imgs', 'jpgencode', 'jpgdecode', 'jpgread', - 'pngencode', 'pngdecode', 'is_numpy_img', 'get_orientation_code' + "imread", + "imwrite", + "imencode", + "imdecode", + "img_to_b64", + "img_to_b64str", + "b64_to_img", + "b64str_to_img", + "b64_to_npy", + "b64str_to_npy", + "npy_to_b64", + "npy_to_b64str", + "npyread", + "pdf2imgs", + "jpgencode", + "jpgdecode", + "jpgread", + "pngencode", + "pngdecode", + "is_numpy_img", + "get_orientation_code", ] jpeg = TurboJPEG() @@ -57,7 +69,7 @@ def jpgencode(img: np.ndarray, quality: int = 90) -> Union[bytes, None]: if is_numpy_img(img): try: byte_ = jpeg.encode(img, quality=quality) - except: + except Exception as _: pass return byte_ @@ -66,16 +78,15 @@ def jpgdecode(byte_: bytes) -> Union[np.ndarray, None]: try: bgr_array = jpeg.decode(byte_) code = get_orientation_code(byte_) - bgr_array = imrotate90( - bgr_array, code) if code is not None else bgr_array - except: + bgr_array = imrotate90(bgr_array, code) if code is not None else bgr_array + except Exception as _: bgr_array = None return bgr_array def jpgread(img_file: Union[str, Path]) -> Union[np.ndarray, None]: - with open(str(img_file), 'rb') as f: + with open(str(img_file), "rb") as f: binary_img = f.read() bgr_array = jpgdecode(binary_img) @@ -86,18 +97,17 @@ def pngencode(img: np.ndarray, compression: int = 1) -> Union[bytes, None]: byte_ = None if is_numpy_img(img): try: - byte_ = cv2.imencode('.png', img, params=[int( - cv2.IMWRITE_PNG_COMPRESSION), compression])[1].tobytes() - except: + byte_ = cv2.imencode(".png", img, params=[int(cv2.IMWRITE_PNG_COMPRESSION), compression])[1].tobytes() + except Exception as _: pass return byte_ def pngdecode(byte_: bytes) -> Union[np.ndarray, None]: try: - enc = np.frombuffer(byte_, 'uint8') + enc = np.frombuffer(byte_, "uint8") img = cv2.imdecode(enc, cv2.IMREAD_COLOR) - except: + except Exception as _: img = None return img @@ -113,7 +123,7 @@ def imdecode(byte_: bytes) -> Union[np.ndarray, None]: try: img = jpgdecode(byte_) img = pngdecode(byte_) if img is None else img - except: + except Exception as _: img = None return img @@ -123,23 +133,21 @@ def img_to_b64(img: np.ndarray, IMGTYP: Union[str, int, IMGTYP] = IMGTYP.JPEG) - encode_fn = jpgencode if IMGTYP == IMGTYP.JPEG else pngencode try: b64 = pybase64.b64encode(encode_fn(img)) - except: + except Exception as _: b64 = None return b64 -def npy_to_b64(x: np.ndarray, dtype='float32') -> bytes: +def npy_to_b64(x: np.ndarray, dtype="float32") -> bytes: return pybase64.b64encode(x.astype(dtype).tobytes()) -def npy_to_b64str(x: np.ndarray, dtype='float32', string_encode: str = 'utf-8') -> str: +def npy_to_b64str(x: np.ndarray, dtype="float32", string_encode: str = "utf-8") -> str: return pybase64.b64encode(x.astype(dtype).tobytes()).decode(string_encode) def img_to_b64str( - img: np.ndarray, - IMGTYP: Union[str, int, IMGTYP] = IMGTYP.JPEG, - string_encode: str = 'utf-8' + img: np.ndarray, IMGTYP: Union[str, int, IMGTYP] = IMGTYP.JPEG, string_encode: str = "utf-8" ) -> Union[str, None]: b64 = img_to_b64(img, IMGTYP) return b64.decode(string_encode) if isinstance(b64, bytes) else None @@ -148,16 +156,12 @@ def img_to_b64str( def b64_to_img(b64: bytes) -> Union[np.ndarray, None]: try: img = imdecode(pybase64.b64decode(b64)) - except: + except Exception as _: img = None return img -def b64str_to_img( - b64str: Union[str, None], - string_encode: str = 'utf-8' -) -> Union[np.ndarray, None]: - +def b64str_to_img(b64str: Union[str, None], string_encode: str = "utf-8") -> Union[np.ndarray, None]: if b64str is None: warnings.warn("b64str is None.") return None @@ -168,42 +172,24 @@ def b64str_to_img( return b64_to_img(b64str.encode(string_encode)) -def b64_to_npy(x: bytes, dtype='float32') -> np.ndarray: +def b64_to_npy(x: bytes, dtype="float32") -> np.ndarray: return np.frombuffer(pybase64.b64decode(x), dtype=dtype) -def b64str_to_npy(x: bytes, dtype='float32', string_encode: str = 'utf-8') -> np.ndarray: +def b64str_to_npy(x: bytes, dtype="float32", string_encode: str = "utf-8") -> np.ndarray: return np.frombuffer(pybase64.b64decode(x.encode(string_encode)), dtype=dtype) def npyread(path: Union[str, Path]) -> Union[np.ndarray, None]: try: - with open(str(path), 'rb') as f: + with open(str(path), "rb") as f: img = np.load(f) - except: + except Exception as _: img = None return img -def read_heic_to_numpy(file_path: str): - heif_file = pyheif.read(file_path) - data = heif_file.data - if heif_file.mode == "RGB": - numpy_array = np.frombuffer(data, dtype=np.uint8).reshape( - heif_file.size[1], heif_file.size[0], 3) - elif heif_file.mode == "RGBA": - numpy_array = np.frombuffer(data, dtype=np.uint8).reshape( - heif_file.size[1], heif_file.size[0], 4) - else: - raise ValueError("Unsupported HEIC color mode") - return numpy_array - - -def imread( - path: Union[str, Path], - color_base: str = 'BGR', - verbose: bool = False -) -> Union[np.ndarray, None]: +def imread(path: Union[str, Path], color_base: str = "BGR", verbose: bool = False) -> Union[np.ndarray, None]: """ This function reads an image from a given file path and converts its color base if necessary. @@ -227,13 +213,11 @@ def imread( The image as a numpy ndarray if successful, None otherwise. """ if not Path(path).exists(): - raise FileExistsError(f'{path} can not found.') + raise FileExistsError(f"{path} can not found.") - if Path(path).suffix.lower() == '.heic': - if platform.system() != 'Linux': - raise ValueError('HEIC file is only supported on Linux system.') - img = read_heic_to_numpy(str(path)) - img = cv2.cvtColor(img, cv2.COLOR_RGB2BGR) + if Path(path).suffix.lower() == ".heic": + heif_file = pillow_heif.open_heif(str(path), convert_hdr_to_8bit=True, bgr_mode=True) + img = np.asarray(heif_file) else: img = jpgread(path) img = cv2.imread(str(path)) if img is None else img @@ -243,8 +227,8 @@ def imread( warnings.warn("Got a None type image.") return - if color_base != 'BGR': - img = imcvtcolor(img, cvt_mode=f'BGR2{color_base}') + if color_base != "BGR": + img = imcvtcolor(img, cvt_mode=f"BGR2{color_base}") return img @@ -252,8 +236,8 @@ def imread( def imwrite( img: np.ndarray, path: Union[str, Path] = None, - color_base: str = 'BGR', - suffix: str = '.jpg', + color_base: str = "BGR", + suffix: str = ".jpg", ) -> bool: """ Writes an image to a file with optional color base conversion. @@ -274,9 +258,9 @@ def imwrite( bool: True if the write operation is successful, False otherwise. """ color_base = color_base.upper() - if color_base != 'BGR': - img = imcvtcolor(img, cvt_mode=f'{color_base}2BGR') - return cv2.imwrite(str(path) if path else f'tmp{suffix}', img) + if color_base != "BGR": + img = imcvtcolor(img, cvt_mode=f"{color_base}2BGR") + return cv2.imwrite(str(path) if path else f"tmp{suffix}", img) def pdf2imgs(stream: Union[str, Path, bytes]) -> Union[List[np.ndarray], None]: @@ -294,6 +278,6 @@ def pdf2imgs(stream: Union[str, Path, bytes]) -> Union[List[np.ndarray], None]: pil_imgs = convert_from_bytes(stream) else: pil_imgs = convert_from_path(stream) - return [imcvtcolor(np.array(img), cvt_mode='RGB2BGR') for img in pil_imgs] - except: + return [imcvtcolor(np.array(img), cvt_mode="RGB2BGR") for img in pil_imgs] + except Exception as _: return diff --git a/docker/Dockerfile b/docker/Dockerfile index 3120c77..43570b9 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -1,5 +1,5 @@ # syntax=docker/dockerfile:experimental -FROM nvcr.io/nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04 +FROM nvidia/cuda:12.8.1-cudnn-runtime-ubuntu22.04 as builder ENV PYTHONDONTWRITEBYTECODE=1 \ DEBIAN_FRONTEND=noninteractive \ @@ -22,11 +22,7 @@ RUN ln -s /usr/bin/python3 /usr/bin/python && \ COPY . /usr/local/Capybara RUN cd /usr/local/Capybara && python setup.py bdist_wheel && \ - python -m pip install dist/*.whl && rm -rf /usr/local/Capybara - -# opencv-fixer -RUN pip install opencv-fixer -RUN python -c "from opencv_fixer import AutoFix; AutoFix()" + python -m pip install dist/*.whl && rm -rf /usr/local/Capybara # Preload data RUN python -c "import capybara" diff --git a/docker/build.bash b/docker/build.bash index 16dded5..8d0c768 100644 --- a/docker/build.bash +++ b/docker/build.bash @@ -1,3 +1,3 @@ docker build \ -f docker/Dockerfile \ - -t capybara_infer_image . + -t capybara_docsaid . diff --git a/docker/pr.dockerfile b/docker/pr.dockerfile index 14e2aa0..841c4cb 100644 --- a/docker/pr.dockerfile +++ b/docker/pr.dockerfile @@ -1,5 +1,5 @@ # syntax=docker/dockerfile:experimental -FROM nvcr.io/nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04 +FROM nvidia/cuda:12.8.1-cudnn-runtime-ubuntu22.04 as builder ENV PYTHONDONTWRITEBYTECODE=1 \ DEBIAN_FRONTEND=noninteractive \ diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..7bcfd26 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,64 @@ +[build-system] +requires = ["setuptools>=61", "wheel"] +build-backend = "setuptools.build_meta" + +[project] +name = "capybara_docsaid" +dynamic = ["version"] +description = "An Image Processing and Deep Learning Toolkit." +readme = {file = "README.md", content-type = "text/markdown"} +license = {text = "Apache License 2.0"} +requires-python = ">=3.10,<3.13" +classifiers = [ + "Development Status :: 5 - Production/Stable", + "License :: OSI Approved :: Apache Software License", + "Intended Audience :: Developers", + "Intended Audience :: Science/Research", + "Operating System :: OS Independent", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Topic :: Software Development :: Libraries", + "Topic :: Software Development :: Libraries :: Python Modules" +] +dependencies = [ + "dacite", + "psutil", + "requests", + "onnx", + "colored", + "numpy", + "pdf2image", + "ujson", + "pyyaml", + "tqdm", + "pybase64", + "PyTurboJPEG", + "dill", + "networkx", + "natsort", + "flask", + "shapely", + "piexif", + "matplotlib", + "opencv-python>=4.12.0.88", + "onnxsim", + "beautifulsoup4", + "onnxruntime==1.22.0; platform_system == 'Darwin'", + "onnxruntime_gpu==1.22.0; platform_system == 'Linux'", + "pillow-heif" +] + +[project.urls] +Homepage = "https://docsaid.org/en/docs/capybara/" +Repository = "https://github.com/DocsaidLab/Capybara" +Issues = "https://github.com/DocsaidLab/Capybara/issues" + +[tool.setuptools] +include-package-data = true + +[tool.setuptools.packages.find] +exclude = ["docker", "demo", "tests", "benchmarks"] + +[tool.setuptools.dynamic] +version = {attr = "capybara.__version__"} \ No newline at end of file diff --git a/setup.cfg b/setup.cfg index 1ba9a23..962694d 100644 --- a/setup.cfg +++ b/setup.cfg @@ -1,7 +1,7 @@ [metadata] name = capybara_docsaid version = attr: capybara.__version__ -description = OpenCV with ONNX Runtime Inference Toolkit. +description = An Image Processing and Deep Learning Toolkit. long_description = file: README.md long_description_content_type = text/markdown license = Apache License 2.0 @@ -16,7 +16,7 @@ classifiers= Programming Language :: Python :: 3.12 Topic :: Software Development :: Libraries Topic :: Software Development :: Libraries :: Python Modules -python_requires = >=3.10,<=3.12 +python_requires = >=3.10,<3.13 url = https://github.com/DocsaidLab/Capybara.git [options] @@ -27,13 +27,12 @@ setup_requires= setuptools wheel install_requires = - pyheif;platform_system=='Linux' dacite psutil requests onnx colored - numpy<2.0.0 + numpy pdf2image ujson pyyaml @@ -47,11 +46,12 @@ install_requires = shapely piexif matplotlib - opencv-python==4.9.0.80 - onnxruntime==1.20.1;platform_system=='Darwin' - onnxruntime_gpu==1.20.1;platform_system=='Linux' + opencv-python>=4.12.0.88 + onnxruntime==1.22.0;platform_system=='Darwin' + onnxruntime_gpu==1.22.0;platform_system=='Linux' onnxsim beautifulsoup4 + pillow-heif [options.packages.find] exclude = diff --git a/tests/onnxruntime/test_engine.py b/tests/onnxruntime/test_engine.py index e07b20b..088dabc 100644 --- a/tests/onnxruntime/test_engine.py +++ b/tests/onnxruntime/test_engine.py @@ -1,9 +1,13 @@ +import platform + import numpy as np +import pytest from capybara import ONNXEngine, get_curdir -def test_ONNXEngine(): +@pytest.mark.skipif(platform.system() != "Linux", reason="Linux only") +def test_ONNXEngine_CUDA(): model_path = get_curdir(__file__).parent / "resources/model_dynamic-axes.onnx" engine = ONNXEngine(model_path, backend="cuda") for i in range(5): @@ -12,3 +16,26 @@ def test_ONNXEngine(): if i: assert not np.allclose(outs["output"], prev_outs["output"]) prev_outs = outs + + +def test_ONNXEngine_CPU(): + model_path = get_curdir(__file__).parent / "resources/model_dynamic-axes.onnx" + engine = ONNXEngine(model_path, backend="cpu") + for i in range(5): + xs = {"input": np.random.randn(32, 3, 224, 224).astype("float32")} + outs = engine(**xs) + if i: + assert not np.allclose(outs["output"], prev_outs["output"]) + prev_outs = outs + + +@pytest.mark.skipif(platform.system() != "Darwin", reason="Mac only") +def test_ONNXEngine_COREML(): + model_path = get_curdir(__file__).parent / "resources/model_dynamic-axes.onnx" + engine = ONNXEngine(model_path, backend="coreml") + for i in range(5): + xs = {"input": np.random.randn(32, 3, 224, 224).astype("float32")} + outs = engine(**xs) + if i: + assert not np.allclose(outs["output"], prev_outs["output"]) + prev_outs = outs diff --git a/tests/onnxruntime/test_engine_io_binding.py b/tests/onnxruntime/test_engine_io_binding.py index 9f39c69..d17a898 100644 --- a/tests/onnxruntime/test_engine_io_binding.py +++ b/tests/onnxruntime/test_engine_io_binding.py @@ -1,9 +1,13 @@ +import platform + import numpy as np +import pytest from capybara import ONNXEngineIOBinding, get_curdir -def test_ONNXEngineIOBinding(): +@pytest.mark.skipif(platform.system() != "Linux", reason="Linux only") +def test_ONNXEngineIOBinding_CUDAonly(): model_path = get_curdir(__file__).parent / "resources/model_dynamic-axes.onnx" input_initializer = {"input": np.random.randn(32, 3, 448, 448).astype("float32")} engine = ONNXEngineIOBinding(model_path, input_initializer) diff --git a/tests/onnxruntime/test_tools.py b/tests/onnxruntime/test_tools.py index f9c922f..23f6016 100644 --- a/tests/onnxruntime/test_tools.py +++ b/tests/onnxruntime/test_tools.py @@ -1,7 +1,7 @@ import numpy as np from capybara import ( - ONNXEngineIOBinding, + ONNXEngine, get_onnx_input_infos, get_onnx_output_infos, make_onnx_dynamic_axes, @@ -34,8 +34,6 @@ def test_make_onnx_dynamic_axes(): output_dims=output_dims, ) xs = {"input": np.random.randn(32, 3, 320, 320).astype("float32")} - engine = ONNXEngineIOBinding( - new_model_path, input_initializer=xs, session_option={"log_severity_level": 1} - ) + engine = ONNXEngine(new_model_path, session_option={"log_severity_level": 1}, backend="cpu") outs = engine(**xs) assert outs["output"].shape == (32, 64, 80, 80) diff --git a/tests/resources/lena.heic b/tests/resources/lena.heic new file mode 100644 index 0000000..b3389eb Binary files /dev/null and b/tests/resources/lena.heic differ diff --git a/tests/vision/test_improc.py b/tests/vision/test_improc.py index 2e4155f..4ed3581 100644 --- a/tests/vision/test_improc.py +++ b/tests/vision/test_improc.py @@ -10,21 +10,26 @@ def test_imread(): # 測試圖片路徑 - image_path = DIR.parent / 'resources' / 'lena.png' + image_path = DIR.parent / "resources" / "lena.png" # 測試 BGR 格式的圖片讀取 - img_bgr = imread(image_path, color_base='BGR') + img_bgr = imread(image_path, color_base="BGR") assert isinstance(img_bgr, np.ndarray) assert img_bgr.shape[-1] == 3 # BGR圖片的channel數為3 # 測試灰階格式的圖片讀取 - img_gray = imread(image_path, color_base='GRAY') + img_gray = imread(image_path, color_base="GRAY") assert isinstance(img_gray, np.ndarray) assert len(img_gray.shape) == 2 # 灰階圖片的channel數為1 + # 測試heif格式的圖片讀取 + img_heif = imread(DIR.parent / "resources" / "lena.heic", color_base="BGR") + assert isinstance(img_heif, np.ndarray) + assert img_heif.shape[-1] == 3 # BGR圖片的channel數為3 + # 測試不存在的圖片路徑 with pytest.raises(FileExistsError): - imread('non_existent_image.jpg') + imread("non_existent_image.jpg") def test_imwrite(): @@ -32,9 +37,9 @@ def test_imwrite(): img = np.zeros((100, 100, 3), dtype=np.uint8) # 建立一個全黑的BGR圖片 # 測試BGR格式的圖片寫入 - temp_file_path = DIR / 'temp_image.jpg' - assert imwrite(img, path=temp_file_path, color_base='BGR') + temp_file_path = DIR / "temp_image.jpg" + assert imwrite(img, path=temp_file_path, color_base="BGR") assert Path(temp_file_path).exists() # 測試不指定路徑時的圖片寫入 - assert imwrite(img, color_base='BGR') # 將會寫入一個暫時的檔案 + assert imwrite(img, color_base="BGR") # 將會寫入一個暫時的檔案