[C] Update opencv for numpy>2.0

README.md

@@ -2,6 +2,8 @@
 
 # Capybara
 
+**An Integrated Python Package for Image Processing and Deep Learning.**
+
 <p align="left">
     <a href="./LICENSE"><img src="https://img.shields.io/badge/license-Apache%202-dfd.svg"></a>
     <a href=""><img src="https://img.shields.io/badge/python-3.10+-aff.svg"></a>
@@ -28,113 +30,66 @@ For more detailed information on installation and usage, please refer to the [**
 
 The document provides a detailed explanation of this project and answers to frequently asked questions.
 
-## Installation
+## Prerequisites
 
-Before starting the installation of Capybara, ensure that your system meets the following requirements:
+Before the installation of Capybara, ensure that your system meets the following requirements:
 
 ### Python Version
 
-- Python 3.10 or later is required.
+3.10+
 
 ### Dependency Packages
 
 Please install the necessary system packages according to your operating system:
 
-- **Ubuntu**
-
-  ```bash
-  sudo apt install libturbojpeg exiftool ffmpeg libheif-dev
-  ```
-
-- **MacOS**
-
-  ```bash
-  brew install jpeg-turbo exiftool ffmpeg
-  ```
-
-  - **Special Notes**: After testing, there are some known issues when using libheif on macOS, including:
-
-    1. **Generated HEIC files cannot be opened**: On macOS, HEIC files generated by libheif may not open with certain applications. This may be related to image dimensions, particularly when the image width or height is odd, causing compatibility issues.
-
-    2. **Compilation errors**: When compiling libheif on macOS, you may encounter undefined symbol errors related to ffmpeg decoders. This could be caused by incorrect compilation options or dependency settings.
-
-    3. **Example programs do not run**: On macOS Sonoma, the example programs of libheif might fail with dynamic link errors, indicating that `libheif.1.dylib` is missing. This might be related to dynamic library path settings.
-
-    Due to these issues, we currently only run libheif on Ubuntu, and macOS support will be addressed in future versions.
-
-### pdf2image Dependency
-
-pdf2image is a Python module used to convert PDF documents to images. Make sure the following tools are installed on your system:
-
-- MacOS: Install poppler
-
-  ```bash
-  brew install poppler
-  ```
-
-- Linux: Most distributions already include `pdftoppm` and `pdftocairo`. If not, install them using:
-
-  ```bash
-  sudo apt install poppler-utils
-  ```
-
-### ONNXRuntime GPU Dependencies
-
-To use ONNXRuntime for GPU-accelerated inference, ensure that you have an appropriate version of CUDA installed. Here's an example:
+#### Ubuntu
 
 ```bash
-sudo apt install cuda-12-4
-# Add to .bashrc
-echo 'export PATH=/usr/local/cuda-12.4/bin${PATH:+:${PATH}}' >> ~/.bashrc
-echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc
+sudo apt install libturbojpeg exiftool ffmpeg libheif-dev poppler-utils
 ```
 
-### Installation via PyPI
+##### GPU Dependencies
 
-1. Install the package from PyPI:
+To use ONNX Runtime with GPU acceleration, ensure that you install a compatible version, which can be found on the official ONNX Runtime CUDA Execution Provider requirements page.
 
-   ```bash
-   pip install capybara-docsaid
-   ```
-
-2. Verify the installation:
-
-   ```bash
-   python -c "import capybara; print(capybara.__version__)"
-   ```
+Here's an example to install cuda-12.8:
 
-3. If the version number is displayed, the installation was successful.
-
-### Installation via Git Clone
+```bash
+wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
+sudo dpkg -i cuda-keyring_1.1-1_all.deb
+sudo apt-get update
+sudo apt-get -y install cuda-toolkit-12-8
+# Post installation, add cuda path to .bashrc or .zshrc
+export shellrc="~/.zshrc"
+echo 'export PATH=/usr/local/cuda-12.8/bin${PATH:+:${PATH}}' >> $shellrc
+echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> $shellrc
+```
 
-1. Clone this repository:
+For more details, please see [Nvidia CUDA](https://developer.nvidia.com/cuda-toolkit).
 
-   ```bash
-   git clone https://github.com/DocsaidLab/Capybara.git
-   ```
+#### MacOS
 
-2. Install the wheel package:
+```bash
+brew install jpeg-turbo exiftool ffmpeg libheif poppler
+```
 
-   ```bash
-   pip install wheel
-   ```
+## Installation
 
-3. Build the wheel file:
+### PyPI
 
-   ```bash
-   cd Capybara
-   python setup.py bdist_wheel
-   ```
+```bash
+pip install capybara_docsaid
+```
 
-4. Install the built wheel file:
+### Git
 
-   ```bash
-   pip install dist/capybara_docsaid-*-py3-none-any.whl
-   ```
+```bash
+pip install git+https://github.com/DocsaidLab/Capybara.git
+```
 
-### Installation via Docker
+## Docker for Deployment
 
-To avoid environment conflicts during deployment or collaborative development, it's recommended to use Docker. Here's a brief guide:
+We provide a Docker script for convenient deployment, ensuring a consistent environment. Below are the steps to build the image with Capybara installed.
 
 1. Clone this repository:
 
@@ -149,17 +104,17 @@ To avoid environment conflicts during deployment or collaborative development, i
    bash docker/build.bash
    ```
 
-   This will build an image using the [**Dockerfile**](https://github.com/DocsaidLab/Capybara/blob/main/docker/Dockerfile) in the project. The image is based on `nvcr.io/nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04` by default, providing the CUDA environment required for ONNXRuntime inference.
+   This will build an image using the [**Dockerfile**](docker/Dockerfile) in the project. The image is based on `nvidia/cuda:12.8.1-cudnn-runtime-ubuntu24.04` by default, providing the CUDA environment required for ONNXRuntime inference.
 
 3. After the build is complete, mount the working directory and run the program:
 
    ```bash
-   docker run -v ${PWD}:/code -it capybara_infer_image your_scripts.py
+   docker run --gpus all -it --rm capybara_docsaid:latest bash
    ```
 
-   To enable GPU acceleration, add `--gpus all` when running the command.
+**PS: If you want to compile cuda or cudnn for developing, please change the base image to `nvidia/cuda:12.8.1-cudnn-devel-ubuntu24.04`.**
 
-#### gosu Permissions Issues
+### gosu Permissions Issues
 
 If you encounter issues with file ownership as root when running scripts inside the container, causing permission problems, you can use `gosu` to switch users in the Dockerfile. Specify `USER_ID` and `GROUP_ID` when starting the container to avoid frequent permission adjustments in collaborative development.
 
@@ -210,3 +165,16 @@ python -m pytest -vv tests
 Once completed, you can check if all modules are functioning properly. If any issues arise, first check the environment settings and package versions.
 
 If the problem persists, please report it in the Issue section.
+
+## Citation
+
+```bibtex
+@misc{lin2025capybara,
+  author       = {Kun-Hsiang Lin*, Ze Yuan*},
+  title        = {Capybara: An Integrated Python Package for Image Processing and Deep Learning.},
+  year         = {2025},
+  publisher    = {GitHub},
+  howpublished = {\url{https://github.com/DocsaidLab/Capybara}},
+  note         = {* equal contribution}
+}
+```

capybara/onnxengine/engine.py

@@ -14,6 +14,7 @@
 class Backend(EnumCheckMixin, Enum):
     cpu = 0
     cuda = 1
+    coreml = 2
 
 
 class ONNXEngine:
@@ -45,19 +46,14 @@ def __init__(
         self.device_id = 0 if backend.name == "cpu" else gpu_id
 
         # setting provider options
-        providers, provider_options = self._get_provider_info(backend, provider_option)
+        providers = self._get_providers(backend, provider_option)
 
         # setting session options
         sess_options = self._get_session_info(session_option)
 
         # setting onnxruntime session
         model_path = str(model_path) if isinstance(model_path, Path) else model_path
-        self.sess = ort.InferenceSession(
-            model_path,
-            sess_options=sess_options,
-            providers=providers,
-            provider_options=provider_options,
-        )
+        self.sess = ort.InferenceSession(model_path, sess_options=sess_options, providers=providers)
 
         # setting onnxruntime session info
         self.model_path = model_path
@@ -74,10 +70,7 @@ def __call__(self, **xs) -> Dict[str, np.ndarray]:
         outs = {k: v for k, v in zip(output_names, outs)}
         return outs
 
-    def _get_session_info(
-        self,
-        session_option: Dict[str, Any] = {},
-    ) -> ort.SessionOptions:
+    def _get_session_info(self, session_option: Dict[str, Any] = {}) -> ort.SessionOptions:
         """
         Ref: https://onnxruntime.ai/docs/api/python/api_summary.html#sessionoptions
         """
@@ -91,30 +84,40 @@ def _get_session_info(
             setattr(sess_opt, k, v)
         return sess_opt
 
-    def _get_provider_info(
-        self,
-        backend: Union[str, int, Backend],
-        provider_option: Dict[str, Any] = {},
-    ) -> Backend:
+    def _get_providers(self, backend: Union[str, int, Backend], provider_option: Dict[str, Any] = {}) -> Backend:
         """
         Ref: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#configuration-options
         """
         if backend == Backend.cuda:
-            providers = ["CUDAExecutionProvider"]
-            provider_option = [
-                {
-                    "device_id": self.device_id,
-                    "cudnn_conv_use_max_workspace": "1",
-                    **provider_option,
-                }
+            providers = [
+                (
+                    "CUDAExecutionProvider",
+                    {
+                        "device_id": self.device_id,
+                        "cudnn_conv_use_max_workspace": "1",
+                        **provider_option,
+                    },
+                )
+            ]
+        elif backend == Backend.coreml:
+            providers = [
+                (
+                    "CoreMLExecutionProvider",
+                    {
+                        "ModelFormat": "MLProgram",
+                        "MLComputeUnits": "ALL",
+                        "RequireStaticInputShapes": "1",
+                        **provider_option,
+                    },
+                )
             ]
         elif backend == Backend.cpu:
-            providers = ["CPUExecutionProvider"]
+            providers = [("CPUExecutionProvider", {})]
             # "CPUExecutionProvider" is different from everything else.
-            provider_option = None
+            # provider_option = None
         else:
             raise ValueError(f"backend={backend} is not supported.")
-        return providers, provider_option
+        return providers
 
     def __repr__(self) -> str:
         import re
@@ -132,11 +135,7 @@ def format_nested_dict(dict_data, indent=0):
                 if isinstance(value, dict):
                     info.append(f"{prefix}{key}:")
                     info.append(format_nested_dict(value, indent + 1))
-                elif (
-                    isinstance(value, str)
-                    and value.startswith("{")
-                    and value.endswith("}")
-                ):
+                elif isinstance(value, str) and value.startswith("{") and value.endswith("}"):
                     try:
                         nested_dict = eval(value)
                         if isinstance(nested_dict, dict):
@@ -153,9 +152,7 @@ def format_nested_dict(dict_data, indent=0):
         title = "DOCSAID X ONNXRUNTIME"
         divider_length = 50
         divider = f"+{'-' * divider_length}+"
-        styled_title = colored.stylize(
-            title, [colored.fg("blue"), colored.attr("bold")]
-        )
+        styled_title = colored.stylize(title, [colored.fg("blue"), colored.attr("bold")])
 
         def center_text(text, width):
             """Center text within a fixed width, handling ANSI escape codes."""

capybara/onnxengine/tools.py

@@ -3,12 +3,12 @@
 
 import onnx
 import onnxsim
-from onnx.helper import (make_graph, make_model, make_opsetid,
-                         tensor_dtype_to_np_dtype)
+from onnx.helper import make_graph, make_model, make_opsetid, tensor_dtype_to_np_dtype
 
 __all__ = [
-    'get_onnx_input_infos',
-    'get_onnx_output_infos',
+    "get_onnx_input_infos",
+    "get_onnx_output_infos",
+    "make_onnx_dynamic_axes",
 ]
 
 
@@ -17,8 +17,8 @@ def get_onnx_input_infos(model: Union[str, Path, onnx.ModelProto]) -> Dict[str,
         model = onnx.load(model)
     return {
         x.name: {
-            'shape': [d.dim_value if d.dim_value != 0 else -1 for d in x.type.tensor_type.shape.dim],
-            'dtype': tensor_dtype_to_np_dtype(x.type.tensor_type.elem_type)
+            "shape": [d.dim_value if d.dim_value != 0 else -1 for d in x.type.tensor_type.shape.dim],
+            "dtype": tensor_dtype_to_np_dtype(x.type.tensor_type.elem_type),
         }
         for x in model.graph.input
     }
@@ -29,8 +29,8 @@ def get_onnx_output_infos(model: Union[str, Path, onnx.ModelProto]) -> Dict[str,
         model = onnx.load(model)
     return {
         x.name: {
-            'shape': [d.dim_value if d.dim_value != 0 else -1 for d in x.type.tensor_type.shape.dim],
-            'dtype': tensor_dtype_to_np_dtype(x.type.tensor_type.elem_type)
+            "shape": [d.dim_value if d.dim_value != 0 else -1 for d in x.type.tensor_type.shape.dim],
+            "dtype": tensor_dtype_to_np_dtype(x.type.tensor_type.elem_type),
         }
         for x in model.graph.output
     }
@@ -54,10 +54,10 @@ def make_onnx_dynamic_axes(
         value_info=None,
     )
 
-    if not any(opset.domain == '' for opset in onnx_model.opset_import):
+    if not any(opset.domain == "" for opset in onnx_model.opset_import):
         onnx_model.opset_import.append(make_opsetid(domain="", version=opset_version))
 
-    new_model = make_model(new_graph, opset_imports=onnx_model.opset_import)
+    new_model = make_model(new_graph, opset_imports=onnx_model.opset_import, ir_version=onnx_model.ir_version)
 
     for x in new_model.graph.input:
         for name, v in input_dims.items():
@@ -72,7 +72,7 @@ def make_onnx_dynamic_axes(
                     x.type.tensor_type.shape.dim[k].dim_param = d
 
     for x in new_model.graph.node:
-        if x.op_type == 'Reshape':
+        if x.op_type == "Reshape":
             raise ValueError("Reshape cannot be trasformed to dynamic axes")
 
     new_model, _ = onnxsim.simplify(new_model)