Initial approach for computer vision

[javiermtorres]

Encoderfile aims at supporting vision models as well as text encoders. Initially, the following huggingface tasks will be supported:

• image classification
• image segmentation
• object detection

Also, no multimodal models or tasks will be taken at this step, until the implementation is well under way.

[javiermtorres]

The pipelines in HF transformers will be followed closely:

• image classification:
  list[dict[str, Any]] | list[list[dict[str, Any]]]: a single or batch list of {label, score} dicts
  • label is the label of each class
  • score is the confidence score for that class
• image segmentation:
  list[dict[str, Any]] | list[list[dict[str, Any]]]: a single or batch list of {label, mask, Option<score>} dicts
  • label is the label of the class assigned to the segment
  • mask is a bitmask over the original image for this specific segment
  • (optional) score is the confidence score of the class
• object detection:
  list[dict[str, Any]] | list[list[dict[str, Any]]]: a single or batch list of {label, score, box} dicts
  • label is the label of the class assigned to the segment
  • score is the confidence score of the class
  • box is a bounding box over the original image for this specific object, using the keys {xmin, ymin, xmax, ymax}