image

package
v0.29.0-beta Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 30, 2024 License: MIT Imports: 19 Imported by: 0

README 

---
title: "Image"
lang: "en-US"
draft: false
description: "Learn about how to set up a VDP Image component https://github.com/instill-ai/instill-core"
---

The Image component is an operator component that allows users to manipulate image files.
It can carry out the following tasks:
- [Draw Classification](#draw-classification)
- [Draw Detection](#draw-detection)
- [Draw Keypoint](#draw-keypoint)
- [Draw OCR](#draw-ocr)
- [Draw Instance Segmentation](#draw-instance-segmentation)
- [Draw Semantic Segmentation](#draw-semantic-segmentation)

## Release Stage

`Alpha`

## Configuration

The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/component/blob/main/operator/image/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/component/blob/main/operator/image/v0/config/tasks.json) files respectively.



## Supported Tasks

### Draw Classification

Draw classification result on the image.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_DRAW_CLASSIFICATION` |
| Category (required) | `category` | string | The predicted category of the input. |
| Image (required) | `image` | string | Input image |
| Show Score | `showScore` | boolean | Show model confidence score on each instance |
| Score (required) | `score` | number | The confidence score of the predicted category of the input. |
</div>






<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Image | `image` | string | Output image |
</div>

### Draw Detection

Draw detection result on the image.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_DRAW_DETECTION` |
| Image (required) | `image` | string | Input image |
| [Objects](#draw-detection-objects) (required) | `objects` | array[object] | A list of detected objects. |
| Show Score | `showScore` | boolean | Show model confidence score on each instance |
</div>


<details>
<summary> Input Objects in Draw Detection</summary>

<h4 id="draw-detection-objects">Objects</h4>

A list of detected objects.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| [Bounding box](#draw-detection-bounding-box) | `bounding-box` | object | The detected bounding box in (left, top, width, height) format.  |
| Category | `category` | string | The predicted category of the bounding box.  |
| Score | `score` | number | The confidence score of the predicted category of the bounding box.  |
</div>
<h4 id="draw-detection-bounding-box">Bounding Box</h4>

The detected bounding box in (left, top, width, height) format.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| Height | `height` | number | Bounding box height value  |
| Left | `left` | number | Bounding box left x-axis value  |
| Top | `top` | number | Bounding box top y-axis value  |
| Width | `width` | number | Bounding box width value  |
</div>
</details>



<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Image | `image` | string | Output image |
</div>

### Draw Keypoint

Draw keypoint result on the image.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_DRAW_KEYPOINT` |
| Image (required) | `image` | string | Input image |
| [Objects](#draw-keypoint-objects) (required) | `objects` | array[object] | A list of keypoint objects, a keypoint object includes all the pre-defined keypoints of a detected object. |
| Show Score | `showScore` | boolean | Show model confidence score on each instance |
</div>


<details>
<summary> Input Objects in Draw Keypoint</summary>

<h4 id="draw-keypoint-objects">Objects</h4>

A list of keypoint objects, a keypoint object includes all the pre-defined keypoints of a detected object.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| [Bounding Box](#draw-keypoint-bounding-box) | `bounding-box` | object | The detected bounding box in (left, top, width, height) format.  |
| [Keypoints](#draw-keypoint-keypoints) | `keypoints` | array | A keypoint group is composed of a list of pre-defined keypoints of a detected object.  |
| Score | `score` | number | The confidence score of the predicted object.  |
</div>
<h4 id="draw-keypoint-keypoints">Keypoints</h4>

A keypoint group is composed of a list of pre-defined keypoints of a detected object.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| Visibility Score | `v` | number | visibility score of the keypoint.  |
| X Coordinate | `x` | number | x coordinate of the keypoint.  |
| Y Coordinate | `y` | number | y coordinate of the keypoint.  |
</div>
<h4 id="draw-keypoint-bounding-box">Bounding Box</h4>

The detected bounding box in (left, top, width, height) format.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| Height | `height` | number | Bounding box height value  |
| Left | `left` | number | Bounding box left x-axis value  |
| Top | `top` | number | Bounding box top y-axis value  |
| Width | `width` | number | Bounding box width value  |
</div>
</details>



<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Image | `image` | string | Output image |
</div>

### Draw OCR

Draw OCR result on the image.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_DRAW_OCR` |
| Image (required) | `image` | string | Input image |
| [Objects](#draw-ocr-objects) (required) | `objects` | array[object] | A list of detected bounding boxes. |
| Show Score | `showScore` | boolean | Show model confidence score on each instance |
</div>


<details>
<summary> Input Objects in Draw OCR</summary>

<h4 id="draw-ocr-objects">Objects</h4>

A list of detected bounding boxes.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| [Bounding Box](#draw-ocr-bounding-box) | `bounding-box` | object | The detected bounding box in (left, top, width, height) format.  |
| Score | `score` | number | The confidence score of the predicted object.  |
| Text | `text` | string | Text string recognised per bounding box.  |
</div>
<h4 id="draw-ocr-bounding-box">Bounding Box</h4>

The detected bounding box in (left, top, width, height) format.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| Height | `height` | number | Bounding box height value  |
| Left | `left` | number | Bounding box left x-axis value  |
| Top | `top` | number | Bounding box top y-axis value  |
| Width | `width` | number | Bounding box width value  |
</div>
</details>



<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Image | `image` | string | Output image |
</div>

### Draw Instance Segmentation

Draw instance segmentation result on the image.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_DRAW_INSTANCE_SEGMENTATION` |
| Image (required) | `image` | string | Input image |
| [Objects](#draw-instance-segmentation-objects) (required) | `objects` | array[object] | A list of detected instance bounding boxes. |
| Show Score | `showScore` | boolean | Show model confidence score on each instance |
</div>


<details>
<summary> Input Objects in Draw Instance Segmentation</summary>

<h4 id="draw-instance-segmentation-objects">Objects</h4>

A list of detected instance bounding boxes.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| [Bounding Box](#draw-instance-segmentation-bounding-box) | `bounding-box` | object | The detected bounding box in (left, top, width, height) format.  |
| Category | `category` | string | The predicted category of the bounding box.  |
| RLE | `rle` | string | Run Length Encoding (RLE) of instance mask within the bounding box.  |
| Score | `score` | number | The confidence score of the predicted instance object.  |
</div>
<h4 id="draw-instance-segmentation-bounding-box">Bounding Box</h4>

The detected bounding box in (left, top, width, height) format.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| Height | `height` | number | Bounding box height value  |
| Left | `left` | number | Bounding box left x-axis value  |
| Top | `top` | number | Bounding box top y-axis value  |
| Width | `width` | number | Bounding box width value  |
</div>
</details>



<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Image | `image` | string | Output image |
</div>

### Draw Semantic Segmentation

Draw semantic segmentation result on the image.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_DRAW_SEMANTIC_SEGMENTATION` |
| Image (required) | `image` | string | Input image |
| Show Score | `showScore` | boolean | Show model confidence score on each instance |
| [Stuffs](#draw-semantic-segmentation-stuffs) (required) | `stuffs` | array[object] | A list of RLE binary masks. |
</div>


<details>
<summary> Input Objects in Draw Semantic Segmentation</summary>

<h4 id="draw-semantic-segmentation-stuffs">Stuffs</h4>

A list of RLE binary masks.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| Category | `category` | string | Category text string corresponding to each stuff mask.  |
| RLE | `rle` | string | Run Length Encoding (RLE) of each stuff mask within the image.  |
</div>
</details>



<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Image | `image` | string | Output image |
</div>

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var IBMPlexSansRegular = []byte{} /* 175232 elements not displayed */

IBMPlexSansRegular is the font byte array

Functions

func Init 

func Init(bc base.Component) *component

Init initializes the operator

Types

type BoundingBox 

type BoundingBox struct {
	Top    int
	Left   int
	Width  int
	Height int
}

BoundingBox holds the coordinates of a bounding box.

func (*BoundingBox) Size 

func (b *BoundingBox) Size() int

Size returns the area of the bounding box.

type Keypoint 

type Keypoint struct {
	// contains filtered or unexported fields
}

Keypoint holds the coordinates of a keypoint.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.