What is Interactive AI and Auto-Edit?

Hanan Othman

Hanan Othman

Content Writer · 2023/3/1 · 3 Min
Interactive data annotation platform

The most commonly used approach to annotate polygons is by placing a series of coordinates along the edge of an object. Polygon annotation is particularly favored for tasks like object detection and image segmentation, ones that require even more attention to detail in emphasizing the boundaries of an object.

 For an even deeper capturing and impression of an object's figure, experienced labelers can employ polygon segmentation techniques; dividing data into separate areas or "segments." Though polygon annotation, labelers can better position coordinates around irregularly shaped or unconventional objects, adjusting the coordinate locations from point-to-point according to an object's unique shape or outline.

 By automating this approach, labelers bypass the manually hassling aspects of coordinate-placing, automatically segmenting objects in both image and video data. Auto-Edit, as the name suggests, is not only intended for automatically placing polygon points, but editing and correcting them as well if needed, whether they were manually placed or through auto-labeling.

Interactive AI Tooling Choices

Another time-consuming element of applying polygon annotations is the QA process. Auto-Edit simplifies the labeling review workflow by applying automated edits to polygons. Meant to be used alongside Auto-Edit, Superb Suite users can utilize the Brush tool for either creating or "drawing" polygons or editing polygons that have been AI-generated.

 The specific technology that automatically applies polygons to data is known as Interactive AI. The basic two-step process for automating polygon generation and revision is to first use Interactive AI to generate the polygons automatically and then edit said polygons through the Suite's Brush tool.

 When to Implement Auto-Label or Interactive AI

The key to carrying out polygon annotation and segmentation expertly is to be able to consider the application for an ML project, recognize that it would benefit most from the polygon segmentation approach.

 Beyond that determination, an experienced labeling team can differentiate between circumstances that call for one automated method over another. In the case of Interactive AI implementation, there are some situations or labeling tasks that suit the use of auto-labeling as an alternative.

 As guiding indications that make the choice of deciding between the two options easy based on their basic functionality; Interactive AI a single object, once, while auto-label labels an entire image at once.

 Auto-label is also capable of estimating the location or boundaries of the annotation (bounding box, polygon, etc), along with the labeling and object type or class of objects. In comparison, Interactive AI doesn't estimate or predict the type of an object's class on it's own, requiring human guidance, but notably can perform labels without pre-labels or training.


Using Auto-Edit

To start using or enable the Auto-Edit annotation tool, a user will need to launch the annotation application and select the Auto-Edit button. Doing so will automatically start the loading process for the AI model.

 Be aware that while an AI model is loading it can be affected by internet network connectivity as well as the operating system of a user's device. If the network is unstable or if you happen to be using Windows OS, the loading time can vary and may take several seconds to execute.

 When loading is in progress, a pop-up window will indicate so but may appear as frozen since other actions cannot be performed during the loading process. Once the AI model is loaded, a notification will appear and the user will then be able to utilize the Auto-Edit function.

 To create a polygon, users should apply a bounding box annotation to an object through the Auto-Edit function, which triggers a polygon to be automatically drawn or applied according to the object shape.

 If the polygon annotation applied through Auto-Edit needs to be adjusted, then simply press the 'Alt' key and click on the section or part that should be adjusted. When adding a part or section to a polygon a user should click on the desired part to add an object to.


Considerations When Applying Auto-Edit

Auto-Edit, along with Interactive AI and their recommended accompanying pair of assistive Pen and Brush tools, are intended to make drawing and editing pixel-precise polygons in a matter of seconds the norm for labeling teams.


In order to fulfill that purpose, users should ask and seek out the answers to how these automated methods and tools can be integrated into their labeling procedures or workflows. A good place to start would be the following questions:

  • Does a labeling team lack domain expertise or knowledge for their build project and its use cases?

  • Is a labeling team interested in automating their pipeline but unsure on how or where to start?

  • If a team is struggling at the start of ML/CV development due to data curation and management, including the effort of establishing ground truth.

  • Is a team faced with a significant volume of data that needs to be preprocessed? Specifically, does a team have doubts that it can handle the data curating needs of their project?

  • Does a team need to reduce labeling time and expense at the start of a project and would benefit from an automated solution that doesn't require pre-labeling and training?


The Answer to the Polygon Problem

The most strenuous parts of data labeling can be narrowed down to a select few that give it a bad name. Striving for the highest standard of datasets leads to the high-performing model that every team dreams of, but struggles to bring to life.

 The first step to making that ambition a reality starts with a lead by example ground truth dataset. Automation helps streamline that effort, but it's only as effective as the pre-labels or existing model used to train it.

 Interactive AI and Auto-Edit are advanced tools that were made to fill in that gap or demand for teams to alleviate the most problematic labeling tasks, by rightfully focusing on polygon segmentation, one that had earned its notoriety. 

The Ground Truth is a community newsletter featuring computer vision news, research, learning resources, MLOps, best practices, events, podcasts, and much more. To get an idea peek at the last issue of the newsletter here.