Building multi-sensor perception data by labeling LiDAR point clouds and multi-camera footage together

Robots are moving outside buildings.

Robots that once followed fixed routes inside logistics centers are now traveling across city sidewalks and crosswalks to deliver services to people. According to MarketsandMarkets, the global autonomous mobile robot (AMR) market is projected to grow from USD 2.75 billion in 2026 to USD 7.07 billion by 2032, at a compound annual growth rate of 14.4%.

But the world outside is far more complex than indoor environments.

Sidewalks and roads, crosswalks and traffic lights, and pedestrians who are difficult to predict — outdoor autonomous robots must understand all of this through LiDAR and cameras. The performance of perception AI ultimately depends on the quality and volume of its training data.

The challenge is how difficult that data is to build. Teams must label 3D point clouds and multi-camera footage while aligning them across time.

Company R, an autonomous robot service company, has been building this data pipeline together with Superb AI. Since the first project began in 2022, the partnership has continued for five consecutive years. This year, Company R has also completed the kickoff for a new engagement and is continuing the build.

The Paths Robots Travel Are More Complex Than Roads

The methodology for building autonomous vehicle datasets has already become established as an industry standard.

But robots that move on sidewalks face a different level of complexity from cars.

The complexity of multi-sensor fusion data

Company R’s robots perceive their surroundings through LiDAR and multiple cameras covering the front, rear, left, and right sides.

To create training data, objects in 3D point clouds must be aligned with the same objects in each camera view using calibration information. Labeling also needs to track the same object across consecutive frames, not just a single frame.

Connecting data with different coordinate systems and viewpoints into one ground truth dataset is itself a difficult engineering task.

Data specifications evolve every year

As perception AI becomes more advanced, the data it requires also changes.

Company R’s requirements have expanded year after year, from 3D cuboids and temporal data to traffic light recognition and more granular segmentation of traversable space.

Every time the specification changes, the labeling guide, deliverable format, and review criteria must be redefined. That burden repeats across projects.

Privacy requirements for urban driving data

Driving data collected in urban areas naturally includes pedestrians and vehicles.

To handle this data with an external partner, companies need a personal data consignment framework and security controls. They also need to verify every year that the partner complies with those standards.

A data partner’s security capability becomes a prerequisite for the project itself.

One Pipeline from Collection to Deliverables, Built to Last Five Years

The collaboration between Company R and Superb AI is not simply labeling outsourcing. It is a joint operation of the full perception data pipeline.

Superb Platform is a Vision Intelligence platform that operates the full AI training data workflow in one pipeline, from data collection and labeling to review and deliverable management.

Company R’s 3D and 2D labeling, as well as batch-level deliverable management, are also carried out on this platform.

Reconstructed perception data pipeline from collection and cleaning to labeling and deliverables

1. 3D LiDAR cuboid and temporal data labeling

Objects are labeled as 3D cuboids on LiDAR point clouds, then reviewed in alignment with multi-camera footage.

In addition to single-frame labeling, the team builds temporal data that tracks the same object across consecutive frames. This creates training data that can capture the trajectories of moving pedestrians and vehicles.

2. 2D labeling tailored to urban pedestrian environments

The pipeline also builds 2D data required for perception in urban environments, including segmentation masks for identifying traversable space and 2D bounding boxes for traffic light recognition.

Data with distortion characteristics, such as footage from fisheye cameras, is also included in the labeling scope.

3. A custom deliverable pipeline tailored to the customer’s format

Company R receives deliverables in its own predefined label format.

Superb AI develops and operates dedicated scripts from PCD upload to custom export, and has continuously reflected additional or changed specifications into the pipeline.

For five years, the operating rhythm has remained consistent: receive data by batch, label it, and deliver the completed outputs.

4. Collection partner management and personal data security

For projects that require on-site data collection, Superb AI also manages and supervises the subprocessor structure with collection partners.

As a data processor under a personal data consignment agreement, Superb AI has passed Company R’s annual security reviews every year. The operation follows consignment standards, including personal data protection training, access control, and deletion policies based on retention periods.

The Best Evidence Is Contract Continuity: A Five-Year Partnership

The clearest measure of this collaboration’s success is continuity.

Since 2022, Company R has carried out data construction projects with Superb AI every year. This year, it has also signed a new engagement and completed the kickoff.

Reconstructed scope of perception data construction accumulated over five years

• The data scope has expanded every year: The collaboration has grown from single-frame 3D cuboids and temporal data to traffic light and traversable-space recognition, as well as data for HD maps. At each stage of perception AI advancement, Company R has continued building the required data on the same pipeline.
• Specification changes are no longer a major risk: The response process — aligning on format guides, reflecting changes in custom scripts, and validating samples — has been repeatedly tested. As a result, new requirements can now be absorbed without rebuilding the pipeline from scratch.
• The partnership has been validated through security reviews: By passing annual security reviews as a data processor, Superb AI has maintained the trust required to handle urban driving data under a personal data consignment framework.

For Autonomous Robot Perception AI, the Data Partner Matters

Outdoor autonomous robot perception AI is not completed through a single dataset-building project.

As service areas expand and scenarios increase, new data becomes necessary. If the pipeline has to be rebuilt from scratch each time, the pace of AI advancement becomes constrained by data operations.

Company R’s case shows that long-term collaboration with a data partner equipped with multi-sensor labeling capabilities, custom format support, and personal data security systems can become core infrastructure for advancing perception AI.

Superb AI supports robotics and autonomous driving companies across the full data journey, from data strategy to construction and operation.

Frequently Asked Questions

Q. Can LiDAR and camera data be labeled together on one platform?

Yes. Superb Platform supports cuboid labeling on 3D point clouds, alignment with camera footage, and object tracking across consecutive frames.

In this case, LiDAR and multi-camera data are being built within a single pipeline.

Q. Can we receive deliverables in our own label format?

Yes. Superb AI configures export pipelines for custom formats such as customer-defined JSON.

In this case, the team operates dedicated scripts from PCD upload to custom deliverable conversion, and has also reflected specification changes into the pipeline.

Q. How is personal data in urban driving data handled?

Personal data is managed under a personal data consignment agreement, including access control, processor training, and deletion based on retention periods.

The process is validated every year through the customer’s annual security review. When collection partners are involved, Superb AI also operates a subprocessor management and oversight structure.

Q. Can Superb AI handle data collection, or only labeling?

Both are possible.

For projects that require on-site collection, Superb AI operates an integrated workflow with verified collection partners, covering collection, cleaning, and labeling. If the customer already has data, the project can begin with labeling and deliverable construction.

Superb AI is a Vision Intelligence company that transforms visual data from industrial environments into intelligence enterprises can use.