Sensor-Independent Autonomous Mobility

Sensor characteristics like the resolution and field of view affect the resulting pixel-by-pixel representation of a given object like a car or a pedestrian. Thus, a neural network trained to distinguish these features in the images from a specific camera will often not perform as well on images from a different one. When implementing a new sensor, training must therefore start from scratch with images from that sensor, which in turn also implies the costly real-world recording and annotation of these images

The invention defines a deflection metric whereby every pixel of an image is assigned an angle with respect to the axis of projection (see image on the left). Geometric properties of the sensor such as its field of view and nonlinear distortion are thereby encoded into the image in the form of an additional channel and taken into account by a neural network trained on images from sensors with differing properties. This in turn improves performance on images from any sensors with geometric properties within the ranges of the training set. Moreover, ambiguity in scale and distortion of objects recorded by any given sensor are also resolved.

The invention makes the perception system of an autonomous vehicle less dependent on the specific sensors providing the data about the environment. This means new sensors can be launched without collecting extensive amounts of new training data first. In a competitive market, where new and improved sensors are continuously developed, this provides a significant competitive advantage, both in terms of costs and time.

Technology Readiness Level 3-4: Experimental proof of concept & Technology validated in lab