Sensor fusion development for automated vehicles

BASELABS Create Embedded

BASELABS Create Embedded is a multi-sensor perception technology for ADAS functions. It uses object-level sensor fusion algorithms to combine data from any combination of radar, camera, and LiDAR sensors. As a result, a unified representation of the vehicle's environment serves as a reliable input to path planning and decision-making algorithms. The library is highly configurable, follows a white box approach, and provides self-contained C-code ready for production usage on CPUs in typical automotive ECUs, including safety certification up to ASIL B.

Benefits at a glance

  • Software library for the fast and efficient development of sensor fusion systems for series production
  • Provides dynamic object fusion for automated driving functions such as AEB, ACC and Highway Pilot
  • Supports all relevant automotive sensors like radar, camera and lidar
  • Scalable from radar-camera front fusion up to 360° object fusion
  • ISO 26262 compliant
  • Compatible with ISO 23150 "Road vehicles - Logical interfaces"
  • Consistent development workflow from prototyping with ROS to series production, e.g. with AUTOSAR Classic/Adaptive
  • Support for typical development platforms, e.g. NVIDIA DRIVE
  • C source code for embedded hardware platforms, e.g. AURIX™ 2G, Renesas RH850 and ARM Cortex-A73
  • Graphical configuration of sensor fusion systems
  • Flexible and easy adaption of sensor fusion applications to different sensor-setups or -types
  • Fully customizable white box solution
  • Built-in sensor fusion diagnostics
  • For more details, see BASELABS Create Embedded FAQ

Find out more about sensor fusion or contact us.

 ADDRESSED USE CASES AND DRIVING FUNCTIONS

  • SAE level 0-3: Automated Emergency Braking (AEB), Adaptive Cruise Control (ACC), Forward Collision Warning (FCW), piloted driving
  • SAE level 3-4: Object fusion as part of a diverse redundant safety architecture, e.g. ASIL D decomposition

For its market relevance and user benefits, BASELABS Create Embedded has been awarded the Frost & Sullivan "2021 Enabling Technology Leadership Award". The award includes a detailed evaluation of best practice criteria and refers to both the product's market performance and technical benefits for software users. BASELABS was particularly named for its innovative strength and far-reaching market strategies, among other things.

More information

Consistent development workflow

BASELABS Create Embedded provides a consistent development workflow - from pre-development and prototyping to series production. Utilizing the tool makes the implementation of sensor fusion systems for embedded platforms faster and much more efficient. The development tool contains sensor fusion algorithms that combine data from radar, camera and lidar sensors. The resulting C source code can be used along the entire development chain. Thus, the development tool drastically reduces the development effort at every stage on the way to series production. BASELABS Create Embedded allows for safety compliant sensor fusion development, including documentation and testing for safety use cases.

More information on how to get your sensor fusion on the road faster and more efficient - contact us.

Supported sensors

BASELABS Create Embedded supports a wide range of sensor technologies and object/track interfaces from various vendors, e.g. Continental ARS, Aptiv ESR, Bosch MRR, Mobileye EyeQ, Valeo Scala and Ibeo HAD.

Custom or extended sensors can be implemented and integrated using the software development kit (SDK).

For details on the supported sensor interfaces, see BASELABS Create Embedded FAQ or contact us.

Sensor fusion library

Sensor fusion designer and generator

With the sensor fusion designer, the radar, camera and lidar sensors of the vehicle setup are configured, customized and parameterized. A specific sensor fusion system for object fusion is generated from this configuration.

Sensor fusion reference architecture

The integrated reference architecture for object fusion algorithms allows to build sensor fusion applications ranging from two sensor systems to large 360° setups with many sensors. The architecture can be customized and extended.

 

Sensor fusion library for embedded systems

The integrated sensor fusion library contains algorithms to build custom object fusion systems, e.g.

  • numerically stable Kalman filters,
  • data association methods,
  • sensor models,
  • existence probability handling,
  • track management algorithms,
  • classification fusion,
  • synchronization of asynchronous sensor measurements and
  • handling of out-of-sequence measurements (OOSM).

The C source code of the library is fully accessible
and ready for embedded platforms:

  • Compatible with relevant series hardware, e.g. AURIX 2G and Renesas RH850
  • Runtime- and memory-efficient
  • Customizable and extensible
  • Human-readable and close to hand-written code
  • MISRA C:2012 compliant
  • Self-contained, no dependencies to external libraries

For more details on the library, see BASELABS Create Embedded FAQ or contact us.

Middleware compatibility and integration

BASELABS Create Embedded is compatible to all middleware systems and provides a seamless integration of the generated sensor fusion algorithm to many platforms and runtime environments:

  • AUTOSAR Classic/Adaptive
  • bare metal environments
  • ROS
  • ADTF
  • RTMaps
  • Matlab/Simulink
  • any custom middleware

For details on how the sensor fusion integrates into middleware systems, see BASELABS Create Embedded FAQ or contact us.

Built-in sensor fusion diagnostics and debugging

To analyze the current system behavior and performance, developers require a rich and customizable visualization that shows all relevant information in an integrated way. Often, sensor fusion issues are hard to find and thus require additional development time. The embedded sensor fusion library BASELABS Create Embedded comes with a built-in visualization of the sensor fusion results, the sensor fields of view, and the sensor data in a birds-eye-view according to the configured sensor setup. Furthermore, it includes a diagnostic tool that provides detailed insights into the sensor fusion system, e.g. to derive sensor parameters or to find the reason why a certain measurement is not associated with a track. By selecting single tracks of interest, a detailed analysis of associated measurements and uncertainties for distinct components of the state is possible. Using diagnostics, you can easily discover if a track is only confirmed by some of the sensors but not by all of them.

ISO 26262 compliance

BASELABS Create Embedded enables the safety-compliant development of sensor fusion algorithms. The contained embedded software library is developed according to ASPICE and ISO 26262​. By that, the software can be directly used in series development and drastically reduces the development effort.

BASELABS Create Embedded has been developed in accordance with a relevant sub-set of ISO 26262:2018 requirements and methods for ASIL B.

It comes with:

  • A safety case that can be integrated into the safety case of the system to be developed.
  • A safety manual that gives users guidelines on how to use the product in a safety-related context.

Please find the detailed safety argumentation in our Safety Concept.

Parts of BASELABS Create Embedded are considered as a software tool, i.e. this tool might need to be qualified when used in the context of the ISO 26262 (see ISO 26262:2018-8 Chapter 11). Create Embedded supports this tool qualification by two means:

  • The software tool itself has been developed according to relevant aspects of ISO 26262 and ASPICE, addressing the possibility for 'evaluation of the tool development process'.
  • A test suite is provided to enable users the 'validation of the software tool'

For the software units that come with the embedded sensor fusion library, users benefit from

  • the extensive documentation containing all relevant algorithmic and design details,
  • the implementation and testing according to ISO 26262 guidelines,
  • the implemented error detection mechanisms (derived from failure mode analysis) that provide a solid basis to perform error handling at the software-architecture level, specific to the system to be developed and
  • the test reports for the intended hardware, compiler, and operating system and the specific version of BASELABS Create Embedded.

The confirmation reviews of relevant ISO 26262:2018 work products were performed by a Principal Safety Engineer of the exida GmbH. The assessment of the compliance of the development process with ASPICE was performed by an intacs™ Certified Competent Assessor. BASELABS ISO26262-2018 Confirmation Reviews Cover and Summary

Loading the player ...

Releases

BASELABS Create Embedded 9.0 (14.03.2023)

New features

  • Measurement models for extended objects

    The SDK now contains measurement models with basic functionality for tracking extended objects. This enables the consideration of information from objects when fusing camera sensors, which in turn already provide size estimation. Thus, the size information is available both during the fusion of sensor data and at the sensor fusion output.

Improvements

  • Tutorial for multiple hypotheses initialization

    A new tutorial explains how to create multiple track hypotheses from a single sensor measurement, e.g. one track hypothesis that accounts for longitudinal traffic and one for cross traffic. It explains how ambiguities are resolved over time, and a single track is provided as the result of the data fusion.
  • Sensor Fusion Diagnostics - with BASELABS Create Embedded 9, the diagnostics tool becomes available from the SDK

    BASELABS Create Embedded comes with a built-in diagnostics tool to find issues related to the parameterization of sensor models or the models themselves. It helps to identify, for instance, too small or large values for measurement or process noise parameters.

BASELABS Create Embedded 8.2 (11.05.2022)

Improvements:

  • Unified User Manual: API documentation, tutorials, and architecture descriptions can be found in a unified and searchable user manual.

BASELABS Create Embedded 8.1 (11.02.2022)

Fixes:

  • SDK: System models did not check for negative time spans. 
  • SDK: The expectation and standard deviation values of pre-defined random numbers did not converge to 0 or 1, respectively.
  • Data fusion designer: Custom categorical measurement models generated an error.
  • Create Embedded compiler: On some computers, an error message appeared if optimization is enabled.

BASELABS Create Embedded 8.0 (15.12.2021)

New features

  • Multiple Model Support: The SDK has been extended so that distinct models can be used for different types of objects. For instance, a constant velocity model might be used for pedestrians, while a constant curvature model is configured for vehicles. When the actual class of an object is not known, e.g., during the initialization, multiple hypotheses with different models can be created. Invalid hypotheses are removed using user-defined removal functions. By this, the motion of different classes can be predicted more precisely to improve the measurement-to-track association and the overall sensor fusion performance.
  • Association uses object class: The SDK has been extended by an option to use categorical values in the association in addition to other quantities. This improves the association quality and better separates objects close to each other.
  • Height Support: The SDK now contains height models and spaces similar to width and length.

Improvements

  • Improved blacklisting: The Trait-C compiler now supports to blacklist identifiers that belong to the Trait-C language, e.g. "const", "int" and "for".

Fixes

  • Replay: When using step-mode in Replay and pressing the 'Next' button, the next item to pause at was incorrectly determined.
  • Replay: The camera visualization applied an existence threshold to the resulting tracks, although this threshold is already applied when retrieving tracks from the sensor fusion.
  • Replay: The generated Replay code could not be compiled if single floating-point precision is used.
  • Replay: After releasing the ‚Pause‘ button in Replay, the replay speed was to high.
  • Replay: The F10 shortcut key did not work reliably.
  • Categorical state proposer wizard: The wizard generated code that could not be compiled.
  • Visualization: The visualization could not be compiled if all configured detection models are dependent on the system's state.
  • Sensor fusion: Negative timespans were not correctly determined.
  • Diagnostics: The diagnostics tool did not display additional measurement spaces.
  • Plausibility check: The plausibility check for duplicate measurements returned an error, although there are no duplicate measurements.
  • Visualization: The visualization was unstable in certain conditions.
  • Replay: The generated code could not be compiled if a sensor does not propose new tracks.

BASELABS Create Embedded 7.3 (10.05.2021)

Fixes:

  • The Windows installer was identified as potentially harmful by the Windows SmartScreen protection.

BASELABS Create Embedded 7.2 (03.03.2021)

Fixes:

  • BASELABS Create Embedded could not be used without an internet connection.
  • BASELABS Create Embedded could not be used if another installation of the same major version is present.

BASELABS Create Embedded 7.1 (26.02.2021)

New features:

  • Replay: BASELABS Create Embedded now contains the environment 'Replay' to replay measurement data from csv files, to process the data in the sensor fusion, and to visualize the sensor fusion results in camera images. To experience this new feature from the Data Fusion Designer, go to 'Project --> Add data fusion item...' and select 'Replay' or follow the instructions in the user manual.

Fixes:

  • Integration: Sensor fusion code could not be compiled using Visual Studio 2019.
  • Data Fusion Designer: Failed to generate a sensor fusion if the project was not explicitly saved before.

BASELABS Create Embedded 7.0 (16.11.2020)

New feature:

  • Multiple hypothesis track initialization (MHTI): Multiple hypotheses for the initializing of tracks can be configured in the Data Fusion Designer. This enables the creation of multiple track hypotheses from a single sensor measurement, e.g. one track hypothesis that accounts for longitudinal traffic and one that accounts for cross traffic. Additionally, ambiguities are automatically resolved over time and a single track is provided as the result of the data fusion.
  • Track prioritization: Tracks are prioritized by a (user-defined) prioritization function. When a new track should be added in the case of a full track list, the track with the lowest priority is removed. By default, tracks closer to the ego vehicle get a higher priority. Custom prioritization functions can be used.
  • Constant curvature and acceleration model: A new motion model is available that better accounts for the motion of vehicles. The so-called constant curvature and acceleration (CCA) motion model can be selected in the data fusion designer. Additionally, compatible sensor models are available.
  • Data fusion architecture documentation: The object fusion architecture generated by the data fusion designer is documented. This documentation is available from the Windows start menu or the Help menu of the data fusion designer.
  • Framework tutorial: A tutorial describing how to create an object fusion using the Create Embedded SDK without the Data Fusion Designer is available from the start menu.
  • Project documentation: For projects created with the Data Fusion Designer, a project documentation is generated when building the project.

Improvements:

  • Data Fusion Designer: Models are sorted according to their compatibility in the drop-down menus.
  • Runtime visualization: The track id is shown in the runtime visualization.
  • Diagnostics: The tooltip for measurements contains all measurement components to find the same measurement in the graphs for other components of the measurement.
  • Continuous persistence model: The discrete persistence model is replaced by a continuous persistence model. The continuous persistence model performs better in multi-sensor fusion systems with varying periods.
  • NPIPDA: The object fusion algorithm uses the non-parametric IPDA algorithm. Thus, configuring the false alarm lambda for each sensor is no longer necessary.
  • Track confirmation: The result of the data fusion contains confirmed tracks only. A track gets confirmed if its existence probability is above a minimum value that can be configured in the data fusion designer.

Fixe:

  • AUTOSAR SWC: Data structures were not properly initialized.
  • Data Fusion Designer: The parameter 'Existence switching rate' was not used in the object fusion.
  • Runtime visualization: The covariance ellipsis of a track was not correctly displayed.

BASELABS Create Embedded 6.1 (24.09.2020)

Fixes:

  • Licensing: CodeMeter Runtime (Wibu Systems) installed with BASELABS Create Embedded contained security vulnerabilities.

BASELABS Create Embedded 6.0 (16.06.2020)

New features:

  • Object fusion diagnostics: BASELABS Create Embedded comes with a built-in diagnostics tool to find issues that are related to the parameterization of sensor models or the models itself. It helps to identify, for instance, too small or large values for measurement or process noise parameters.
  • AUTOSAR SWC: An AUTOSAR Classic software component (SWC) can be added to a data fusion system. It contains code to run the data fusion algorithm in AUTOSAR. The SWC provides application and implementation data types as well as application port interfaces.

fixes 6.0

Fixe:

  • vADASdeveloper component: Data fusion shutdown was not called when playback is stopped.
  • ROS node: Data fusion shutdown was never called.

BASELABS Create Embedded 5.1 (20.05.2020)

Fixes

  • Data Fusion Designer: Adding data fusion items with dependencies on other data fusion items caused an error - the dependencies were not added.

BASELABS Create Embedded 5.0 (24.04.2020)

New

Exida confirms ISO 26262 compliance of BASELABS Create Embedded

  • BASELABS Create Embedded – has been confirmed by exida to be compliant with the regulations of the ISO 26262 safety norm. With its comprehensive understanding of ISO 26262, exida provides ISO 26262 certification for the development of safer products.
  • BASELABS Create Embedded has been developed in accordance with a relevant sub-set of ISO26262:2018 requirements and methods for ASIL B. Its resulting software algorithms are directlyapplicable to the production of safety-related ADAS and automated driving functions.

In its recently released version 5.0, BASELABS Create Embedded comes now with:

  • A safety case that integrates into the safety case of the system to be developed.
  • A safety manual that gives users guidelines on how to use the product in a safety-related context.

 

Improvements

  • Data Fusion Designer: Improved versioning of data fusion items to avoid unnecessary data fusion item updates in the future.
  • Data Fusion Designer: Existence of tracks is provided as log odds instead of probabilities in track structure to avoid rounding to 1.0. Furthermore, functions to convert between log odds and probabilities are provided.
  • ARM support: The support for ARM platforms has been improved.
  • User Manual: Hyphens from command line examples can now be copied correctly.

Fixes

  • ROS Node: package.xml_ contained invalid xml.
  • Data Fusion Template: C keyword warn_unused_result was used for instantiation of Trait-C type.
  • License Manager: Manage license button was not working with CodeMeter runtime reduced.
  • User Manual: Wrong object type model name in user manual.
  • Data Types Template: TimeStamp struct unnecessarily has Precision.
  • Linux: Create Embedded library could not be build for Debug and Release.

BASELABS Create Embedded 4.2 (31.01.2020)

New features

  • Support for NVIDIA DRIVE and ARM platforms: Many modern driving functions depend on AI and deep learning methods whose results are often fed into a data fusion. While AI-based methods require extensive parallelization and therefore often run on GPUs, object fusion can be sequentially executed on CPUs. With the support of the ARM CPU family, the object fusion of BASELABS Create Embedded can now be run on the ARM part of NVIDIA DRIVE platforms.
  • Built-in height estimation: Besides determining the width and length, BASELABS Create Embedded now provides a built-in way to estimate the object’s height if at least one of the configured sensors provides information about the height.
  • Access to classification when creating new tracks: In state proposers, it is now possible to reference the classification value or object type that might be provided by a sensor. With this, new tracks can be better initialized, e.g. its initial velocity range or uncertainty might be smaller for pedestrians than cars.

Improvements

  • Data Fusion Designer: When upgrading a project, a backup copy of the templates is made to avoid loss of customizations.
  • Data Fusion Designer: Support for Visual Studio 2017 and 2019 when creating a project for building the data fusion library.

Fixes

  • Data Fusion Designer: Font size for error messages was too small.
  • Data Fusion Designer: Error message for unset system model parameters did not disappear after setting this parameter.
  • vADASdeveloper component: Invalid data types for some of the C functions.
  • vADASdeveloper component: Out of sequence measurements caused out of sequence tracks even though deterministic buffering was used.
  • vADASdeveloper component: Template error when model properties did not have the display name attribute.
  • Installer: Incomplete uninstallation when multiple versions of BASELABS Create Embedded were installed.

BASELABS Create Embedded 4.1 (02.01.2020)

Fixes:

  • Template engine: Adding file headers caused errors for some file types.
  • User manual: Underscores could not be copied from the user manual.
  • Installer: Installer did not check for the needed .Net version.
  • Linux: User manual not contained in documentation folder.
  • ROS node: Message conversion stub contained error in local variable declaration.

BASELABS Create Embedded 4.0 (18.10.2019)

New features

  • Classification fusion: Sensors such as cameras often provide information on the class of a certain object. This information is highly relevant to improve the data fusion performance, e.g. to specify class-specific sensor characteristics. The Data Fusion Designer of BASELABS Create Embedded now supports to design data fusion systems that include classification information from any of the configured sensors. If multiple sensors provide a classification for an object, the built-in classification fusion resolves potential conflicts. Furthermore, the runtime visualization has been enhanced and includes the object classification.
  • Out-of-sequence measurements handling: When using multiple sensors, data may arrive delayed at the data fusion, e.g. due to pre-processing or communication. To handle these so-called out-of-sequence measurements, BASELABS Create Embedded now provides a deterministic buffering approach. When using this buffering method, the measurements of the different sensors are buffered and then processed in chronological order.
  • Runtime calibration: The parameters configured in the Data Fusion Designer now can be changed at runtime as well.
  • Visualization of ego motion data: The velocity and yaw rate of the host vehicle are shown in the runtime visualization.
  • Track statistics: For each configured sensor, the resulting track structure of the data fusion contains the information whether an object has been seen by this sensor for the last eight time steps.
  • Traceability: Generated files contain a header containing information on the time of creation and the used version of BASELABS Create Embedded.
  • Host vehicle Parameters: Dimensions of the host vehicle can be configured in the Data Fusion Designer and are used in the visualization during development and runtime.

Improvements

  • vADASdeveloper component: The classes used at the component's pins check that the maximum number of measurements is not exceeded.
  • Unique error codes: The error codes contain information on their origin.
  • Association: When multiple system models are configured, the measurements from all continuous models are used for the measurement to track association.
  • Data fusion template: Less stack memory consumption to avoid stack overflow exceptions.
  • Example projects: ROS and vADASdeveloper example project estimate the object's width and object class.
  • Template dependencies: Dependencies between templates can be expressed. Missing dependencies are automatically added, when adding a new data fusion item.

Fixes

  • vADASdeveloper example project: The example project could not be built when the Trait-C data fusion template is used.
  • Runtime visualization: The fields of view were not correctly drawn but covered the whole visualization area.
  • Runtime visualization: Predicted measurements were shown even if a sensor was not active.
  • vADASdeveloper component: Large measurement data structures could not be marshalled to C structures.
  • ROS node: Extended state spaces, e.g. by an additional width space, were not supported in the track message.
  • Linux: Custom modules were not found when building a data fusion project on Linux machines.

BASELABS Create Embedded 3.0 (19.07.2019)

New features

  • Smart radar measurement model: The Data Fusion Designer now contains a 'Smart radar measurement model' which can be used for typical internally tracking radar sensors.
  • "Expert mode": The control flow of the data fusion is now available both in C and Trait-C. An introduction on Trait-C is included.
  • Improved model extensibility and reusability: Existing models can now be easily extended right from the data fusion designer, e.g. the width and the length of an object can be added while keeping or reusing an existing model. For that, additionally required quantities from other model parts can now be accessed and read by the extended model, e.g. the dynamic position of the object.
  • Visualization of object length and width: If an object contains width and/or length information, it is shown in the visualization. This is especially useful if one or more sensors provide width and/or length information, e.g. lidar or camera sensors.
  • Improved workflow for series development: The creation of data fusion projects for series production has been simplified and improved.

Improvements

  • Multiple versions installable: Multiple versions of BASELABS Create Embedded can be installed on the same machine to enable the user to work on projects created with different versions of BASELABS Create Embedded.
  • Width estimation in vADASdeveloper example project: The object width is estimated and visualized in the vADASdeveloper example project.
  • vADASdeveloper example project: Increased replay speed.

Fixes

  • Data Fusion Designer: Model names did not fit in the drop down.
  • Data Fusion Designer: Could not generate code if deactivated sensor was not complete.
  • vADASdeveloper example project: Player did not pause on tracks.
  • Model wizards: Wizards for different models were not consistently formatted and documented.
  • Visualization: Sporadic stack overflow while zooming or panning.

BASELABS Create Embedded 2.0 (15.04.2019)

New features:

  • Width and length estimation: The Data Fusion Designer now allows to add system models additionally to the motion model. Models for width and length estimation can be selected.
  • Model wizards: The Data Fusion Designer now provides wizards for creation of spaces, system models, measurement models and detection models.

Improvements:

  • Visualization: Associations and predicted measurements are shown.

 

 

Fixes

  • vADASdeveloper example application: Camera image was too small.
  • vADASdeveloper example application: Silent fail on exception.
  • vADASdeveloper example application: Incorrect restoring of settings.
  • vADASdeveloper component: Pin names of component contained sensor ID.
  • vADASdeveloper component: Float was not supported.
  • vADASdeveloper component: Incorrect out of sequence measurements handling.
  • Smart camera models: Insufficient descriptions of properties.

BASELABS Create Embedded 1.0 (01.03.2019)

Initial release

Contact & further information

Top of page