Skip to main content

1.1. Raw Data Capture

info

Goal: Get to grips with SENSUS ROS and capture some raw signal files (.xml)
Level: Beginner
Time: 15 minutes

Background​

The Calyo PULSE sensor is software defined so in this first lesson we will learn how to store the raw sensor data signals without any processing.

This is a very useful technique:

  • Creating datasets
  • Sensor/Parameter tuning in post
  • Extracting raw signals for custom data processing

Before starting its beneficial to understand the data format of the PULSE's raw data.

32 (Channels) by 4000-20,000 (Time-points)

Depending which package in the SENSUS suit you are using this raw data can the storage format of this raw data can vary.

  • XML
  • CalyoMAT2

Prerequisites​

  • A computer running Linux Ubuntu 22.04 (or higher)
  • A Calyo PULSE Sensor
  • ROS2 (Humble >)
  • SENSUS ROS downloaded and installed (see SENSUS ROS Installation)
note

While you can perform simple data capture without ROS (See Advanced Data Capture) the rest of the tutorials will be ROS based so this is a got time to set this up and get familiar with the interface.

Tasks​

Preparing Configuration​

All the SENSUS software used the same configuration which is a .YAML file used to configure the sensor and software for the user specific use case.

All the configurations take the same format

type: calyosensus
pipelines:
  - id: example_pipeline_object
    reader:
      type: <reader_type>
    writer:
      type: <writer_type>
    processor:
      type:
        stage: <processor_stage>
      runtime_parameters:
        <parameter list>

For raw data capture using a live sensor we first configire the reader to use the DeviceReader type and configure it's serial number to any (This will work assuming you only have one Calyo PULSE plugged in, else you can replace this with your devices serial number).

Next the writer is configured with the XMLWriter type and set output path (feel free to change this path as this is where our raw data will be stored).

Lastly the process is configured to passthrough (no additional parameters needed), here we have told the software to output the raw data from the sensor with no processing.

tip

All the details about configuration types and their corresponding parameters for readers, writers and processor can be found on the SDK Reference (or click the types above they have been linked for easy access). Have a go, and read about the types we have just added to the config!

Here is the full YAML with all that configured, please create this file using your favourite text editor and save it somewhere easy to access.

raw-data-capture-tutorial.yaml
type: calyosensus
pipelines:
    - id: signal_capture
      reader:
        type: DeviceReader
        serial_number: any
        num_cycles: 20
        max_distance: 8
      writer:
        type: XMLWriter
        output_path: "~/Calyo-Tutorials/1.1-RawDataCapture/"
      processor:
        type:
          stage: passthrough

Capturing Data​

Once we have created that file we can run it. We will be using ROS in this example so open up a terminal window and source ROS:

source /opt/ros/humble/setup.bash

Next navigate to the folder where you downloaded the install files for ROS

cd ~/Downloads/calyo-ros/

Make sure your sensor is plugged in at this point ready to capture some data

In here you will find a file called meta.launch.py. This is the ROS launch script we will be using to launch the config.

ros2 launch meta.launch.py engine_config:=~/raw-data-capture-tutorial.yaml

You should now be capturing data!

Example logs of successfully running ROS driver
[INFO] [1738603062.335225761] [calyosensus_node]: Path at:
[INFO] [1738603062.335437982] [calyosensus_node]: <your_path>/config/plug_and_play.yaml
[INFO] [1738603062.335929699] [calyosensus_node]: Initialising Pipelines...
[INFO] [1738603062.336006058] [calyosensus_node]: Creating pipeline 'signal_capture'...

[INFO] [1738603062.336094015] [calyosensus_node]: Pipeline type is 'PassThroughPipeline'
[INFO] [1738603063.464681214] [calyosensus_node]: Creating pipeline 'signal_filtering'...

[INFO] [1738603063.464825603] [calyosensus_node]: Pipeline type is 'FilteredSignalCapturePipeline'
[INFO] [1738603063.464923337] [calyosensus_node]: Creating pipeline 'imager_2d'...

[INFO] [1738603063.465002681] [calyosensus_node]: Pipeline type is 'ResolutionImager2LogConvertedPipeline'
[INFO] [1738603063.465666961] [calyosensus_node]: Creating pipeline 'imager_3d'...

[INFO] [1738603063.465785496] [calyosensus_node]: Pipeline type is 'ResolutionImager3PointCloudPipeline'
[INFO] [1738603063.467114927] [calyosensus_node]: Pipelines initialised

[INFO] [1738603063.645239294] [calyosensus_node]: Processed frame # 0
[INFO] [1738603063.768747667] [calyosensus_node]: Processed frame # 1

Press Ctrl+C in your terminal to quit the running ROS process once you have collected 20 or so frames.

Inspecting the data​

Now navigate to the folder we stored the XML files in and inspect one of the XML files.

cd ~/Calyo-Tutorials/1.1-RawDataCapture/
cat ExampleFile.xml # (Or open in your favourite text editor)

At the top of the file you will observe some metadata about the file and how the data was collected. This is often useful blah blah blah

The rest of the file containes the raw bytes captured from the sensor! While it may not appar very useful in this format this is a useful technquie as you can focus on calibrating and tuning the software later on now that you have captured some useful data as all the SENSUS software will be able to injest these XML files and process them further.

Reconstructed Data Capture (Extensions)​

Currently we are capturing completely raw bytes directly from the sensor, however sometimes it is useful to record the recontucted bytes (i.e. the float values). This saves us having to recontconstuct every packet when we try and read the data + its more human readable if you want to inspect the data, however does mean each of the saved XML files will be much larger in file size so if you are capturing large amounts of raw sensor data the previoly described method would be much better suited.

raw-data-capture-tutorial.yaml
type: calyosensus
pipelines:
    - id: signal_capture
      reader:
        type: DeviceReader
        serial_number: any
        num_cycles: 20
        max_distance: 8
      writer:
        type: XMLWriter
        output_path: "~/Calyo-Tutorials/1.1-RawDataCapture/"
      processor:
        type:
          stage: passthrough
          stage: signal_reconstruction

Summary​

We have learnt quite a bit about the SENSUS software and the basics of how it works in this tutorial

  • Configoration using YAML
  • SENSUS ROS
  • Calyo XML storeage files

In the next tutorial we will use the some of this raw data to generate some 2-D intensity maps of the environment.

Next steps​

Related content​