Home / Datasets / FieldSAFE

04.1 · Open dataset

FieldSAFE

Dataset for Obstacle Detection in Agriculture

Download →

GitHub ↗

2 h

Raw sensor data

6

Sensing modalities

403 GB

Rosbags

This dataset is a re-hosting of FieldSAFE, originally co-created by Mikkel Fly Kragh and Peter Christiansen and hosted by Aarhus University at https://vision.eng.au.dk/fieldsafe (archived version). We thank Department of Electrical and Computer Engineering, Aarhus University for their foundational contribution to this data.

The FieldSAFE dataset is a novel multi-modal dataset for obstacle detection in agriculture. It comprises 2 hours of raw sensor data from a tractor-mounted sensor system in a grass mowing scenario in Denmark, October 2016.

Sensing modalities include stereo camera, thermal camera, web camera, 360-degree camera, lidar, and radar, while precise localization is available from fused IMU and GNSS. Both static and moving obstacles are present including humans, mannequin dolls, rocks, barrels, buildings, vehicles, and vegetation. All obstacles have ground truth object labels and geographic coordinates.

Sensors

Sensor	Model	Resolution	FOV / °	Range / m	Datarate / fps
Stereo camera	Multisense S21 CMV2000	1024 x 544	85 x 50	1.5-50	10
Web camera	Logitech HD Pro C920	1920 x 1080	70 x 43	n/a	20
360 camera	Giroptic 360cam	2048 x 833	360 x 292	n/a	30
Thermal camera	Flir A65, 13 mm lens	640 x 512	45 x 37	n/a	30
Lidar	Velodyne HDL-32E	2172 x 32	360 x 40	1-100	10
Radar	Delphi ESR	16 targets/frame 16 targets/frame	90 x 4.2 20 x 4.2	0-60 0-174	20 20

Usage

Recorded with ROS, shipped as rosbags

The entire dataset was recorded using the Robot Operating System (ROS) and is therefore contained in rosbags.

For detailed installation instructions and examples for how to use the dataset, see github.com/mikkelkh/FieldSAFE.

In the ROS Details section below, ROS topics and data types for all sensors are listed along with a ROS transform tree.

Download

Filename	Size / GB	Duration / mm:ss	Description
static_map_rgb.png	0.5	n/a	Static RGB orthophoto of the field.
static_ground_truth.png	0.002	n/a	Static ground truth map the field.
2016-10-25-11-41-21_example.bag	3.6	01:00	1 minute example bag. Extract from sequence with dynamic obstacles.
2016-10-25-11-09-42.bag	73.9	17:25	Static obstacle session #1. 3 laps around the grass field with several static obstacle along the way.
2016-10-25-11-34-25.bag	26.6	06:00	Repositioning of the tractor. At the end, the setup is ready for recording dynamic obstacles.
2016-10-25-11-41-21.bag	91.3	21:54	Dynamic obstacle session #1. Humans move in random patterns, crossing the path of the tractor.
2016-10-25-12-07-22.bag	95.2	22:17	Dynamic obstacle session #2. Humans and a mini digger move in random patterns, crossing the path of the tractor.
2016-10-25-12-37-57.bag	112.2	26:05	Static obstacle session #2. The rest of the field is traversed and mowed, while passing a number of static obstacles.

Ground truth

Labels in global
coordinates.

Ground truth information on object location and class labels for both static and moving obstacles is available as timestamped global (geographic) coordinates. By transforming local sensor data from the tractor into global coordinates, a simple look-up of class label into the annotated ground truth map is possible.

Static and dynamic obstacles

**Static obstacles** Orthophoto with tractor tracks overlaid. Black tracks include only static obstacles, whereas red and white tracks also have moving obstacles. Currently, red tracks have no ground truth for moving obstacles annotated.

**Static Obstacles** Labeled orthophoto.

Animation of the tractor path and moving obstacles on the ground truth map — **Dynamic obstacles** Path of tractor and moving obstacles (humans) overlaid on the static ground truth map.

License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International – Licens.

Contact

For questions, contact Mikkel Fly Kragh.

Citation

If you use this dataset in your research or elsewhere, please cite/reference the following paper:

FieldSAFE: Dataset for Obstacle Detection in Agriculture ↗

@article{kragh2017fieldsafe,
AUTHOR   = {Kragh, Mikkel Fly and Christiansen, Peter and Laursen, Morten Stigaard and Larsen, Morten and Steen, Kim Arild and Green, Ole and Karstoft, Henrik and Jørgensen, Rasmus Nyholm},
TITLE    = {FieldSAFE: Dataset for Obstacle Detection in Agriculture},
JOURNAL  = {Sensors},
VOLUME   = {17},
YEAR     = {2017},
NUMBER   = {11},
ARTICLE NUMBER = {2579},
URL      = {http://www.mdpi.com/1424-8220/17/11/2579},
ISSN     = {1424-8220},
DOI      = {10.3390/s17112579}
}

Transforms,calibration,topics.

ROS details

Below, ROS-specific details are given for which transforms and topics are available. Extraction of the estimated extrinsic parameters for all sensors is described in the Transforms section. Intrinsic camera parameters are described in the Camera Calibration section. All available topics for each sensor are described in the Topics section.

More information and example code is available in the git repository.

Transforms

Below, a screenshot from rviz illustrates the sensor platform and sensor coordinate frames in ROS. A full TF (transform) tree is available here.

To extract specific extrinsic parameters, the tf_echo package in ROS can be used. Below, an example is given for finding translation and rotation between the lidar (velodyne) and the stereo camera (cam_stereo_left_frame). The example requires the provided demo to be run in another terminal at the same time as described in the git repository.

rosrun tf tf_echo velodyne cam_stereo_left_frame

At time 0.000
- Translation: [0.012, 0.427, -0.295]
- Rotation: in Quaternion [0.564, -0.539, 0.428, -0.457]
            in RPY (radian) [-1.788, 0.010, -1.517]
            in RPY (degree) [-102.468, 0.590, -86.895]

Screenshot from rviz showing the sensor platform and coordinate frames — **rviz** sensor platform and coordinate frames

Camera Calibration

The stereo and thermal cameras have been registered and calibrated using the camera calibration method available in the Computer Vision System Toolbox in MATLAB.

The intrinsic camera parameters are available as MATLAB mat files:

Stereo camera parameters

.mat

Thermal camera parameters

.mat

Details for how to show reprojection errors and visualize extrinsic parameters are available from the MathWorks Documentation.

Reprojection error plot from the camera calibration — **Reprojection error** per calibration image

Topics

The recorded rosbags include a number of different topics for each sensor. The table below lists all the topics and their message types in ROS.

Sensor	Topics	Message type	Description
Stereo camera	/Multisense/depth	sensor_msgs/Image	Depth image
	/Multisense/imu/accelerometer_vector	geometry_msgs/Vector3Stamped	Accelerometer data
	/Multisense/imu/gyroscope_vector	geometry_msgs/Vector3Stamped	Gyroscope data
	/Multisense/imu/imu_data	sensor_msgs/Imu	Packed accelerometer and gyroscope data
	/Multisense/imu/magnetometer_vector	geometry_msgs/Vector3Stamped	Magnetometer data
	/Multisense/left/camera_info	sensor_msgs/CameraInfo	Intrinsic calibration data (factory settings)
	/Multisense/left/image_rect_color	sensor_msgs/Image	Rectified left color image
	/Multisense/left/image_rect_color/camera_info	sensor_msgs/CameraInfo	Intrinsic calibration data (copy of above)
	/Multisense/pps	std_msgs/Time	Pulse per second signal
	/Multisense/right/camera_info	sensor_msgs/CameraInfo	Intrinsic calibration data (factory settings)
	/Multisense/right/image_rect	sensor_msgs/Image	Right grayscale image
	/Multisense/right/image_rect/camera_info	sensor_msgs/CameraInfo	Intrinsic calibration data (copy of above)
	/Multisense/stamped_pps	multisense_ros/StampedPps	Timestamped pulse per second signal
	/Multisense/status	multisense_ros/DeviceStatus	Device status message
Web camera	/Logitech_webcam/camera_info	sensor_msgs/CameraInfo	Intrinsic calibration data
	/Logitech_webcam/image_raw/compressed	sensor_msgs/CompressedImage	Compressed color image
360 camera	/giroptic_360/image/compressed	sensor_msgs/CompressedImage	Compressed color image
Thermal camera	/FlirA65/camera_info	sensor_msgs/CameraInfo	Intrinsic calibration data (do not use)
	/FlirA65/image_raw	sensor_msgs/Image	Grayscale image
Lidar	/velodyne_packets	velodyne_msgs/VelodyneScan	Raw laser data packets. Use provided software to generate 3D point clouds.
Radar	/fmData/can_rx_radar	msgs/can	Raw CAN messages
	/Delphi_ESR/RadarData	htf_delphi_esr/Delphi_radar	Radar detections parsed from CAN messages
GPS	/fmData/gps_rx	msgs/serial	Raw serial messages from GPS. Use provided software to generate sensor_msgs/NavSatFix messages.
IMU	/fmData/imu_rx	msgs/serial	Raw serial messages from IMU. Use provided software to generate sensor_msgs/Imu messages.
	/fmData/imu_tx	msgs/serial	Raw serial message requests sent to the IMU.
System	/diagnostics	diagnostic_msgs/DiagnosticArray	Diagnostics
	/tf	tf/tfMessage	Outdated transform messages. Must be ignored when replaying bag. Use provided software to generate updated transforms.

Note that intrinsic calibration data for cameras are included directly in ROS messages. These values, however, are factory defaults and not the ones estimated during camera calibration. For the calibrated values, see the Calibration section, above.

Below, an example is given for extracting the intrinsic parameters for the left image of the stereo camera. The example requires the provided demo to be run in another terminal at the same time as described in the git repository.

rostopic echo /Multisense/left/camera_info

header:
  seq: 21419
  stamp:
    secs: 1477388586
    nsecs: 798077000
  frame_id: /Multisense/left_camera_optical_frame
height: 544
width: 1024
distortion_model: plumb_bob
D: [0.0030753163155168295, 0.002497022273018956, 0.0003005412872880697,
    0.001575434347614646, -0.003454494522884488, 0.0, 0.0, 0.0]
K: [555.9204711914062, 0.0, 498.1905517578125, 0.0, 556.6275634765625,
    252.35089111328125, 0.0, 0.0, 1.0]
R: [0.9999634027481079, -0.000500216381624341, 0.00853759702295065,
    0.0005011018947698176, 0.9999998807907104, -0.00010158627264900133,
    -0.00853754486888647, 0.00010586075950413942, 0.9999635219573975]
P: [580.6427001953125, 0.0, 512.0, 0.0, 0.0, 580.6427001953125, 254.5, 0.0,
    0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi:
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False
---