读论文系列：The TNO Multiband Image Data Collection

文章目录

读论文系列：The TNO Multiband Image Data Collection1️⃣ 资料2️⃣ 文章翻译摘要Specifications Table 规格表Value of the Data 数据的价值Data 数据Experimental design, materials, and methods 3️⃣ 我的笔记目录结构TRICLOOBSTNOKayak第一部分第二部分 我的总结

读论文系列：The TNO Multiband Image Data Collection

1️⃣ 资料

说明：非常老牌的红外可见光融合数据集，值得一看官方仓库：https://figshare.com/articles/dataset/TNO_Image_Fusion_Dataset/1008029大佬笔记：https://blog.csdn.net/qq_43249953/article/details/139769505

2️⃣ 文章翻译

摘要

Despite of the ongoing interest in the fusion of multi-band images for surveillance applications and a steady stream of publications in this area, there is only a very small number of static registered multi-band test images (and a total lack of dynamic image sequences) publicly available for the development and evaluation of image fusion algorithms. To fill this gap, the TNO Multiband Image Collection provides intensified visual (390–700 nm), nearinfrared (700–1000 nm), and longwave infrared (8–12 μm) nighttime imagery of different military and surveillance scenarios, showing different objects and targets (e.g., people, vehicles) in a range of different (e.g., rural, urban) backgrounds. The dataset will be useful for the development of static and dynamic image fusion algorithms, color fusion algorithms, multispectral target detection and recognition algorithms, and dim target detection algorithms.

尽管人们对用于监控应用的多波段图像融合持续感兴趣，并且该领域的出版物源源不断，但只有极少数静态注册的多波段测试图像（完全缺乏动态图像序列）可供公开用于图像融合算法的开发和评估。为了填补这一空白，TNO多波段图像集提供了不同军事和监视场景的增强视觉（390-700nm）、近红外（700-1000nm）和长波红外（8-12μm）夜间图像，显示了不同背景（如农村、城市）中的不同物体和目标（如人、车辆）。该数据集将有助于开发静态和动态图像融合算法、颜色融合算法、多光谱目标检测和识别算法以及弱目标检测算法。

Specifications Table 规格表

标题	内容
Subject area	Digital image processing - Image fusion
Type of data	Visual, near-infrared (NIR) and longwave infrared (LWIR) digital images representing different nighttime military and surveillance scenarios.
How data was acquired	The images were acquired with different multiband camera systems. 这些图像是用不同的多波段相机系统采集的。
Data format	BMP, TIF, MP4
Experimental factors	The images have been geometrically warped and registered so that corresponding image pairs have pixelwise correspondence. 图像已经过几何扭曲和配准，使得相应的图像对具有像素对应关系。
Experimental features	The imagery was collected in (semi-)darkness during several outdoor field trials in both rural and urban areas. 这些图像是在农村和城市地区的几次户外田间试验中在（半）黑暗中收集的。
Data source location	The imagery was collected at different sites in the Netherlands. 这些图像是在荷兰的不同地点收集的。
Data accessibility	https://doi.org/10.6084/m9.figshare.c.3860689.v1
[1]	A. Toet, J.K. IJspeert, A.M. Waxman, M. Aguilar, Fusion of visible and thermal imagery improves situational awareness, Displays 18 (2) (1997) 85–95. http://dx.doi.org/10.1016/S0141-9382(97)00014-0.
[2]	A. Toet, Detection of dim point targets in cluttered maritime backgrounds through multisensor image fusion, in: W. R. Watkins, D. Clement, W.R. Reynolds (Eds.), Targets and Backgrounds: Characterization and Representation VIII, The International Society for Optical Engineering, Bellingham, WA, http://dx.doi.org/10.1117/12.478798.
[3]	A. Toet, M.A. Hogervorst, A.R. Pinkus, The TRICLOBS dynamic multi-band image data set for the development and evaluation of image fusion methods, PLoS One 11 (12) (2016) e0165016. http://dx.doi.org/10.1371/journal.pone.0165016.

Value of the Data 数据的价值

The dataset will be useful for the development of

static and dynamic image fusion algorithms, 静态和动态图像融合算法color fusion algorithms, 颜色融合算法multispectral target detection and recognition algorithms, 多光谱目标检测和识别算法dim target detection algorithms. 弱小目标检测算法

Data 数据

The TNO Multiband Image Collection currently consists of three individual image sets:

The TNO Image Fusion Dataset The Kayak Image Fusion Sequence (parts I and II) The TRICLOBS Dynamic Multiband Image Dataset

The TNO Image Fusion Dataset [1] contains intensified visual (390–700 nm), near-infrared (7001000 nm), and longwave infrared (8–12 μm) nighttime imagery of different military and surveillance scenarios, showing different objects and targets (e.g., people, vehicles) in different (e.g., rural, urban) backgrounds.

TNO图像融合数据集[1]包含不同军事和监视场景的增强视觉（390-700nm）、近红外（7001000nm）和长波红外（8-12μm）夜间图像，显示了不同背景（如农村、城市）中的不同物体和目标（如人、车辆）。

The multimodal Kayak Image Fusion Sequence [2] contains registered visual, near-infrared and longwave infrared image sequences showing three approaching kayaks in a cluttered maritime background. Because of the variation in distance the targets (kayaks) vary from dim point targets to easily distinguishable objects.

多模式皮划艇图像融合序列[2]包含注册的视觉、近红外和长波红外图像序列，显示了三艘正在接近的皮划艇在杂乱的海上背景中。由于距离的变化，目标（皮划艇）从暗淡的点目标到易于区分的物体各不相同。

The TRICLOBS Dynamic Multiband Image Dataset [3] contains registered visual (400–700 nm), near-infrared (NIR, 700–1000 nm) and longwave infrared (LWIR, 8–14 μm) motion sequences of dynamic surveillance scenarios in an urban environment. To enable the development or realistic color remapping procedures, the dataset also contains color photographs of each of the three scenes. This dataset was collected during several field trials at three different locations and contains 16 motion sequences representing different military and civilian surveillance scenarios.

TRICLOBS动态多波段图像数据集[3]包含城市环境中动态监控场景的注册视觉（400-700nm）、近红外（NIR，700-1000nm）和长波红外（LWIR，8-14μm）运动序列。为了实现开发或逼真的颜色重映射过程，数据集还包含三个场景中每个场景的彩色照片。该数据集是在三个不同地点的几次现场试验中收集的，包含代表不同军事和民用监视场景的16个运动序列。

All three datasets include publications describing the registration conditions and the used camera systems in full detail. The data collection will be incrementally extended with new imagery when this becomes available. The images in this data collection can freely be used for research purposes, and may be used in publications without prior notice, provided this paper is properly referenced.

所有三个数据集都包括详细描述注册条件和使用的相机系统的出版物。当新的图像可用时，数据收集将逐步扩展。本数据集中的图像可自由用于研究目的，并可在不事先通知的情况下用于出版物，前提是正确引用本文。

Experimental design, materials, and methods

The original sensor signals were warped and subsampled to achieve pixelwise image registration.
原始传感器信号被扭曲和二次采样，以实现像素图像配准。

3️⃣ 我的笔记

目录结构

我感觉有必要分别列举一下目录结构。因为其一是TNO 中包含了三个数据集，内容混乱，其二是本身图片并不多，一一列举是实际的。

可见，并不是 TNO、Kayak、TRICLOBS 三个文件夹，而是一堆杂乱的结构。

TRICLOOBS

TRICLOBS子数据集对应着 Triclobs_images文件夹。

(base) kimshan@MacBook-Pro Triclobs_images % lsBallsMarne_06ReekBosniaMarne_07VeluweFarmMarne_09VlasakkersHouseMarne_11barbed_wire_1Kaptein_01Marne_15barbed_wire_2Kaptein_1123Marne_24houses_with_3_menKaptein_1654Movie_01jeep_in_smokeKaptein_19Movie_12pancake_houseMarne_01Movie_14soldier_behind_smokeMarne_02Movie_18soldiers_with_jeepMarne_03Movie_24square_with_housesMarne_04REFRENCES

总结一下，

场景	可见光	近红外	远红外	其他
Balls 屋外的黑色铁球	VIS.bmp	NIR.bmp		photo.bmp
barbed_wire_1 人跑过铁丝网	R_Vis.tif	G_NIR.tif	B_LWIR.tif	RGB.tif
barbed_wire_2 人跑过铁丝网	a_VIS-MarnehNew _24RGB_1110.tif	b_NIR-MarnehNew _24RGB_1110.tif	c_LWIR-MarnehNew _24RGB_1110.tif
Bosnia 草地上的房子	VIS_R.bmp	NIR_G.bmp	LWIR_B.bmp	daylight_image.bmp
Farm 草地上的房子	Farm_Vis.bmp	Farm_II.bmp	Farm_IR.bmp	photo.bmp
House 灯热影响红外	VIS.bmp	NIR.bmp		photo.bmp
houses_with_3_men 房子和三个人	VIS.bmp	NIR.bmp	LWIR.bmp
jeep_in_smoke 雾中的吉普	VIS_R.bmp	NIR_G.bmp	LWIR_B.bmp
Kaptein_01 黑夜中的门	Vis01.bmp	NIR01.bmp	IR01.bmp	photo.bmp
Kaptein_19 黑夜中的帐篷	Vis19.bmp	NIR19.bmp	IR19.bmp	photo.bmp
Kaptein_1123 黑夜中的门和人	Kaptein_ 1123_Vis.bmp	Kaptein_ 1123_II.bmp	Kaptein_ 1123_IR.bmp	photo.bmp
Kaptein_1654 黑夜中的帐篷和人	Kaptein_ 1654_Vis.bmp	Kaptein_ 1654_II.bmp	Kaptein_ 1654_IR.bmp	Kaptein_1654_ ref_with_man.bmp, Kaptein_ 1654_REF.bmp
Marne_01 白天的房子	Marne_01_Vis.bmp	Marne_01_II.bmp	Marne_01_IR.bmp	Marne_01_REF.bmp
Marne_02 白天的房子	Marne_02_Vis.bmp	Marne_02_II.bmp	Marne_02_IR.bmp	Marne_02_REF.bmp
Marne_03 白天的房子	Marne_03_Vis.bmp	Marne_03_II.bmp	Marne_03_IR.bmp	Marne_03_REF.bmp
Marne_04 著名的吉普	Marne_04_Vis.bmp	Marne_04_II.bmp	Marne_04_IR.bmp	Marne_04_REF.bmp
Marne_06 白天的房子	Marne_06_Vis.bmp	Marne_06_II.bmp	Marne_06_IR.bmp	Marne_06_REF.bmp
Marne_07 白天的房子	Marne_07_Vis.bmp	Marne_07_II.bmp	Marne_07_IR.bmp	Marne_07_REF.bmp
Marne_08 白天的房子	Marne_08_Vis.bmp	Marne_08_II.bmp	Marne_08_IR.bmp	Marne_08_REF.bmp
Marne_09 白天的房子	Marne_09_Vis.bmp	Marne_09_II.bmp	Marne_09_IR.bmp	Marne_09_REF.bmp
Marne_11 白天的房子	Marne_11_Vis.bmp	Marne_11_II.bmp	Marne_11_IR.bmp
Marne_15 白天的房子	Marne_15_Vis.bmp	Marne_15_II.bmp	Marne_15_IR.bmp
Marne_24 铁丝和奔跑的人	Marne_15_Vis.bmp	Marne_15_II.bmp	Marne_15_IR.bmp
Movie_01 白天的街道	Movie_01_Vis.bmp	Movie_01_II.bmp	Movie_01_IR.bmp	Movie_01_REF.bmp
Movie_12 房子	Movie_12_Vis.bmp	Movie_12_II.bmp	Movie_12_IR.bmp	Movie_12_REF.bmp
Movie_14 房子和人	Movie_14_Vis.bmp	Movie_14_II.bmp	Movie_14_IR.bmp	Movie_14_REF.bmp
Movie_18 房子和人和车	Movie_18_Vis.bmp	Movie_18_II.bmp	Movie_18_IR.bmp	Movie_18_REF.bmp
Movie_24	房子	Movie_24_Vis.bmp	Movie_24_II.bmp	Movie_24_IR.bmp
pancake_house 白天的街道	VIS.tif	NIR.tif		photo.tif
Reek 白天的房子	Reek_Vis.bmp	Reek_II.bmp	Reek_IR.bmp	Reek_REF.bmp
soldier_behind_smoke 雾中的士兵	VIS-MarnehNew_ 15RGB_603.tif	NIR-MarnehNew_ 15RGB_603.tif	LWIR-MarnehNew_ 15RGB_603.tif	RGB-MarnehNew_ 15RGB_603.tif
soldiers_with_jeep 士兵和吉普	Jeep_Vis.bmp	Jeep_II.bmp	Jeep_IR.bmp
square_with_houses 房子和方框	VIS.bmp	NIR.bmp	LWIR.bmp
Veluwe 望远镜中的房子	VIS.bmp	NIR.bmp		photo.bmp
Vlasakkers 房子	VIS.tif	NIR.tif		photo.tif

TNO

在Athena_imagesz合格文件夹中的REFRENCES里边，有这样的 pdf：Report_TNO-DV-2007-A329.pdf。所以我认为这个文件夹里边都是 TNO 下的TNO的。

(base) kimshan@MacBook-Pro Athena_images % ls2_men_in_front_of_househelicopterAPC_1lakeAPC_2man_in_doorwayAPC_3soldier_behind_smoke_1APC_4soldier_behind_smoke_2REFRENCESsoldier_behind_smoke_3airplane_in_treessoldier_in_trench_1bunkersoldier_in_trench_2heather

还是整理一下：

场景	可见光	近红外	远红外	其他
2_men_in_front _of_house 房子和两个人	VIS_meting003_r.bmp		IR_meting003_g.bmp	meting003_rg.bmp
airplane_in_trees 树中的飞机	vis.bmp		ir.bmp
APC_1/view_1 装甲车	VIS_fk_06_005.bmp		IR_fk_06_005.bmp	fk_06_005_rg.bmp
APC_1/view_2 装甲车和人	VIS_fk_ref_01_005.bmp		IR_fk_ref_01_005.bmp	fk_ref_01_005_rg.bmp
APC_1/view_3 装甲车	VIS_fk_ref_02_005.bmp		IR_fk_ref_02_005.bmp	fk_ref_02_005_rg.bmp
APC_2/view_1 装甲车	1_fk_ge_03_005.bmp		2_fk_ge_03_005.bmp	fk_ge_03_005_rg.bmp
APC_2/view_2 装甲车	1_fk_ge_04_005.bmp		2_fk_ge_04_005.bmp	fk_ge_04_005_rg.bmp
APC_2/view_3 装甲车	1_fk_ge_06_005.bmp		2_fk_ge_06_005.bmp	fk_ge_06_005_rg.bmp
APC_3view_1 装甲车	VIS_fk_bar_06_005.bmp		IR_fk_bar_06_005.bmp	fk_bar_06_005_rg.bmp
APC_3view_2 装甲车	VIS_fk_bar_01_005.bmp		IR_fk_bar_01_005.bmp	fk_bar_01_005_rg.bmp
APC_3view_3 装甲车	VIS_fk_bar_05_005.bmp		IR_fk_bar_05_005.bmp	fk_bar_05_005_rg.bmp
APC_4 装甲车	VIS_fennek01_005.bmp		IR_fennek01_005.bmp	fennek01_005_rg.bmp
bunker 树中建筑	bunker_r.bmp		IR_bunker_g.bmp	VIS_bunker-rg.bmp
heather 树中道路	VIS_hei_vis_r.bmp		IR_hei_vis_g.bmp	hei_vis-rg.bmp
helicopter 直升机	VIS_helib_011.bmp		IR_helib_011.bmp	helib_011_rg.bmp
lake 湖	VIS_lake_r.bmp		IR_lake_g.bmp	lake_rg.bmp
man_in_doorway 男人在门口	VIS_maninhuis_r.bmp		IR_maninhuis_g.bmp	maninhuis_rg.bmp
soldier_behind_smoke_1 烟中狙击手	VIS_meting012-1200_r.bmp		IR_meting012-1200_g.bmp	meting012-1200_rg.bmp
soldier_behind_smoke_2 烟中狙击手	VIS_meting012-1500_r.bmp		IR_meting012-1500_g.bmp	meting012-1500_rg.bmp
soldier_behind_smoke_3 烟中狙击手	VIS_meting012-1700_r.bmp		IR_meting012-1700_g.bmp	meting012-1700_rg.bmp
soldier_in_trench_1 桥里的男人	VIS_meting016_r.bmp		IR_meting016_g.bmp	meting016_rg.bmp
soldier_in_trench_2 桥里的男人	VIS_meting055_r.bmp		IR_meting055_g.bmp	meting055_rg.bmp

Kayak

那我就认为剩下的都属于Kayak子集。那这就又很杂乱了。我还是分两部分：

第一部分是Kayak的特色，一些序列，第二部分是剩下的杂项。

第一部分

Introduction Intensified visual images have the extensions VIS and/or RNear Infrared images have the extensions NIR and/or GThermal images have the extensions IR , RAD and/or BDHV images are false color (R,G,0) images corresponding to (VIS,NIR,0) Datasets DHV_images/Fire_sequenceFEL_images/Duine_sequenceFEL_images/Nato_camp_sequenceFEL_images/Tree_sequence

总结

名称	介绍	VIS	DHV	Thermal/RAD
DHV_images/Fire_sequence/Part1	烟火		DHV2.bmp ~ HDV29.bmp	RAD2.bmp ~ RAD29.bmp
DHV_images/Fire_sequence/Part2	烟火		DHVheli0.bmp ~ DHVheli16.bmp	RADheli0.bmp ~ RADheli16.bmp
DHV_images/Fire_sequence/Part3	烟火		DHVheli20.bmp ~ DHVheli80.bmp	RADheli20.bmp ~ RADheli80.bmp
FEL_images/Duine_sequence	路上人	7400v.bmp ~ 7422v.bmp		7400i.bmp ~ 7422i.bmp
FEL_images/Nato_camp_sequence	路上人	1800v.bmp ~ 1831v.bmp		1800i.bmp ~ 1831i.bmp
FEL_images/Tree_sequence	林中人	4900v.bmp ~ 4918i.bmp		4900i.bmp ~ 4918i.bmp

第二部分

场景与路径	可见光	近红外	远红外	其他
tank 著名的坦克	Vis.tif		LWIR.tif
DHV_images/bench 河边人	VIS_37dhvR.bmp	NIR_37dhvG.bmp	IR_37rad.bmp
DHV_images/sandpath 林中人	VIS_18dhvR.bmp	NIR_18dhvG.bmp	IR_18rad.bmp
DHV_images/wall 墙	VIS_163dhvR.BMP	NIR_163dhvG.BMP	IR_163rad.bmp

我的总结

TNO其实是一个散装的数据集，里边包含了三个小数据集。TNO/TNO，TNO/TRICLOOBS 以及 TNO/Kayak 的一部分，结构比较简单，就是可见光，近红外，远红外，照片或者 fask color 组成。其中会有缺省的内容。TNO/Kayak的另一部分都是连续视频的截图，包含了好几个序列。这里边的“烟火”序列，他的可见光和近红外分别组成了 R 通道和 G 通道的 fake color 图片，需要再解析一下。TNO 数据集中文件的命名很随意，有的是 VIS，IR 这样的，其他大部分也加入了场景描述以及序列 id，并且文件夹嵌套混乱，通过程序自动读入比较困难，需要手工挑选图片。