\documentclass[conference,final]{IEEEtran}
%\IEEEoverridecommandlockouts
% The preceding line is only needed to identify funding in the first footnote. If that is unneeded, please comment it out.
%Template version as of 6/27/2024

\usepackage[l2tabu, orthodox, abort]{nag}%%  warnt vor outdated packages und LaTeX-Sünden und bricht bei Fehlern ab
\usepackage{xspace} % to get the spacing after macros right

% run this only on final as this makes LaTeX slow (albeit precise)
\usepackage{ellipsis} % ... ;-) %% besserer Satz von \dots, v.a. space around
\usepackage{relsize} % scale font up or down  
\usepackage{microtype}% micro typogragraphy - place this after the font has been loaded

%\usepackage{cite}% we use BibLaTeX instead

\usepackage{amsmath,amssymb,amsfonts}
\usepackage{mathtools}%fix some Problems of AMS Math and some additional tools; load after AMSmath
% This package only works, if formulas are set by \( \) and \begin/end{equation} or \[ \]
\DeclareMathOperator{\e}{e}
\renewcommand{\Re}{\operatorname{Re}}
\renewcommand{\Im}{\operatorname{Im}}

\usepackage{diffcoeff} % differentials
\usepackage[OMLmathbf,rmdefault=mdput,sfdefault=arev,scaled=0.85]{isomath} % conflicts with bm
\renewcommand{\vec}{\vectorsym}
\newcommand{\mat}{\matrixsym}

\usepackage{xfrac}% required by siunitx
\usepackage[parse-units=false,
per-mode=symbol,%fraction,\frac, \tfrac, \dfrac, \sfrac
]{siunitx}% SI-Einheiten setzen und Komma richtig setzten
\DeclareSIUnit{\dBm}{dBm}
\DeclareSIUnit{\dBc}{dBc}

%\usepackage{algorithmic}

\usepackage[style=ieee,url=false,isbn=false]{biblatex}
\addbibresource{bibliography_zotero.bib}
\addbibresource{bibliography_manual.bib}
\AtEveryBibitem{\clearfield{note}}

\usepackage[acronym,nomain,shortcuts,nonumberlist,xindy]{glossaries}%style=long; translate=false - if babel is loaded an you should not
\loadglsentries{abbreviations}
%\makeglossaries

\usepackage{graphicx}
\usepackage{textcomp}
\usepackage{xcolor}

\usepackage{tikz}
\usepackage{pgfplots}
\pgfplotsset{compat=1.8}

%\usepackage{changes}
%\definechangesauthor[name={Stephan}, color=orange]{sl}
%%\setremarkmarkup{(#2)}

\usepackage{tabularx} % better tables
\usepackage{multirow}

\usepackage{cleveref}

% Verhinderung von "Schusterjungen"
% = einzelne Absatzzeilen auf der Seite unten
\clubpenalty = 10000

% Verhinderung von "Hurenkindern"
% = einzelne Zeilen eines Absatzes am Kopf der Seite
\widowpenalty = 10000
\displaywidowpenalty = 10000

% gibt ein besseres Bild im Math-Mod (vgl. Voss: Math Mode)
% mach Klammern werden nach außen hin immer größer
\delimitershortfall=-1pt

\newcommand{\eg}{e.\,g.\xspace}
\newcommand{\ie}{i.\,e.\xspace}


\begin{document}

\title{An Attention-based Architecture for Early Fire Detection% \\
%{\footnotesize \textsuperscript{*}Note: Sub-titles are not captured for %https://ieeexplore.ieee.org  and
%should not be used}
%\thanks{Identify applicable funding agency here. If none, delete this.}
}

\author{\IEEEauthorblockN{Tim Raunegger-Müller}
\IEEEauthorblockA{\textit{Faculty of Electronics and Computer Science}\\
\textit{Aalen University of Applied Sciences}\\
Aalen, Germany\\
ORCID 0009-0005-9279-0146}%tim.raunegger-mueller@studmail.htw-aalen.de
\and
\IEEEauthorblockN{Ala' Khalifeh}
\IEEEauthorblockA{\textit{School of Computer Science} \\
\textit{German Jordanian University}\\
Amman, Jordan\\
ORCID 0000-0003-3600-8090}
\and
\IEEEauthorblockN{Stephan Ludwig}
\IEEEauthorblockA{\textit{Faculty of Electronics and Computer Science}\\
\textit{Aalen University of Applied Sciences}\\
Aalen, Germany\\
ORCID 0000-0002-0233-1706}%stephan.ludwig@hs-aalen.de
}

\maketitle

\begin{abstract}
Early fire detection is essential for preventing fatalities and large-scale economic and environmental damage.
Traditional systems like smoke alarms suffer from delayed detection, especially in complex environments.
This work proposes an attention-based machine learning architecture for early fire detection using multivariate time-series sensor data from distributed sensor nodes.
It leverages multi-headed attention as a way of capturing long-ranging dependencies in data, enabling more effective discrimination between fire and nuisance scenarios.
An evaluation is carried out using a dataset of fire event types recorded in an industrial hall.
After preprocessing the data, the attention model is trained and compared against a feed-forward neural network baseline.
Results show that the attention-based approach achieves superior performance across all evaluated metrics, with \qty{99.6}{\percent} accuracy, higher precision (\num{0.89}), improved F1-score (\num{0.937}), and a significantly reduced false positive rate.
%These findings demonstrate that attention mechanisms are highly effective for modeling multivariate sensor dynamics and provide a promising foundation for early fire detection systems of the next generation.
\end{abstract}

\begin{IEEEkeywords}
Early fire detection, Multivariate sensor data, Attention mechanisms, Time-series analysis, Deep learning, Residual networks, Recurrent neural networks, 
Gradient descent, Sensor fusion, Anomaly detection, 
Distributed sensor networks, Industrial safety systems, 
False alarm reduction
\end{IEEEkeywords}

\section{Introduction}

Fires have a multitude of root causes and can spread extremely fast if not noticed and contained early.
Fire safety remains a critical public health challenge, with an estimated \num{300000} deaths annually due to fire, and many more people suffering long-term disabilities and disfigurement \cite{WorldHealthOrganization_Burnpreventionsuccess_2011}.
%Fires cause not only immediate physical destruction to buildings and infrastructure, but also economic, health, and environmental costs. 
Systematic reviews of residential fire incidents reveal that losses include medical costs, lost productivity, emergency response, and long-term quality-of-life impacts \cite{RahmanEtAl_EconomicCostsResidential_2023} \cite{RichardsonEtAl_hiddencostwildfires_2012}.
Beyond the tragic human toll, fires impose enormous economic costs \cite{USFA_statistics}.

Traditional fire alarm systems primarily rely on threshold-based smoke, heat, or flame sensors.
These are inexpensive and widely deployed but face several limitations like delayed detection in large or obstructed indoor spaces, high false-positive rates due to nuisances like cooking aerosols, dust, steam, and ineffectiveness in outdoor environments, which calls for next-generation multimodal approaches \cite{RajendranChenniappan_comprehensivesurveyseveral_2024}.
Additional reviews evaluating detection performance in indoor areas show significant variations depending on air flow, sensor placement and fire \cite{DengEtAl_LargeSpaceFireDetection_2025, WangEtAl_ExperimentalStudyEarly_2024}.

In parallel to conventional devices, modern research investigates video-based fire and smoke detection, which offers earlier detection than traditional sensors by recognizing visual cues.
These systems have evolved rapidly, especially in the context of forest monitoring \cite{SunEtAl_forestfiredetection_2024}.
Indoor-focused methods include computer-vision pipelines as well as deep learning models, such as intelligent video-based smoke and flame detectors \cite{SharmaKanwal_Intelligentvideobasedfire_2025}, convolutional neural network-based approaches like FLAME \cite{GragnanielloEtAl_FLAMEfiredetection_2025}, the work proposed in \cite{Khalifeh}, and real-time indoor fire and smoke recognition systems \cite{YangEtAl_Realtimeintelligentfire_2025,RahmanEtAl_Realtimesmokefire_2024}.
These models may struggle under challenging lighting conditions, occlusions, or sensor noise.
Additionally, video data alone may not generalize across domains.
Beyond vision, many studies explore non-visual fire detectors, including gas and temperature sensors, CO and CO\textsubscript{2} measurements and infrared detectors \cite{KhanEtAl_RecentAdvancesSensors_2022}.
Furthermore, multi-sensor data fusion has become an important research direction.
Classical fusion techniques include Dempster–Shafer theory, Bayesian fusion and Kalman filtering \cite{ZervasEtAl_Multisensordatafusion_2011, WuEtAl_MultiSensorDataFusion_2021}.

With the advent of modern \gls{ml}, deep learning-based fire detection has become a rapidly expanding field.
Another trend involves generating synthetic fire events due to the scarcity of labelled data, particularly in early-stage fire scenarios.
Kim et al. demonstrate that training with photorealistic synthetic smoke and flame images significantly improves detection accuracy and robustness \cite{KimEtAl_EarlyFireDetection_2024}.
Complementing these efforts, several curated datasets have emerged \cite{NazirEtAl_EarlyFireDetection_2022}.
Given these developments, early detection remains a crucial research area.

Modern sensors generate rich multimodal and multivariate data that existing fire detection systems are not designed to interpret, presenting both a challenge and an opportunity for advanced detection algorithms.
\gls{ml} algorithms, however, can learn complex relationships between these sequential measurements and the underlying fire events.
Simple models quickly reach their limits in this context, motivating the development of more sophisticated architectures. One promising approach involves attention mechanisms \cite{VaswaniEtAl_Attentionallyou_2017}, which can capture long-range dependencies in sequential data.
These advances have not yet been fully explored in all fire detection applications.
Therefore, in this paper, we propose a novel fire detection framework using attention-based architectures to model multivariate sensor sequences, aiming to improve early detection performance compared to traditional methods.

\section{Background}
Within language, words appear in sequences and convey meaning from one party to another.
What makes this work is that we learned a language and internalized its way of conveying information.
Machines can learn similarly using data acting as experiences of a specific domain.
In this paper, data ordered in a time-series is used.
Simple \gls{ml} models tend to struggle with longer sequences of data, which is why different approaches have been brought forward, with attention mechanisms being the newest and as of now most capable one.

\subsection{Vanishing Gradient Problem}
\Gls{ml} models of all kinds use gradient descent and backpropagation to minimize a cost function, which outputs the difference or error between ground truth and predicted values. 
In a neural network with \(L\) layers, each layer is indexed by \(l \in \{1,\dots,L\}\), where \(l=1\) denotes the first hidden layer and \(l=L\) denotes the output layer. 
Each layer computes a hidden representation, also called an activation, which is written as \(\vec{h}^{(l)}\). 
The expression \(\vec{h}^{(l-1)^\text{T}}\) refers to the transpose of the previous layer’s activation vector and appears naturally when computing gradients with respect to the weight matrix. 
Gradient descent differentiates the cost function \(\mathcal{L}\) with respect to all trainable parameters \(\mat{W}^{(l)}\) at every layer \(l\). 
This yields a gradient vector that points in the direction of the steepest increase of \(\mathcal{L}\) (cf.\ red dotted path in \cref{sntheticfig}). 
Backpropagation then uses these partial derivatives to update each weight matrix \(\mat{W}^{(l)}\) in the opposite direction, thereby iteratively reducing the model’s loss \cite{RumelhartEtAl_Learningrepresentationsbackpropagating_1986}
\begin{equation}
\diffp{\mathcal{L}}{\mat{W}^{(l)}}
= \diffp{\mathcal{L}}{\vec{h}^{(L)}} 
\prod_{k=l+1}^{L} \diffp{\vec{h}^{(k)}}{\vec{h}^{(k-1)}} 
\vec{h}^{(l-1)^\text{T}}.
\end{equation}

Since cost is to be decreased, the negative gradient is commonly used.
Over the course of a long sequence of time-series data, the gradient tends to vanish for earlier layers of the model architecture.
This phenomenon occurs, because the backpropagation step updates the weights of a deep neural network by repeatedly applying the chain rule of differentiation to compute the gradient of the loss with respect to each individual weight, \ie
\begin{equation}
    \diff*{{f\left( u \right)}}{x} = \diff*{\left[{f\left( u \right)}\right]}{u} \diff{u}{x}.    
\end{equation}

\begin{figure}
\centering
\includegraphics[width=\columnwidth]{3d_gradientdescent.png}
\caption{Gradient descent applied to an arbitrary cost function. Each step towards the minimum is marked by a red dot.}
\label{sntheticfig}
\end{figure}

Inside popular frameworks like PyTorch, the \texttt{autograd} feature \cite{pytorch_autograd_2025} is taking advantage of this rule heavily by logging all operations performed on tensors.
In recurrent or very deep architectures, this process involves multiplying many small derivatives together as the gradient is propagated backward through time or through many layers.
If the derivatives \(\difsp{\vec{h}^{(k)}}{\vec{h}^{(k-1)}}\) are less than one, the product of many such terms shrinks exponentially. 
As a result, gradients for early layers or earlier time steps become tiny, effectively freezing those weights.
Thus, earlier components of the network receive close to no updates during training.
This makes it difficult to learn long-range dependencies in sequential data, due to the parameters responsible for processing earlier information not being adjusted.
Simple \gls{ml} models struggle with longer data sequences due to the vanishing gradient problem, so over time several solutions have been proposed.

Residual networks \cite{HeEtAl_DeepResidualLearning_2016} are designed to learn the difference, or “residual”, that each layer contributes to the overall representation, rather than trying to learn a complete transformation from input to output at every layer.
This approach makes it easier to train deep networks by mitigating the vanishing gradient problem.
In contrast, recurrent neural networks maintain and update a hidden state over time, which allows them to carry information across longer sequences, making them suitable for tasks where temporal or sequential dependencies are important, such as language modeling or time series prediction \cite{PascanuEtAl_difficultytrainingrecurrent_2013}.
Even though they were intended to model long-range dependencies, in practice they fail to do so reliably because of the vanishing gradient problem.

\subsection{Attention}
Attention mechanisms avoid vanishing gradients by bypassing the sequential multiplication of gradients.
Instead of propagating information step by step, attention directly computes dependencies between all positions in the sequence.
This avoids vanishing gradients, making it easier to learn global dependencies, while allowing for better parallelization.
Parallelization is possible, because the attention process extracts meaning from a sequence.
However, another process might lead to a different kind of understanding.
Each of these processes is called an attention head, with many of them being run in parallel.
An example of this in language are adjectives that change the meaning of what a subject is, or positional indicators changing where a subject is.
The core idea is to not keep and update a hidden state, but to create contextualized embeddings. 
An embedding is a often low-dimensional, continuous vector representation of a discrete item, which is in this case a token.
These are generated from the input sequence through tokenization and vectorization.

Tokenization splits an input into chunks of data and is the designer's choice, which can be tricky for sequences of measurements.
Trial-and-error is often applied when no findings from related experiments are available.
Vectorization brings these tokens into a vector representation by mapping each token to an integer index through a lookup table.
An embedding layer then transforms these integer IDs into continuous vectors by indexing a trainable embedding matrix, which is randomly initialized at the start of training.
Each embedding vector constitutes a trainable parameter that is updated during training when using attention mechanisms.
The dimensionality of this embedding space is a hyperparameter.
The maximum number of tokens processed per sequence is another hyperparameter.

Since the complexity of attention is \( \mathcal{O}(n^{2}) \), longer sequences (of length \(n\)) lead to increased computational cost.
Let us take the sentence “A black cat and a fast horse” with the spaces between the words constituting the dividing lines for tokenization.
It can easily be imagined as one scene.
However, without attention, there is no such thing as whole sequence understanding of this sentence.
Let the adjective “black” attend to the following “cat” and “fast” to “horse”.
Attention between tokens is measured by attention scores.
This publication utilizes Multihead Attention using self-attention, which means that only the set of embedding vectors is used as input query, key and value\footnote{as opposed to a transformer architecture, where the constituent decoder obtains the matrices \(\mat{Q},\mat{K}\) from its respective encoder \cite{VaswaniEtAl_Attentionallyou_2017}.} \cite{VaswaniEtAl_Attentionallyou_2017}.
Every token is first converted into an embedding vector \(\vec{q}\in\mathbb{R}^{d}\) of length \(d\).
\(Q\) embedding vectors of different, typically subsequent, tokens are combined into the queue matrix \(\mat{Q}\in\mathbb{R}^{Q\times d}\), where \(Q=\mathbb{N}\cdot H\) is an integer multiple of the number of heads \(H\).
\(\mat{Q}\) is sliced by every \(\sfrac{d}{H}\) row into disjunct sub-matrices \(\mat{Q}^{(i)}\in\mathbb{R}^{Q \times \sfrac{d}{H}}\), each being the input to its attention head indexed by \(i\in1,\dots,H\).
Every head builds three input linear projections, namely the head's queue matrix \(\tilde{\mat{Q}}_{i}\), key matrix \(\tilde{\mat{K}}_{i}\), and value matrix \(\tilde{\mat{V}}_{i}\), defined as
\begin{equation}
\tilde{\mat{Q}}_{i}=\mat{Q}^{(i)}\mat{W}_\text{q}^{(i)},\quad\tilde{\mat{K}}_{i}=\mat{Q}^{(i)}\mat{W}_\text{k}^{(i)},\text{ and }\tilde{\mat{V}}_{i}=\mat{Q}^{(i)}\mat{W}_\text{v}^{(i)},
\end{equation}
all being from \(\mathbb{R}^{Q\times \sfrac{d}{H}}\).
Their independent (per head) weight matrices \(\mat{W}_\text{k}^{(i)}, \mat{W}_\text{v}^{(i)},\) and \(\mat{W}_\text{q}^{(i)}\in\mathbb{R}^{\sfrac{d}{H}\times \sfrac{d}{H}}\) are learnt, too.
Dividing the task into \(H\) heads, reduces overall complexity, while still allowing to consider embedding elements (s.\ attention output).
 
From these matrices, a matrix \(\mat{O}^{(i)}\in\mathbb{R}^{Q\times \sfrac{d}{H}}\) of attention values is computed using the scaled dot-product attention function
\begin{equation}\label{eq_attention}\begin{split}
 \mat{O}^{(i)}&=\text{Attention}\left(\tilde{\mat{Q}}_{i}, \tilde{\mat{K}}_i, \tilde{\mat{V}}_i\right)\\
    &= \text{softmax}\left(\frac{\tilde{\mat{Q}}_i \tilde{\mat{K}}_i^\text{T}}{\sqrt{d_k}}\right) \tilde{\mat{V}}_i,
\end{split}\end{equation}
The elements of the \((Q\times Q)\) attention weight matrix in \cref{eq_attention},
\begin{equation}
    \mat{Z}=\frac{\tilde{\mat{Q}}_i\tilde{\mat{K}}_i^\text{T}}{\sqrt{d_k}},
\end{equation}
are computed as the dot product between each element of \(\tilde{\mat{Q}}_i\) and every element of \(\tilde{\mat{K}}_i\).
As a common choice, the “SoftMax” function is used to normalize its rows:
Let \(\vec{z} \in \mathbb{R}^{K}\) denote one of every row vector of \(\mat{Z}\).
Then, the softmax function is applied to every element of \(\vec{z}\) at index \(k\) as
\begin{equation}
\tilde{\vec{z}}_k=\mathrm{softmax}_{k}(\vec{z}) = 
\frac{e^{z_k}}{\sum_{j=1}^{K} e^{z_j}}
\end{equation}
to form the normalized attention vector \(\tilde{\vec{z}}\) corresponding to \(\vec{z}\).
Repetition by row finally forms the normalized attention matrix \(\tilde{\mat{Z}}\) (cf.\ \cref{tab:masked_attention_matrix} for an arbitrary example using the complete example sentence from above as \(\mat{Q}\)).
The non-negative elements of \(\tilde{\vec{z}}\) (rows of \(\tilde{\mat{Z}}\)) sum up to unity \cite{Bridle_ProbabilisticInterpretationFeedforward_1990}.
These attention scores in \(\tilde{\mat{Z}}\) are used as weights to compute a weighted sum of the value matrix \(\tilde{\mat{V}}_i\), producing a contextualized embedding  (sub-vector) for each token that captures relevant information from other tokens in the sequence.

\begin{table}
\caption{Exemplary normalized attention matrix for the sentence “a black cat and a fast horse”. Each column sums to 1.}
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|}
\hline
\textbf{Query \textbackslash Key} & \textbf{a} & \textbf{black} & \textbf{cat} & \textbf{and} & \textbf{a} & \textbf{fast} & \textbf{horse} \\
\hline
\textbf{a}      & 0.5 & 0   & 0   & 0   & 0   & 0   & 0   \\
\hline
\textbf{black}  & 0.4 & 0.1 & 0   & 0   & 0   & 0   & 0   \\
\hline
\textbf{cat}    & 0.1 & 0.9 & 1.0 & 0.5 & 0   & 0   & 0   \\
\hline
\textbf{and}    & 0   & 0   & 0   & 0.5 & 0   & 0   & 0   \\
\hline
\textbf{a}      & 0   & 0   & 0   & 0   & 0.3 & 0   & 0   \\
\hline
\textbf{fast}   & 0   & 0   & 0   & 0   & 0.7 & 0.1 & 0   \\
\hline
\textbf{horse}  & 0   & 0   & 0   & 0   & 0   & 0.9 & 1.0 \\
\hline
\end{tabular}
\label{tab:masked_attention_matrix}
\end{table}

Each attention head has its own set of learned matrices \(\mat{W}_\text{k}^{(i)}\), \(\mat{W}_\text{v}^{(i)}\), and \(\mat{W}_\text{q}^{(i)}\), allowing different heads to capture different types of relationships between their embedding sub-elements within the sequence.
The \(H\) output matrices \(\tilde{\mat{O}}^{(i)}\) of every head are concatenated column-wise into \(\tilde{\mat{O}}\) and then the attention sub-scores are linearly combined into an output matrix \(\mat{O}\in\mathbb{R}^{Q\times d}\) 
\begin{equation}
    \mat{O}=\text{concat}_{i=1}^{H}\left[\tilde{\mat{O}}^{(i)}\right]\cdot \mat{W}_\text{o}
\end{equation}
of \(Q\) token's size-\(d\) overall attention score using learnt weights \(\mat{W}_\text{o}\in\mathbb{R}^{d x d}\).

\section{Use case}
The goal is to use attention to reach better performance in the domain of detecting fires than models which do not use it for reasons explained in the introducing chapter.
For this, the real-world labeled dataset “Indoor Fire Dataset with Distributed Multi-Sensor Nodes (Industrial Hall)” \cite{Vorwerk_IndoorFireDataset_2024} is used.
It contains four fire scenarios along with eight different nuisance scenarios carried out in an \(\qty{10}{\m} \times \qty{22}{\m} \times \qty{8}{\m}\) industrial hall without ventilation.
The fire scenarios include the ignition of wood, candles, cables, and lunts, whereas nuisance scenarios comprise smoke from a fog machine, deodorant, ethanol, \(\mathrm{CO}\) release, exhaust gases, green waste, welding, and abrasive cutting.
Each scenario is repeated at three random times, with background sequences in between, to reduce the influence of prehistory.
It contains measurements from eight individual sensor nodes sensors (\(\mathrm{CO_2}\), \(\mathrm{CO}\), \(\mathrm{H_2}\), humidity, particulate matter of different sizes, air temperature, volatile organic compound/VOC, and ultraviolet light/UV).
It contains \num{248502} samples sampled at 1 Hz and provides diverse temporal and spatial patterns across multiple fire types and nuisance conditions.
This dataset is particularly suitable for training attention-based models, because it provides multivariate and labeled sensor data collected under realistic conditions, allowing the model to learn meaningful patterns for early fire detection.
Furthermore, the presence of multiple sensor types allows models to capture different aspects of fire dynamics. 
Moreover, the inclusion of nuisance scenarios challenges the model to distinguish between actual fire events and false alarms, reflecting the complexity of real-world environments. 
Finally, the dataset’s repeated scenarios and interleaved background sequences support the development of models capable of detecting fires at an early stage, as they provide sufficient temporal context and variation for training attention-based architectures to focus on the most informative features over time.
This dataset requires preprocessing before being used for training.
Sensor values are first checked for missing or non-numeric values.
If found, these are linearly interpolated with the average value of the previous and following measurement.
The columns 'scenario\_label' and 'anomaly\_scenario' are dropped due to them containing information about what causes the current sensor values.
The column 'progress\_label' is the designated label to judge the models' performance and employ supervised learning.

Before feeding the multivariate sensor data into the attention-based model, each feature is normalized using z-score normalization \cite{AlFaiz2019ZScoreNN}. 
For each feature \(x\), the normalized value \(x_{\text{norm}}\) is computed as
\begin{equation}
x_{\text{norm}} = \frac{x - \mu}{\sigma},
\end{equation}
where \(\mu\) and \(\sigma\) are the empiric mean and standard deviation of the feature, respectively,  obtained from the training data. 
After normalization, each feature has approximately zero mean and unit variance, and all features contribute comparably to the attention mechanism, which improves numerical stability and allows the model to converge faster during training.
This step is especially important since the sensor measurements have different scales (e.g., \(\mathrm{CO_2}\) in ppm, temperature in \unit{\degreeCelsius}, particulate matter in \unit{\ug\per\m^3}). 
The choice fell on having 50 measurements per sequence with a stride of 10, which means sequences are selected from the overall measurement dataset by a window of 50 samples that is advanced by 10 samples.
The choice of 50 measurements per sequence with a stride of 10 provides a balance between capturing sufficient temporal context for the attention mechanism and generating many sequences for training.
The result is an array of shape of \num{24846} sequences with 50 measurements each across 16 different physical quantities.
There is no unified approach on how data should be split into training, validation and test data, but a common approach is to have training at \qty{60}{\percent} and the rest at \qty{20}{\percent}, respectively.
Imbalances are not present at alarming rates, so cross-validation is not used.
Following architectures are to be trained and compared on the same data.

\begin{table}
\centering
\caption{Model architectures}
\label{tab:model_architectures}
\begin{tabularx}{\columnwidth}{l l X}
\hline
Model & Layer & Shape / Description \\
\hline
\multirow{5}{*}{Attention-based} 
  & Input               & 16 features $\times$ 50 steps \\
  & Linear Embedding    & 16 $\rightarrow$ 64 \\
  & Temporal Attention  & MultiheadAttention(64, 4 heads) \\
  & Dropout + LayerNorm & Dropout 0.1 + Residual + LayerNorm \\
  & Output              & Linear 64 $\rightarrow$ 1 \\
\hline
\multirow{3}{*}{Baseline}
  & Input  & 16 features $\times$ 50 steps \\
  & Hidden & Linear 16 $\rightarrow$ 64 + ReLU \\
  & Output & Linear 64 $\rightarrow$ 1 \\
\hline
\end{tabularx}
\end{table}

\section{Evaluation}
The Attention-based model (cf.\ \cref{tab:model_architectures}) consists of an input embedding layer which maps the sensor features at each time step into a higher-dimensional embedding space; then, a temporal attention module, which captures sequential dependencies across sensor readings; finally an output layer which is a fully connected linear layer mapping the attention output to a single scalar representing the probability of a fire event.
The Temporal Attention module is designed to capture dependencies across time steps in multivariate sensor sequences.
It uses a multi-head self-attention mechanism with 4 heads, where each head uses scaled dot-product attention, allowing the model to weight the importance of each time step when making predictions.

As a baseline, a simple feed-forward neural network is employed, following common practice in multivariate time-series classification \cite{popov_simple_2024}. 
This baseline allows us to quantify the benefits of modeling temporal dependencies explicitly using attention.
It consists of one hidden layer with 64 neurons and ReLU activation, followed by a fully connected output layer producing a single scalar representing the probability of a fire event.
Both models were trained for 250 epochs, where an epoch is one complete pass through the entire training dataset. 
A batch size of 64 is used, meaning that the model parameters are updated after processing 64 sequences at a time. 
Early stopping is applied to prevent overfitting.
Both models were trained on a laptop computer equipped with an NVIDIA GeForce MX550 GPU and converged after \qty{754}{\s} runtime (cf.\ \cref{firefig}), while full-dataset inference required less than a second, indicating that the approach is computationally feasible for near real-time fire detection.

\begin{figure}
\centering
\includegraphics[width=\columnwidth]{loss_plot_fire.png}
\caption{Training and validation curves of the attention-based and baseline fire detection models}
\label{firefig}
\end{figure}

The attention-based model achieves an accuracy of \qty{99.6}{\percent}, with a precision of \num{0.89}, a recall of \num{0.989}, and an F1-score of \num{0.937} (cf.\ \cref{tab:metrics}). 
The false positive rate is low at \num{0.004}, indicating that the model rarely produces false alarms.
However, it must be kept in mind that typically an actual fire that is not detected has a higher cost than a false alarm when there is no fire present.

\begin{table}
\centering
\caption{Evaluation metrics of the attention-based and baseline fire detection models}
\label{tab:metrics}
\begin{tabular}{lccccc}
\hline
Model & Accuracy & Precision & Recall & F1-score & FPR \\
\hline
Attention & \num{0.996} & \num{0.890} & \num{0.989} & \num{0.937} & \num{0.004} \\
Baseline  & \num{0.968} & \num{0.503} & \num{0.981} & \num{0.665} & \num{0.033} \\
\hline
\end{tabular}
\end{table}

In comparison, the baseline model achieves a lower accuracy of \qty{96.8}{\percent} and a significantly lower precision of \num{0.50}, despite a similar recall of \num{0.981}. 
The F1-score drops to \num{0.665}, and the false positive rate increases to \num{0.033}. 
The confusion matrix (cf.\ \cref{tab:confusion}) indicates many more false positives here, demonstrating that the baseline model is less reliable for practical fire detection where false alarms are critical to minimize.

\begin{table}
\centering
\caption{Confusion matrices of the attention-based and baseline models. TN = True Negative, FP = False Positive, FN = False Negative, TP = True Positive}
\label{tab:confusion}
\begin{tabular}{lcccc}
\hline
Model & TN & FP & FN & TP \\
\hline
Attention & \num{239360} & \num{992} & \num{89} & \num{8009} \\
Baseline  & \num{232496} & \num{7856} & \num{150} & \num{7948} \\
\hline
\end{tabular}
\end{table}

To improve interpretability, \cref{attentionscores} visualizes the learned attention weights, highlighting which sensor features most strongly contribute to early fire detection, showcasing that multivariate data leads to a more holistic approach to prediction, due to no single feature being of dominant importance.
While recent diffusion-based architectures have shown promise in modeling complex temporal dynamics, they were not included here due to significantly higher computational demands,
However, incorporating such models remains an interesting direction for future benchmarking.
Overall, these results show that the attention-based model outperforms the baseline in all major metrics, particularly in precision and F1-score, while maintaining very high recall. 
This indicates that incorporating temporal attention allows the model to better capture the complex relationships in multivariate sensor sequences, improving early fire detection performance.

\begin{table}
\centering
\caption{Feature Attention Values}
\label{attentionscores}
\begin{tabular}{lc}
\hline
Feature & Attention Weight \\
\hline
CO2 & \num{0.46} \\
CO & \num{1.00} \\
H2 & \num{0.77} \\
Humidity & \num{0.96} \\
VOC & \num{0.91} \\
Temperature & \num{0.72} \\
Particles $\le$ 100 µm & \num{0.99} \\
Particles $\le$ 40 µm & \num{0.80} \\
Particles $\le$ 25 µm & \num{0.33} \\
Particles $\le$ 10 µm & \num{0.59} \\
Particles $\le$ 0.5 µm & \num{0.59} \\
Total detected Particles & \num{0.57} \\
\hline
\end{tabular}
\end{table}

\section{Conclusion}
This work demonstrated that attention-based architectures provide a substantial improvement in early fire detection using multivariate sensor sequences.
Traditional detection systems use threshold-based heuristics or models that struggle with long-range dependencies, making them vulnerable to delayed detection and high false positive rates.
Compared to that, the temporal attention mechanism used in this study models global relationships across sensor readings, which allows the system to identify subtle temporal patterns that precede a fire event.
Using an industrial fire dataset with multiple fire and nuisance scenarios, the proposed attention-based model achieved superior performance across all key metrics.
Most notably, the model’s precision and F1-score markedly exceeded those of the baseline feed-forward network.
This demonstrates that attention helps reduce false alarms without compromising the system’s ability to reliably detect actual fires, which is an essential requirement in safety-critical applications.
%The results indicate that attention mechanisms are well-suited for complex multivariate time-series problems where sequential context is necessary.
Hence, they might serve as a foundation for next-generation fire detection solutions, particularly in environments where conventional sensors and models are limited by architecture, scale, or noise.
Future work includes evaluating the model under outdoor conditions and integrating edge computing capabilities to enable on-node inference.
Lightweight attention variants could further reduce latency and make the system deployable on embedded devices.
%Additionally, testing the architecture in environments with variable airflow, sunlight, and sensor noise would provide deeper insights into real-world robustness.
%Overall, the attention-based approach provides a promising direction toward more reliable, scalable, and early detection of fires in real-world settings.

\printbibliography

\end{document}
