Neural Networks

Neural networks are computational models inspired by the organisation and function of biological neurons. They consist of layers of interconnected units (neurons), each computing a weighted sum of inputs and passing the result through a nonlinear activation function. Simple feed-forward networks, also known as multilayer perceptrons, can approximate arbitrary functions given sufficient capacity. Convolutional neural networks (CNNs) add local spatial filtering and pooling operations to exploit structure in images and time-series, while recurrent and transformer architectures capture sequential and contextual dependencies in data such as language or speech. Training proceeds by minimising a loss function—typically via gradient-based backpropagation—adjusting millions of parameters so that the network learns to map inputs to desired outputs. Modern neural networks can be very deep (tens or hundreds of layers), harnessing large datasets and specialised hardware (GPUs, TPUs) to achieve state-of-the-art performance across vision, language, scientific simulation and beyond.

Research from Nature Portfolio

A new non-parametric neural approach to power-meter coefficient estimation employs a back-propagation network to model complex, device-specific relationships in household electricity data. By combining careful feature extraction, preprocessing and network tuning, the study reports a 40 % reduction in mean absolute error compared with classical least-squares methods, offering real-time metering improvements for smart grids. Another study presents a lightweight CNN-based segmentation model for high-resolution remote sensing of burned areas. By integrating a transitive transfer-learning strategy and attention modules within a Deeplab V3+ backbone, the network achieves over 92 % overall accuracy with a single-second inference time per frame, enabling rapid, large-scale forest fire monitoring. In industrial diagnostics, a domain-mixed enhancement network uses mixup-based augmentation and domain discrepancy metrics to generalise bearing-fault classification across unseen operating conditions. The resulting architecture generates domain-invariant features, demonstrating superior F1 scores under variable loads without additional labelled data.

Research from all publishers

Hybrid optimisation frameworks now couple particle swarm methods with neural network training to optimise both architecture and weights of radial basis function networks. One approach uses logistic-map-driven inertia adjustments and a spiral local search operator, yielding compact RBF structures with faster convergence on nonlinear benchmarks. Multi-kernel fusion methods assign learnable weights to each basis function within RBFNNs, allowing the network to adaptively combine Gaussian, cosine and other kernels. This flexibility accelerates training, escapes poor local minima and improves classification accuracy in pattern-recognition and system-identification tasks. Advances in information-bottleneck regularisation embed variational compression objectives into deep networks. By balancing mutual information between inputs, latent codes and outputs, these techniques promote representation sparsity, mitigate overfitting and enhance robustness to adversarial and out-of-distribution samples.

Neural Networks publication trend

The graph below shows the total number of articles in neural networks across all publications each year (not limited to Nature Index journals).

Technical terms

Activation function: A nonlinear mapping (for example, sigmoid, tanh or ReLU) applied to a neuron’s weighted input sum, enabling the network to learn complex functions. Backpropagation: A gradient-based algorithm that computes derivatives of the loss with respect to each network weight, propagating error signals from the output layer back to the input. Convolutional layer: A network layer that applies a set of learnable filters across local windows of the input, extracting spatially localised features. Pooling: A down-sampling operation (for example, max-pooling) that reduces spatial dimensions and enforces local translation invariance. Domain generalization: The ability of a model to maintain performance on data drawn from distributions not seen during training, often addressed via augmentation or adversarial alignment. Information bottleneck: A principle for representation learning that seeks a latent encoding minimising mutual information with inputs while preserving information relevant to outputs.

References

1. Artificial Neural Networks.
2. Construction and optimization of non-parametric analysis model for meter coefficients via back propagation neural network. Scientific Reports (2024).
3. A lightweight Deeplab V3+ network integrating deep transitive transfer learning and attention mechanism for burned area identification. Scientific Reports (2025).
4. A domain generalization network for imbalanced machinery fault diagnosis. Scientific Reports (2024).
5. Adaptive Hyperparameter Fine-Tuning for Boosting the Robustness and Quality of the Particle Swarm Optimization Algorithm for Non-Linear RBF Neural Network Modelling and Its Applications. Mathematics (2023).
6. Multi-Kernel Fusion for RBF Neural Networks. Neural Processing Letters (2022).
7. Variational Information Bottleneck for Semi-Supervised Classification. Entropy (2020).