Introduction

In the modern digital environment, the security of multimedia data—particularly color images—has become a critical concern due to the rapid escalation of cyber threats. Image encryption plays a vital role in ensuring confidentiality, integrity, and authenticity in such applications. Traditional symmetric encryption algorithms, including the Advanced Encryption Standard (AES) and Data Encryption Standard (DES), are not ideally suited for image data because of large data sizes and strong pixel correlations, which weaken their diffusion and confusion capabilities. Consequently, researchers have explored specialized image encryption methods, such as chaos-based systems, DNA-inspired cryptography, and algebraic frameworks employing advanced mathematical structures.

Among these, Substitution–Permutation Networks (SPNs) have emerged as one of the most effective frameworks for image encryption. SPN architectures efficiently combine substitution and permutation operations to provide strong confusion and diffusion while maintaining computational efficiency. The nonlinear component of SPN—the S-box—plays a decisive role in enhancing resistance to differential and linear cryptanalysis. Numerous studies have focused on designing robust S-boxes with high NL, strong avalanche characteristics, and uniformity properties1,2,3,4,5,6,7. However, most S-boxes designed over conventional Galois fields or using chaotic maps remain limited in algebraic richness and dimensionality, resulting in vulnerabilities to advanced cryptanalytic attacks.

To overcome these limitations, researchers have recently investigated hypercomplex algebraic structures-particularly QI’s \(\:H\left(\mathbb{Z}\right)-\)for cryptographic applications. The quaternion system, introduced by Hamilton in 1843, extends complex numbers into four-dimensional space, providing noncommutative properties that naturally enhance the algebraic complexity of cryptographic primitives. RCQI’s offer a well-defined ring structure suitable for constructing bijective transformations with strong nonlinear properties. Recent studies11,12,13,14 have shown that quaternion-based S-boxes and key schedules significantly strengthen the confusion and diffusion capabilities of block ciphers. Moreover, the multidimensional quaternion domain allows independent transformations of RGB channels, making it particularly well-suited for color image encryption. Sajjad et al.4,6,19,20 have demonstrated the effectiveness of algebraic approaches using Gaussian and Eisenstein integers for both coding and cryptographic systems. Their results motivated the extension toward RQCI’s, which provide richer algebraic operations and stronger key sensitivity. These findings highlight that quaternion-based SPN frameworks can outperform chaos-based or conventional algebraic schemes in achieving comparable entropy, larger key spaces, and improved resistance to linear and differential attacks.

Extending this foundation, the present study proposes a novel RGB image encryption algorithm that integrates SPN architecture with S-boxes constructed over the RCQI’s. The rationale for adopting this approach lies in combining the proven cryptographic robustness of SPN structures with the multidimensional algebraic diversity of RCQI’s. This integration results in superior NL, enhanced diffusion, and expanded key space, offering a mathematically grounded solution to modern image security challenges. The remainder of the paper is organized as follows: Sect. Hamilton quaternion integers introduces the fundamental concepts of Hamilton QI’s and their residue class structures. Section Redesigning of S-boxes over QI’s and their analysis presents the proposed construction of nonlinear S-boxes based on QA and their cryptographic strength. Section SPN-Based RGB image encryption over QI’s and their experimental results describes the complete SPN-based RGB image encryption algorithm utilizing RCQI, their comprehensive experimental evaluation and performance comparisons. Finally, Sect. Conclusion and future directions concludes the study and outlines potential future directions for quaternion-based cryptographic designs.

Hamilton quaternion integers

Definition 2.110,11,12,13,14:

Let \(\:H\left(\mathbb{R}\right)\:\)be the Hamilton QA over the real number \(\:\mathbb{R}\) is the unital algebra if it fulfills the subsequent axioms:

  1. i.

    \(\:H\left(\mathbb{R}\right)=\{a+bi+cj+dk:\:\forall\:\:a,\:b,\:c,\:d\in\:\mathbb{R}\}\) is a free \(\:\mathbb{R}\) module over the bases elements \(\:\pm\:1,\:\pm\:i,\:\pm\:j\:\)and \(\:\pm\:k\).

  2. ii.

    1 is the multiplicative identity.

  3. iii.

    Multiplication of \(\:\pm\:1,\:\pm\:i,\:\pm\:j\:\)and \(\:\pm\:k\) are given in Table 1.

Table 1 Multiplication operation in QI’s

Definition 2.213,14:

Let \(\:H\left(\mathbb{Z}\right)=\{a+bi+cj+dk:\forall\:\:a,\:b,\:c,\:d\in\:\mathbb{Z}\}\) be a QI ring contained in\(\:\:H\left(\mathbb{R}\right).\) And QI ring \(\:H\left(\mathbb{Z}\right)\) is not commutative. If the vector parts of QIs are parallel to each other then, commutativity holds, which is given in Eq. 1.

$$\:H\left(K\right)=\left\{a+bU:\:U=i+j+k,\:\forall\:\:a,b\in\:\mathbb{Z}\right\}.$$
(1)

Thus, \(\:H\left(K\right)\:\)is the subring of the QI ring \(\:H\left(\mathbb{Z}\right).\).

Conjugate and norm of quaternion integer (QI)13,14

If \(\:q={a}_{0}+{a}_{1}i+{a}_{2}j+{a}_{3}k\) be a QI, then \(\:\stackrel{-}{q}{=a}_{0}-{a}_{1}i-{a}_{2}j-{a}_{3}k\) is the conjugate of\(\:\:q\). So, the norm of QI\(\:\:N\left(q\right)\) is defined in Eq. 2.

$$\:N\left(q\right)=q\stackrel{-}{q}={a}_{0}^{2}+{a}_{1}^{2}+{a}_{2}^{2}+{a}_{3}^{2}.$$
(2)

Theorem 2.110,11:

The addition and multiplication of two QI’s is a QI.

Theorem 2.213,14:

The set of natural numbers for each odd rational prime\(\:\:p,\) there is a QP\(\:\:{\uppi\:}\in\:H\left(\mathbb{Z}\right),\) such that\(\:\:N\left({\uppi\:}\right)=p={\uppi\:}\stackrel{-}{{\uppi\:}}\). Specifically, \(\:p\) is not prime in\(\:\:H\left(\mathbb{Z}\right).\)

Theorem 2.313,14:

If the norm of a QI \(\:N\left({\uppi\:}\right)\) is prime in\(\:\mathbb{\:}\mathbb{Z}\), then the QI \(\:{\uppi\:}\) is prime in\(\:\:H\left(\mathbb{Z}\right).\)

Theorem 2.413,14:

If \(\:a,\:b\in\:\mathbb{Z},\) such that \(\:a\) and \(\:b\) are co-prime, then \(\:H\left(K\right)/<a+b(i+j+k)>\) is isomorphic to\(\:\:{\mathbb{Z}}_{{a}^{2}+{3b}^{2}}\).

RCQI10,11,12,13,14

Let \(\:H\left(\mathbb{Z}\right)\) be the ring of a QI then the RCQI’s \(\:{H\left(\mathbb{Z}\right)}_{\pi\:}\) modulo \(\:\varvec{\pi\:}\) is defined in Eqs. 3 and 4.

$$\:{\Psi\:}:\:H\left(\mathbb{Z}\right)=\left\{a+bi+cj+dk\::\forall\:a,b,\:c,\:d\in\:\mathbb{Z}\right\}\to\:{H\left(\mathbb{Z}\right)}_{{\uppi\:}}.$$
(3)

Then,

$$\:{\Psi\:}\left(u\right)=v\left(mod\:{\uppi\:}\right)=u-\lfloor\frac{u\stackrel{-}{{\uppi\:}}}{{\uppi\:}\stackrel{-}{{\uppi\:}}}\rfloor{\uppi\:},$$
(4)

where\(\:\:v\in\:{H\left(\mathbb{Z}\right)}_{{\uppi\:}}\) and \(\lfloor\cdot\rfloor\) represents the rounding to the nearest integer. The rounding of a QI can be done by rounding the each part of QI to the closest integer.

Theorem 2.514:

Let \(\:{\uppi\:}\) be a QP, and the number of QIs modulo \(\:{\uppi\:}\) is the norm of \(\:{\uppi\:}.\) Let \(\:\beta\:\ne\:0\) (mod\(\:\:{\uppi\:}\)) is the generator of the residue class over QP \(\:{\uppi\:},\) then \(\:{\beta\:}^{N\left({\uppi\:}\right)-1}\equiv\:1\:\)(mod\(\:\:{\uppi\:})\).

Quaternion Mannheim weight and Distance25

Let \(\:\alpha\:,\:\delta\:\in\:{H\left(\mathbb{Z}\right)}_{\pi\:}\), and define \(\:\beta\:=\delta\:-\alpha\:=({b}_{0}+{b}_{1}i+{b}_{2}j+{b}_{3}k)\left(mod\:\pi\:\right)\), where \(\:{b}_{0},{b}_{1},{b}_{2},\:{b}_{3}\in\:\mathbb{Z}.\) The quaternion Mannheim weight, denoted by \(\:{W}_{QM}\left(\beta\:\right)\), is defined in Eq. 5.

$$\:{W}_{QM}\left(\beta\:\right)=\left|{b}_{0}\right|+\left|{b}_{1}\right|+\left|{b}_{2}\right|+\left|{b}_{3}\right|.$$
(5)

The quaternion Mannheim distance between \(\:\alpha\:\) \(\:\delta\:\), denoted by \(\:{d}_{QM}(\alpha\:,\:\delta\:)\), which is given in Eq. 6.

$$\:{d}_{QM}(\alpha\:,\delta\:)={W}_{QM}\left(\beta\:\right).$$
(6)

Remark 2.125:

The quaternion Mannheim weight \(\:{W}_{QM}\) satisfies the properties of a metric, and therefore the corresponding distance function \(\:{d}_{QM}\) defines a valid metric space over \(\:{H\left(\mathbb{Z}\right)}_{\pi\:}\).

Redesigning of S-boxes over QI’s and their analysis

The S-boxes are critical nonlinear components in block ciphers that enhance resistance against cryptanalytic attacks by introducing confusion. The unique algebraic properties of QI’s provide a novel avenue for S-boxes design, enabling strong avalanche effects, and improved resistance to linear and differential attacks. The following Algorithm 1 summarizes the construction process of quaternion-based S-boxes.

Algorithm 1

Quaternion-Based S-boxes Construction.

Input

QP \(\:\pi\:,\) quaternion generator \(\:\beta\:=a+bi+cj+dk,\) integer \(\:n\)

Output

\(\:\text{T}\text{w}\text{o}\:\text{S}-\text{b}\text{o}\text{x}\text{e}\text{s}\:{S}_{1}\:\text{a}\text{n}\text{d}\:{S}_{2}\)

Step

Description

1. Select QP

Choose a QP \(\:\pi\:\) from \(\:H\left(\mathbb{Z}\right)\) that cannot be factored into non-unit pairs.

2. Generate Residue Classes

Construct the \(\:RCQI=\{q\:\left(mod\:\pi\:\right)\::q\in\:\langle\beta\:\rangle\},\) where \(\:\langle\beta\:\rangle\) is the set generated by \(\:\beta\:\) under quaternion multiplication.

3. Apply Affine Mapping

For each \(\:x\in\:QI,\) compute \(\:g\left(x\right)=(ax+b)\:\left(mod\:{2}^{n}\right),\) where \(\:a,\:b\in\:QI\) and \(\:gcd(a,\:{2}^{n})=1.\)

4. Decompose Quaternion Coefficients

For each \(\:q\) in QI, decompose \(\:q={q}_{0}+{q}_{1}i+{q}_{2}j+{q}_{3}k\:\)and store \(\:\left({q}_{0},\:{q}_{1},\:{q}_{2},\:{q}_{3}\right).\)

5. Apply Modulo and Ensure Distinct Elements

Apply mod \(\:{2}^{n}\) to each \(\:{q}_{j}\:\)to keep values within range and ensure \(\:{2}^{n}\) unique entries.

6. Construct S-boxes

Assign S-box \(\:{S}_{1}\) from \(\:{q}_{0}\) components and S-box \(\:{S}_{2}\:\)from \(\:\left({q}_{1},\:{q}_{2},\:{q}_{3}\right)\) merged set. Ensure bijectivity (no repetition).

The quaternion-based S-boxes generated through this process exhibit superior resistance against linear and differential attacks, making them suitable for modern symmetric ciphers.

\(\:8\times\:8\)S-boxes over QI’s and analysis

Let\(\:\:\varvec{\pi\:}=75+35\varvec{i}+35\varvec{j}+35\varvec{k},\:\varvec{p}=\varvec{N}\left(\varvec{\pi\:}\right)={75}^{2}+{35}^{2}+{35}^{2}+{35}^{2}=9151,\) and\(\:\:\varvec{\beta\:}=2+13\varvec{i}+13\varvec{j}+13\varvec{k}\) be the generator of residue class which satisfies\(\:\:\varvec{N}\left(\varvec{\beta\:}\right)<\varvec{N}\left(\varvec{\pi\:}\right)\). Then apply the all steps of proposed algorithm get two S-boxes \(\:{\varvec{S}}_{1}\) and\(\:{\:\varvec{S}}_{2}\), which are given in Tables 2 and 3.

Table 2 8\(\:\times\:\)8 S-box over the 1 st part of QI (S1).
Table 3 8\(\:\times\:\)8 S-box over the 2nd, 3rd and 4th parts of QI (S2).

Nonlinearity (NL)

The NL of an S-box, a critical cryptographic property, measures its resistance to linear approximations. An S-box is modeled as a vectorial Boolean function \(\:f:{F}_{2}^{n}\to\:{F}_{2}^{m}\). The \(\:NL\left(f\right)\) is the minimum Hamming distance between \(\:f\) and all affine functions \(\:A:{F}_{2}^{n}\to\:{F}_{2}^{m}\), quantified via the Walsh-Hadamard transform, which is given in Eq. 7.

$$\:{W}_{f}\left(a,\:b\right)=\sum\:_{x\in\:{F}_{2}^{n}}{\left(-1\right)}^{f\left(x\right).b\oplus.x},$$
(7)

where \(\:a\in\:{F}_{2}^{n}\) and \(\:b\in\:{F}_{2}^{m}\) denotes the dot product. The NL is computed using Eq. 8.

$$\:{N}_{L}\left(f\right)={2}^{n-1}-\frac{1}{2}\underset{a\in\:{F}_{2}^{n},\:b\in\:{F}_{2}^{m}}{\text{max}{\:\:\:\:\:\:}}\left|{W}_{f}\left(a,\:b\right)\right|.$$
(8)

A \(\:NL\left(f\right)\) implies stronger resistance to linear cryptanalysis. The NL of proposed study with different eight functions \(\:{f}_{1},\:{f}_{2},\:\dots\:,\:{f}_{8}\) is given in Table 4. Thus, NL is essential in S-box design to ensure cryptographic strength. The S-boxes proposed in our scheme and6,7,8,9,16 exhibit such desirable nonlinear characteristics, as shown in Table 5; Fig. 1.

Table 4 NL of the proposed work over QI.
Table 5 NL comparison with existing work.
Fig. 1
figure 1

NL comparison with existing work.

Bit independent criteria (BIC)

BIC evaluates how independently the output bits of an S-box behave when a single input bit is flipped. Formally, it analyzes the statistical independence of every pair of output bits by calculating their mutual correlation. A BIC-compliant S-box ensures that each output bit changes independently, enhancing resistance to cryptanalytic attacks such as linear and differential cryptanalysis. To assess BIC, all input combinations are tested, and the correlation between output bit pairs is examined. High BIC implies strong randomness and entropy, contributing to secure encryption. Thus, alongside NL, avalanche, and differential uniformity, BIC is essential in designing robust cryptographic S-boxes. The BIC analysis of our proposed S-boxes over QI’s and other schemes is presented in Tables 6, 7 and 8, with comparative references in6,7,8,9,16,20.

Table 6 BIC of first proposed S-box (S1) over QI.
Table 7 BIC of first proposed S-box (S2) over QI.
Table 8 BIC comparison with existing literature.

Strict avalanche criterion (SAC)

SAC is a key property in evaluating S-boxes for block ciphers. An S-box satisfies SAC if flipping a single input bit causes each output bit to change with a probability of 0.5. This ensures that small input changes lead to significant, unpredictable output changes, enhancing diffusion and making the cipher resistant to differential cryptanalysis. SAC compliance indicates that output bits respond uniformly and randomly to input changes, helping distribute input influence across the entire output. Although testing SAC is time-consuming due to exhaustive input configurations, it ensures robustness and prevents structural weaknesses. S-boxes meeting SAC also support high NL, reinforcing overall cryptographic strength. The SAC analysis for our proposed S-boxes over QI’s is provided in Tables 9, 10 and 11, with comparisons to existing methods in6,7,8,9,16,20.

Table 9 SAC of first proposed S-box (S1) over QI.
Table 10 SAC of second proposed S-box (S2) over QI.
Table 11 SAC comparison with existing literature.

Linear Approximation Probability (LAP): LAP is a key metric for evaluating an S-box’s resistance to linear cryptanalysis. It measures the maximum probability that a linear relation between input and output bits holds: \(\:Pr[a\cdot\:x=b\cdot\:f(x\left)\right]\) for fixed non-zero bit masks \(\:a,b\in\:{F}_{2}^{n}\), where \(\:\cdot\:\) denotes the dot product over \(\:{F}_{2}\). A low LAP indicates stronger resistance, as it implies that any linear approximation succeeds only with a probability near 0.5 (random chance). High NL helps achieve minimal LAP, making it harder for attackers to deduce key information. Table 12 presents the LAP comparison of the proposed S-boxes over QI’s with previous methods in6,7,8,9,16,20.

Table 12 LAP comparison with existing literature.

Differential approximation probability (DAP)

The DAP measures of S-box resistance to differential cryptanalysis by evaluating the probability that a specific input difference \(\:\varDelta\:x\) leads to a specific output difference \(\:\varDelta\:y,\) which is given in Eq. 9.

$$\:DAP=Pr\left(S\left(x\right)\oplus\:S\left(x\oplus\:\varDelta\:x\right)=\varDelta\:y\right),$$
(9)

where \(\:\oplus\:\) denotes the XOR operation. A low DAP indicates strong diffusion and resistance to differential attacks, as output changes become unpredictable for any given input difference. Achieving low DAP requires high NL and even distribution of output differences. Tables 13, 14 and 15 compare the DAP results of the proposed S-boxes over QI’s with those from existing methods6,7,8,9,16,20, highlighting improved cryptographic strength.

Table 13 DAP of first proposed S-box (S1) over QI.
Table 14 DAP of second proposed S-box (S2) over QI.
Table 15 DAP comparison with existing literature.

SPN-Based RGB image encryption over QI’s and their experimental results

To ensure robust image protection against statistical and differential attacks, the proposed encryption method employs a Substitution–Permutation Network (SPN) architecture enhanced with quaternion-based S-boxes. These S-boxes introduce higher algebraic complexity and multidimensional diffusion due to their four-component quaternion structure. The encryption process operates independently on each color channel and combines substitution, permutation, and XOR layers for strong confusion and diffusion.

Algorithm 2

SPN-Based RGB Image Encryption over QI’s.

Input

lain RGB image \(\:I\) of size \(\:m\times\:n\times\:3,\) quaternion-based S-boxes \(\:{S}_{1}\) and \(\:{S}_{2}\)

Output

Encrypted RGB image \(\:C\)

Step

Description

1. Channel Decomposition

Decompose \(\:I\) into \(\:R,\:G,\:B\) channels (each \(\:m\times\:n\)).

2. Substitution Layer

Substitute each pixel \(\:p\) in each channel \(\:M\) using \(\:{S}_{1}\left[p\right]\) to obtain \(\:M{\prime\:}.\)

3. Permutation Layer

Flatten \(\:M{\prime\:}\) to a \(\:1D\) array \(\:A.\) Generate a permutation vector \(\:P\) from \(\:{S}_{2}\) ensuring bijectivity. Rearrange \(\:A\) according to \(\:P\to\:A{\prime\:}\) and reshape to \(\:M{\prime\:\prime\:}.\)

4. XOR Combination Layer

Compute \(\:{S}_{3}=({S}_{1}+{S}_{2})\left(mod\:{2}^{n}\right).\) For each pixel \(\:p\) in \(\:M{\prime\:\prime\:},\) perform \(\:p{\prime\:\prime\:\prime\:}=p\oplus\:S_{3}\left[index\right(p\left)\right]\) to get \(\:M{\prime\:\prime\:\prime\:}.\)

5. Channel Integration

Combine \(\:R{\prime\:\prime\:\prime\:},\:G{\prime\:\prime\:\prime\:},\) and \(\:B{\prime\:\prime\:\prime\:}\) to form final cipher image \(\:C.\)

This Algorithm 2 ensures the key sensitivity, expanded key space, and excellent resistance to differential, statistical, and brute-force attacks. The quaternion-based SPN framework achieves near-optimal entropy and correlation characteristics, demonstrating its strength for secure color image encryption. Flowchart of the proposed study is also given in Fig. 2.

Fig. 2
figure 2

Proposed Algorithm of S-boxes and RGB Image Encryption over QI’s.

The experimental results demonstrate the effectiveness and robustness of the proposed quaternion integer (QI)-based RGB image encryption method. Quaternions, being higher-dimensional extensions of complex numbers, enhance mathematical complexity and encryption depth, making them highly suitable for multi-channel data like RGB images. The method encrypts the red, green, and blue channels concurrently, ensuring strong diffusion and confusion compared to conventional techniques. Performance evaluation considers key factors such as encryption quality, computational efficiency, key sensitivity, and resistance to attacks. Metrics including histogram, Peak Signal-to-Noise Ratio (PSNR), Mean Squared Error (MSE), NPCR, and UACI confirm high randomness, low correlation, and strong key dependence. The quaternion structure also supports parallel processing, achieving faster encryption suitable for real-time applications4,17,23,24,27. The first row of Fig. 3 displays images obtained from the USC-SIPI Image Database (https://sipi.usc.edu/database/), which were resized and utilized in this study. The second row presents the corresponding encrypted images, generated using the proposed encryption algorithm implemented in MATLAB.

Fig. 3
figure 3

Different Original and Encrypted Images.

Histogram analysis (HA)

Image encryption algorithm assessment heavily relies on the analysis technique known as HA. Images have their pixel intensity values displayed through the histogram methodology that demonstrates their frequency distribution. The natural (unencrypted) RGB image contains distinguishing histogram patterns in its red, green and blue channels that correspond to the image structure. The encrypted histograms should present a flat distribution after encryption since this pattern proves pixel intensities have been randomized to protect useful information from extraction. A secure encrypted image requires all RGB channel histograms to appear evenly distributed while showing no indications of original image content. Any remaining structure in the histogram after encryption indicates weaknesses in the encryption algorithm which makes the system vulnerable to statistical attacks. The security of encrypted data is strengthened through using uniform histograms which cause total modifications to the output when the plaintext undergoes minimal changes. The principle analysis of histograms helps to establish the protective measures and reliability of security systems employed in RGB image encryption techniques3,6,21,22. The original and encrypted image histograms, generated using MATLAB programming, are presented in Fig. 4.

Fig. 4
figure 4

The HA of Female and Pepper original and encrypted images.

QA and Sensitivity Enhancement: The QA enhances the key and plaintext sensitivity of the proposed SPN-based RGB image encryption scheme. Due to the four-dimensional nature of QI’s, even minor variations in the input or secret key cause complex, nonlinear changes throughout the encryption process. This high sensitivity is validated using two key metrics: NPCR and UACI.

NPCR measures how many pixels change in the ciphertext when a single pixel in the plaintext is modified, which can be computed by Eqs. 10 and 11.

$$\:NPCR=\frac{\sum\:_{i=1}^{M}\sum\:_{j=1}^{N}D\left(i,\:j\right)}{M\times\:N}\times\:100\%,$$
(10)

where

$$\:D\left(i,\:j\right)=\left\{\begin{array}{c}1\:\:\:\:\:\:\:\:\:\:if\:{C}_{1}(i,\:j)\ne\:{C}_{2}(i,\:j)\\\:0\:\:\:\:\:\:\:\:\:\:if\:{C}_{1}(i,\:j)={C}_{2}(i,\:j)\end{array}\right.,$$
(11)

Here, \(\:{C}_{1}(i,j)\) and \(\:{C}_{2}(i,j)\) are the pixel values of encrypted images generated from the original and slightly modified plaintexts.

UACI quantifies the average intensity variation between the two encrypted images, which can be calculated by Eq. 12.

$$\:UACI=\frac{1}{M\times\:N}\sum\:_{i=1}^{M}\sum\:_{j=1}^{N}\frac{{|C}_{1}\left(i,\:j\right)-{C}_{2}\left(i,\:j\right)|\:}{255}\times\:100\%.$$
(12)

High NPCR (close to 99%) and UACI (approaching 33% for grayscale or similar values for RGB channels) indicate strong diffusion and resistance to differential cryptanalysis.

Moreover, the quaternion-based S-boxes and key-dependent permutation patterns ensure that even a single-bit change in the key generates completely different ciphertexts, highlighting strong key sensitivity. These effects confirm that the use of QA provides superior robustness against statistical, differential, and brute-force attacks. The observed NPCR and UACI results3,6,21,22,33 are detailed in Table 16.

Table 16 NPCR and UACI of different RGB images.

Maximum deviation (MD) and irregular deviation (ID)

The MD and ID are key metrics for evaluating the strength of image encryption.

MD measures the largest pixel intensity difference between the original and encrypted images, shown in Eq. 13.

$$\:MD=\text{max}\left|{C}_{1}\left(i,\:j\right)-{C}_{2}\left(i,\:j\right)\right|,$$
(13)

where, \(\:{C}_{1}(i,j)\) and \(\:{C}_{2}(i,j)\) are pixel values at position \(\:(i,j)\) in the original and encrypted images, respectively. High MD values across RGB channels indicate effective disruption of the original image structure.

The ID captures the average inconsistency of pixel differences, reflecting regional randomness, which can be calculated by Eq. 14.

$$\:ID=\frac{1}{M\times\:N}\sum\:_{i=1}^{M}\sum\:_{j=1}^{N}\left|{C}_{1}\left(i,\:j\right)-{C}_{2}\left(i,\:j\right)-{\mu\:}_{D}\right|,$$
(14)

where \(\:{\mu\:}_{D}\) is the mean deviation. High ID suggests non-uniform, unpredictable changes—vital for resisting statistical attacks. Both MD and ID are computed separately for R, G, and B channels. Table 17 summarizes these values, showing the effectiveness of the proposed encryption in ensuring randomness and image security4,25,26,33.

Table 17 MD and ID of different RGB images.

Correlation analysis

The correlation analysis evaluates the relationship between adjacent pixels in an image. Natural images exhibit high pixel correlation, but a strong encryption algorithm should reduce this to near zero, ensuring randomness. The correlation coefficient is given in Eq. 15.

$$\:r=\frac{\sum\:\left({x}_{i}-\stackrel{-}{x}\right)\left({y}_{i}-\stackrel{-}{y}\right)}{\sqrt{{\left({x}_{i}-\stackrel{-}{x}\right)}^{2}{\left({y}_{i}-\stackrel{-}{y}\right)}^{2}}},$$
(15)

where, \(\:{x}_{i}\) ​ and \(\:{y}_{i}\) are values of adjacent pixels.

  • Horizontal correlation: between neighboring pixels in the same row.

  • Vertical correlation: between pixels in adjacent rows.

  • Diagonal correlation: between diagonally adjacent pixels.

A secure encryption significantly reduces all these correlations, making the encrypted image appear random. Figure 5; Table 18 present the correlation results for plain and encrypted images4,15,18,23,24,27.

Fig. 5
figure 5

Diagonal, Vertical and Horizontal Correlation of Female original and their ciphered images.

Table 18 Correlation analysis of different RGB encrypted Images.

Contrast and energy

Contrast measures the intensity variation between pixels and reflects the degree of randomness introduced by encryption. It is calculated by Eq. 16.

$$\:Contrast=\sum\:_{i=1}^{N}\sum\:_{j=1}^{N}P\left(i,\:j\right).{\left(i-j\right)}^{2},$$
(16)

where, \(\:P(i,j)\) is the probability of pixel intensities. In encrypted images, high contrast indicates disrupted spatial patterns, making the image unrecognizable and resistant to statistical attacks. Contrast is evaluated separately for R, G, and B channels to ensure uniform encryption.

Energy reflects image uniformity and texture, defined in Eq. 17.

$$\:Energy=\sum\:_{i=1}^{N}\sum\:_{j=1}^{N}P{\left(i,\:j\right)}^{2},$$
(17)

In encrypted images, low energy signifies uniform pixel distribution and randomness. High energy may expose residual structure, weakening security. Energy is also computed for each RGB channel to assess encryption performance. Table 19 presents the contrast and energy results for different images, confirming the strength of the proposed encryption method25,26,33.

Table 19 Contrast and energy of different RGB images.

Homogeneity

The homogeneity measures the uniformity of pixel intensity distribution in an image and is defined in Eq. 18.

$$\:Homogeneity=\sum\:_{i=1}^{N}\sum\:_{j=1}^{N}\frac{P\left(i,\:j\right)}{1+|i-j|},$$
(18)

where, \(\:P(i,\:j)\) is the probability of pixel intensity pairs. Lower homogeneity in encrypted images indicates disrupted structure and improved randomness. For RGB images, homogeneity is computed separately for R, G, and B channels to assess encryption effectiveness.

Standard deviation (SD)

The SD quantifies the spread of pixel intensities and is calculated by Eq. 19.

$$\:\sigma\:=\sqrt{\frac{1}{N}{\sum\:_{i=1}^{N}{\left({x}_{i}-\mu\:\right)}^{2}}^{\:}\:},$$
(19)

where, \(\:{x}_{i}\:\)are pixel values and \(\:\mu\:\) is the mean. A higher SD in encrypted images reflects strong randomness and reduced visual structure, enhancing resistance to statistical attacks. It is also evaluated per color channel for RGB images. Table 20 shows the homogeneity and SD results for various images, validating encryption strength25,26.

Table 20 Homogeneity and SD of different RGB images.

PSNR

The PSNR is widely used to assess image quality in encryption. A lower PSNR between the original and encrypted images indicates higher encryption strength, as it reflects minimal visual similarity. It is calculated using Eq. 20.

$$\:PSNR=10.{\text{log}}_{10}\left(\frac{MA{X}^{2}}{MSE}\right),$$
(20)

where \(\:MAX=\:255\) for 8-bit images. For RGB images, PSNR is computed separately for each color channel and then averaged. Effective encryption schemes produce PSNR values below 10 dB, confirming significant image distortion and resistance to visual attacks.

MSE

The MSE measures pixel-wise differences between original and encrypted images, which can be calculated by Eq. 21.

$$\:MSE=\frac{1}{M\times\:N}\sum\:_{i=1}^{M}\sum\:_{j=1}^{N}{\left[I\left(i,\:j\right)-K\left(i,\:j\right)\right]}^{2},$$
(21)

Higher MSE values indicate stronger encryption, as they reflect greater deviation and randomness. Like PSNR, MSE is evaluated per RGB channel and averaged. A low MSE suggests weak encryption, while a high MSE ensures robust protection against statistical and cryptanalytic attacks. Table 21 presents PSNR and MSE values for various encrypted images, validating the security strength of the proposed method25,26,33.

Table 21 PSNR and MSE of different RGB images.

Information entropy (IE)

The IE measures the randomness or unpredictability of an RGB image, which is critical for evaluating encryption strength. It is calculated using Eq. 22.

$$\:H=-\sum\:({p}_{i}*{\text{log}}_{2}\left({p}_{i}\right),$$
(22)

where \(\:{p}_{i}\) is the probability of each pixel value. Higher entropy (close to 8 for 8-bit images) indicates greater randomness, making the image more secure against statistical attacks. Effective encryption increases entropy by removing patterns from the original image. Low entropy suggests residual structure, weakening security. Table 22 presents entropy results for various encrypted images to assess encryption quality17,18,23,24,32.

Table 22 IE of different RGB images.

Encryption and decryption speed analysis

The proposed SPN-based color image encryption scheme over RCQI’s was evaluated for computational efficiency on an Intel(R) Core(TM) i5-1135G7 @ 2.40 GHz CPU with 8 GB RAM. The algorithm combines quaternion-based algebraic operations with SPN principles, performing substitution through nonlinear S-boxes, permutation for diffusion, and modular arithmetic over RCQI’s. Speed tests measured encryption and decryption times for images of different sizes to assess scalability. While parallel execution benefits from multi-core processing, performance is bounded by hardware constraints. The results of Table 23 show that the method achieves efficient runtime and scalability, confirming its suitability for practical image security applications leveraging the mathematical strength of RCQI’s28,29,33.

Table 23 Encryption and decryption speed comparison in Seconds.

Randomness cipher test

The randomness of the QI-based RGB encrypted images was evaluated using the NIST statistical test suite to assess their cryptographic strength. This suite includes multiple tests—such as Frequency, Block Frequency, Rank, Runs, Long Runs, Template Matching, Spectral DFT, Approximate Entropy, Universal, Serial, and Cumulative Sums (forward and reverse)—each measuring different aspects of randomness and unpredictability. Passing these tests confirms that the encrypted data lacks detectable patterns, correlations, or repetitive structures, ensuring strong resistance against statistical attacks. The results, summarized in Table 24, demonstrate that the proposed encryption method achieves high randomness and robust security performance4,18,23,24,27,28,29,30,31,32,33.

Table 24 NIST test of female encrypted Image.

Conclusion and future directions

This work introduced a novel RGB image encryption algorithm based on an SPN structure over the RCQI’s \(\:H{\left(\mathbb{Z}\right)}_{\pi\:}.\)​ By exploiting the noncommutative algebraic properties of quaternions, robust S-boxes were generated to achieve high confusion and diffusion. The experimental evaluation confirms the effectiveness of the proposed approach: entropy values approach the theoretical maximum value, NPCR exceeds 99%, UACI remains close to the ideal 33%, correlation coefficients between adjacent pixels fall near zero, and PSNR/MSE analyses demonstrate strong resilience against noise and statistical attacks. Compared with existing SPN-based and algebraic cryptosystems, the proposed method consistently delivers superior numerical performance, offering stronger resistance against differential and linear cryptanalysis while maintaining efficiency. These numerical achievements highlight the potential of quaternion-based SPNs as a more powerful alternative to conventional image encryption methods and establish a new direction for secure multimedia communications.

Future research on the proposed encryption framework should focus on multiple avenues that would improve its performance and extend its capabilities. The efficiency of S-box construction based on quaternions requires enhancement to achieve faster real-time encryption operations. The security measure can be enhanced through the integration of chaotic maps along with alternative dynamic key generation mechanisms. The method could gain potential security benefits through an application extension to QA dimensions beyond octonions. Researchers need to study the practical applications of this encryption scheme by examining its use in multimedia security alongside cloud storage and secure transmission to understand its implementation capacity. Future work in quaternion-based encryption research will strengthen both reliability and practicality of the method as it progresses the field of secure digital communication technologies.