Introduction

Programmable metasurfaces (PMs)1, which are also called reconfigurable intelligent surfaces (RISs)2,3,4,5,6,7,8,9 in the communications community, are artificial planar structures that can dynamically control electromagnetic (EM) waves, reshaping the wavefronts and creating diverse scattering patterns. These advanced field-manipulation capabilities not only facilitate signal modulations but also enable communication and sensing systems to achieve adaptive and intelligent wireless environments1,2,3,4,5,6,7,8,9. Another emerging technology, known as integrated sensing and communication (ISAC), stands out as a prominent candidate for the sixth-generation (6G) of wireless communication systems, drawing considerable interest from researchers across different disciplines. In essence, ISAC aims to seamlessly integrate sensing and wireless communications by leveraging shared spectra, hardware platforms, and signal processing functions. This integration seeks to achieve a balance in performance, while significantly reducing the hardware cost and power consumption10,11,12,13,14,15,16,17,18. The synergy between PM/RIS and ISAC systems holds promise for advancing the development of 6G wireless network technologies. Specifically, PMs/RISs not only contribute to enhancing and balancing multiple performance metrics in an ISAC system19,20,21,22 but also possess inherent capabilities for autonomously integrating sensing and wireless communication functions23,24,25,26. The design of platforms that integrate communications and sensing functionalities is also considered an enabling technology to tackle some key challenges for the adoption of PMs/RISs in future wireless communication systems, which include channel estimation and the availability of power supply27,28. Given the interest in PM/RIS and ISAC in future communication networks, industry specification groups in pre-standardization organizations have been established as well (https://www.etsi.org/committee/1966-ris, https://www.etsi.org/committee/2295-isac). A recent overview on the synergy of PM/RIS and ISAC can be found in ref. 29.

PMs/RISs are two-dimensional versions of metamaterials. These artificial structures are composed of suitably designed sub-wavelength scattering units, and offer advantages in terms of ultra-low electrical thickness, resulting in higher integration, lower cost, and reduced insertion loss30,31,32,33. Through a careful design and arrangement of the scattering units, metasurfaces enable advanced control of EM wave properties, including amplitude, phase, and polarization. The concept of digital coding metasurfaces, initially introduced in 20141, employs basic scattering units with a limited number of states that can be represented as discrete digits1,34. This digital approach makes coding metasurfaces ideal for integrating active devices, such as positive-intrinsic-negative (PIN) diodes, varactors, or tunable materials such as liquid crystals35,36. These tunable elements can be independently controlled by field programmable gate arrays (FPGA) to dynamically configure the EM responses of scattering units or supercells. This enables flexible and programmable real-time manipulations of EM waves, e.g., via a precise phase shift introduced by each scattering unit or supercell.

The advent of digital coding architectures not only streamlines the design and optimization process of metasurfaces but also empowers PMs to serve as versatile platforms for numerous applications, including reprogrammable holograms37,38, EM information theory39,40, reflect- and transmit-arrays41,42, vortex beam generations43, information processing44,45, wireless communications46,47,48, adaptive metasurfaces49, smart imaging50,51, and programmable artificial intelligence (AI) devices52. PMs/RISs offer a cost-effective, compact, lightweight, and low-power solution for integrating various functions. The integration of PMs/RISs into communication systems has been extensively researched, highlighting its significant advantages. Beyond communication, PMs/RISs can also be employed in sensing applications due to their cost-effective reconfigurability. When the direct path between the sensing target and a base station (BS) is obstructed, a PM/RIS can establish a non-line-of-sight link, reflecting signals to facilitate sensing operations19,20,53. In such a scenario, a PM/RIS cannot typically gather environmental sensing information on its own and usually relies on the BS to transmit external control signals via a dedicated control link for configuring its phase pattern. Alternatively, a PM/RIS can function as a physical sampling device by switching between multiple phase patterns. In this mode, it requires only a single receive channel to analyze the received signals from each phase pattern, thereby extracting sensing information24,50,54,55. However, in this configuration, it cannot simultaneously support communication functions. To address this limitation, sensors or detectors can be integrated into PMs/RISs to enable concurrent sensing and communication 26,56,57. This allows for dynamic configuration of phase patterns, potentially eliminating the need for a dedicated control link from the BS. Nevertheless, this integration requires additional peripheral circuitry, leading to an increase in hardware costs.

Digital PMs/RISs conventionally operate with fixed space-coding (SC) patterns, controlled by a central module, to implement some desired functions for spatial beamforming. To manipulate the frequency spectra of EM waves, temporal encoding has gained attention, and the concept of space-time-coding (STC) digital metasurface has emerged as a rapidly growing research field58,59. This idea has found applications in various domains including programmable nonreciprocity60, harmonic controls61, spectrum camouflage62, sensing63,64,65, and wireless communications66,67. Through a suitable design and optimization of the STC matrix for spatio-temporal modulations, these platforms can effectively manipulate the EM waves in both spatial and frequency domains simultaneously.

In this context, we introduce a low-cost and efficient ISAC approach based on STC metasurfaces (STCMs), which can harness some specified harmonics for wireless sensing and can control the propagation of the fundamental frequency wave to establish reliable sensing and wireless communication links simultaneously on the same platform by controlling the STC matrix. To this end, we propose two coding strategies for STCM to simultaneously manipulate the fundamental-frequency wave and harmonics, which are referred to as “adjustable partitioning” and “full-aperture” schemes. Compared with conventional ISAC systems relying on phased array antennas, STCM-based ISAC systems offer advantages in terms of reduced hardware cost and power consumption. Moreover, leveraging the harmonics for sensing eliminates the need for additional sensors or a dedicated control link from the BS, enabling simultaneous execution of sensing and communication functions without interference.

Results

Principle of STCM-based communication and sensing

Referring to the concept illustrated in Fig. 1, the proposed scheme features an STCM that manipulates the incoming EM wave, controlling the direction of propagation of the fundamental frequency and simultaneously generating spatially distributed harmonics. The fundamental-frequency wave is used for communication, while the spatial-spectral characteristics of the harmonics are processed to accurately estimate the direction of arrival (DOA) of some targets. Using the DOA information, the STCM’s coding pattern is adjusted to ensure that the direction of the beam of the fundamental-frequency (carrier) wave is consistently steered towards the user’s location. This configuration allows the dual functions of wireless sensing and communication to operate simultaneously without mutual interference. Even changing the direction of the incoming waves, the STCM can continuously sense the DOA information in real time and can autonomously configure the corresponding coding pattern to establish a reliable communication link with the users, ensuring the required communication quality.

Fig. 1: Conceptual illustration of the proposed space-time-coding metasurface (STCM)-based integrated sensing and communication (ISAC) scheme.
Fig. 1: Conceptual illustration of the proposed space-time-coding metasurface (STCM)-based integrated sensing and communication (ISAC) scheme.
Full size image

in which an STCM controls the propagation direction of the fundamental-frequency wave for communication while generating spatially distributed harmonics for wireless sensing.

Without loss of generality, we consider a two-dimensional scenario featuring a 2-bit reflection-type STCM consisting of N columns of unit cells. Each unit cell provides four discrete operating states with relative reflection phases {−π/2, 0, π/2, π}, corresponding to the digits {0, 1, 2, 3}, respectively. Assuming identical reflection coefficients for all the unit cells in the nth-column, denoted as \({\varGamma }_{{{\rm{S}}}}^{n}\), and considering a time-harmonic plane-wave illumination with suppressed time-dependence \(\exp \left(j2{{\rm{\pi }}}{f}_{{{\rm{c}}}}t\right)\) and an angle of incidence θi, the far-field pattern at the fundamental frequency is expressed as

$${f}_{{{\rm{S}}}}(\theta,{\theta }_{i})={f}_{e}(\theta ){\sum }_{n=1}^{N}{\varGamma }_{{{\rm{S}}}}^{n}\exp \left [j{k}_{{{\rm{c}}}}d(n-1)(\sin \theta+\,\sin {\theta }_{i})\right]$$
(1)

where \({f}_{e}(\theta )\) denotes the scattering pattern of the unit cell at frequency \({f}_{c}\), \({k}_{c}=2\pi {f}_{c}\,/c\) is the wavenumber in free space, c is the speed of light, and d is the spatial period of the unit cell. Given the angle of incidence \({\theta }_{i}\) and the specified beam-pointing direction \({\theta }_{r}^{0}\), the phase response of the nth-column unit \({\varPhi }_{n}\) can be calculated in a systematic fashion, as detailed in Supplementary Note 1. This phase response is then quantized into discrete values and represented in the corresponding digital codes to generate the SC patterns (see Supplementary Note 1 for more details). Figure 2a shows a set of SC sequences for N = 16, while the corresponding spatial scattering patterns (theoretically calculated for an angle of incidence \({\theta }_{i}=0^\circ\)) are illustrated in Fig. 2b. In this example, the beam-pointing direction varies from −60° to +60° with a 5° step. It is evident that the STCM exhibits precise and flexible control over the beam-pointing direction.

Fig. 2: Coding matrices and corresponding far-field scattering patterns of the proposed space-time-coding metasurface (STCM).
Fig. 2: Coding matrices and corresponding far-field scattering patterns of the proposed space-time-coding metasurface (STCM).
Full size image

a, b Space-coding sequences for N = 16 and corresponding theoretical far-field patterns for the fundamental-frequency wave at \({\theta }_{i}=0^\circ\), respectively. c, d Optimized STC matrix for controlling the harmonics when N = 16 and L = 32, and corresponding far-field scattering patterns at \({\theta }_{i}=0^\circ\), respectively. e, f Optimized STC matrix for simultaneously controlling the fundamental-frequency wave and for generating multiple harmonics, and corresponding far-field scattering patterns, respectively.

When utilizing spatio-temporal modulation, the operating state of the nth-column of unit cells undergoes periodic changes according to a given time-coding sequence. The reflection coefficient of each nth-column of unit cells is represented by a periodic function \({\varGamma }_{{{\rm{T}}}}^{n}(t)\)58, which can be expanded into a Fourier series, with the generic coefficient expressed as \({a}_{v}^{n}\). The time-coding sequences of all N columns can be collectively represented as an STC matrix \({{\bf{Z}}}\in {{{\bf{R}}}}^{N\times L}\). Accordingly, for the assumed plane-wave illumination, the far-field scattering pattern at the vth harmonic frequency \(({f}_{c}+\nu {f}_{0})\) (with \({f}_{0}=1/{T}_{0}\) denoting the modulation frequency) is expressed in the slow-modulation limit (\({f}_{0}\ll {f}_{{{\rm{c}}}}\)) as58,59

$${f}_{{{\rm{T}}}}^{v}(\theta,{\theta }_{i})={f}_{e}(\theta ){\sum }_{n=1}^{N}{a}_{v}^{n}\exp \left(\,j(n-1)d\left[{k}_{v}\,\sin \theta+{k}_{{{\rm{c}}}}\,\sin {\theta }_{i}\right]\right)$$
(2)

where \({k}_{\nu }=2\pi ({f}_{c}+\nu {f}_{0})/c\) is the corresponding wavenumber (see Supplementary Note 2 for more details).

Via a suitable design of the STC matrix, we can modulate the spatial distribution of the harmonics while suppressing the fundamental-frequency component of the reflected wave. Figure 2c displays an optimized STC matrix with N = 16 and L = 32, and the corresponding far-field harmonic scattering patterns (for \({\theta }_{i}=0^\circ\)) are shown in Fig. 2d. It can be observed that the harmonics from the −5th to the +5th are uniformly distributed in space [−60°, +60°], each pointing to a different direction, while the power of the fundamental-frequency wave is significantly suppressed (see Supplementary Note 3 for more details on the STC matrix optimization).

A STCM has the ability to simultaneously control the propagation of the fundamental-frequency wave and the spatial distribution of the harmonics. To achieve this, we define an STC matrix \({{{\bf{W}}}}_{{{\bf{T}}}}\,\), which maintains a uniform duty cycle \({\tau }_{0}\) across the unit cells of the N-column. Additionally, we introduce a time shift \({t}_{s}\) between the n-th column and the (n + 1)th-column of unit cells, i.e., \({\varGamma }_{{{\rm{T}}}}^{n+1}(t)={\varGamma }_{{{\rm{T}}}}^{n}(t-{t}_{s})\). Given the desired beam-pointing direction of the fundamental-frequency wave \({\theta }_{r}^{0}\) and the angle of incidence \({\theta }_{i}\), the corresponding SC sequences can be represented as a time-invariant STC matrix \({{{\bf{W}}}}_{{{\bf{S}}}}\). By combining (adding) the two STC matrices, \({{{\bf{W}}}}_{{{\bf{T}}}}\) and \({{{\bf{W}}}}_{{{\bf{S}}}}\), we can obtain a new STC matrix. An example assuming N = 16, \({\tau }_{0}={T}_{0}/32\), \({t}_{s}={T}_{0}/16\), \({\theta }_{r}^{0}=-30^\circ\), is shown in Fig. 2e and the corresponding far-field scattering patterns of the fundamental-frequency and harmonic waves are presented in Fig. 2f. As can be observed, the obtained STC matrix effectively directs the fundamental-frequency wave to the desired angle \({\theta }_{r}^{0}={-30}^{\circ }\) while generating multiple harmonics with distinct beam-pointing directions. Moreover, by adjusting the duty cycle \({\tau }_{0}\) and the time shift \({t}_{s}\), the beam-pointing directions and powers of the harmonics can be finely tuned with the minimal impact on the beam-pointing direction of the fundamental-frequency wave. More details on the design of the STC matrix are provided in Supplementary Note 4.

By analyzing the spatial-spectral characteristics of the harmonics generated by the spatio-temporal modulation, the DOA information can be accurately estimated. Denoting as \({s}_{v}({\theta }_{i})=|\,{f}_{{{\rm{T}}}}^{v}(\theta={0}^{\circ },{\theta }_{i})|\) the scattered amplitudes at the harmonic frequencies \({f}_{c}+\nu {f}_{0}\) (\(\nu=\pm 1,\,\pm 2\ldots \pm M\)) along the normal direction, and arranging them into a vector \({{\bf{s}}}({\theta }_{i})\) of dimension \(2M\), we can define the corresponding normalized vector as \({{\bf{x}}}({\theta }_{i})={{\bf{s}}}({\theta }_{i})/\max [{{\bf{s}}}({\theta }_{i})]\). The relationship between \({{\bf{x}}}({\theta }_{i})\) and \({\theta }_{i}\) can be exploited to estimate the direction of the incoming wave. Specifically, utilizing \({{\bf{x}}}({\theta }_{i})\) as the input and \(\theta i\) as the output, we train an artificial neural network (ANN) model to establish a robust and generalized mapping between them (see Supplementary Note 5 for details). Alternatively, this relationship can be characterized by constructing a sensing matrix for DOA estimation, as outlined in Supplementary Note 6. These DOA estimation methods rely solely on the amplitude of the harmonics, avoiding the use of phase information. This approach ensures low latency and is inherently suitable for in-situ estimation.

STCM-based ISAC system

Considering the capability of STCM to manipulate the fundamental-frequency wave and harmonics, we propose two coding strategies for the STCM-based ISAC system, namely, the adjustable partitioning and full-aperture schemes. The adjustable partitioning scheme consists of partitioning the STCM by using different modulation strategies, as shown in Fig. 3a. The STCM is split into two regions: Region 1 performs spatio-temporal modulation, periodically switching its operating states according to the STC matrix, while Region 2 is configured with an SC pattern that can be switched on demand. Hence, the far-field pattern of the fundamental-frequency wave is the combination of the far-field patterns of both regions. To reduce the interference with the beam shaping of the space-coding region of the STCM, it is essential to minimize the fundamental frequency component in the reflected wave of the spatio-temporal-modulated region. This approach enables the STCM to simultaneously optimize the fundamental-frequency wave and its harmonics, hence dynamically optimizing the power allocation between the communication and sensing by controlling the size of the two regions. Various power allocation strategies in ISAC systems have been extensively investigated under different optimization criteria68,69,70.

Fig. 3: Space-time-coding metasurface (STCM)-based integrated sensing and communication (ISAC) system architecture.
Fig. 3: Space-time-coding metasurface (STCM)-based integrated sensing and communication (ISAC) system architecture.
Full size image

a Adjustable partitioning scheme of the STCM, which is divided into two regions for sensing and communication. The size of the two regions can be flexibly adjusted based on the required needs. b Full-aperture scheme of the STCM without partitioning, which can utilize the entire STCM aperture by using STC matrices. c Architecture of the ISAC system, in which a USRP is used for signal modulation and demodulation, and the STCM establishes a reliable communication link by sensing the direction of the incoming wave in real-time and configuring the coding matrices.

The full-aperture scheme for the STCM applies spatio-temporal modulation across the entire metasurface, as shown in Fig. 3b. Given the desired beam-pointing direction of the fundamental frequency wave \({\theta }_{r}^{0}\), the angle of incidence \({\theta }_{i}\), the duty cycle \({\tau }_{0}\), and the time shift \({t}_{s}\), the corresponding STC matrix can be generated (see the design flow in Supplementary Fig. 3). By designing the STC matrix, the propagation of the fundamental-frequency wave can be controlled for communication while generating multiple harmonics with distinct spatial distributions for sensing. More details on this full-aperture coding strategy are provided in Supplementary Note 6.

Figure 3c illustrates the ISAC system architecture for simultaneous direction sensing and wireless communication. A commercial software-defined radio transceiver (NI USRP-2943R) is utilized for signal modulation and demodulation, by connecting each of its three ports to an antenna. Given that the operating frequency \({f}_{c}\) of the STCM exceeds the upper bandwidth limit of the USRP, a mixer is integrated between the USRP port and the antenna. The modulated (finite-bandwidth) signal generated by the USRP undergoes up-conversion to frequency \({f}_{c}\) before being transmitted via the antenna ANT1 to the STCM at an angle of incidence \({\theta }_{i}\). The incoming wave illuminating the spatio-temporal-modulated region of the STCM is converted into multiple harmonics and received by the antenna ANT2, which is positioned perpendicularly to the STCM. These received harmonics are down-converted to the frequency \({f}_{{IF}}\) and are analyzed by the USRP. By analyzing the amplitudes of the harmonics, the angle of incidence \({\theta }_{i}\) of the incoming wave can be estimated. Based on this DOA information, the FPGA module adjusts the SC pattern of the STCM, so that the incoming wave is deflected to the direction \({\theta }_{r}^{0}\), is received by antenna ANT3, and is demodulated by the USRP to recover the signal. Any change in the position of ANT1 leads to a variation of the angle of incidence. Consequently, the amplitudes of the harmonics received by ANT2 are analyzed in real-time, and the coding pattern of the STCM is adjusted autonomously. This ensures the establishment of a reliable communication link between ANT1 and ANT3, irrespective of the angle of incidence \({\theta }_{i}\).

Experimental verification and analysis

To assess the viability of the proposed approach, we fabricated a prototype of a 2-bit reflection-type STCM and tested its capability to manipulate the fundamental-frequency wave and the harmonics in a microwave anechoic chamber, as shown in Fig. 4a. The fabricated STCM prototype, operating at frequency \({f}_{c}=10.3\) GHz, comprises 16 × 16 unit cells, and each unit cell is integrated with two PIN diodes (MACOM MADP-000907-14020x), as illustrated in Fig. 4b. The unit cells are designed such that a 90° phase difference is achieved when the two PIN diodes are switched among the four “OFF-OFF”, “ON-OFF”, “OFF-ON”, and “ON-ON” states, corresponding to the four digital states “0”, “1”, “2”, and “3”, respectively (see Supplementary Note 7 for more details).

Fig. 4: Experimental validation of 2-bit space-time-coding metasurface (STCM) prototype and measured far-field patterns for the full-aperture scheme.
Fig. 4: Experimental validation of 2-bit space-time-coding metasurface (STCM) prototype and measured far-field patterns for the full-aperture scheme.
Full size image

a Measurement setup in an anechoic chamber. b Fabricated prototype of a 2-bit reflection-type STCM. c, Measured far-field scattering patterns of the fundamental-frequency wave for different STC matrices as detailed in Supplementary Fig. 4a. d–f, Measured far-field scattering patterns corresponding to three STC matrices for different parameters of the duty cycle and time shift, as detailed in Supplementary Fig. 4d–f.

We first measured the far-field scattering patterns of the fundamental-frequency wave and harmonics of the STCM, by utilizing the coding strategy for the full-aperture scheme (see Supplementary Note 8 for more details). Assuming that \({\theta }_{r}^{0}\) varies from −60° to +60° with a 10° increment and \({\theta }_{i}={0}^{\circ }\), \({t}_{s}={T}_{0}/16\), \({\tau }_{0}={T}_{0}/32\), a series of STC matrices can be generated. In this case, the entire aperture of the STCM is exploited, relying only on STC matrices. The STCM is illuminated by a transmitting horn antenna with a monochromatic wave at 10.3 GHz, and the measured far-field scattering patterns of the fundamental frequency wave are shown in Fig. 4c. We also measured the far-field scattering patterns of the STCM with STC matrices for different values of the duty cycle \({\tau }_{0}\) and time shift \({t}_{s}\). Assuming \({\theta }_{r}^{0}={30}^{\circ }\), three matrices are generated, using the parameters \({t}_{s}={T}_{0}/16\), \({\tau }_{0}={T}_{0}/32\); \({t}_{s}={T}_{0}/24\), \({\tau }_{0}={T}_{0}/24\); and \({t}_{s}={T}_{0}/24\), \({\tau }_{0}={T}_{0}/48\), and the corresponding measured far-field scattering patterns are shown in Fig. 4d–f. The experimental results show the STCM capability of using the full-aperture scheme to manipulate the propagation of the fundamental-frequency wave and the spatial distribution of the harmonics. By analyzing the spatial-spectral characteristics of the harmonics, the DOA can be accurately estimated. Further details and experimental results for the full-aperture scheme are provided in Supplementary Note 6.

When the STCM is designed based on the adjustable partitioning scheme, two regions of the metasurface are configured with the SC patterns and the STC matrix, as shown in Fig. 2a and c, respectively. As an illustrative example, we consider that the STCM is divided into two regions of equal area, ensuring equal power allocation for wireless communication and sensing. The measured far-field scattering patterns of the fundamental-frequency wave are displayed in Fig. 5a. Figure 5b and c show the far-field harmonic scattering patterns (from the −5th to the +5th orders), with the reflected beams at the fundamental frequency pointing to the directions of −15° and −30° (corresponding to different space-coding patterns), respectively. We observe that these measurements closely resemble the simulation results in Fig. 2d. Additionally, the reflected power at the fundamental frequency (from Region 1 of the metasurface) is significantly suppressed, and different SC patterns have minimal impact on the spatial-spectrum characteristics of the harmonics. Moreover, the optimization of the direct-current (DC) voltage bias network on the STCM can enable the independent control of each unit cell, thereby enhancing the flexibility in adjusting the area of each STCM region and the power distribution for communication and sensing, as illustrated in Supplementary Fig. 11. Further details on the optimization of the STCM can be found in Supplementary Note 9.

Fig. 5: Measured far-field patterns for the adjustable partitioning scheme and direction of arrival (DOA) estimation results using the artificial neural network (ANN) model.
Fig. 5: Measured far-field patterns for the adjustable partitioning scheme and direction of arrival (DOA) estimation results using the artificial neural network (ANN) model.
Full size image

a Measured far-field patterns of the fundamental-frequency wave for different SC sequences. b, c Measured far-field harmonic scattering patterns for the reflected fundamental-frequency wave pointing to −15° and −30°, respectively. d Measured signal spectra received by antenna ANT2 (for θi=0°). e Distribution of the normalized harmonic amplitude for θi=−50° and θi=+20°, respectively. f Results and errors for DOA estimation by using the ANN model.

To generate the training data for the ANN model to perform DOA estimation, we measure the far-field harmonic scattering patterns under varying transmission powers and diode switching speeds. The trained model demonstrates high prediction accuracy on the test set across different transmission powers and diode switching speeds (see Supplementary Note 5 for details). In practical applications, the incoming wave is usually not monochromatic but carries a modulated finite-bandwidth signal. Hence, we further assess the performance of the ANN model by using a USRP-modulated signal, where a color image is encoded into a binary stream and modulated with a quadrature phase shift keying (QPSK) scheme. The modulated signal is up-converted to 10.3 GHz and transmitted through the antenna. Subsequently, after the STCM modulates the incoming wave, the generated harmonics are received and down-converted to the input port of the USRP for signal spectrum analysis. Here, the diode switching speed is set to 64 MHz, corresponding to a modulation frequency \({f}_{0}=2\) MHz, and the symbol rate is 0.5 Mbps, while the angle of incidence \({\theta }_{i}\) is varied in the range [−65°, +65°] with a step of 5°. Then, the received signal spectrum corresponding to each angle of incidence is measured sequentially (see Supplementary Note 8 for more details). Figure 5d illustrates the received signal spectra for \({\theta }_{i}=0^\circ\), demonstrating no overlap between the signal spectra at the fundamental and harmonic frequencies. The signal spectra are normalized to form the input vector for the ANN model, as shown in Fig. 5e for \({\theta }_{i}=-50^\circ\) and \({\theta }_{i}=+ 20^\circ\). The results and errors of the estimated DOA of the ANN model are presented in Fig. 5f, indicating that the error of the estimated angles is limited within 3°. Despite the fact that the ANN model is trained with a dataset for monochromatic incoming waves, the ANN architecture accurately estimates the angle of incidence even for finite-bandwidth modulated signals.

Subsequently, in line with the architecture in Fig. 3c, we implement the STCM-based system in an indoor setting to validate the proposed ISAC scheme, as illustrated in Fig. 6a. This proof-of-concept STCM-based ISAC system utilizes the adjustable partitioning scheme. The signal generated by the USRP at the frequency of 1.5 GHz is up-converted to 10.3 GHz before illuminating the STCM via antenna ANT1. Antenna ANT2 is located in the normal direction to the STCM for receiving the harmonic signals, while antenna ANT3 is positioned at +30° for receiving the original modulated signal. The signals received by antennas ANT2 and ANT3 are down-converted to 1.5 GHz and are then sent to the two input ports (RX1(RF1) and RX2(RF1)) of the USRP for analyzing the spectra of the harmonics and for demodulating the original signals, respectively. The position of antenna ANT1 varies across 10 selected measurement points, corresponding to the directions [−61°, −45°, −38°, −29°, −11°, 12°, 23°, 42°, 50°, 62°] based on the DOA estimation by the ANN architecture. Figure 6b illustrates three examples of different angles of incidence. When the STCM is operational, the space-coding pattern is determined based on the angle of incidence, hence ensuring that the reflected beam at the fundamental frequency points to the direction of antenna ANT3. Consequently, the original modulated signal can be successfully recovered. Conversely, when the STCM is inactive, the angle of incidence cannot be determined. As a result, the incoming wave is reflected to the specular direction with respect to antenna ANT1, leading to an incorrect data demodulation. Figure 6c–f shows the demodulated constellation diagrams, error vector magnitudes (EVMs), and bit error ratios (BERs) when the STCM is operational and not operational across the 10 antenna positions, respectively. Notably, when the STCM is inactive, the EVM remains relatively low only for angles of incidence very close to −30° (i.e., specularly symmetrical with respect to antenna ANT3). However, deviations from -30° result in the inability to recover the transmitted image, as it is apparent from the cluttered constellation diagrams and the increased EVM and BER. Conversely, when the STCM is operational, the demodulated constellation diagrams consistently exhibit good quality, and the EVM and BER are maintained at low levels, thus facilitating a satisfactory reconstruction of the transmitted image irrespective of the position of antenna ANT1. Hence, the STCM facilitates the establishment of reliable communication links between ANT1 and ANT3 by real-time sensing of the DOA. Further details can be found in Supplementary Note 10 and Supplementary Movie 1. We further validated the system’s performance in real-world scenarios, including an outdoor environment and an indoor setting without absorbers, as detailed in Supplementary Note 11.

Fig. 6: Implementation and testing of the space-time-coding metasurface (STCM)-based integrated sensing and communication (ISAC) system.
Fig. 6: Implementation and testing of the space-time-coding metasurface (STCM)-based integrated sensing and communication (ISAC) system.
Full size image

a Experimental setup. b Received images corresponding to different angles of incidence when the STCM is not operational (upper panel) and operational (lower panel). c–f Measurement results for 10 positions of antenna ANT1. Demodulated constellation diagrams when the STCM is not operational and operational, respectively. e, f Measured error vector magnitudes (EVMs), and bit error ratios (BERs), respectively, when the STCM is not operational and operational, for different angles of incidence.

Moreover, although the scenario in Fig. 1 involves a single transmitter and requires the estimation of a single DOA, the proposed scheme is also capable of estimating the DOAs of multiple incoming signals (see Supplementary Note 6). Finally, it is also important to note that the proposed STCM-based ISAC scheme is not the only possible option. For instance, an alternative scheme utilizing the same platform is discussed in Supplementary Note 12, which is focused on establishing a fixed communication link and locating a passive target.

Discussion

We have introduced a cost-effective and efficient ISAC scheme leveraging the STCM technology. The proposed approach operates by simultaneously controlling the fundamental-frequency and high-order harmonic waves, which enables the analysis of the spatial-spectral characteristics of the harmonics for DOA estimation while ensuring efficient communication through the fundamental-frequency wave. By leveraging the shared sensing information, this system enhances communication through beamforming, embodying the resource-sharing principles of ISAC. We have proposed two coding strategies based on the STCM technology and, with the aid of experiments, we have demonstrated their capability of precisely and flexibly shaping the fundamental-frequency wave and estimating the DOA, and dynamically adjusting the power allocation between the communication and sensing. Moreover, we have realized and tested an ISAC system based on the implemented STCM platform. The obtained experimental results have demonstrated real-time beam steering and DOA estimation capabilities. The proposed scheme offers several advantages, such as a simplified hardware architecture, the absence of additional sensors, and cost-effectiveness. These benefits significantly enhance the potential of STCMs and RISs, positioning them as pivotal technologies for future communication systems.

Methods

ANN implementation

The ANN model, illustrated in Supplementary Fig. 5a, consists of a 10-dimensional input layer, followed by four hidden layers with 128, 64, 32, and 16 neurons, respectively, and terminated by a 1-dimensional output layer. ReLu activation functions are employed across the ANN model, and the MSE is used as the loss function. This model is trained and optimized using TensorFlow71 on a desktop computer equipped with an NVIDIA Quadro RTX 8000 Graphical Processing Unit (GPU), and an AMD Ryzen Threadripper 3970 × 32-Core Processor @3.69 GHz, and 256 GB of RAM, operating on Windows 10 (Microsoft). More details are available in Supplementary Note 5.

Experimental setup of the USRP

The USRP is connected to a portable computer used for modulating and demodulating the data, analyzing the spectrum, and managing the USRP. The control software for the USRP is developed in LabVIEW 2023 from National Instruments (https://www.ni.com/zh-cn/shop/ labview.html). For transmission, the USRP encodes a color image into a binary data stream, which is then segmented into data frames containing synchronization bits, frame order, and guard bits.