Introduction

Topological insulators represent a class of materials that support excitations propagating along their boundaries, whose properties and existence are guaranteed by the nontrivial topology of the bulk1,2. The broad class of topological insulators includes materials in which topologically non-trivial phases arise due to the breaking of certain symmetries of the system, including time-reversal and inversion symmetries, or due to specific deformations of the underlying lattice structure and dynamical modulation of the system parameters3,4,5,6,7,8. To date, topological insulators have been demonstrated in solid-state physics9, acoustics10, and mechanics11, in cold atoms and matter waves12, in various optoelectronic and photonic systems13,14. Photonic topological insulators extend the concept of electronic topological insulators to the electromagnetic domain and offer unique possibilities for controlling the structure of optical fields, their ultrafast switching, routing, and diffraction control15,16,17,18,19,20. Robust in-gap edge states in such materials protected by the topology of the system can resist disorder and defects, making them ideal for the design of topologically protected transmission and lasing devices. The central concept in the theory of topological insulators is that the symmetry of the lattice and the geometry of its boundaries are key to determining the properties, domains of existence, and dimensionality of topological boundary states. A conventional d-dimensional topological insulator supports (d − 1)-dimensional topological states at its boundaries.

Recently, a class of higher-order topological insulators (HOTIs) was introduced. Such systems support nontrivial topological boundary modes characterized by a codimension of at least two21,22,23,24. Thus, d-dimensional HOTI of order n supports (d − n)-dimensional localized states at its boundaries, and these states also enjoy topological protection against disorder and defects. Symmetry plays a pivotal role in the study of these topological states, and symmetry-based analysis resulted in the prediction and classification of a multitude of crystalline insulators25,26, see classification in27,28. Photonic HOTIs based on lattices with different symmetries have also been reported29,30,31,32,33,34,35,36,37,38,39,40. Usually, topologically nontrivial phases in crystalline HOTIs are associated with shifts of the Wannier centers to topologically nontrivial positions within the unit cells of periodic lattice, while truncation of the lattice that keeps the number of the unit cells in its integer may lead to filling anomaly for certain types of truncation and appearance of fractional spectral charges, indicating the existence of localized corner states in cells characterized by such charges. The emergence of higher-order corner states in such systems is expected only in certain configurations, tightly connected with non-trivial positions of the Wannier centers.

In this paper, we present a more general configuration of photonic HOTI with unconventional boundary truncations that result in the intersection of the interior of several unit cells, leading to the coexistence of complete and incomplete cells at the boundary. To this end, we use the femtosecond (fs) laser writing technique to generate waveguide arrays with a desired internal structure and external geometry. Remarkably, the nontrivial topological properties in our HOTIs emerge not from the intrinsic structure of the material, as in electronic topological insulators, but from the waveguide array inscribed in it, where the key factors are the geometric arrangement of the waveguides, the coupling strengths, and the boundary truncation. We show that a boundary state protection mechanism still exists in this system, which can be associated with fractional Wannier centers, and that it supports localized states of topological origin in a much wider parameter range compared to standard HOTIs (including in the regime where the latter become trivial insulators). Furthermore, we exploit the fact that our experimental system, unlike electronic and acoustic topological systems, possesses a nonlinear response that is crucial for controlling the localization and propagation dynamics of topological excitations41,42,43,44, the realization of topological lasers45,46, the generation of higher harmonics47,48 and topological solitons49,50,51,52,53 and experimentally observe the formation of a variety of stable nonlinear higher-order topological states with different symmetries and stability properties bifurcating from linear corner and edge modes. Our results extend the understanding of HOTIs and provide avenues for the observation and characterization of topological states in such systems.

Results and discussion

The appearance of the topological phase in crystalline HOTIs is closely related to the structure of Wannier functions—a set of orthogonal functions that provide a convenient representation of the eigenstates of the crystalline system54. Wannier centers, as central points of maximally localized Wannier functions, are a representation of mode densities in real space protected by crystalline symmetry55,56. For a Wannier center in a topologically trivial position, the associated mode density is restricted to a single unit cell (no fractional charge can arise if the lattice is truncated to obtain an integer number of unit cells). For a Wannier center in a topologically non-trivial position, the associated mode density is evenly distributed over neighboring unit cells (fractional charge occurs when the lattice is truncated). The occurrence of fractional charge can be visualized by integrating the local density of states in the occupied band per unit cell. A fractional charge is a signature of the “filling anomaly” in the terminology of solid-state physics. The theory in refs. 57,58,59 analyzes the locations of the Wannier centers for different system parameters and explains the topological origin of HOTIs via the fractional charges. Thus, in crystalline HOTIs containing an integer number of unit cells, the transition to a topologically nontrivial phase can be controlled by introducing dimerization into the intracell (t1) and intercell (t2) coupling strengths, resulting in a topological phase with corner modes at t2 > t1 and a trivial phase corresponding to t2 < t1.

In contrast, here we present a HOTI with unconventional edge truncation that generates a structure with multiple complete and incomplete unit cells and show that the higher-order edge state protection mechanism exists in it in both t2 > t1 and t2 < t1 regimes. Our HOTI is based on a honeycomb waveguide array with a triangular configuration created using the fs laser writing technique15,37. To control the coupling strengths t1,2 (which determine the structure of the eigenmodes of the system), we adjust the position of the waveguides within each unit cell by changing the ratio γ = d1/d2 (dimerization parameter) of the waveguide spacing within the cell d1 and between the cells d2. Microphotographs of the arrays with γ < 1 (in this case t2 < t1), γ = 1 (t2 = t1), and γ > 1 (in this case t2 > t1) are shown in Fig. 1a–c, while Fig. 1d illustrates the notations. The blue lines in Fig. 1a, b, and c mark complete unit cells of the structure. As can be seen, the waveguides in the corners and some waveguides at the edge belong to incomplete cells.

Fig. 1: Construction of higher-order topological insulators with unconventional boundary truncations.
Fig. 1: Construction of higher-order topological insulators with unconventional boundary truncations.
Full size image

Microphotographs of the triangular femtosecond laser-written waveguide arrays with complete and incomplete honeycomb unit cells (the former are indicated by blue contours superimposed on the photomicrographs) with different degrees of dimerization γ = 0.6 (a), γ = 1.0 (b) and γ = 1.6 (c), where γ = d1/d2 defines the ratio between the distance d1 between the nearest waveguides within the unit cell and the distance d2 between the waveguides of neighboring cells (d). The insets schematically show weak and strong couplings in such arrays. e Propagation constants β of the linear eigenmodes as a function of t2/(t1 + t2), calculated with the tight-binding model. Note the existence of boundary states with β = 0 both at t2 < t1 and t2 > t1. f, g Schematic representation of the concept of fractional Wannier center, density of states, and distributions of mode density and Wannier centers within the array in type-I [t2 < t1] and type-II [t2 > t1] phases of the system, shown with different background colors in (e).

Figure 1e shows the eigenvalues β of the linear modes of such an array with 81 sites as a function of t2/(t1 + t2). Here the eigenvalues of array modes were calculated using the standard tight-binding Hamiltonian H (see the tight-binding approximate model in Supplementary Note 1), which only considers the couplings between the nearest neighbors, and takes into account that all waveguides are shallow and support only single mode defined by amplitude am,n, where m, n is the waveguide index in two-dimensional honeycomb array (in contrast to tight-binding models in electronic systems that usually apply the concepts of multiple orbitals on a single site60). Unlike conventional HOTIs on honeycomb arrays, where the topological phase only occurs at t2 > t1, in our system, boundary states are found both at t2 < t1 (i.e. γ < 1) and t2 > t1 (i.e. γ > 1). There is still a transition point t2 = t1, which distinguishes between type I and type II phases, which are shown in Fig. 1e with different colors (in a very large array, the gap would close at this point, while in our system it remains open due to its finite size). We also study two special cases t1 = 3t2 in the type-I phase and t2 = 3t1 in the type-II phase, as indicated by the dashed lines in Fig. 1e. Since both phases (see Fig. 1f and g) support coexisting corner and edge states in the gap, as confirmed by the calculation of the density of states \({{{\rm{DOS}}}}(\beta )=-{\lim }_{\delta \to 0}{{{\rm{Im}}}}[{{{\rm{Tr}}}}{(\beta -H+i\delta )}^{-1}]\), this implies that even if the truncation occurs through the interior of several unit cells, there is still a topological mechanism supporting the existence of localized modes. This also implies that the Wannier centers, which are usually considered as a whole, can be decomposed into parts, i.e., the topological properties can be explained by the concept of fractional Wannier center. The left images in Fig. 1f and g show the positions of the Wannier centers (red circles) within the unit cells. In the type I phase (γ < 1), the Wannier centers are located at so-called topologically trivial positions (reflecting the positions of strong coupling links), in the centers of the cells, while truncation leads to the appearance of fractional Wannier centers in the corners and at the edges of the lattice (Fig. 1f). In the type II phase (γ > 1), the Wannier centers are located at so-called topologically non-trivial positions (again reflecting the positions of strong coupling links), at the edges of the cells, and truncation of the array cutting some of these centers now leads to the appearance of fractional Wannier centers (partly white, partly red circles) at the edges (Fig. 1g), but within other unit cells compared to the type I case. The analysis of the eigenmodes allows the general conclusion that topologically non-trivial localized boundary modes (edge or corner modes) only occur where the truncation passes through the Wannier centers, i.e., their occurrence is directly related to the occurrence of the fractional Wannier centers. Thus, Fig. 1 illustrates that the key difference between the γ < 1 and γ > 1 phases lies in the positions of the Wannier centers, which lead to different topological origins of boundary modes: For γ < 1 both corner and edge states arise from fractional Wannier centers located within incomplete unit cells, resulting in unconventional topological states, while for γ > 1 they correspond to conventional topological states similar to those that arise in conventional higher-order insulators and associated with Wannier centers near the boundaries of complete unit cells.

Although conventional topological invariants that rely on symmetry are poorly defined in this context, we can define the fractional spectral charge Qb corresponding to the boundary cell b:

$${Q}_{b}={\sum}_{m}{\sum}_{(x,y)\in b}\left\vert \left\langle {\psi }_{m}\right\vert {P}_{x,y}\left\vert {\psi }_{m}\right\rangle \right\vert \quad (\,{{\mathrm{mod}}}\,\,1),$$
(1)

where \({P}_{x,y}=\left\vert x,y\right\rangle \left\langle x,y\right\vert\) represents the position operator and \(\left\vert {\psi }_{m}\right\rangle\) denotes the m-th eigenmode for the corresponding occupied band61. To generalize the concept of fractional charge, we consider the incomplete unit cells as if they were complete, with completely filled additional imaginary vacant areas. The spectral charges defined according to this procedure are shown in the right panels in Fig. 1f, g. In Fig. 1f the fractional charges for orange (corner) zones and green (edge) zones are given by 1/6 and 1/2, respectively. In Fig. 1g, the fractional charges for green edge zones are equal to 5/6, while for green cells near the corners, they are equal to 2/3 (to define corresponding spectral charges we take into account that Wannier centers shared between cells contribute charge 1/2 to each cell, except for Wannier centers that fall onto outer boundaries, and normalize the charge for all cells such that bulk cells possess spectral charge 1). They immediately allow the identification of incomplete (in the type I phase) or complete (in the type II phase) cells in which the charge is fractional and localized boundary modes can arise. It can also be seen that such cells always contain fractional Wannier centers.

Linear modes and soliton families

To confirm the above conclusions based on the tight-binding model and to demonstrate the appearance of new types of topological boundary modes in HOTI arising from unconventional truncation, we further use the continuous dimensionless nonlinear Schrödinger equation describing the paraxial propagation of light along the z-axis:

$$i\frac{\partial \it{\Psi} }{\partial z}=-\frac{1}{2}{\nabla }^{2}\it{\Psi} -{{{\mathcal{V}}}}(x,y)\it{\Psi} -| \it{\Psi} {| }^{2}\it{\Psi} ,$$
(2)

where Ψ(x, y, z) is the field amplitude, z is the propagation distance, x and y are the transverse coordinates and ∇ = (∂/∂x, ∂/∂y) is the transverse Laplacian (for the normalization of all quantities see “Methods”). The function \({{{\mathcal{V}}}}(x,y)\) describes the refractive index distribution inside the array with a honeycomb inner structure and triangular outer geometry consisting of Gaussian waveguides:

$$\begin{array}{r}{{{\mathcal{V}}}}(x,y)=p\sum\limits_{m,n}{e}^{-{\left(x-{x}_{m,n}\right)}^{2}/{w}_{x}^{2}-{\left(y-{y}_{m,n}\right)}^{2}/{w}_{y}^{2}}.\end{array}$$

Here, the parameter p represents the array depth proportional to the refractive index contrast δn within the structure, the positions of the waveguide centers (xm,n, ym,n) define a honeycomb array with introduced Kekulé distortion, whose strength is characterized by the ratio γ = d1/d2 of the intra- and intercell distances of the waveguides [see Fig. 1d], where γ = 1 corresponds to an undistorted structure. The side length of the unit cell is denoted by a. In addition, the waveguides are elliptical with widths wy > wx due to the writing process. In accordance with the actual parameters of the laser-written structures, the dimensionless parameters are set as a = 5.72, wx = 0.25, wy = 0.75, and p = 4.7. The proposed HOTIs are composed of 144 waveguides forming 19 complete unit cells and several incomplete cells. Microphotographs in Fig. 1a–c show three typical waveguide arrays with different γ values written by the fs laser. Note that the continuous model takes into account all details of the refractive index landscape and thus accounts for the coupling between all waveguides in the system, which is an advantage over the tight-binding description.

To characterize the topological properties of this structure, it is crucial to determine its linear spectrum, which we achieve by searching for linear eigenmodes Ψ(x, y, z) = ψ(x, y)eiβz, where ψ(x, y) is the modal field and β is the eigenvalue (propagation constant). The eigenmodes result from the subsequent linear eigenvalue problem:

$$\beta \psi =\frac{1}{2}{\nabla }^{2}\psi +{{{\mathcal{V}}}}(x,y)\psi ,$$
(3)

solved using the finite-difference method.

The linear spectrum of our array is shown in Fig. 2a–c, while typical modal profiles are shown in Fig. 2d. The most representative feature of the spectrum is the presence of localized boundary modes of topological origin in the spectral gap for all values of the dimerization parameter γ (except for the point γ = 1), see Fig. 2a. This is in sharp contrast to conventional HOTIs, where localized states only occur for γ > 1. Note that for very large structures with incomplete cells at the edge, the gap closes at γ = 1, whereas in Fig. 2a it only remains open due to the finite size effect, since we model the structure of the same size as in the experiment (where certain technological constraints exist that limit the size of the lattice that can be inscribed). From Fig. 2b and c it can be seen that there are 12 corner or edge modes within the spectral gap for γ < 1 and for γ > 1. Among these modes, corner modes appear in triplets that become degenerate for sufficiently small or large γ values (see, for example, modes n = 67 ~ 69 at γ = 0.6). Since these modes are degenerate, one can use their linear combinations to construct states residing in only one, two, or three corners of the array. It is noteworthy that the internal structure of the corner modes at γ < 1 and γ > 1 is qualitatively different, as their primary maxima are located in different waveguides (compare the corner modes in Fig. 2d, top row). Such modes gradually delocalize when γ → 1. In addition, the system is characterized by the existence of bound states in the continuum (BICs) in the spectral band, which only occur for sufficiently large values of γ (see e.g. the blue arrows in Fig. 2c, which show the location of such states at γ = 1.6 in the spectrum, and the typical profiles in Fig. 2d, bottom row). A detailed description of the modes within the spectral gap of the array with unconventional boundary can be found in Supplementary Fig. S1 and Supplementary Note 2. It should be also stressed that emergence of strongly localized corner modes, like mode n = 69 at γ = 0.6, or BICs n = 22 and n = 121 at γ = 1.6, cannot be explained as a result of hybridization of conventional edge states because all these corner modes have considerable intensity in a waveguide from incomplete cell in the corner and disappear if this waveguide is removed.

Fig. 2: Linear spectrum of the system and its eigenmodes.
Fig. 2: Linear spectrum of the system and its eigenmodes.
Full size image

Linear spectrum of the array showing the eigenvalues of all linear modes as a function of the dimerization parameter γ (a) as well as the eigenvalues at γ = 0.6 (b) and γ = 1.6 (c), which correspond to the red and blue dashed lines in the β(γ) dependence, respectively. The top line in (d) shows representative linear modes ψ with eigenvalues that fall into the gap at γ = 0.6 and 1.6. At γ = 0.6, the modes with indices n = 67 ~ 69 correspond to the corner modes, while the modes with n = 70 ~ 78 are edge modes. For γ = 1.6, the indices n = 67 ~ 69, 74, 77, 78 correspond to the corner modes (including the modes n = 67, 68, and 74 with two out-of-phase peaks, while the modes n = 69, 77 and 78 have two in-phase peaks near the corner), while the indices n = 70 ~ 73, 75, and 76 correspond to the edge modes. The bottom row in (d) shows linear modes from two bands with indices n = 22 and 121, which are localized at γ = 1.6 (i.e., represent bound states in the continuum) and delocalized at γ = 0.6. The field distributions of all linear modes in the spectral gap can be found in Supplementary Fig. S1.

For the sake of comparison, in Supplementary Fig. S2 we show the linear spectrum and representative eigenmodes of the array with the same parameters, but with usual truncation along the boundaries of the unit cells, which keeps the integer number of the unit cells. By comparing this linear spectrum with the spectrum of the array with unconventional truncation presented in Fig. 2a, one arrives at the following conclusions. In striking contrast to unconventionally truncated HOTIs (i) no localized modes emerge when γ < 1.0, (ii) at γ > 1.0 strongly localized on two outermost waveguides modes (akin to modes n = 22 and n = 121 in HOTI with unconventional truncation) disappear from the spectrum, and (iii) the spread of eigenvalues of the modes residing in the spectral gap of array with usual truncation at γ > 1.0 increases considerably, while corner and edge modes in the latter structure acquire more complex and broad shapes with multiple peaks having comparable amplitudes. Thus, one can conclude that unconventional truncation substantially changes the spectrum of the system and structure of modes (also see relevant discussion in Supplementary Note 3).

It should be stressed that the arrays considered in this work are sufficiently large and finite-size effects that could lead to coupling between localized states in different corners are not visible in them (in practice, such coupling occurs only when modes strongly expand as γ → 1), so that significant reduction of the size of the structure would be needed for observation of such effects. Moreover, being states of topological origin, corner modes persist in our case even in the presence of disorder, as long as the spectral gap, where they emerge, remains open. To illustrate this, we show the transformation of the linear spectrum of the array with unconventional boundary truncation for increasing disorder levels in Supplementary Fig. S3. The gap shows the tendency for reduction with the increase of disorder level (as observed for both diagonal and off-diagonal disorder), in the case of off-diagonal disorder, it is more pronounced at γ < 1 due to the way in which we set the disorder. Nevertheless, corner modes persist in the gap and are not destroyed until disorder becomes sufficiently strong to close it completely. For γ > 1 we also observe remarkable insensitivity of the eigenvalues of corner and edge modes to disorder (see Supplementary Note 4).

Since our system possesses focusing nonlinearity, we try to observe solitons of topological origin bifurcating from the above states in the spectral gap. We consider solitons of the form Ψ(x, y, z) = ψ(x, y)eiβz, where β is now an independent parameter that determines the soliton profile ψ and its power U = ∬∣ψ(x, y)∣2dxdy. If we insert this expression into Eq. (2), we obtain the equation \((1/2){\nabla }^{2}\psi +{{{\mathcal{V}}}}(x,y)\psi -\beta \psi +| \psi {| }^{2}\psi =0\), which was solved using the Newton iteration method. In Fig. 3, we show representative examples of soliton families U(β) and profiles of solitons of four different types. Because all such solitons bifurcate from linear edge or corner modes, their power U vanishes when the soliton propagation constant β approaches the eigenvalue of the linear state, from which bifurcation occurs. This means that all such solitons can form even at low powers, i.e., they are thresholdless, since they do not require a minimal power threshold for their formation, as it happens for usual two-dimensional lattice solitons. At γ < 1, we present two different families of solitons bifurcating from the corner mode and the edge mode in the incomplete unit cell, see Fig. 3a and b. As the power U increases, the propagation constant β of the solitons shifts upwards from the linear eigenvalue towards the bulk band. Most of the power of the solitons bifurcating from the corner mode is concentrated in the corner waveguide, while a small part of the power is localized in the adjacent cell (see panel a1 in Fig. 3a). When U increases, the soliton gradually widens (see panel a2), and when β enters the band, it spreads over the entire array (see panel a3). The behavior of soliton bifurcating from the edge mode is somewhat different - because the propagation constant of edge mode is lower than that of corner state, the family of edge solitons emanating from edge mode remains well-localized at the edge (see panel b1 in Fig. 3b) only until β crosses the eigenvalue of corner state. The crossing leads to a coupling of the edge soliton with the corner state and a progressive increase of the intensity in the corner (see the panels b2 and b3 in Fig. 3b), followed by a coupling with the bulk modes when β enters the band. Remarkably, both corner and edge solitons are dynamically stable in their entire existence region, even when they couple with bulk modes (this means the input perturbations imposed on such solitons do not grow upon propagation over huge distances z exceeding sample length by orders of magnitude). The stability of the soliton families was checked using linear stability analysis, and the details of which are summarized in “Methods”.

Fig. 3: Families of thresholdless topological solitons arising at different dimerization values γ.
Fig. 3: Families of thresholdless topological solitons arising at different dimerization values γ.
Full size image

Soliton power U versus propagation constant β, illustrating soliton families bifurcating from different linear localized modes: a the corner mode with n = 69 at γ = 0.6; b the edge mode with n = 70, which is located in the incomplete edge cell at γ = 0.6; c the edge mode with n = 75, which is located in the complete cell at γ = 1.6; and d the out-of-phase mode near the corner with n = 74 at γ = 1.6. The bottom line shows the intensity distributions ∣ψ∣2 corresponding to the points in the U(β) dependencies. Stable branches are shown with solid black lines, while unstable branches are shown with red dashed lines. Gray regions indicate bulk bands, and vertical gray dashed lines represent the eigenvalues of the linear modes.

For γ > 1, a representative family of solitons bifurcating from the edge mode in the complete unit cell at the boundary is shown in Fig. 3c. Such solitons are typically strongly confined to three waveguides of the complete unit cell for the propagation constants β in the gap (see panels c1 and c2 in Fig. 3c) and extend into the bulk only at sufficiently high powers (see panel c3 in Fig. 3c). As the linear stability analysis shows, this nonlinear family is completely stable in the gap [see solid part of the U(β) curve indicating the stable branch], but destabilizes within the band [see dashed part of the U(β) curve]. A representative example of a corner soliton family at γ > 1 bifurcating from out-of-phase linear corner mode is shown in Fig. 3d. The transformation of shapes of such solitons with the increase of β is illustrated in panels d1-d3 of Fig. 3d. Interestingly, this solution becomes unstable shortly after the bifurcation from the linear corner mode (see the dashed part of the family in Fig. 3d). This instability is analogous to classical symmetry-breaking instability that in a finite gap occurs for a corner soliton with out-of-phase spots. It is a result of bifurcation from the present anti-symmetric family of stable asymmetric corner soliton family characterized by larger intensity in one of the near-corner waveguides (not shown in Fig. 3d). These results show that topological solitons in HOTIs with unconventional truncations occur in a variety of forms with different internal structures and for any value of the dimerization parameter γ. Several different stable soliton branches can coexist in the same gap.

It should be stressed that the corner solitons studied here are robust nonlinear states that persist in the presence of disorder in the underlying array. As long as the spectral gap remains open and a linear corner or edge mode persists in the spectral gap, this mode gives rise to the family of solitons bifurcating from it. Such soliton families obtained in arrays with different realizations and levels of the off-diagonal disorder are presented in Supplementary Fig. S4 for both γ = 0.6 and γ = 1.6 values of dimerization parameter. Remarkably, the studied corner and edge soliton families remained stable within the spectral gap despite the presence of disorder in the array (see relevant discussion in Supplementary Note 4).

Experimental observation of the nonlinear states

To verify the existence of such linear localized modes and solitons bifurcating from them for each value of the dimerization parameter γ in an array with unconventional boundary truncation, we used the fs laser direct writing technique to inscribe such arrays with triangular shapes in 10 cm long fused silica slabs. A detailed description of the writing parameters can be found in the “Methods” section. For excitation, we used 280 fs high-intensity laser pulses at a wavelength of λ = 800 nm with an energy E per pulse originating from a 1 kHz Ti:sapphire laser system that can be focused into selected waveguides. The input peak power of such pulses can be evaluated as a ratio of the pulse energy to its 280 fs duration, so that 1 nJ of input pulse energy can be equated to 2.5 kW of peak power. The error in measured pulse energy and evaluated peak power does not exceed 5% and is mainly connected with fluctuations of energy of pulses, accuracy of determination of their duration, and focusing efficiency into waveguides. The dispersion for such pulses at 10 cm can be neglected, allowing us to consider only the spatial dynamics in Eq. (2). Nevertheless, one should take into account that measured output intensity distributions represent averaged patterns containing contributions from regions around the pulse peak propagating in a strongly nonlinear regime, and contributions from linearly diffracting pulse tails. For this reason, experimental patterns in the figures below may appear somewhat more extended than corresponding theoretical output intensity distributions obtained from a purely spatial model (2). For two-site excitations with a controllable phase difference between two beams, the Michelson interferometer scheme was employed (see “Methods”).

First, we examined the excitation dynamics in an array with a regular honeycomb structure, without introducing Kekulé distortion into it, i.e., for d1 = d2 and γ = 1.0. Fig. 4 compares experimentally recorded output intensity distributions (a-d) for four different and typical types of excitations of the edge and corner waveguides of the array with results of numerical modeling in the frames of Eq. (2) (e-h). Due to the absence of localized modes in the array at γ = 1, the beams in linear (pulse energy E = 10 nJ) and weakly nonlinear regime (pulse energy E = 200 nJ) exhibit dramatic broadening across the array irrespective of the position of the input excitation. This is observed in the first two columns of Fig. 4a–c and e–h. With an increase of pulse energy to the values about E = 500 nJ the diffraction is considerably suppressed in some cases, and signatures of nonlinearity-induced localization start to appear in other cases (third column). For all types of excitation pulse energy of E = 800 nJ was sufficient for confinement of light practically in one waveguide at the edge or in the corner (fourth column in Fig. 4a–c and e–g). This is a clear signature of the formation of a nontopological soliton existing in a semi-infinite gap and requiring a considerable power threshold for its excitation. The threshold was largest for the case of out-of-phase excitation of two waveguides in the bottom corner. These experimental results are in excellent agreement with the results of theoretical simulations, illustrating that the system is indeed in a trivial phase at γ = 1.

Fig. 4: Excitation of linear and non-linear states at dimerization parameter γ = 1.0.
Fig. 4: Excitation of linear and non-linear states at dimerization parameter γ = 1.0.
Full size image

Comparison of the experimental output intensity distributions [(a–d), maroon background] with theoretically calculated output distributions [(e–h), white background] for different excitation positions indicated by the arrow and the circles. The contours superimposed on the intensity distributions show only complete (not truncated) unit cells of the lattice. a, e Waveguide in the bottom corner; b, f the seventh waveguide from the top left corner at the left edge; c, g the ninth waveguide from the top left corner at the top edge; and d, h two waveguides in the bottom corner, out-of-phase excitation. The total input pulse energies in the experiment and the input powers in the simulations are given on each panel. Here and below, the corresponding E values are typically doubled in the case of two-site excitation to obtain the same nonlinear contribution to the refractive index in each waveguide.

The arrays with γ < 1 and γ > 1 support localized topological boundary modes, but their structure is significantly different, as shown above, so that qualitatively different propagation scenarios can be expected in these cases. In Fig. 5 we show experimental results for the same excitation positions, but in an array with γ = 0.6. In this configuration, even at low energy inputs E = 10 nJ, we observe the excitation of the topological corner mode in the bottom corner (panels a and e), which corresponds to the linear mode with index n = 69, and the edge mode (panels b and f), which is associated with the state with index n = 73 (see distributions in Supplementary Fig. S1a). Note that in the latter case, the single input beam excites a combination of several localized edge modes. For this reason, one observes light beating between three waveguides in the excited incomplete cell, but no diffraction into the bulk. Note that both these types of localized modes form on waveguides from incomplete cells. In contrast, when a corner or edge waveguide is excited within a complete unit cell, strong diffraction into the bulk is observed (see panels c and g), since such cells do not support localized topological modes for γ < 1. When two bottom waveguides are excited (panels d and h), one observes a mixed scenario, where light remains in the outermost waveguide, but leaves the excited waveguide just above it (this dynamic persists even in the nonlinear regime). As the pulse energy increases, localization of the soliton in the bottom corner increases as well (panels a and e). Edge mode in a nonlinear regime demonstrates beatings depending on pulse energy E with output remaining within the incomplete cell at the edge (panels b and f). For excitation of the complete unit cell on the edge, even pulse energy E = 800 nJ was not sufficient to obtain good localization (panels c and g).

Fig. 5: Excitation of linear and nonlinear states at dimerization parameter γ = 0.6.
Fig. 5: Excitation of linear and nonlinear states at dimerization parameter γ = 0.6.
Full size image

The arrangement and meaning of the panels are as in Fig. 4. a, e show the intensity distributions for the single-site excitation of the waveguide in the lower corner (where the mode with index n = 69 is located); b, f show the single-site excitation of the seventh waveguide from the upper left corner at the left edge (strong overlap with the edge mode with n = 72 in the incomplete cell); c, g single-site excitation of the ninth waveguide at the upper edge belonging to the complete unit cell; d, h out-of-phase excitation of two lower waveguides.

Figure 6 compares the experimental and theoretical excitation dynamics for the dimerization parameter γ = 1.6. As the investigation of the linear spectrum shows (see Fig. 2), for this value of γ, there are no topological modes in the spectral gap that would be predominantly confined in a single outer corner waveguide (see Supplementary Fig. S1). For this reason, the excitation of the corresponding corner (belonging to the incomplete cell) leads to a dynamical change of the pattern with the increase of the pulse energy E (see Fig. 6a and e), quite in contrast to the strong localization observed for such an input at γ = 0.6. Remarkably, in this case, light switches mainly between two waveguides, the corner one and the waveguide adjacent to it, but there is no diffraction into the bulk. This is a clear indication of the excitation of the linear combination of two BICs associated with linear in-band modes with indices n = 121 and n = 22 (see Fig. 2d, bottom row), localized at γ = 1.6 and characterized by two out-of-phase and in-phase maxima in the two outermost corner waveguides. While single-site excitation leads to nearly equal efficiency of excitation of these two modes, which results in beatings, using two out-of-phase input beams has enabled very clean excitation of the nonlinear corner state of this type, illustrated in Fig. 6d and h. It should be emphasized that despite the presumed location of this state in the band, increasing the input pulse energy did not lead to radiation into the bulk, as can be seen by comparing the output patterns at E = 400 nJ and E = 1600 nJ. In contrast to the γ = 0.6 case, for γ = 1.6 bright spots in edge modes are located in waveguides belonging to the complete unit cells. Thus, when a waveguide in the incomplete cell is excited, one observes strong diffraction (see Fig. 6b and f) that is arrested only at the highest pulse energy E = 800 nJ used here for single-waveguide inputs. In clear contrast, similar excitation of a waveguide in the complete unit cell on the edge (see Fig. 6c and g) results in the formation of a thresholdless topological edge soliton that remains well-confined at all pulse energies.

Fig. 6: Excitation of linear and non-linear states at dimerization parameter γ = 1.6.
Fig. 6: Excitation of linear and non-linear states at dimerization parameter γ = 1.6.
Full size image

The arrangement and meaning of the panels are as in Fig. 5. a, e Single-site excitation of the waveguide in the lower corner waveguide; b, f single-site excitation of the seventh waveguide from the upper left corner at the left edge; c, g single-site excitation of the ninth waveguide at the upper edge (overlapping with the edge mode with index n = 75 in the whole unit cell); d, h phase-shifted excitation of two lower waveguides overlapping with the phase-shifted corner mode with n = 121.

Here, we have discussed and presented experimental evidence for the formation of only some types of topological edge and corner solitons from a variety of nonlinear states possible in this topological system. Additional experimental results illustrating the excitation of other types of solitons (see relevant discussions in Supplementary Note 5 and Supplementary Note 6), such as two-spot states bifurcating from the topological corner mode n = 69 at γ = 1.6, are presented in Supplementary Fig. S6. These results show far-reaching perspectives for the realization of states of topological origin with different symmetries, which are opened up by unconventional edge truncations in HOTIs.

Conclusions

We have proposed an approach for the realization of HOTI with unconventional edge truncations, which allows the observation of linear modes and solitons of topological origin with new types of symmetry arising for values of dimerization parameters that are usually considered topologically trivial (when Wannier centers are located in the center of the unit cell). We have proposed an explanation for the topological origin of these edge modes based on the concept of fractional Wannier centers. Thus, unconventional boundary truncation not only allows us to obtain localized corner modes at any value of the dimerization parameter except for γ = 1, but it also leads to the appearance of corner states with new structure and generally better localization, associated with fractional spectral charges (i.e., fractional mode densities) on corresponding cells, in comparison with modes in HOTIs with usual truncation. We have also shown that in this relatively simple waveguiding system, linear boundary states lead to coexisting rich families of stable topological solitons bifurcating from linear states. Being thresholdless, such solitons inherit the internal structure of linear topological corner modes, from which they bifurcate, and therefore their variety is richer in arrays with unconventional boundary truncations in comparison with usual HOTIs. Our results expand the category of topological insulators and pave the way for their realization in other areas of science beyond optics, including cold atoms62,63, Bose-Einstein condensates1,2, acoustics64,65, and polariton condensates66,67,68.

Methods

Normalization of parameters in theoretical model

The dimensionless Eq. (2) in the main text is derived from the following dimensional version:

$$i\frac{\partial {{{\mathcal{E}}}}}{\partial Z}=-\frac{1}{2k}\left(\frac{{\partial }^{2}}{\partial {X}^{2}}+\frac{{\partial }^{2}}{\partial {Y}^{2}}\right){{{\mathcal{E}}}}-\frac{k}{{n}_{0}}(\delta n+{n}_{2}| {{{\mathcal{E}}}}{| }^{2}){{{\mathcal{E}}}},$$
(4)

where X, Y are dimensional transverse coordinates and Z is the propagation distance. We use the standard “soliton” units and introduce dimensionless transverse coordinates as x = X/r0, y = Y/r0, where r0 = 10 μm is the characteristic transverse scale, which also determines the relationship between the real propagation distance Z and the dimensionless propagation distance as Z = zdz, where the diffraction length \({z}_{d}=k{r}_{0}^{2}\approx 1.14\,{{{\rm{mm}}}}\). The real field amplitude is related to the dimensionless amplitude as \({{{\mathcal{E}}}}={({n}_{0}/{k}^{2}{r}_{0}^{2}{n}_{2})}^{1/2}\psi\). Where k = 2πn0/λ is the wavenumber in the medium with undisturbed refractive index n0 (for fused silica n0 ≈ 1.45 and the non-linear refractive index n2 ≈ 2.7 × 10−20 m2/W), λ = 800 nm is the experimental working wavelength and δn is the refractive index contrast that defines the structure of the shallow optical potential \({{{\mathcal{V}}}}(x,y)\). The dimensionless depth of this potential is given by \(p={k}^{2}{r}_{0}^{2}\delta n/{n}_{0}=4.7\), which corresponds to the refractive index contrast δn ≈ 5.3 × 10−4. The side length of the unit cell of the lattice in all the structures considered is a = 5.72, which corresponds to 57.2 μm, the waveguide widths wx = 0.25, wy = 0.75 correspond to 2.5 μm, 7.5 μm wide elliptical waveguides, while the sample length of 10 cm corresponds to z ≈ 84.3 (taking into account a slight reduction in length due to edge polishing).

Fs-laser inscription of the waveguide arrays

Truncated honeycomb waveguide arrays with a triangular outer geometry were fabricated using the fs laser direct writing technique. We focused laser pulses from a fiber laser system (Avesta Antaus, 515 nm wavelength, 1 MHz repetition rate, 230 fs pulse duration, 320 nJ pulse energy) through an aspherical lens (NA = 0.3) into the bulk of fused silica glass (JGS1, 10 cm long). To create a waveguide, the glass sample was translated relative to the waist of the writing beam with a scanning velocity of 1 mm/s using a high-precision positioner (Aerotech FiberGlide 3D). This procedure was repeated to create the array in the desired configuration with complete and incomplete unit cells.

Experimental excitation of the waveguide arrays

In our experimental setup, we excited one or two waveguides with femtosecond pulses of variable energy E. We used the 1 kHz Ti:Sapphire CPA laser system Spitfire HP (Spectra-Physics), which delivers ultrashort pulses with a duration of 40 fs, a central wavelength of 800 nm and 40 nm spectral width. Before focusing into selected waveguides of the sample, the femtosecond radiation passed through an active beam position stabilization system (Avesta), an attenuator, and was further narrowed in a 4f single grating pulse spectrum slicer with a variable slit. Narrowing the broad pulse spectrum to 5 nm increased its duration to 280 fs and made it possible to avoid strong spectral broadening associated with self-phase modulation in the nonlinear regime. The sample with the waveguide arrays was coupled to the focusing system using a precise 6-axis positioning system (Luminos). The output radiation distribution of the arrays was recorded with a 12MP CMOS camera Kiralux (Thorlabs). For in-phase and out-of-phase excitations of two waveguides, we used a Michelson interferometer where the phase difference between the two beams was controlled by a precise rotation of the compensation plate in one of its arms. We defined the input peak power of the pulse as the ratio of the pulse energy to its duration (which in the experiment was 280 fs), so that 1 nJ of input pulse energy can be equated to 2.5 kW of peak power, taking into account the losses during beam focusing into a waveguide. For example, the maximum excitation energy of E = 800 nJ in the experimental samples presented here corresponds to the peak power of 2.0 MW.

Linear stability analysis of soliton families

The analysis of the stability of soliton families in this topological system was performed not only using direct propagation of perturbed inputs but also using rigorous linear stability analysis. For this purpose, the shapes of the perturbed soliton solutions were represented in the form Ψ(x, y, z) = [ψ(x, y) + u(x, y)eδz + iv(x, y)eδz]eiβz, where ∣u, v∣ ≪ ∣ψ∣ are small perturbations, while δ is the growth rate of the perturbation (which can be complex). Substituting this expression into Eq. (2) and linearizing the equation around the stationary solution ψ(x, y), the following linear eigenvalue problem can be derived:

$$\delta u = -\frac{1}{2}{\nabla }^{2}v+\beta v-{\psi }^{2}v-{{{\mathcal{V}}}}v,\\ \delta v = + \frac{1}{2}{\nabla }^{2}u-\beta u+3{\psi }^{2}u+{{{\mathcal{V}}}}u.$$
(5)

By solving Eq. (5) with a linear standard eigenvalue solver, we obtain the dependencies δ(β) of the growth rate of the perturbation on the propagation constant β. If Re(δ) ≤ 0 holds for all perturbation modes, then the soliton solution with a given β is stable, while it is unstable if Re(δ) > 0. Stable soliton branches are shown in Fig. 3 with solid lines, while unstable branches are shown with dashed lines.