Introduction

Cognitive map theory proposes that the brain represents spatial information for efficient spatial navigation in the complex environment1. Two distinct types of spatial reference frames are believed to be used in spatial navigation: Allocentric and egocentric spatial reference frames2. While allocentric neural representations of an animal’s location, orientation, or distance relative to environmental features are well characterized3,4,5,6,7,8,9,10, egocentric neural representations of environmental features relative to the animal’s current location and head direction have only recently been discovered2,11,12,13,14,15,16,17,18. Interestingly, many of the egocentric spatial-selective neurons use vector coding of space2 where neurons spike to generate spatial receptive fields when spatial structures are at a certain distance and orientation relative to the animal, tuned by its head direction2. The egocentric vector coding cells include egocentric boundary cells11,12,13,17,18, egocentric border cells14, egocentric center bearing/distance cells15, and egocentric object vector cells11, which may collectively contribute to constructing an egocentric cognitive map of the spatial layout of the external world that is important in guiding goal-directed navigation19 and episodic memory2.

The environmental features in the external world are geometric, which are represented by combinations of edges and vertices. Therefore, an egocentric neural representation of edges and vertices should help the efficient construction of a geometrically detailed egocentric cognitive map of egocentric space that can guide movement transition20 for goal-directed navigation19. Although egocentric vector coding of edges has been reported in the form of boundary vector or border cells11,12,13,14,17,18, egocentric neural representation of vertices has not been identified yet.

The retrosplenial cortex (RSC) is a good neural circuit candidate for studying egocentric neural representations of vertices. RSC is well established to serve critical roles in goal-directed navigation19,21,22,23,24,25,26, egocentric spatial navigation12,14,27,28, episodic memory29, as well as transformation between allocentric and egocentric spatial reference frames14,27, all of which require an egocentric cognitive map. Anatomically, the granular RSC has dense reciprocal connections with multiple neural circuits that map allocentric space via spatially-tuned cells, including the hippocampus27,30,31 (place cells3), medial entorhinal cortex27,32 (grid cells4), anterior thalamic nuclei27,30,31 (head direction cells6), and subiculum27,30,31 (boundary vector cells7 and corner cells33). Furthermore, egocentric boundary/border cells have already been discovered in the RSC12,14.

Results

Neural representation of geometric vertex in the granular RSC

First, we investigated whether the neural representation of geometric vertices exists in the RSC. To do so, we injected AAV-CaMKII-GCaMP6s into the granular layers of RSC and recorded Ca2+ signals from a total of 740 GCaMP6s-expressing excitatory neurons using a miniaturized microscope in nine mice (Fig. 1a, b). From the ΔF/Fo Ca2+ signals acquired from the granular RSC neurons, we extracted deconvoluted spikes (Fig. 1b) to plot neural activities while mice freely explored open chambers of different environmental geometries. We used a circular open chamber (radius of 25 cm) and three different shapes of regular polygon open chambers that had distinct numbers of vertices. The dimensions of regular polygon-shaped open chambers were chosen so that they can be inscribed into the circular open chamber, so as to keep the center-to-vertex distance constant (25 cm) (Fig. 1c). In a circular open chamber, the granular RSC neuronal spikes tessellated along the entire circumference (Fig. 1c, d), which is similar to RSC neuronal responses previously reported in rats34. However, in regular polygon-shaped open chambers, subpopulations of granular RSC neurons spiked exclusively near the vertices (Fig. 1c, d, Supplementary Movie 1). When the regular polygon-shaped open chambers were morphed from a square to a 24-gon using 8 cm-wide walls while keeping the total perimeter constant across chambers, vertex-selective spiking was still observed in the square, hexagon, and octagon open chambers, but vertex-selective spiking diminished in the 12-gon and 24-gons (Supplementary Fig. 1), indicating there may be a spatial resolution (edge length > 16 cm) required in detecting vertices of the environmental geometry. The spatial patterns of vertex-selective RSC neuronal receptive fields (Fig. 1c, d) were different from grid cells, as all RSC neurons had low grid scores4 in a square chamber (Supplementary Fig. 2). Additionally, they were distinctively different from boundary vector cells7 or border cells8 as subpopulations of RSC neurons fired exclusively near the vertex located farthest away from the center, as shown in the population responses of RSC neurons as a function of distance from vertex (Fig. 1e) and at either end of the edge (Fig. 1f). To quantitatively analyze the vertex-selective neuronal characteristics, we defined the vertex score. The vertex score was calculated by taking the distance from the spatial bin with the maximum spike firing rate within the spatial receptive field to the nearest vertex, which was then normalized by the largest possible distance to the vertex in a given chamber. The resulting vertex scores ranged from 0 to 1, where 0 indicated that the spatial receptive fields were located farthest away from all vertices, and 1 indicated that the spatial receptive fields were located exclusively at any of the vertices. A given RSC neuron was defined as a vertex cell within each chamber if the vertex score exceeded the 95th percentile of the probability distribution of randomly shuffled vertex scores in any of the regular polygon-shaped open chambers (Fig. 1g), and if it showed stable spatial receptive fields during the entire 30-min recording sessions. Based on vertex score analysis, in each chamber, 32 %, 32 %, and 33 % of RSC neurons (234, 234, and 247 out of 740 neurons) were vertex cells in triangle, square, and hexagon chambers, respectively (Fig. 1g) (Supplementary Fig. 3), while 9 % of RSC neurons (66 out of 740 neurons) sustained vertex cell responses in all three different open chambers (Fig. 1h, left). Overall, across the three open chambers, granular RSC neurons that are defined as vertex cells in any one of the chambers were 60 % (443 out of 740 neurons), with similar ratio of vertex cells observed in each animal (Fig. 1h, right). Among the imaged granular RSC neuronal populations, a high proportion of granular RSC neurons were vertex cells that spiked exclusively near the vertices while being inhibited at the center of the open chamber as well as the edge (Fig. 1i). However, subpopulations of non-vertex cells in the granular RSC were found to be border cells (27%, 200 out of 740 neurons, Supplementary Fig. 4a–c) while subpopulations of vertex cells showed conjunctive responses to both vertices and borders (16%, 122 out of 740 neurons, Supplementary Fig. 5).

Fig. 1: Neural representation of geometric vertices in the retrosplenial cortex.
figure 1

a Schematic of AAV-CaMKII-GCaMP6s injection, GRIN lens implantation, and miniaturized microscope for Ca2+ imaging in the granular retrosplenial cortex (RSC, left). Fluorescent image of GRIN lens track (middle) with GCaMP6s-expressing neurons (right). b Example field of view (FOV) of Ca2+ imaging (left). Representative raw Ca2+ signals and deconvoluted spikes extraction (right). c, d Representative spike-trajectory plot (c, mouse trajectory (gray line), spike locations (red dots)) and the firing rate maps (d, inset: maximum firing rate) of a single RSC neuron during free exploration in circular, triangle, square, and hexagon open chambers. e, f Firing rate map as a function of distance from a given edge or vertex (e, two-sided paired Student’s t test) and position along a given edge (f, two-sided repeated measures ANOVA with post hoc Tukey’s test), ordered by the mean firing rates at the center or edge center (n = 740 neurons in 9 mice). Black line: Mean firing rate. g Probability distributions of vertex scores in three open chambers (colored) and the randomly shuffled vertex scores (gray). Red line: 95th percentile of randomly shuffled vertex score distribution. Inset: Ratio of vertex cells and non-vertex cells in each open chamber. h Venn diagram, pie chart, and bar graph showing vertex cell ratio across three open chambers (left) and in each mouse (right). i Mean firing rate of vertex cells (left) and non-vertex cells (right) along the distance from vertex (two-sided two-way ANOVA with post hoc Tukey’s test). Solid lines: mean firing rate. Shade: SEM. j–l Violin plot of vertex fields number (j), vertex fields distance from vertex (k), and the vertex field size (l) (two-sided one-way ANOVA with post hoc Tukey’s test; (j): F(2, 712) = 17.33, p < 0.001, (k): F(2, 712) = 0.81, p = 0.44, (l): F(2, 712) = 1.99, p = 0.14). Box plot (j–l) shows 25th (lower box line), 50th (middle line), and 75th (upper box line) percentile values. Firing rates (d, e) are color-coded. ***p < 0.001, n.s.: p > 0.05. Source data are provided as a Source Data file.

Full size image

Analyses of spatial receptive fields of vertex cells, termed vertex field henceforth, revealed that they significantly increased in number as the number of environmental vertices within the chamber increased (Fig. 1j). Vertex field locations were ~5 cm proximal to the vertex (Fig. 1k), and vertex field sizes were similarly small (mean: ~80 cm2) across the chambers (Fig. 1l). Thus, vertex fields covered a small fraction of the edge exclusively near the vertex, which contrasts with border cells (Supplementary Fig. 4). Vertex cells did not show any changes in spike firing rates across different environmental geometries (Supplementary Fig. 6) and even at different vertices within the same context (Supplementary Fig. 6). Also, vertex cell responses were independent of whether the vertex angle was convex (<180o) or concave (> 180o) (Supplementary Fig. 7a–d), indicating that vertex cell responses are independent of context and angle of the vertex. Also, vertex cells’ spatial receptive field properties were robust over time (Supplementary Fig. 8), in different sizes of open chambers (Supplementary Fig. 9), in dark conditions (Supplementary Fig. 10), and with visual cue rotation (Supplementary Fig. 11). Behavioral analyses revealed that mice behaviors were similar along the walls of polygon open chambers or of the circular chamber (Supplementary Fig. 12), consistent with a previous report35. Even when spikes evoked during rearing, sniffing, and grooming behaviors35,36 were excluded, vertex-selective spiking was still preserved (Supplementary Fig. 12), suggesting that mice behavior did not contribute to vertex-selective firing per se. Moreover, there was little correlation between angular velocity and spiking firings of vertex cells (Supplementary Fig. 12). In addition to deconvoluted spikes, other features of Ca2+ data (Ca2+ events and raw Ca2+ signals) could also reliably capture the vertex cell responses as well as the vertex cell ratio within the population (Supplementary Fig. 13). Together, these results show that neurons in the granular RSC provide a robust neural representation of geometric vertices using spike firing rate that remains stable over time and across different geometric environments.

Egocentric vector coding of vertex by granular RSC neurons

Next, to examine the spatial reference frame of vertex cells in granular RSC, we analyzed the tuning of vertex cell’s spikes in either allocentric head direction (Fig. 2a, b) or egocentric bearing relative to the vertex (Fig. 2a, c). Compared with head direction cells in the granular RSC that showed uni-directional allocentric head direction tuning (Supplementary Fig. 14), a given vertex cell’s spikes showed discretization of allocentric head direction tuning at each vertex (Fig. 2d), suggesting a non-allocentric spatial reference frame. When we calculated the egocentric bearing of vertex cell’s spikes as the angle of the mouse head direction relative to the closest vertex, the tuning curves of egocentric bearing were similar across all vertices (Fig. 2d). Thus, vertex cells were tuned to specific egocentric bearing relative to the vertex, indicating that they use an egocentric spatial reference frame. The egocentric reference frame of vertex cells was further characterized by determining the egocentric vertex vector using the egocentric bearing and egocentric distance of spikes12,13,14 relative to the closest vertex (Fig. 2e). This egocentric vertex vector was then used to plot an egocentric vertex rate map (Fig. 2f), from which the vector length (VL) of the egocentric vertex vector was analyzed to quantify the strength of egocentric vector coding of vertex cells. Using these analyses, a given vertex cell was classified as an egocentric vector coding vertex cell, termed egocentric vertex cell (EVC), if the VLs exceeded the 95th percentile of the probability distribution of randomly shuffled VLs within each open chamber (Fig. 2g) (Supplementary Fig. 15). In triangle, square, and hexagon open chambers, 63 % (148 out of 234 vertex cells), 60 % (140 out of 234 vertex cells), and 59 % of vertex cells (145 out of 247 vertex cells) were defined as EVCs, respectively (Fig. 2g). Overall, across the chambers, 66% of vertex cells were EVCs (Fig. 2h, left) and the proportion of EVCs in each animal was similar (Fig. 2h, right). The distribution of preferred bearing (θego) and distance (Dego) were statistically similar across the three open chambers (Fig. 2i). The preferred θego was distributed between −180o to 180o, covering the anterior as well as lateral angles (Fig. 2i, left) and Dego was distributed dominantly <10 cm (Fig. 2i, right), which is proximal to the vertex. Preferred θego and Dego were sustained across chambers (Fig. 2j) with high correlations (Supplementary Fig. 16a–b). On the other hand, VLs of EVCs had no correlations with neither θego nor Dego (Supplementary Fig. 16c) and there was no correlation between θego and Dego in each chamber either (Supplementary Fig. 16d), indicating that closeness or bearing of the animal relative to the vertex was independent of egocentric vertex vector itself. The preferred θego and Dego of EVCs sustained over time, in different sizes of open chambers, in dark condition, with visual cue rotation (Supplementary Fig. 17), and at concave and convex angles in the concave polygon-shaped open chamber (Supplementary Fig. 7e-h). Also, sniffing, rearing, and grooming behaviors had no effect on the egocentric vector coding of the vertex (Supplementary Fig. 18). Together, these results indicate that EVCs perform robust and stable egocentric vector coding of the vertex across different environmental geometries.

Fig. 2: Egocentric vector coding of geometric vertices in the retrosplenial cortex.
figure 2

a–c Illustration of the mouse allocentric head direction (θallo), egocentric bearing (θego), and egocentric distance (Dego) relative to the closest vertex from the current location (a), θallo (b) and θego (c). d Representative spike-trajectory plot (mouse trajectory (gray line), spike locations (colored dots)) of a single RSC vertex cell during free exploration in triangle, square, and hexagon open chambers. Each spike is color-coded for θallo (inset, North: N, East: E, South: S, and West: W). Tuning curves of θallo and θego are color-coded for the closest vertex (Vi: ith vertex). e Illustration of an egocentric vertex vector plot. f The egocentric vertex rate map of the same vertex cell in (d). Firing rate is color-coded. Insets: mean resultant vector length (VL), preferred θego, preferred Dego, and maximum firing rate. g Probability distributions of VLs of all vertex cells (triangle: n = 234, square: n = 234, and hexagon: n = 247 in 9 mice) in each three open chambers (colored) and the randomly shuffled VLs (gray). Red line: 95th percentile of randomly shuffled VLs distribution. Inset: ratio of egocentric vertex cells (EVCs) and non-EVCs in each open chamber. h EVC ratio across three open chambers (left) and in each mouse (right). i Probability distribution of preferred θego (left, two-sided Watson–Williams multi-sample tests) and Dego (right, two-sided one-way ANOVA) of EVCs in each open chamber. j Box plot showing differences in preferred θego (left, two-sided Watson–Williams tests; triangle–square: F(1, 122) = 0.002, p = 0.96, triangle–hexagon: F(1, 108) = 0.002, p = 0.96, square–hexagon: F(1, 110) = 0.04, p = 0.84) and Dego (right, two-sided repeated measures ANOVA; triangle–square: F(1, 61) = 0.31, p = 0.58, triangle–hexagon: F(1, 54) = 0.28, p = 0.60, square–hexagon: F(1, 55) = 1.14, p = 0.29) of EVCs between two chambers. Box plot (j) shows 25th (lower box line), 50th (middle line), 75th (upper box line) percentile values and minimum and maximum values (whiskers). n.s.: p > 0.05. Source data are provided as a Source Data file.

Full size image

Egocentric vector coding of vertex requires physical boundaries

As vertices are formed at the intersections of boundaries that define the geometry of an environment, the necessity of environmental boundaries in the emergence of EVCs was investigated by removing the physical boundaries (Fig. 3a). Analysis of vertex-selective firing and egocentric vertex vector of EVCs revealed that spike firing rate (Fig. 3b) and vertex score (Fig. 3c) significantly decreased in the absence of physical boundaries. Moreover, VL significantly decreased (Fig. 3d), contributed by significant changes in preferred Dego and θego (Fig. 3e–f), which led to the weakening of egocentric vector coding of vertex when physical boundaries were removed. However, when the physical boundaries were re-introduced, egocentric vector coding of vertices was restored (Fig. 3a–f ), as shown by the increase in firing rates at vertices (Fig. 3b), vertex scores (Fig. 3c), VLs in egocentric vertex rate map (Fig. 3a, d), and preferred Dego and θego (Fig. 3e, f), underscoring the requirement of physical boundaries in adaptive egocentric vertex vector coding of environmental geometry. A simple in silico computational model of RSC neuron receiving subthreshold synaptic inputs carrying boundary information could simulate vertex-selective firing characteristics observed in both the convex (triangle, square, and hexagon) and concave polygon open chambers (Supplementary Fig. 19). Also, this in silico model could mimic the angle-independent spike firing rates of vertex cells (Supplementary Fig. 19) that is observed from the in vivo data in convex polygons (Supplementary Fig. 6) and in concave polygons (Supplementary Fig. 7). Thus, integration of boundary vector cell inputs conveying environmental boundaries could be one of the possible input sources in the emergence of vertex cell responses for the coding of both concave and convex vertices.

Fig. 3: Requirement of environmental physical boundaries for egocentric vector coding of vertices in the retrosplenial cortex.
figure 3

a Representative spike-trajectory plot (top, mouse trajectory (gray line), spike locations (colored dots)) and the corresponding firing rate maps (middle) of a single egocentric vertex cell (EVC) in the granular retrosplenial cortex during free exploration in a square open chamber with (Wall), without (No wall), and with re-introduction of physical boundaries (Wall’). Each spike is color-coded for allocentric head direction (θallo) (a). Firing rate is color-coded (inset: maximum firing rate). The corresponding egocentric vertex rate map (bottom) of the same EVC. Firing rate is color-coded. Insets: mean resultant vector length (VL), preferred egocentric bearing (θego), preferred egocentric distance (Dego), and maximum firing rate. b Mean firing rate of EVCs (n = 70 EVCs in 5 mice) as a function of distance from a given vertex in each chamber (two-sided two-way ANOVA with post hoc Tukey’s test; F(28, 6003) = 23.26, p < 0.001, interaction effect: F(56, 6003) = 5.72, p < 0.001). Solid lines: mean firing rate. Shade: SEM. c, d Violin plot of EVCs’ vertex scores (c, two-sided repeated measures ANOVA with post hoc Tukey’s test; F(2, 138) = 10.44, p < 0.001) and mean VLs in each open chamber (d, two-sided repeated measures ANOVA with post hoc Tukey’s test; F(2, 138) = 30.22, p < 0.001). e Probability distribution of preferred Dego (left) and θego (right) of EVCs in each open chamber. f Box plot of differences in preferred Dego (left, two-sided repeated measures ANOVA with post hoc Tukey’s test; F(2, 138) = 62.91, p < 0.001) and θego (right, two-sided Watson–Williams multi-sample tests; F(2, 207) = 7.17, p < 0.001). Box plot (c, d, f) shows 25th (lower box line), 50th (middle line), and 75th (upper box line) percentile values. Whiskers in box plot (f) show minimum and maximum values. ***p < 0.001, *p < 0.05, n.s.: p > 0.05. Source data are provided as a Source Data file.

Full size image

Goal-directed navigation strengthens egocentric vector coding of vertex at goal location

RSC has been proposed to be important in goal-directed navigation19,21,22,23,24,25,26 using egocentric cognitive map2. To directly test EVC’s contribution to goal-directed navigation, we designed a goal-directed navigation task in a complex environment consisting of a square open chamber with three connecting inner walls (Fig. 4a), in which mice were trained to learn the goal location (top-left vertex, V1) where a reward was given (Fig. 4a, b). We monitored the EVCs responses during free exploration in the complex environment before and after the goal location learning. Before goal-directed navigation, EVCs responded to both the vertices of the square open chamber and the inner walls (Fig. 4c) with the same preferred egocentric bearing and distance across the vertices (Supplementary Fig. 20). However, after learning the goal location, spike firing rate (Fig. 4d, e) and VLs of the same EVCs significantly increased exclusively at V1 compared to before learning (Fig. 4d, f, g). Moreover, after goal location learning, EVC’s preferred egocentric distance at V1 significantly increased while egocentric bearings were sustained (Fig. 4h). The goal location learning-induced changes in egocentric vector coding of the vertex was independent of the mice occupancy time and mice behaviors at the goal location (Supplementary Fig. 21). Overall, these results indicate that goal location learning induces strengthening of egocentric vector coding of vertices in the complex environment to guide efficient goal-directed navigation.

Fig. 4: Strengthening of egocentric vector coding of geometric vertices in the retrosplenial cortex by goal-directed navigation.
figure 4

a Goal-directed navigation task design (top) and the representative mouse trajectories (gray line) in a square open chamber with a triangle-shaped inner walls (Vi: ith vertex) during pre-training, training, and post-training sessions (bottom). b Averaged trajectory distance (left, two-sided repeated measures ANOVA with post hoc Tukey’s test; F(3, 18) = 16.15, p < 0.001) and latency (right, two-sided repeated measures ANOVA with post hoc Tukey’s test; F(3, 18) = 11.25, p < 0.001) from start position to goal location during training sessions (n = 7 mice). Error bar: SEM. c, d Representative spike-trajectory plot (left). Mouse trajectory (gray line) superimposed with spike locations (colored dots) and the corresponding firing rate maps (middle) of a single egocentric vertex cell (EVC) during free exploration before (c, Pre-training) and after (d, Post-training) training sessions. Each spike is color-coded for allocentric head direction (θallo) (inset, North: N, East: E, South: S, and West: W). Firing rate is color-coded (inset: maximum firing rate). The egocentric vertex rate map (right) of the same EVC at the vertex near the goal location (V1, white line in firing rate map). Firing rate is color-coded. Insets: mean resultant vector length (VL), preferred egocentric bearing (θego), preferred egocentric distance (Dego), and maximum firing rate. (e-g) Mean firing rate (e, two-sided two-way ANOVA with post hoc Tukey’s test; interaction effect: F(6, 1974) = 4.86, p < 0.001), VL map (f), and mean VL (g, two-sided two-way ANOVA with post hoc Tukey’s test; interaction effect: F(6, 1974) = 13.66, p < 0.001) of all EVCs (n = 142 EVCs in 7 mice) at each vertex in each condition. VL in (f) is color-coded. h Probability distribution of preferred Dego (left, two-sided paired Student’s t test) and θego (right, two-sided Wilcoxon signed-rank test) of EVC at V1 in each condition. Box plot (e, g) shows 25th (lower box line), 50th (middle line), and 75th (upper box line) percentile values. ***p < 0.001, n.s.: p > 0.05. Source data are provided as a Source Data file.

Full size image

Discussion

In our study, we identified a type of spatially tuned neuron in the granular RSC that represents geometric vertices of the environmental features in an egocentric spatial reference frame (Figs. 1, 2). EVCs required physical boundaries (Fig. 3), and goal location learning through goal-directed navigation selectively strengthened the egocentric vector coding at the nearest vertex to the goal location (Fig. 4). Together, these results suggest that EVCs identified in our study may help construct an egocentric cognitive map of the vertex that could guide goal-directed navigation.

In the granular RSC, subpopulations of neurons that showed vertex-selective spike firing could be defined as vertex cells (Fig. 1). They had distinctly small spatial receptive field sizes (~80 cm2) (Fig. 1) compared to other spatial selective neurons4,8,10, and generated multiple spatial receptive fields (Fig. 1). Additionally, the sizes and locations of vertex cell’s spatial receptive fields were robust across different environmental geometries (Fig. 1), different from other spatial navigation neurons that exhibit spatial receptive field remapping20,37,38. This suggests that vertex cells are ideal for providing a stable and robust one-to-one mapping of vertices present in the environmental features using spike firing rates (Fig. 1). The vertex-selective firing was maintained in open chambers of different sizes (Fig. 1, Supplementary Fig. 9), but when the edge length was <16 cm in the 12-gon and the 24-gon open chambers, the vertex-selective firing was diminished (Supplementary Fig. 1), indicating that there may be a fine spatial resolution required for encoding vertex in spikes, which will be interesting to further investigate in the future.

Overall, 60% of granular RSC neurons imaged in our study had vertex-selective responses in any one of the open chambers (Fig. 1), among which 66% of them were egocentrically tuned (Fig. 2), resulting in overall 39% of the total imaged granular RSC neurons (291 neurons out of 740 neurons) being EVCs. It has been reported that 24.1% of granular and dysgranular RSC neurons were egocentric boundary cells12 while in another study, 36% of dysgranular RSC neurons were found to be egocentric boundary cells39. Thus, the population of EVC in our study is comparable to the population of egocentrically tuned neurons found in the RSC. However, here we found that egocentric boundary cell population in the granular RSC was 18 % (Supplementary Fig. 4d–f), which is smaller than those have been reported in dysgranular RSC39. Such discrepancy may suggest that granular and dysgranular RSC may have distinct roles in coding vertex and boundaries, respectively, which needs further confirmation in future studies. Interestingly, 16 % of granular RSC neurons were conjunctive vertex/border cells (Supplementary Fig. 5), thus, it seems that the majority of granular RSC neurons are encoding edges and vertices of the environment. Such conjunctive coding of vertex and boundaries may enhance the distinction of environmental geometry such as square and circular environment or even help construct route-centric map40 as has been suggested by previous study12,34.

Analysis of the spatial reference frame of vertex cells revealed that they encode the vertices using egocentric vertex vector (Fig. 2). VL, θego, and Dego of EVCs were preserved across different environmental geometries with different contexts (Fig. 2j, Supplementary Fig. 16) as well as at different environmental manipulations (Supplementary Fig. 17), indicating that EVCs provide stable representations and detections of environmental vertices. However, distinctly different from subicular corner cells33, RSC vertex cells did not differentiate between different convex angles in different contexts (Supplementary Fig. 6) nor did they differentiate concave and convex angles in the same context (Supplementary Fig. 7). As egocentric boundary cells fire differently at corners enough to help distinguish square and circular environments34 and as RSC is also reciprocally connected with subiculum30 where corner cells are found33, it will be interesting to further dissect experimentally how RSC EVCs, RSC egocentric boundary cells, and subicular corner cells together integrate spatial information to code the geometry of the environment.

In the absence of physical boundaries, EVCs lost their sensitivity to vertex as well as egocentric tuning (Fig. 3), supporting the requirement of physical boundaries in EVC generations. Interestingly, θego was decreased after the wall removal (Fig. 4f, right), indicating that θego was rotated anticlock-wise. Although further experiments are required, as all Ca2+ imaging was performed on the left hemisphere of the granular RSC in this study, the decrease in contralateral whisker stimulation input detecting the boundaries may have contributed to the change in anticlock-wise rotation in θego. Indeed, our simple in silico model demonstrated that subthreshold synaptic inputs carrying boundary vector information27 are sufficient to simulate vertex cell responses (Supplementary Fig. 19). Thus, somatosensory input-triggered boundary information may be one possible source of input to vertex cell responses. Interestingly, our in silico model could also simulate vertex-selective spiking at both convex and concave vertices (Supplementary Fig. 19), indicating that boundary vector/border cell inputs to vertex cells could serve as an input candidate. However, further experiments are required to fully underpin how vertex cells, unlike subicular corner cells33, show similar responses to both convex and concave angles in terms of spike firing rate (Supplementary Fig. 7) and egocentric vector coding (Supplementary Fig. 7). Interestingly, there were subpopulations of EVCs that selectively responded to either convex angles of the square open chambers or the concave angles of the triangular inner walls (Supplementary Fig. 20). Thus, while RSC neurons may not be able to differentiate concave and convex corners by spike firing rates and egocentric vertex vectors, population coding may be used in differentiating concave and convex angles, which may need further investigation. Also, as granular RSC neurons receive direct projections from multiple neural circuits that process spatial information27,30,31,32, visual information19, and egocentric boundary/border information12,14, neural circuit dissections of these inputs in future studies may help elucidate how EVC responses arise. Such investigations could also shed light on how EVCs relate to the known functions of RSC including goal-directed navigation19,21,22,23,24,25,26, visual cues/landmarks processing41,42,43, episodic spatial memory27, and motor planning19,27.

Egocentric vector coding of vertices was strengthened selectively at the vertex where the goal was located after goal-directed navigation (Fig. 4), which is in line with RSC’s role in goal-directed navigation19,21,22,23,24,25,26. Especially, it is interesting to note that the Dego and spike firing rates were selectively increased at the goal location after learning even in the absence of the reward (Fig. 4d–h) without changes in behavior at the goal location (Supplementary Fig. 21). Thus, the increased spike firing at the reward location may serve as a reward prediction signal and the increased Dego may help compute the egocentric goal location further away from the vertex location, which may both effectively contribute to the efficient goal location coding in the egocentric reference frame. These results are in line with previously reported roles of RSC neurons such as encoding of goal location22,23 and value discrimination44. As RSC receive dopaminergic inputs from VTA45, it will be interesting to investigate how value and reward signals are integrated into the changes in the egocentric vector coding of a vertex in the future. In fact, such strengthening of egocentric vector-coding of goal location could provide egocentric vectorial information of the goal location to hippocampal place cells or neurons in the lateral entorhinal cortex, so as to help them construct the goal-oriented vectorial map11,46,47,48 and even perform goal-related reorganization of allocentric49,50,51,52,53,54,55 and egocentric cognitive maps11. Considering that granular RSC output is well known to project to the motor cortex19, EVCs may also help fine-tune motor output that guides movements toward the goal location.

The main aim of this study was to characterize the neurons that show vertex-selective spiking that generate vertex field proximal to the vertex and identify their spatial reference frame. Thus, we limited the egocentric vertex vector analysis in Fig. 2 to neurons that were classified as vertex cells with above-threshold vertex score in Fig. 1. However, similar to egocentric boundary cells in the RSC12,13, there were subpopulations of neurons that are egocentrically tuned to vertices distally with Dego > 10 cm when extending the egocentric vertex vector analysis to all imaged neurons (Supplementary Fig. 22). Thus, EVCs tuning could be both proximal and distal, indicating that vector coding of vertices is not solely dependent on physical interaction with physical boundaries. As whisker inputs arise dominantly at the proximal distances from the vertices, somatosensory inputs could be critical for EVC responses with proximal vertex field while distal egocentric vector coding cells may use other sensory modalities depending on the distance to the vertex. Thus, dissociating the neural circuit mechanisms of proximal and distal vector coding of vertex will help better understand the functional roles of EVCs. Moreover, how the proximal and distal vector coding of vertices change in the absence of walls in Fig. 3 and during goal-directed navigation in Fig. 4 will be required to understand how the egocentric cognitive map dynamically modifies to adapt to the changes in the environmental features.

Despite these results, there are many limitations to our study. Firstly, we based our results on Ca2+ imaging data, which has lower temporal resolution than spike recordings. Moreover, some corner-selective neurons tuned by turning direction20 and internal states56 have been reported previously. Therefore, careful dissociation of self-motion and internal states in vertex cell analysis with in vivo-recorded spikes may be needed in the future. Nevertheless, EVCs identified here provide insight into the egocentric neural coding of geometric features in the external environment. These neurons could serve pivotal functions in constructing a geometrically detailed and spatial memory-based egocentric cognitive map that is updated continually in changing environment, which in turn could help guide adaptive goal-directed navigation and process episodic spatial memory in the complex environment.

Methods

Animals

A total of nine C57BL/6 mice (3–6 months old; DBL, South Korea) were used for in vivo Ca2+ imaging during free exploration with the approval of Institutional Animal Care and Use Committee (IACUC) at Korea University (KUIACUC-2020-0099, KUIACUC-2021-0052). All mice were individually housed in a temperature- and humidity-controlled vivarium, which was kept in a 12 h/12 h light/dark cycle. All experiments were performed during the dark cycle. Food and water were available ad libitum unless it is mentioned otherwise.

Virus and stereotaxic surgery

For surgical procedure, mice were deeply anesthetized using 2% isoflurane (2 ml/min flow rate) and head-fixed into a stereotaxic frame (51730D, Stoelting Co., USA). AAV9-CaMKII-GCaMP6s57 (#107790, Addgene, USA) in solution (500 nl per injection site, 2.5 ×1013 virus molecules/ml diluted with saline at 3:1 ratio) was injected into three sites along the dorsoventral axis of the retrosplenial cortex (RSC) (AP: –2.5 mm, ML: –0.3 mm, DV: –0.8, –0.6, and –0.4 mm, Fig. 1a) using a Hamilton syringe (#87930, Hamilton, USA) controlled by an automated stereotaxic injector (100 nl/min; #53311, Stoelting Quintessential Injector, Stoelting Co., USA). The syringe was left at the injection coordinates for more than 5 min to allow viral diffusion.

After viral injection, a Gradient-index (GRIN) lens (1 mm diameter, 4 mm length; #1050-004605, Inscopix, USA) attached to a stereotaxic was slowly lowered into the RSC (AP: –2.5 mm; ML: –0.3 mm; DV: –0.5 mm), after which it was fixed to the skull with dental cement (Self Curing, Vertex, Netherlands) for the lens implant. After two weeks of recovery time following the surgery, mice were deeply anesthetized using 2% isoflurane (2 ml/min flow rate) to attach a magnetic baseplate (#1050-004638, Inscopix, USA), on top of which miniaturized microscope (nVoke, Inscopix, USA) was mounted. If GCaMP6s-expressing neurons were visible in the field of view (FOV), baseplate was fixed to the skull using dental adhesive resin cement (Super bond, Sun Medical, Japan) to perform in vivo longitudinal Ca2+ imaging of the same FOV. After baseplanting, at least one week of recovery time was allowed before performing in vivo Ca2+ imaging.

Behavior and chamber design

Prior to in vivo Ca2+ imaging during free exploration, mice were adapted to the head-mounted miniaturized microscope connected to the commutator systems (Inscopix, USA) for at least five days. All experiments were performed under the luminescence of 30 Lux unless it is mentioned otherwise.

In Figs. 1, 2, each mouse was allowed to freely explore four different shapes of open chambers with different geometry: circular open chamber (radius: 25 cm, height: 30 cm), a regular triangle open chamber (edge: 43.3 cm, height: 30 cm), a square open chamber (edge: 35.3 cm, height: 30 cm), and a regular hexagon open chamber (edge: 25 cm, height: 30 cm). The dimensions of regular polygon-shaped open chambers were designed to be inscribed into a circular open chamber of 25 cm-radius to keep the center-to-vertex distance in each regular polygon-shaped open chamber constant (25 cm). In Fig. 3, the square open chamber (edge: 40 cm, height: 30 cm) was inverted to create an arena with no physical boundaries.

In Fig. 4 and in Supplementary Figs. 20, 21, goal-directed navigation task was performed. For this, mice were food-restricted until they reached 80–85% of their free-feeding weight22. A day before (pre-training session) the goal-directed navigation training started, each mouse was allowed to freely explore a complex environment consisting of a square open chamber (edge: 60 cm, height: 30 cm) with three walls inserted within the chamber that made a regular triangle-shape (edge: 20 cm, height: 20 cm) for 30 min. Goal-directed navigation training was performed over four days where each mouse was trained to forage in the chamber which had a food reward (Froot Loops, Kellogg’s, USA) located close to the top-left vertex (V1) of the square chamber (Fig. 4A). Each training began by placing the mouse at the start position and ended when the mouse reached the goal location and consumed the Froot Loops. The training was repeated 10 times per day. A day after the end of goal-directed navigation training (post-training session), each mouse was allowed to freely explore the same chamber for 30 min in the complex environment without a reward/goal. No Froot Loops were presented during neither pre-training nor post-training sessions.

In Supplementary Fig. 1, to keep the total perimeter constant across different shapes of environmental geometries, five different regular polygon-shaped open chambers were configured using twenty four identical pieces of rectangular foam board (width: 8 cm, height: 30 cm, total perimeter 192 cm). The shapes of the open chambers were continuously transformed or “morphed” from a square (edge: 48 cm) to hexagon (edge: 32 cm), octagon (edge: 24 cm), 12-gon (edge: 16 cm), and 24-gon open chamber (edge: 8 cm) in a series. In Supplementary Fig. 7, a regular concave open chamber (edge: 25 cm, height: 30 cm) with three different vertex angles (60o, 120 o, and 240 o) was used. In Supplementary Fig. 8 and 17, a square open chamber (edge: 35.3 cm, height: 30 cm) was used. In Supplementary Figs. 9, 17, two different sizes of square open chambers were used: one with 35.3 cm edge and the other with 60 cm edge, both with 30 cm height. In Supplementary Fig. 10 and 17, a square open chamber (edge: 35.3 cm, height: 30 cm) was used in the presence (30 Lux) or absence of visible light (0 Lx). In Supplementary Fig. 11 and 17, a square open chamber (edge: 35.3 cm, height: 30 cm) with circular and striped patterned visual cues (10 cm × 10 cm for each) was used. Visual cues were attached to one side of the chamber first (Supplementary Fig. 11a, middle, 17 h), which was then rotated by 90o (Supplementary Fig. 11a, right, 17i).

Each mouse was allowed to freely explore each open chamber or arena for 30 min, after which each open chamber was cleaned using 70% ethanol to eliminate any odors. Froot Loops were randomly scattered on the floor of the open chambers to facilitate the full coverage of behavior trajectory on the testing arena. Mouse behavior and trajectories were recorded by a video camera placed on the ceiling above the chambers or arenas at a sampling rate of 7.5 Hz. All behavior data including (\({{{\rm{x}}}}\), \({{{\rm{y}}}}\)) coordinates, velocity, and head direction were analyzed using deep learning-based body points detection methods included in Ethovision XT 16 program (Noldus, Netherlands). The detected body points and head direction of the mouse through the behavior data analysis program were manually verified, and any incorrectly detected frames were corrected. The \({{{\rm{x}}}}\), \({{{\rm{y}}}}\) coordinates and velocity of the mouse were determined based on the body center point of the mouse. Head direction was measured based on the nose point and the body center point. Epochs of data with velocity slower than 2 cm/s were excluded from data analysis, which ensured analysis of data during active exploration.

In vivo Ca2+ imaging and signal processing

In vivo Ca2+ imaging data from GCaMP6s-expressig RSC neurons were acquired using nVoke acquisition software (Inscopix, USA) at a sampling rate of 20 Hz (exposure time: 50 ms). An optimal LED power (0.1–0.3 mW/mm2) and digital focus (0–1000) was selected for each mouse based on GCaMP6s expression on neurons in the FOV and the same parameters were used for each mouse for longitudinal imaging.

Ca2+ image processing was performed using Inscopix Data Processing Software (Inscopix, USA), following other studies58,59. For all data, motion correction was applied using an image registration method58,60. The mean fluorescence signal value (F) of each pixel across the entire Ca2+ image video was used to calculate the reference, termed Fo. Then, changes in the fluorescence signal value (△F) of each pixel were normalized by Fo to acquire △F/Fo. Putative neurons and their Ca2+ signals were isolated using an automated cell-segmentation algorithm on △F/Fo data, which is based on principal component analysis-independent component analysis58,61. Only putative neurons with a single spherical shape and Ca2+ dynamics with fast onset and slow decay were included for further analysis, following a method described in a previous study58.

Putative spikes were inferred from △F/Fo signal using spike deconvolution algorithm called “Online Active Set methods to Infer Spikes”59,62,63,64. Spikes with an amplitude less than 3 signals-to-noise ratio of deconvolved Ca2+ signals were removed from the data. Ca2+ transient events (Supplementary Fig. 13b–d) were defined using a Ca2+ event detection algorithm which detects the large amplitude peaks (3 standard deviation (sd) from △F/Fo signal) with slow decays (minimum: 0.2 s) by thresholding at the local maxima of the △F/Fo. For △F/Fo-based analysis (Supplementary Fig. 13b, i), sample data with the value of △F/Fo lower than 1.5 sd was excluded.

Histology

To confirm the position of GRIN lens track and GCaMP6s expression in RSC neurons after Ca2+ imaging, the brain was removed from the mice which were deeply anesthetized using Avertin (T48402, Sigma-Aldrich, USA) and perfused with 10 ml chilled 4% paraformaldehyde (158127, Sigma-Aldrich, USA) transcardially. The brain was fixed overnight in 4% paraformaldehyde and transferred to 30% sucrose solution in phosphate-buffered saline (PBS) for cryoprotection for 24–48 h. Brains were frozen in Optimum Cutting Temperature (OCT) compounds (Tissue-Tek O.C.T. Compound, SAKURA, Japan) at –50 °C prior to cryosectioning in 50 μm thick coronal slices on a cryostat (YD-2235, Jinhua Yidi Medical Appliance Co., China). Slices were washed three times with PBS to wash out the OCT compounds and mounted on slide glasses with an antifade mounting medium with DAPI (Vectashield, Vector Laboratories, USA). Ca2+ imaging locations were identified based on the position of GRIN lens track. Expression of GCaMP6s in neurons was verified by detecting the fluorescent signal using a confocal microscope (LSM-700, ZEISS, Germany) and fluorescent microscope (DM2500, Leica, Germany).

Spatial receptive field and firing rate map analyses

Ca2+ imaging data from Inscopix and behavior data from Ethovision XT 16 were synchronized using Noldus-IO box (Mini USB-IO box, Noldus, Netherlands) system, which were then analyzed with custom-made Matlab (Mathworks, USA) code. To analyze the spatial receptive fields of GCaMP6s-expressing RSC neurons, positional occupancy of each mouse within an open chamber was discretized into 1.5 cm × 1.5 cm spatial bins. Within each spatial bin, the number of spikes evoked was divided by the total time spent within the bin to derive firing rate maps. Only spatial bins in which a mouse spent more than 133 ms (temporal resolution of behavior data) were used in spatial receptive field analysis. Firing rate maps were smoothed by applying boxcar average (5 × 5 bins) with a Gaussian smoothing kernel (σ = 1). A spatial receptive field was defined by detecting the adjacent spatial bins of at least 20 cm2 that showed firing rates which were above 30% of the maximum firing rate in the smoothed firing rate map, following the definition of spatial receptive fields in other spatial navigational neurons8,65 with modification of minimum spatial receptive field size criteria. To analyze spatial stability of RSC neuronal firing rate map over time (Figs. 1–4, Supplementary Fig. 8) and across different conditions (Supplementary Fig. 10, 11), spatial correlation coefficients (r) between firing rate maps were calculated using “Corr2” function in Matlab toolbox where r is defined as below:

$$r=\frac{\,{\sum}_{m}\,{\sum}_{n}\,\left({A}_{{mn}}-\bar{A}\right) * \left({B}_{{mn}}-\bar{B}\right)}{\sqrt{\left({\sum}_{m}\,{\sum}_{n}\,{\left({A}_{{mn}}-\bar{A}\right)}^{2}\right) * \,({\sum}_{m}\,{\sum}_{n}\,{\left({B}_{{mn}}-\bar{B}\right)}^{2})}}$$
(1)

where \(m\) and \(n\) are the number of spatial bins for horizontal and vertical axis, respectively, \({A}_{{mn}}\) and \({B}_{{mn}}\) are mean firing rates of two firing rate maps for (\(m\), \(n\)) coordinate spatial bin, respectively, and \(\bar{A}\) and \(\bar{B}\) are mean firing rates of two firing rate maps for all spatial bins, respectively. A neuron with r between the first half (0–15 min) and the second half (15–30 min) of each imaging session greater than the 95th percentile of randomly shuffled r distribution of all imaged RSC neurons was included in data analysis following the definition of spatial stability in other spatial navigation neurons8,12. Shuffled r distribution was generated by circularly time-shifting the spike trains of neurons with randomly selected period along the mouse trajectory with 500 repetitions following the method used in previous studies12,13. This random time-shuffling preserved the temporal structures of both Ca2+ signals and behavior data while disrupting the temporal correlation between them. Random time-shuffling was applied for all randomly shuffled distribution of other metrics.

Vertex score analysis

To define and characterize vertex-selective spatial receptive fields, vertex score was introduced. Vertex score was computed by measuring the distance from a single neuron’s peak firing location within spatial receptive fields to the nearest vertex, a similar approach used in defining the corner scores33 of subicular neurons that show corner-selective spatial receptive fields. Thus, vertex score (v) of a given neuron was calculated as follows:

$$v=1-d$$
(2)

where \(d\) is the distance from the (x, y) coordinate within the spatial receptive field with the peak firing rate to the nearest vertex, which was normalized by the largest possible distance to vertex in a given chamber. In this way, \(v\) value ranged between 0 to 1, where \(v\) = 0 indicated peak firing locations of all spatial receptive fields were located farthest away from all vertices and \(v\) = 1 indicated peak firing locations of all spatial receptive fields were located exclusively at any of the vertices. A neuron was defined as a vertex cell within each chamber if the vertex score for each chamber exceeded the 95th percentile of randomly shuffled vertex score distribution of all imaged RSC neurons.

Head direction tuning analysis

To analyze the head direction tuning of RSC neuronal spikes near each vertex (Fig. 2d), individual allocentric head direction tuning curves at each vertex were plotted by taking the normalized spike firing rate as a function of allocentric head direction (θallo). For each vertex, spikes that were located within the half-length of edge from the corresponding vertex (triangle: 21.65 cm, square: 17.65 cm, and hexagon: 12.50 cm) were used in the tuning curve analysis. In Supplementary Fig. 14, allocentric head direction tuning curves were plotted by taking the normalized firing rate of all spikes in the open chamber as a function of θallo. Allocentric head direction tuning curve was plotted using 15° bin and smoothed using a moving average filter.

Egocentric vertex vector analysis

To analyze the egocentric bearing of the mouse to vertex (θego), the angle of the closest vertex relative to θallo at each sample point of behavior data was computed. θego was set to 0°, 90°, –180°/180°, and –90°, when the closest vertex is in front, right, back, and left to mice, respectively. Egocentric distance to vertex (Dego) was calculated by computing the distance from mice to the closest vertex at each sample point of behavior data. In order to analyze the tuning of egocentric bearing (θego) of RSC neuronal spikes (Fig. 2d), individual egocentric bearing tuning curves at each vertex were plotted by taking the normalized firing rate of spikes as a function of θego. For each vertex, spikes that were located within the half-length of edge from the corresponding vertex (triangle: 21.65 cm, square: 17.65 cm, and hexagon: 12.50 cm) were used in the tuning curve analysis. The egocentric bearing tuning curve was plotted using a 15° bin and smoothed using a moving average filter.

To analyze RSC neuronal spike firing characteristics in terms of θego and Dego, the egocentric vertex rate map was analyzed (Figs. 2, 3, 4, Supplementary Fig. 15, 17, 18, 20, 21, 22). The egocentric vertex rate map was plotted by calculating the firing rate in terms of θego and Dego in a polar coordinate in polar plot. θego was divided into 15° bins and egocentric distance was divided into 1.5 cm bins, which yielded 15° × 1.5 cm bins. Firing rates for each bin were calculated for each neuron, by dividing the number of spikes within a given bin by the whole time spent in the bin. Only the bin in which mice spent more than 133 ms (temporal resolution of behavior data) was included in the data analysis. Egocentric vertex rate maps were smoothed using 2D Gaussian kernel σ = 0.7 pixels.

To quantify the stability of egocentric vertex rate maps of RSC neurons, correlation coefficients \(\left(r\right)\) between egocentric vertex rate maps were calculated using “Corr2” function in Matlab toolbox where r is defined as below:

$$r=\frac{\,{\sum}_{m}\,{\sum}_{n}\,\left({A^{\prime} }_{{mn}}-\bar{A^{\prime} }\right) * \left({B^{\prime} }_{{mn}}-\bar{B}^{\prime} \right)}{\sqrt{\left({\sum}_{m}\,{\sum}_{n}\,{\left({A^{\prime} }_{{mn}}-\bar{A}^{\prime} \right)}^{2}\right) * \,({\sum}_{m}\,{\sum}_{n}\,{\left({B^{\prime} }_{{mn}}-\bar{B}^{\prime} \right)}^{2})}}$$
(3)

where \(m\) and \(n\) are the number of degree and distance bins, respectively, \({A^{\prime} }_{{mn}}\) and \({B^{\prime} }_{{mn}}\) are mean firing rates of two egocentric vertex rate maps for (\(m\), \(n\)) coordinate degree × distance bin, respectively, and \(\bar{A}^{\prime}\) and \(\bar{B^{\prime} }\) are mean firing rates of two egocentric vertex rate maps for all degree × distance bins, respectively.

To quantify egocentric vertex vector sensitivity of RSC neurons (Figs. 2, 3, 4, Supplementary Fig. 15, 17, 18, 20, 21, 22), the mean resultant egocentric vertex vector was calculated by adopting the previous definition of egocentric boundary vector12,13,14 with slight modification as:

$${{{\rm{Egocentric}}}}\; {{{\rm{vertex}}}}-{{{\rm{vector}}}}=\left({{\sum}_{\theta=1}^{n}}{{\sum}_{D=1}^{m}}{F}_{\theta,D} * {e}^{i * \theta }\right)/(n * m * \bar{F})$$
(4)

where \(\theta\) indicates the egocentric bearing to vertex, \(D\) indicates the egocentric distance to vertex, \({F}_{\theta,D}\) indicates the firing rate in a given \(({{{\rm{egocentric\; bearing}}}} * {{{\rm{distance}}}})\) bin, \(n\) indicates the number of degree bins, \(m\) indicates the number of distance bins, \(e\) is the Euler constant, i is the imaginary constant, and \(\bar{F}\) indicates the mean firing rates of all bins.

The vector length of egocentric vertex vector was defined as the absolute value of the mean resultant egocentric vertex vector, which quantifies the strength of egocentric tuning to vertex. The preferred θego of the egocentric vertex vector was calculated as the mean resultant bearing (MRB):

$${{{\rm{MRB}}}}= \arctan 2({{{\rm{imag}}}}\left({{{\rm{egocentric}}}}\; {{{\rm{vertex}}}}-{{{\rm{vector}}}}\right)) / \\ {{{\rm{real}}}}({{{\rm{egocentric}}}}\; {{{\rm{vertex}}}}-{{{\rm{vector}}}})\left)\right.$$
(5)

Preferred Dego of egocentric vertex vector was estimated by fitting a Weibull distribution12 to the firing rates corresponding to the MRB and finding the distance bin with the maximum firing rate.

The vector length of egocentric vertex vector ranged from 0 to 1, where 0 indicated neurons with no egocentric vector coding of vertices and 1 indicated neurons with perfect egocentric vector coding of vertices. A neuron was defined as egocentric vertex cell (EVC) within each chamber if the egocentric vertex vector lengths for each chamber exceeded the 95th percentile of shuffled distribution of the vector length of all imaged neurons, and if the neuron’s correlation coefficient (r) of egocentric vertex vector map between the first half (0–15 min) and the second half (15–30 min) of each imaging session is greater than the 95th percentile of randomly shuffled r' distribution of all imaged RSC neurons.

Grid score analysis

To investigate whether spatial receptive fields of RSC neurons have grid cell-like characteristics, grid score was computed (Supplementary Fig. 2) following the same method described in other grid cell studies4,66. First, spatial autocorrelograms were calculated as:

$${{{\rm{r}}}}\left({\tau }_{x},{\tau }_{y}\right)=\frac{n {\sum } {\lambda }_{1}(x,y){\lambda }_{2}(x-{\tau }_{x},y-{\tau }_{y})-{\sum } {\lambda }_{1}(x,y) {\sum }{\lambda }_{2}(x-{\tau }_{x},y-{\tau }_{y})}{\sqrt{n {\sum } {{\lambda }_{1}(x,y)}^{2}({\sum }{{\lambda }_{1}(x,y)})^{2}}\sqrt{n{\sum }{{\lambda }_{2}(x-{\tau }_{x},y-{\tau }_{y})}^{2}({\sum }{{\lambda }_{2}(x-{\tau }_{x},y-{\tau }_{y})})^{2}}}$$
(6)

where \({{{\rm{r}}}}\left({\tau }_{x},{\tau }_{y}\,\right)\) is the autocorrelation between bins with spatial offset \({\tau }_{x}\) and \({\tau }_{y}\), \(\lambda\) is the firing rate map of a neuron with no smoothing applied, and \(n\) is the number of overlapping bins in two offset copies of the map. The autocorrelogram was smoothed using boxcar averaging of spatial bins (5 × 5 bins) using a Gaussian smoothing kernel with σ = 1. Grid score was calculated by repeatedly rotating the smoothed autocorrelation plot by 6o and the spatial correlation between the rotated autocorrelation plot and the original autocorrelation was calculated as the difference between minimum correlation coefficient for rotations of 60° and 120° and the maximum correlation coefficient for rotations of 30°, 90°, and 150°. A neuron was defined as grid cell if grid score for square open chamber exceeded the 95th percentile of randomly shuffled grid score distribution of all imaged neurons. Grid scores measured in triangle and hexagon open chambers were not used in defining grid cell to avoid the possibility that geometric shape of triangle and hexagon chambers could result in high grid scores of vertex-selective firing neurons, as presented in Supplementary Fig. 2.

Border score analysis

Border cell analysis was performed (Supplementary Figs. 4, 5) following the same method described in the previous study8. Putative border fields were defined first by detecting the adjacent spatial receptive field along the border with firing rate higher than 30% of the peak firing rate of the smoothed firing rate map. For all experiments, the maximum value of any single coverage of border field over any environmental border (c) was calculated by computing the fraction of spatial bins along the border which was occupied by the spatial receptive fields. The mean firing distance (e) indicates the mean distance of all spatial bins, containing receptive fields, to the nearest border, weighted by the normalized firing rate in the spatial bins. \({{{\rm{e}}}}\) was divided by the maximum distance to borders to obtain \({{{\rm{e}}}}\) value ranging from 0 to 1. A border score (b) for a neuron was defined by comparing \({c}\) and \({{{\rm{e}}}}\), adopted from the previous research8:

$$b=(c-{{{\rm{e}}}})/(c+{{{\rm{e}}}})$$
(7)

\(b\) ranged from –1 to 1, where \(b\) = –1 indicated no spatial receptive fields along any of the borders and \(b\) = 1 indicated all spatial receptive fields perfectly located along any of borders. A neuron was defined as border cell within each chamber if border score for each chamber exceeded the 95th percentile of randomly shuffled border score distribution of all imaged neurons.

Egocentric boundary vector analysis

Egocentric boundary vector analysis was performed following the previous described definition of egocentric boundary cell12,13,14. To analyze the egocentric bearing of the mouse to border (θ’ego), the angle of the closest border relative to θallo at each sample point of behavior data was computed. θ‘ego was set to 0o, 90o, –180/180o, and –90o, when the closest border is in front, right, back, and left to mice, respectively. Egocentric distance to border (D’ego) was calculated by computing the distance from mice to the closest border at each sample point of behavior data.

To analyze RSC neuronal spike firing characteristics in terms of θ’ego and D’ego, egocentric boundary rate map was analyzed (Supplementary Fig. 4d). Egocentric boundary rate map was plotted by calculating the firing rate in terms of θ’ego and D’ego in a polar coordinate in polar plot. θ‘ego was divided into 15° bins and egocentric distance was divided into 1.5 cm bins, which yielded 15° × 1.5 cm bins. Firing rates for each bin were calculated for each neuron, by dividing the number of spikes within a given bin by the whole time spent in the bin. Only the bin in which mice spent more than 133 ms (temporal resolution of behavior data) was included in data analysis. Egocentric boundary rate maps were smoothed using 2D Gaussian kernel σ = 0.7 pixels.

To quantify the egocentric boundary vector sensitivity of RSC neurons (Supplementary Fig. 4d), the mean resultant egocentric boundary vector was calculated by adopting the previous definition of egocentric boundary vector12,13,14 as

$${{{\rm{Egocentric}}}}\; {{{\rm{boundary}}}}-{{{\rm{vector}}}}=\left({{\sum}_{\theta {\prime}=1}^{n}}{{\sum}_{D{\prime}=1}^{m}}{F^{\prime} }_{\theta {\prime},D{\prime} } * {e}^{i * \theta {\prime} }\right)/(n * m * \bar{F^{\prime} })$$
(8)

where \(\theta {\prime}\) indicates the egocentric bearing to border, \(D{\prime}\) indicates the egocentric distance to border, \({F^{\prime} }_{\theta {\prime},D{\prime} }\) indicates the firing rate in a given \(({{{\rm{egocentric\; bearing}}}} * {{{\rm{distance}}}})\) bin, \(n\) indicates the number of degree bins, \(m\) indicates the number of distance bins, \(e\) is the Euler constant, i is the imaginary constant, and \(\bar{F}^{\prime}\) indicates the mean firing rates of all bins.

The vector length of egocentric boundary vector was defined as the absolute value of the mean resultant egocentric boundary vector, which quantifies the strength of egocentric tuning to border. Preferred θ’ego of egocentric boundary vector was calculated as the mean resultant bearing (MRB):

$${{{\rm{MRB}}}}= \arctan 2({{{\rm{imag}}}}\left({{{\rm{egocentric}}}}\; {{{\rm{boundary}}}}-{{{\rm{vector}}}}\right)) / \\ {{{\rm{real}}}}({{{\rm{egocentric}}}}\; {{{\rm{boundary}}}}-{{{\rm{vector}}}})\left)\right.$$
(9)

Preferred D’ego of egocentric boundary vector was estimated by fitting a Weibull distribution12 to the firing rates corresponding to the MRB and finding the distance bin with the maximum firing rate.

The vector length of egocentric boundary vector ranged from 0 to 1, where 0 indicated neurons with no egocentric vector coding of borders and 1 indicated neurons with perfect egocentric vector coding of borders. A neuron was defined as egocentric boundary cell within each chamber if the egocentric boundary vector lengths for each chamber exceeded the 95th percentile of shuffled distribution of the vector length of all imaged neurons.

Behavior analysis

Mice behavioral states in square and circular open chambers (Supplementary Fig. 12, 18) were categorized into sniffing, rearing, and grooming by manual inspection via Ethovision XT 16 program following the previously described criteria35,36. Grooming was defined as the behavior during which a mouse scratching or cleaning its body quickly with the forelegs while sitting still, commonly involving the cleaning of the face and whiskers. This behavior also includes licking the tail or scratching the head with the hind legs. Sniffing was defined as the exploratory behavior involves a mouse approaching its nose close to the wall, vertex, or the floor, often pausing, or moving slowly. Rearing was defined as the behavior during which a mouse lifting its forelegs off the floor and standing upright on its hind legs, either with support from a wall or vertex in the open chamber, or without support.

Head direction score analysis

To characterize the strength of allocentric head direction tuning of a RSC neuron, head direction score67 was computed (Supplementary Fig. 14) based on the mean vector length from circular distributed firing rates in terms of θallo. A neuron was defined as head direction cell within each chamber if head direction score for each chamber exceeded the 99th percentile of randomly shuffled head direction score distribution of all imaged neurons.

In silico computational model of vertex cell

To investigate whether RSC vertex cell responses can be simulated simply with a neuron receiving synaptic inputs carrying boundary vector information, in silico computational model of RSC and boundary vector cells were constructed (Supplementary Fig. 19). RSC neuron model was adopted from Brennan et al.68 (ModelDB accession number 260192), which was a conductance-based simple multi-compartment Hodgkin-Huxley-type neuron model of excitatory neuron in RSC described by the following equation:

$${C}_{m}\frac{{{dV}}_{m}\left(t\right)}{{dt}}=-\left({I}_{{Leak}}+{I}_{{Na}}+{I}_{{Kdr}}+{I}_{d}\right)$$
(10)

where \({C}_{m}\) is the membrane capacitance, \({I}_{{Leak}}\) is the leak current, \({I}_{{Na}}\) is the fast sodium channel current, \({I}_{{Kdr}}\) is the delayed-rectifier potassium channel current, and \({I}_{d}\) is the d-current69. The parameters used for the RSC neuron model are shown in Supplementary Table 1, adopted from Brennan et al.68.

In silico computational model of boundary vector cell was built as a conductance-based single-compartment Hodgkin-Huxley-type neuron model where the membrane potential of boundary vector cell model was described by the following equation:

$${C}_{m}\frac{{{dV}}_{m}\left(t\right)}{{dt}}=-\left({I}_{{Leak}}+{I}_{{Na}}+{I}_{{Kdr}}+{I}_{{Km}}\right)$$
(11)

where \({C}_{m}\) is the membrane capacitance, \({I}_{{Leak}}\) is the leak current, \({I}_{{Na}}\) is the fast sodium channel current, \({I}_{{Kdr}}\) is the delayed-rectifier potassium channel current, and \({I}_{{Km}}\) is the M-type potassium channel current70. The parameters used for the boundary vector cell model are shown in Supplementary Table 2, adopted from Jang et al.71.

Boundary vector cell’s spike was evoked by giving a current input that is modulated by a Gaussian tuning curve72,73 using the following equation:

$${I}_{{boundary}}=\alpha * {e}^{-\frac{{d}^{2}}{2{\gamma }^{2}}}$$
(12)

Where \({I}_{{boundary}}\) is the amount of input current to the soma of boundary vector cell model, \(\alpha\) is a constant input current value (0.0525 nA), \(d\) is the distance from current position to the boundary in the environment, and \(\gamma\) is the boundary vector cell’s spatial receptive field size (10 cm). The peak spike firing rate of boundary vector cell model was set to 15–17 Hz, similar to in vivo observations7. To implement in vivo-like noise, boundary vector cell received Gaussian noise current input (mean = 0 nA, sd = 0.101 nA).

As RSC neuron has been reported to receive direct excitatory synaptic input from boundary vector cells74, RSC neuron was modeled to receive boundary vector cell spike-evoked excitatory postsynaptic current (\({I}_{{syn}}\)) using double-exponential conductance model75:

$${I}_{{syn}}(t)={g}_{{syn}} * {factor} * ({e}^{-t/{\tau }_{{decay}}}-{e}^{-t/{\tau }_{{rise}}}) * ({V}_{m}-{E}_{{rev}})$$
(13)

where \({g}_{{syn}}\) is the maximal conductance of synapse (0.001087 μS), \({factor}\) is the normalizing constant, \({\tau }_{{decay}}\) is the decay time constant (5 ms), \({\tau }_{{rise}}\) is the rise time constant (4 ms), \({V}_{m}\) is the membrane potential, and \({E}_{{rev}}\) is the reversal potential of the synapse model (0 mV). To simulate a simple vertex cell model receiving boundary vector cell inputs, we assumed that an in silico RSC neuron model received inputs from boundary vector cells that represent each edge of the open chamber. Thus, three, four, and six boundary vector cell-evoked subthreshold synaptic inputs were given to the RSC neuron model in triangle, square, hexagon, and concave polygon open chambers, respectively (Supplementary Fig. 19). All computational models were simulated using NEURON 8.276 with a temperature of 30°C and an integration time step of 0.1 ms. In vivo mouse trajectory data with linear interpolation (0.1 ms interval) was used to build 2D virtual environments for in silico simulation. Simulations were repeated using mice trajectories within each chamber recorded from nine mice in Fig. 1. Simulated spikes were analyzed using the same method that was used for analyzing in vivo experimental data.

Behavior data and Ca2+ signal quality metrics

To validate whether a mouse actively explored behavior chambers, average velocity and angular velocity were calculated for each mouse (Supplementary Table 3). Mice with average velocity less than 2 cm/s in a given chamber were excluded for data analysis as 2 cm/s is approximately 25 percentile value of average velocity distribution of C57BL/6 mice in an open chamber77. To validate whether a mouse explored enough area of a given open chamber, the area of each chamber covered by a mouse trajectory (chamber coverage) was calculated for each mouse (Supplementary Table 3). The positional occupancy of each mouse within an open chamber was discretized into 1.5 cm × 1.5 cm spatial bins, and the number of spatial bins covered by a mouse trajectory was divided by the number of total spatial bins to calculate chamber coverage. Mice with chamber coverage less than 80% were excluded for data analysis.

To validate the quality of Ca2+ signals, average signal-to-noise ratio, average firing rate, and inter-spike interval of deconvoluted spikes of each neuron were calculated. The signal-to-noise ratio of a neuronal spike was quantified as △F/Fo value of the corresponding spike over the standard deviation of the △F/Fo values during 1 s before the corresponding spike57. Neurons with average signal-to-noise ratio of spikes lower than 3 were excluded from data analysis, following the previous study62. Also, neurons with firing rates lower than 0.028 Hz (minimum of 50 spikes for 30 min) in the spike raster plots (Supplementary Fig. 23) were excluded following the previously described criteria used in characterizing spatially tuned cells11.

Statistics and reproducibility

Statistical significance was measured using paired or unpaired Student’s t test, or one-way, two-way, repeated measures ANOVA with post hoc Tukey’s test. For circular data, Watson–Williams tests, Watson–Williams multi-sample tests, or Wilcoxon signed-rank test78,79 were used. p value less than 0.05 was considered statistically significant. All statistical tests were two-sided. Immunohistochemistry of GCaMP6s expression (Fig. 1a, Supplementary Fig. 13a) and Ca2+ imaging FOV was checked in all mice (n = 9 mice).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.