Investigating visual search mechanisms and enhancing the diagnostic potential of the trail making test using eTMT

Del Gatto, Claudia; Indraccolo, Allegra; Delogu, Franco; May, Maliea; Pedale, Tiziana; Brunetti, Riccardo

doi:10.1038/s41598-025-19125-0

Download PDF

Article
Open access
Published: 29 September 2025

Investigating visual search mechanisms and enhancing the diagnostic potential of the trail making test using eTMT

Claudia Del Gatto¹,
Allegra Indraccolo¹,
Franco Delogu²,
Maliea May²,
Tiziana Pedale³ &
…
Riccardo Brunetti¹

Scientific Reports volume 15, Article number: 33445 (2025) Cite this article

1114 Accesses
1 Citations
Metrics details

Subjects

Abstract

The Trail Making Test (TMT) is a complex task involving visual search and is widely used to assess various cognitive functions. While existing literature often emphasizes task switching, the nuances of visual search within the TMT remain debated. In this study, we developed a new digital version of the TMT, the eTMT, to more effectively analyze visual search performance dynamics. We hypothesized that critical aspects of visual search, particularly the display set-size effect, have been overlooked in prior research — specifically the impact of distractors in the visual field on participants’ performance. Additionally, we explored how the line drawn by participants in the traditional paper-and-pencil version may influence performance. To test these hypotheses, we conducted two experiments (data collected in 2022). The first involved modifying the standard TMT by removing the connecting line between targets and turning reached targets into black discs. In the second experiment, we introduced variations such as having reached targets disappear entirely and altering their positions, alongside adjustments to the spatial relationships between successive targets. Our findings confirm the significant influence of the display set-size effect on TMT performance, providing insights into the mechanisms underlying visual search and underscoring the potential of the digitally redesigned TMT for improved assessment.

Unveiling Trail Making Test: visual and manual trajectories indexing multiple executive processes

Article Open access 22 August 2022

Active visual search in naturalistic environments reflects individual differences in classic visual search performance

Article Open access 12 January 2023

Comparing attentional disengagement between Prolific and MTurk samples

Article Open access 23 November 2023

Introduction

The Trail Making Test (TMT) is one of the most widely used tests in clinical and research settings as a tool to assess executive functions and brain impairments quickly and easily^1,2,3,4,5. The original TMT is a paper-and-pencil task in which participants graphically connect, in ascending order, a series of 25 circles labeled with numbers and/or letters as quickly and accurately as possible. If errors occur, the examiner highlights them in real time, allowing the examinee to make corrections. The TMT is divided into two parts: the first part (TMT-A) consists of 25 circles randomly arranged on the page and numbered from 1 to 25. The second part (TMT-B) also contains 25 circles, but half are numbers (from 1 to 13) and the other half are letters (from A to L); the participants’ task is to connect the circles by alternating between numbers and letters in ascending order (i.e., 1 to A, A to 2, 2 to B, etc.^2,4,6. In both TMT-A and TMT-B, the primary measure of performance is the total time required to connect all 25 circles⁶. The paper-and-pencil version was originally included in the Army Individual Test Battery⁷: researchers have found that TMT scores correlate strongly with intelligence, specifically the results have shown that individuals with lower IQ scores take longer to complete the TMT⁸.

Although the TMT appears to be an excellent tool for indicating the presence of brain impairment, poor performance on the task is a non-specific result, as it does not accurately pinpoint which aspects of the performance are deficient in an individual with cognitive impairment. Due to the complexity and the diverse nature of the cognitive processes involved in the TMT, poor performance can be attributed to different causes (e.g. motor slowing, poor coordination, visual scanning difficulties². Therefore, although most of the studies agree on the multifactorial nature of the processes involved in the TMT, there is a lack of consensus regarding their exact roles and influence. Sànchez-Cubillo and collaborators⁹ presented an overview of 24 studies that aimed to clarify the processes underlying TMT scores and found that visual search, motor skills, working memory, and general intelligence are the most cited processes that contribute to TMT performance. Moreover, other studies highlight that the TMT-B involves additional executive function resources^2,10,11,12 such as cognitive flexibility, inhibition/interference control, working memory, mental tracking, and attentional set-shifting^{13,14,15,16,17,18}.

In recent years, two paradigms in visual cognition have emerged as particularly relevant to understanding performance in the Trail Making Test: The Multiple Item Localization (MILO) task and hybrid foraging. Both paradigms involve serial visual search under constraints of memory and target al.ternation conditions that closely mirror those found in the TMT, especially in part B. In hybrid foraging tasks¹⁹, participants search for multiple targets from two or more categories simultaneously. A consistent finding is the tendency to select items from the same category in bursts before switching reflecting the cognitive cost of alternating between target types. These switch costs suggest that participants optimize their search by minimizing category changes, revealing a trade-off between memory load and attentional control. The MILO task²⁰ was designed to simulate sequential search across interleaved target categories, much like the letter-number alternation in TMT-B. In MILO, participants must select targets in a fixed order (e.g., 1-A-2-B.), resulting in longer response times at category-switch points. Although the so-called “saw-tooth” pattern has not always been directly reported, the RT dynamics in MILO have been interpreted in terms of switch costs and inhibitory tagging, a mechanism whereby recently selected items are suppressed in memory to prevent their re-selection²¹.

These experimental paradigms have substantially advanced our understanding of the cognitive architecture underlying the TMT, offering insights into how visual search, category switching, and memory interact. Their relevance is further supported by a long-standing body of research that has consistently highlighted visual search as a central component of TMT performance. Ehrenstein and collaborators²², for example, investigated the role of visual search in Part A of the TMT, by correlating the TMT-A performance with different verbal and non-verbal tests, such as the Objects Finding Test (OFT). The results of the study showed a significant correlation between performance in TMT-A and the visual search skills required by the OFT, leading the researchers to conclude that the primary process underlying the TMT-A performance is visual search²². Later, Crowe²³ investigated the differential contributions of visual search and attentional functions on TMT performance, using a modified version of the TMT-A and TMT-B without numbers and letters. The results of the study demonstrated that TMT-A uniquely measured visual search and motor speed, while TMT-B measured visual search and cognitive alternation of operations²³.

To better understand the mechanisms underlying visual search in the TMT, it is useful to turn to the extensive body of research on visual search paradigms, which has long examined how task and display properties affect performance. It has been long known that, in visual search tasks, some factors can affect performance in terms of reaction times (RTs), including the number of distractors displayed and the target that can have unique features or conjunctions of features with other distractors²⁴. The demonstration that visual search response time frequently increases with the number of distractors has been a fundamental aspect of attention theories^{24,25,26,27,28}. In other words, it is well known that the number of distractors affects serial visual search producing the so-called Display Set-size Effect(DSE, e.g²⁹.). This happens because the level of interference in encoding a stimulus may increase with the quantity of stimuli, possibly as a result of the dispersal of attentional resources^30,31. This phenomenon, often labeled as “resource limitation” or “limited capacity”³², does not restrict the number of stimuli encoded, but results in a more rapid decline in performance as a function of set-size. However, in some visual search tasks, the number of distractors has minimal impact, and the target seems to “pop out” from the display³³. This can even capture attention without being the search target^34,35. These patterns underlie Anne Treisman’s Feature Integration Theory (FIT)²⁴. Treisman proposed an initial “preattentive” processing stage in which basic features like color and orientation are processed in parallel across the visual field.

While the number of distractors in a visual display can significantly impact search efficiency, it is also important to consider how individuals engage with the display. In other words, the strategy adopted to visually scan the environment can influence performance independently of stimulus quantity. Boot et al.³⁶, for instance, demonstrated that different scanning strategies ranging from highly active to more passive, can lead to significantly different outcomes. Interestingly, participants who made fewer eye movements, relying instead on a radial “sit and wait” strategy, achieved better results. These findings are relevant not only in the broader context of visual search research but also for understanding performance in tasks such as the TMT, where visual scanning plays a central role. Such strategic components may therefore offer a useful lens through which to interpret how individuals handle varying display complexities and demands an aspect further explored in our experimental investigation.

In addition to strategic variations, perceptual features of the stimuli, particularly their salience, also play a critical role in modulating visual attention. Traditional interactions of TMT-A exhibit a uniform level of target salience, given the perceptual homogeneity of all display elements. However, experimental modifications of the TMT like the one we are going to introduce in this study can induce differential salience between the current target and previously acquired targets. Specifically, the alteration of prior target colors can result in a prominent “pop-out” phenomenon, thereby creating irrelevant singletons distinguished by their immediate physical dissimilarity from the present target. In this conceptual framework, TMT-A can be conceived as a series of single target searches where every time a target is found it gets turned into a distractor and one of the distractors becomes the new target. Lamy and Zoaris³⁷, in a study that does not use the TMT, show that stimulus salience influences attention even when it is task-irrelevant. These results offer evidence that contradicts Theeuwes’³⁸ proposal, which posits that attentional capture by a salient distractor does not occur in the context of serial search. Also, Lamy and Tsal³⁹ showed that distractors with salient features did not interfere with performance when the participant was engaged in searching for the target. Additionally, the hypothesis of salient-signal suppression, proposed that individuals inhibit signals originating from items that are salient but irrelevant while searching the target^40,41,42. Another study in a similar direction is by Proulx and Egeth⁴³, who investigated whether the similarity between the target and the distractors can influence performance. In this case, they revealed that distractors dissimilar to the target facilitate visual search.

In line with Lamy and Zoaris findings, salient items in the TMT could facilitate participants attention, since salient items are dissimilar from the subsequent targets and easy to filter out from future search⁴³. In contrast, other studies show that salient distractors are not able to capture attention during serial search³⁸, because individuals tend to narrow their attentional focus and inhibit signals originating from items that are salient but irrelevant while searching for the target^40,41,42. However, Burnham⁴⁴ showed that in a dual-task condition, attention is captured by a task-irrelevant color, but this happens in a single-task condition. This suggests that manipulating the display set-size might affect performance more in TMT part B than in TMT part A. In Part B, participants need to not only search visually for targets but also to switch between target types (i.e., letters and numbers) during the search and keep track of the last item reached for each target type. Such a switching has been shown to be time-consuming, with longer response times when successive selections involve different target types compared to when they involve the same type. For example, studies on Hybrid Foraging tasks have demonstrated robust effects of category switching. Responses tend to be slower when switching between target types than when selecting successive items from the same category⁴⁵. Further support for the impact of switching between target categories comes from research using the MILO task^20,46,47. In MILO, participants must search for a fixed sequence of targets from two or more categories, often revealing increased response times following category switches. This has been interpreted as evidence for switch costs and mechanisms such as inhibitory tagging²¹. In a variant of the MILO task, participants search for interleaved sequences of targets belonging to different categories, similar to the alternation required in TMT-B. Trial by trial results from the MILO task showed a characteristic “saw-tooth” pattern of response times, with slower responses when switching to a different target type and faster responses on the following targets, suggesting a cost when switching target and a preliminary pianification of the search of the upcoming targets. Moreover, the results obtained with the MILO seem to suggest the interleaving targets also disrupts retrospective inhibitory tagging, a mechanism that helps participants suppress interference from previously selected targets²⁰. Disruption of inhibitory tagging may contribute to the difficulties observed in TMT-B performance, as switching between categories appears to interfere with the ability to chunk upcoming targets and to exert inhibitory control over previously selected items. Finally, based on Burnham’s findings, it can be hypothesized that participants’ attention in Part B will be more influenced by irrelevant information. This means their attention could be drawn more to salient irrelevant items, making it easier to recognize them as distractors and exclude them from the search⁴³. Just like in a dual-task condition where irrelevant items are quickly excluded, heightened attentional sensitivity could help in Part B, but might also make it harder when no items can be excluded. Therefore, the comparison between Parts A and B will be interesting precisely due to the presence of the switch task and the influence of the attentional and perceptual components.

Digitized TMT and eTMT

The digitization of paper-and-pencil tests has the advantage of increasing the ease of administration, standardizing the presentation of the task, but it also allows the acquisition of additional parameters for a more thorough evaluation of cognitive processes. Digital versions enable the collection of data and the processing of single items in customizable research settings. In the last decade, numerous digitized versions of the original paper TMT have been implemented and have successfully addressed these aspects^48,49,50. For example, Reynolds created the Comprehensive Trail Making Test (CTMT⁵¹; that was designed to expand the task and address any issues of norming and standardization of the TMT. The CTMT consists of a sequence of five trails: Trails 1–3 are similar to TMT-A, but Trails 2 and 3 introduce increasing numbers of distractor circles that participants must ignore. Trails 4 and 5 are more similar to TMT-B: in Trail 4, participants are asked to draw a line between Arabic numerals in circles and number words in rectangles; in Trail 5, similar to TMT Part B, there are empty circles provided as distractors. Results show that the task gets progressively more difficult with the addition of distractors⁵¹. However, this study did not analyze what might happen instead with the gradual elimination of distractors in terms of a possible facilitation effect.

Custom versions of the TMT were created digitally to test specific hypotheses, for example, the Connection Test⁵². This test involves an array of circles where the targets are numbers, letters, or alternating numbers and letters. The task is to draw lines connecting the circles in the given sequence, with targets positioned above, below, to the left, to the right, or diagonally from the previous one. The study helps to understand individual differences in alternating and non-alternating versions of the TMT. Perceptual speed accounts for most of these differences, particularly age-related ones. Although alternating versions take more time, both versions share significant variance, indicating minimal age-related effects on switching processes.

Beyond the display set-size effect, other factors seem to influence participants’ performance. In 2015, Woods and colleagues⁵³ proposed a new version of the Computerized TMT (C-TMT) with the aim, among others, of reducing the interference caused by visual clutter of the lines drawn. The authors hypothesized that participants who draw imprecisely or make errors will add visual clutter, which complicates the detection of subsequent targets and, consequently, increases completion times. In their C-TMT, errors were automatically corrected, and erroneous paths were removed to decrease visual cluttering. Results showed that completion times on both C-TMT-A (numbers only) and C-TMT-B (alternating letters and numbers) were significantly influenced by age. However, this study does not clarify whether the correct pen-drawn line may also produce visual cluttering, making the task more difficult or, conversely, whether the line drawn with the pen can facilitate the task. In other words, once a line has been drawn with the pen on and between the items, it is possible to hypothesize that they are functionally “canceled” and therefore no longer taken into consideration in the visual serial search task, effectively reducing the display set-size.

In summary, while various digital versions of the TMT have explored the role of visual clutter, it remains unclear under which specific conditions it helps or hinders performance. No studies so far have looked at how the number of distractors specifically affects TMT performance. Reynolds⁵¹ noted that adding distractors makes the task harder but did not examine what happens when they are reduced. Woods et al.⁵³ studied the number of distractors, but the exact role of visual clutter remains uncertain. Salthouse et al.⁵² addressed visual disturbance and motor interference but excluded distance as a variable. Therefore, a study is needed to disentangle factors like strategy, the number of distractors, and whether the lines drawn connecting targets might actually help by “canceling” them. The main aim of this study is to verify how the item set-size affects performance in the TMT task. We intend to test set-size effects by manipulating the appearance of the already-found targets. Additionally, we will investigate the influence of visuospatial memory by adding a condition in which all items change position after each correct target-reaching action. Furthermore, we aim to delve deeper into the cognitive processes involved in this task by controlling for the influence of inter-target distance and path continuity. To do so, we implemented a new digital version of the TMT called eTMT to test the influence of the following factors on visual search:

1)
item set-size
2)
visual cluttering produced by the lines drawn
3)
the permanence/cancellation of already-reached targets
4)
visuospatial memory
5)
Spatial arrangement of subsequent targets

To investigate these properties, in Experiment 1 we created three different versions of the TMT:

1.
Standard version: replicates the original paper-and-pencil task.
2.
No-Line (NL) condition: participants still draw lines with the mouse between target circles, but as soon as the next target is reached, the previously drawn line disappears, allowing all distractors to remain fully visible throughout the task. This condition aims to verify whether the line drawn by participants contributes to “canceling” items, thereby reducing the display set-size and the number of items among which participants must search. We hypothesize that this version of the TMT will be more difficult than the original, as the absence of a persistent line may make the visual search task more demanding.
3.
Black Discs (BD) condition: once the next target is reached, the line disappears, and the just-reached circle turns black. This condition allows us to disentangle contrasting results and interpretations. Following Lamy and Zoaris³⁷, black discs may influence participants’ attention by facilitating the task⁴³. Conversely, attentional capture by a salient distractor might not occur in the context of serial search³⁸. With this condition, we aim to understand whether transforming previous targets into black discs—making them more salient but also more dissimilar from other targets facilitates or interferes with the serial visual search task in the TMT.

Moreover, we speculated that if the main influence on the visual search task of the TMT is given by the display set-size, then we should get a facilitation by the transformation of previous targets into black discs, consequently reducing the set-size. If this were the case, we should observe a progressive decrease in response time as the task progresses due to the increasingly narrower display of distractors.

To summarize, the aim of this study is to verify the hypothesis that display set-size is a variable that could significantly affect TMT performance. Hence, we expect the first condition (Standard) to replicate the classic results of the TMT; the second condition (No-Line), is expected to be more difficult as the just-drawn line disappears preventing the participant from “canceling” the targets already reached and therefore forcing them to search for the next target throughout the screen without being able to neglect those already analyzed; finally, the third condition (Black Discs), the one in which the circles are transformed into black discs after they have been reached, could be the simplest condition because it allows the participant to easily filtering the black disks out of the search. This should happen for both parts A and B of TMT, since they are both based on a visual search task. Burnham⁴⁴ found that attention is drawn to task-irrelevant colors in dual-task conditions, but not in single-task conditions. Based on this, we suggest that in Part B of the TMT, which involves a dual task, our manipulations could have a more pronounced impact. Drawing from Burnham’s study, we hypothesize that participants’ attention in part B will be more susceptible to task-irrelevant information, such as the black discs used in our study⁴³. This heightened attentional bias towards the black discs may facilitate their quick identification as distractors or non-targets, aiding in their exclusion from the search. Conversely, this heightened attentional sensitivity might pose challenges in conditions where items cannot be excluded from the search, such as when connecting lines between successive targets are removed after drawing. Therefore, comparing parts A and B will be particularly insightful due to the dual-task nature of part B and its impact on attentional and perceptual processes^43,44. Considering all previous results, we expect:

1.
To replicate results of previous experiment with our Standard condition (part A easier than part B);
2.
To find slower reaction time in the No-Line condition, since the absence of the line that passes and cancels items makes the visual search task more challenging;
3.
To find a facilitation in the Black-Discs condition^37,43;
4.
To obtain different influences of our manipulations (NL and BD) on the performance of Part A and Part B, given that the two parts involve different cognitive processes, and Part B is considered more challenging.

Experiment 1

Method

Power analysis. This study was powered to detect moderate-to-large differences in test performance between test variants. The power to detect differences between testing variants was examined using the program Gpower 3⁵⁴. This indicated that detecting an effect size of 0.2, at 80% power, would require a sample of minimum 68 participants.

Participants

Sixty-nine participants (29 males; mean age = 25.59 years, SD = 11.91 years; range = 16–62) took part in the experiment. All participants reported normal or corrected-to-normal vision. Although previous studies⁵⁵ did not find an influence of handedness on TMT performance, we chose to include only right-handed participants in our sample to avoid potential confounds. They were naïve as to the purpose of the study and gave written informed consent (for minors informed consent have been obtained by a parent) in accord with the declaration of Helsinki; the experimental protocol has been approved by the Ethical Committee of Università Europea di Roma (approval number 3/2024). Data were collected in 2022.

Materials

We created the eTMT, a digital version of the TMT, and manipulated it to generate three different conditions of the test. In the eTMT the test display is no longer vertical, as in the paper-and-pencil version, but has been adapted for a horizontal screen while maintaining the same spatial relationships between the targets, adjusted to fit a horizontal layout. Experiment 1 includes three conditions of the TMT test: The Standard (ST) condition is analogous to the paper-and-pencil version (both version A and B; see Fig. 1); The No-Line (NL) condition has the pen stroke removed: as soon as the participant reaches a target, the line just drawn with the mouse disappears (see Fig. 2); The Black Discs (BD) condition removes the lines and transform the reached targets into black discs (see Fig. 3).

Procedure

After receiving on-screen instructions, each participant performed the three conditions Standard, No-Line, and Black Discs (ST, NL, and BD) in the two different task versions (Parts A and B) resulting in a total of six trials per participant. All participants completed Part A first and then Part B, so the order of the two tasks was the same for everyone. The administration order of the 3 conditions was instead randomized for each participant to control for potential order effects. Before starting each condition, participants had the opportunity to undergo training in a TMT version containing only eight items (see Figs. 1, 2 and 3). A separate training trial was provided for each manipulation of the task, to ensure that participants were familiar with the specific format of the upcoming condition. Participants were asked to complete all tasks as accurately and as quickly as possible. In the TMT-A, the participants’ task was to connect, using the mouse, 25 encircled numbers distributed on the screen from 1 to 25 in ascending order (1-2-3, etc.). For the TMT-B, participants were asked to connect numbers and letters in alternating, ascending order (1-A; 2-B; 3-C, etc.). To connect one circle to the next, participants had to keep the mouse button pressed and could release it and press it again as they wished.

Apparatus

The study was carried out using a laptop computer (MacBook Pro 15’) and a mouse. The computer was running eTMT: a custom-made script in Max 8 (Cycling ’74 (https://cycling74.com/)).

Results

Firstly, a correlation analysis was conducted to determine whether age had any effect on the total time taken to complete the TMT tasks. We calculated the total time elapsed to complete the TMT tasks (i.e., the A and B parts of the Standard, No-Line, Black Discs TMT versions) and assessed whether these times correlated with the participants’ age. After verifying that age did not significantly influence the total time (r =.129, p =.294), we calculated the z-scores associated with the total time elapsed to complete all tasks for each participant. The maximum z-score was 2.41, thus all participants were included in the following analyses, and we proceeded to analyze the mean reaction times (RTs) taken to reach the targets for each participant. For each participant, we analyzed the mean reaction times (RTs) taken to reach the targets. In other words, the program calculates each individual RTs, from the moment the previous target is reached to the moment the next target is touched. Repeated measures analysis of variance (ANOVA) was used to evaluate whether the mean RT, varied as a function of TMT parts, (2 levels: “A” and “B”), the TMT conditions (3 levels: Standard, No-Line, Black Discs) and the comparison between the individual average reaction time taken to reach the first 12 targets (from 1 to 12) and the average time taken to reach the last 12 targets (from 13 to 24) (2 levels: Early and Late targets); see Figs. 1, 2 and 3. We have included this last variable to assess whether participants’ performance benefited from the reduction in display set-size (in the BD condition as well as in the ST condition) and thus corroborated the results obtained so far regarding the influence of distractors once eliminated from the display. In case of violation of the sphericity assumption (Mauchley’s sphericity test; p <.05), a Greenhouse-Geisser correction was performed. The results of the ANOVA revealed, as expected, a main effect of parts [F(1, 67) = 90.73, p <.001, η²_p = 0.575], indicating in general faster RT in the TMT-A (M = 1400, SE = 34.5 ms) than in the TMT-B (M = 2222, SE = 98.6 ms) were also task-switching abilities are required. Moreover, the ANOVA revealed a main effect of conditions [F(2, 134) = 11.97, p <.001, η²_p = 0.152]. Post-hoc Bonferroni comparisons showed significantly longer RT in the NL condition (M = 2038, SE = 90.8 ms) as compared to the ST (M = 1771 ms, SE = 58.4 ms, Mean difference = − 267 ms, 95% CI [− 439, − 95], t(67) = − 3.11, p =.008, d = 0.38) and the BD (M = 1624 ms, SE = 80.3, mean difference = − 413 ms, 95% CI [− 564, − 262], t(67) = − 5.48, p <.001, d = 0.66) conditions (see Fig. 4). We observed no significant main effect in the comparison between the average reaction times between early (positions 1–12) and late targets (13–24) [F(1, 67) = 0.590, p =.445, η²_p = 0.009]. Analysis revealed a significant interaction between the conditions (ST, NL and BD) and parts (A and B) [F(1.82, 122.23) = 5.18, p =.009, η²_p = 0.072]. Indeed, post-hoc comparison showed within the part A, significant longer reaction time for the ST condition compared to the BD condition (mean difference = 165.2, 95% CI [80.2, 250.2], t(67) = 3.89, p =.004, d = 0.47). Moreover, within Part B, post-hoc comparisons showed longer reaction times in the NL condition compared to the ST condition (mean difference = 242 ms, 95% CI [91, 393], t(67) = 3.21, p =.031, d = 0.39) and to the BD condition (mean difference = 560 ms, 95% CI [412, 708], t(67) = 7.58, p <.001, d = 0.92). Participants in part B experienced an 11.5% slowdown in the NL condition compared to the ST condition, while conversely, they sped up their reaction times by 15.4% in the BD condition compared to the ST condition. In Part A instead participants were faster by 11.8% in the BD condition compared to the ST one and were slower in the NL condition by 3.2% compared to the ST condition. Finally, analysis revealed another significant interaction between TMT parts (A and B) and Early and Late targets [F(1, 67) = 46.93, p <.001, η²_p = 0.412]. Post-hoc comparisons (Bonferroni) showed a significant difference between early and late targets within Part A (mean difference = 414 ms, 95% CI [338, 490], t(67) = 10.94, p <.001, d = 1.33) and within Part B (mean difference = − 344 ms, 95% CI [− 531, − 157], t(67) = − 3.67, p =.003, d = 0.4; see Fig. 5). Therefore, these analyses indicate that participants take less time to reach the next target in the second half of the task compared to the first. However, this only occurs in Part A of the TMT because this pattern reverses in Part B of the test; in this case, reaction times in the second half of the task increase compared to the first half. This result seems to precisely reflect the presence of the switch task in part B of the test. As the task progresses, the switch task becomes more demanding, as it becomes increasingly challenging to keep track of the running count with increasing magnitudes. To more precisely characterize the response patterns in the first and second halves of the task, we conducted a more complex ANOVA that included serial reaction time (SRT) defined as the sequential time taken to reach each of the 24 targets. This ANOVA incorporated three factors: TMT parts (2 levels: “A” and “B”), TMT conditions (3 levels: Standard, No-Line, Black Discs), and SRT (target number as a factor: 24 levels, from 1 to 24; see similar approaches in^20,46,47. As expected, this analysis confirmed all previous findings: specifically, a main effect of parts [F(1, 67) = 89.68, p <.001, η²_p = 0.572], a main effect of conditions [F(1.82, 122.29) = 12.38, p <.001, η²_p = 0.156], and a significant interaction between conditions and parts [F(1.79, 120.14) = 4.81, p =.012, η²_p = 0.067]. In addition, the analysis revealed a main effect of target number [F(4.81, 322.89) = 15.08, p <.001, η²_p = 0.184], as well as significant interactions between target number and part [F(5.10, 341.98) = 12.98, p <.001, η²_p = 0.162], and between target number and condition [F(46, 3082) = 1.91, p <.001, η²_p = 0.028]. Figure 5b illustrates the SRT and clearly shows the significant time variability required to reach each sequential item in both parts of the TMT. Notably, the “saw-tooth” pattern characterized by alternating slower and faster responses in consecutive pairs was absent in TMT-B, in contrast to the pattern reported by Thornton and Horowitz²⁰ in a MILO variant that resembled the features of TMT-B with 12 sequential targets.

Transparency and openness

We report how we determined our sample size, all manipulations, and all measures in the study. All research materials are available upon request from the corresponding author. Data were analyzed using Jamovi, version 2.3.21. This study’s design and its analysis were not pre-registered. All data have been made publicly available on the Open Science Framework- OSF and can be accessed at https://osf.io/kwjxu.

Discussion of exp. 1

The results confirm, as expected, that reaction times are slower in Part B of the TMT suggesting that the switching requirement included in TMT-B makes it more difficult than TMT-A^52,56,57. The removal of the drawn line specifically affects Part B: the No-Line condition, where all the distractors remain visible, and line feedback is removed, is the most difficult one. These results confirm our hypothesis regarding the influence of distractors on this task: When distractors remain on screen, reaction times are delayed. This is a remarkable result, as it allows for deeper exploration into the type of processes and variables that influence this visual search task. Our NL manipulations highlights that in TMT-B, a task already complicated by the requirement of switching between numbers and letters, the participants slow down even more (an 11.5% slowdown in the No-line condition Part B compared to the Standard condition Part B; a 3.2% slowdown in the No-line condition Part A compared to the Standard condition Part A). We argue that in TMT-B participants could be forced to analyze the entire visual display without being able to neglect previously selected targets. This experiment shows a strong set-size effect, clearly affecting performance and creating a consistent pattern of results. This evidence is further supported by the fact that the Black Discs condition replicates the effect of the Standard one in canceling already selected targets. Our finding confirms the hypothesis that both in the original paper-and-pencil and in the digital versions, visual search is facilitated by the drawn line or by the canceling of the items (turned in black discs) which gradually reduces the field of distractors among which to detect the next target. Moreover, these results confirm that cognitive load interacts more prominently with the difficulty introduced by the absence of the drawn line. As hypothesized and consistent with the literature⁴⁴, the most significant effects are observed in TMT-B. These findings also align with the literature on visual stimulus salience: at a perceptual level, the removal of the line prevents the formation of visual prominence within the field of stimuli, making it impossible to visually exclude already-processed (and blackened) items⁴³. MILO studies have shown that making previously selected items disappear (Vanish condition) can facilitate sequential search [20; 46]. In Experiment 1 of the present study, by contrast, visited targets remained visible but were transformed into black discs, thereby variating their salience without reducing overall set-size. This manipulation differs from the MILO Vanish condition, yet our results suggest that even without full distractor removal, altering the visual status of visited items can modulate search efficiency, pointing to a graded role of display dynamics in sequential search.

The final aspect we aimed to investigate pertains to the difference in average reaction times between completing the initial and latter halves of the test. We conducted this analysis to confirm the effect of distractor set-size on task performance. Based on the results we obtained comparing the three different conditions, we expected to find a gradual reduction in reaction times within the Standard and Black Discs conditions. This expectation stemmed from our hypothesis that the presence of black discs (BD) or the removal of targets via the line (ST) would progressively diminish distractor magnitude, thereby easing the task and resulting in quicker reaction times, for instance, comparing Early targets (where most distractors are present in the visual field) with Late targets (where half the distractors are absent due to darkening or line erasure). However, we found no significant interaction between our conditions and average reaction times across the test’s two halves (Early and Late targets). These null results could be interpreted as an evidence that participants had more time to process the position of late items due to their increased exposure time. However, a significant interaction emerged between the two parts of the TMT (A and B) and Early/Late targets showing a contrasting patterns of faster performance in the second half of TMT A, and slower responses in the second half of TMT-B. This discrepancy notably reflects the presence of the switching task in TMT-B. As complexity increases, maintaining a running count of numbers and letters becomes more arduous, consequently delaying performance in Late targets of TMT-B. Alternatively, or in conjunction, this finding could be explained by considering the presence of automatic processing of the beginning part of the alphabet (A < B < C < D) and more effortful processing of later letters (H < K < I < L) (see also⁵⁸. Remarkably, this interaction persists, regardless of our experimental manipulations. Based on the results of Experiment 1, it remains unclear why, despite the observed general facilitation from cancelling reached items, the gradual reduction of active distractors did not result in a progressive decrease in reaction times as the task advanced. Consequently, we conducted another experiment to assess whether other variables, such as the distance between successive targets and the degree of continuity in the direction taken to move from one target to another, could affect reaction times.

Experiment 2

The main purpose of Experiment 2 was to test why the TMT task does not become simpler over time (as observed in Experiment 1, where no significant interactions emerged between the conditions, ST/BD/NL and the stage of the task, Early/Late targets) but remain equally demanding throughout all repetitions. Additionally, Experiment 2 aims to control for variables that may influence TMT performance, which were not accounted for in previous versions of the test.

It has been demonstrated that the implicit learning of distractor configurations, when repeatedly presented, can provide a “contextual cueing” effect that facilitates target search in visual search tasks. A study conducted by Manginelli and Pollmann⁵⁹ investigated how search behavior adjusts when the target location shifts from a constant position to a new location. The results showed that altering implicitly learned spatial relations between target and distractor configurations eliminated the contextual cueing advantage.

Thus, to control for this possible effect in the TMT task, we introduced a new manipulation: once the next target is reached, not only does the drawn line disappear (a condition that, as we saw in Experiment 1, increased reaction times in the NL condition), but all the items also change their position on the screen. More specifically, while the position of the circles on the screen remains constant, the items are shuffled among them each time a target is reached. In this way, the potential facilitation provided by memorizing item positions is eliminated, as targets switch positions pseudo-randomly with each new target.

Previous studies have indicated that the distance between targets and the spatial arrangement of distractors correlates with the difficulty of visual search. Specifically, Gaudino and collaborators⁵⁶ identified variables related to spatial positioning that influence performance in TMT-B. They showed that part B differs from part A in terms of average distances (part B has 2.4 cm more between each target) and in the relationships between one target and another (part B involves more visual interference as there are more distractors between targets compared to part A). Moreover, in a study by Linari and collaborators⁴⁹, a new digitized version of the TMT was proposed, in which they systematically controlled the accuracy and speed of task execution and added a measure for tracking eye movements. In this study, in addition to replicating the results of the original version, they were able to provide indices that more specifically reflect measures of executive control. Furthermore, they found differences between Parts A and B, specifically in the number of fixations, which was greater in Part B. The authors of this study do not exclude the possibility that some differences may have emerged due to the influence of the different spatial organization of the stimuli in the test. Regarding proximity, Salthouse and collaborators⁵² demonstrated that it is possible to reduce irrelevant influences of visual search and hand movements by using a version of the test in which the targets are adjacent. On the other hand, Reddy and VanRullen⁶⁰ investigated whether items being close together make the task harder due to visual “clutter” or, conversely, increasing the spacing between objects results in better search performance. Their results showed that increasing the distance between objects improves visual search performance. Regarding spatial continuity between subsequent targets, Fischer and Whitney⁶¹ demonstrated that after seeing a particular pattern, people’s judgments about the orientation of similar patterns that follow are influenced by the orientation of the initial pattern. They suggested that this serial dependence arises from a spatiotemporal integration window, termed the continuity field, where stimuli observed a few seconds ago interact with the perception of current visual stimuli. This concept has been supported by additional research, which found that judgments related to numerosity⁶², eye gaze⁶³, shape⁶⁴, motion coherence⁶⁵, facial identity⁶⁶, gaze direction⁶³, and emotional expressions⁶⁷ also exhibit serial dependence based on past perceptual experiences. Moreover, Fritsche and colleagues^68,69 as well as Rafiei and collaborators⁷⁰ demonstrated the existence of an attractive or repulsive bias concerning the proximity between an item and another item processed earlier. Their studies indicate that an attractive bias in perceived orientation occurs when preceding items share similar orientations, while a repulsive bias occurs when they exhibit dissimilar orientations. Conversely, it is possible to consider the “Inhibition of Return” (IOR) effect that suggests that items that have been attended to are afterwards inhibited in a way that enables attention to shift away, creating a tendency to avoid selecting that specific item again, at least for a certain period. Posner and Cohen⁷¹ were the pioneers in presenting proof for a mechanism wherein, after attention is directed to a location and subsequently withdrawn, there is a delay in response time (RT) when a target emerges at the previously focused spot. They coined the term “Inhibition of Return” to convey the concept that attention may be restrained from returning to locations that have already been examined. The interaction with the IOR effect with visual search is well established: utilizing a set of visual search tasks modeled after Treisman and Gelade²⁴, Klein⁷² evaluated reaction time to a dot-probe displayed immediately following a visual search trial. When the search task was challenging (requiring effort to locate the target), but not when it was easy (with the target easily standing out), observers were slower to respond to the dot-probe if it appeared at the same location as a distractor in the search array. Klein⁷² and others employing this task, such as Müller and Mühlenen⁷³ and Takeda and Yagi⁷⁴, attributed this pattern to IOR under the assumption that attention assesses distractors to figure out if they are the target in situations where the search is demanding. Boot and collaborators³⁶ also discovered a similar IOR-like pattern in saccadic responses to sequentially presented stimuli: participants were less likely and slower to fixate on the target when it appeared at a location previously fixated. This effect could also influence a task like the TMT, and it is possible to imagine that this phenomenon is related to direction as well as position, as other studies have discussed oculomotor suppression related to movement and direction (e.g⁷⁵.,.

Considering all the above-mentioned studies, in Experiment 2, we decided to introduce a manipulation in the eTMT where we control the distance between successive targets and their spatial arrangement among distractors. Following Salthouse and colleagues⁵², it is possible to hypothesize that the task could be simpler, with reduced reaction times, when the next target is adjacent to the target just reached and, conversely, harder, with longer reaction times, when the next target is farther away. However, following Reddy and VanRullen⁶⁰, it is plausible to hypothesize the opposite: when items are adjacent the task is harder due to visual “clutter”, and increasing the spacing between objects improves search performance. Indeed, in this new condition, we systematically manipulated target positions to ensure that half of the targets were near, and half were distant from the previous target. Moreover, following studies on serial dependence⁶¹ and attractive and repulsive bias^68,69,70 it is possible to hypothesize that, in the context of a serial visual search task, a condition of spatial continuity or discontinuity between successive targets can affect performance. Specifically, if the line we have drawn to reach the last target maintains continuity with the direction of the subsequent target (e.g. within a 90° variation), it could be easier to reach the latter with shorter reaction times compared to when spatial discontinuity in the orientation exists between the two targets (e.g. more than 90°). Conversely, based on studies considering the IOR phenomenon^36,72,73,74, it is possible to speculate that participants could be less likely and slower to fixate on the subsequent target when it is in the same direction as the just-reached one. In other words, it is possible to hypothesize that the IOR effect could extend to the inhibition of gaze direction.

Finally, following the results of Experiment 1 regarding the better performance obtained in the black discs condition, we aim to further simplify the task by creating an additional condition in which, once the target has been reached, it completely disappears from the screen. It is possible to hypothesize that although there is a benefit in Experiment 1 from turning the targets into black discs, they may still interfere with visual search, as they are all still visible. With this additional condition, we can verify whether making an item disappear completely can further improve visual search performance, confirming the fact that in TMT visual search can be influenced by the number of distractors. For example, a similar manipulation was performed in a previous digital implementation of the Multi-Item Localization (MILO) task⁴⁶, a task designed to mimic the A and B variants of the TMT [20, 47]. In the digital version of the MILO task, targets either vanished or remained visible after being touched, revealing a greater facilitation effect for vanishing targets, thus an improved visual search performance, specifically in the more complex switching condition, which resembled the demands of TMT-B.

In summary, in Experiment 2, we administered two new conditions: a hypothesized simple one called Disappearing Discs where the just reached target disappears, and a complex one we called MeDiSp (controlling for Memory, Distractors, and Spatial positioning). In the MeDiSp condition it is impossible for participants to memorize the positions of the items (see Manginelli & Pollmann⁵⁹, because each time the participant must search for the next target in a new field of distractors in which the latter are always effective and do not have salient features that can be easily eluded (see Proulx & Egeth⁴³. Additionally, in the MeDiSp condition, we systematically manipulated the spatial arrangement of the test items following precise criteria crossing the two variables Distance (near/far) and Continuity (continuous/discontinuous). Each time the participant reaches a target, all the other items change their position on the display in such a way that the next target can be near or far compared to the just reached target. Moreover, each time the participant reaches a target, all the other items change their position on the display so that the next target will require a continuation of movement in the same direction (Continuous) or in a different one (Discontinuous).

Experiment 2 is thus designed to confirm and expand our understanding of how the display set-size affects TMT performance (Exp. 1) and to explore the influence of spatial positioning of targets, while controlling for implicit learning and discrepancies between Parts A and B.