Introduction

Stroke remains the second leading cause of death and the third leading cause of disability worldwide, significantly impacting global public health1,2. According to the World Stroke Organization-Lancet Neurology Commission, global stroke deaths are projected to increase by 50% from 2020 to 2050, while stroke-related disabilities will rise by 31% during the same period3. In the face of this increasingly concerning trend, access to reliable stroke-related health information is becoming especially important for both healthcare providers and patients.

The rapid development of social media platforms has enabled more people to access health information. Users can now share their experiences and express health needs on these platforms4,5. Social media has become an important channel for health information due to its convenience and interactivity6,7. However, this method of access also brings potential risks, including varying information quality and the spread of misinformation8,9. These risks make it particularly important to evaluate the reliability of various health information on social media platforms. China’s mainstream social media platforms include BiliBili, Douyin, Xiaohongshu, Kuaishou, WeChat, and Weibo, each serving different purposes in the digital landscape. For this study, we selected BiliBili, Douyin, and Xiaohongshu as our research platforms. BiliBili is a video-sharing platform popular among younger users, with 348 million active users monthly. It focuses on long-form professional content10. Douyin is a leading short-form platform with 1.5 billion monthly active users. It provides concise and easy-to-understand information snippets11. Xiaohongshu integrates socialization and content sharing with over 300 million users12. These platforms were selected because they are primarily designed for public video content sharing and discovery, providing systematic search capabilities essential for comprehensive health information analysis. WeChat was excluded as it primarily functions as a private messaging platform13, while Weibo focuses on text-based microblogging14 and Kuaishou has significant user overlap with Douyin15, making these three selected platforms most suitable for video-based health information research. These selected platforms, through their diverse content formats and audience reach, constitute the main channels for health information dissemination in China.

With the proliferation of health content on social media, researchers have been systematically evaluating the information quality of health-related videos on mainstream social media platforms, including BiliBili, Douyin and Xiaohongshu. Previous research results indicate varying information quality of videos across different diseases. Studies on breast cancer, lung cancer and thyroid nodules found that the information quality of video information on BiliBili and Douyin platforms was significantly inadequate16,17,18. Meanwhile, studies on diabetes, thyroid cancer and gastroesophageal reflux considered the information quality on relevant platforms to be satisfactory19,20,21. Additionally, a study on colorectal polyp-related videos on Douyin, WeChat and Xiaohongshu platforms similarly found that the information quality and reliability of these videos were insufficient22. While existing research has evaluated the information quality of health videos related to various diseases, studies assessing local social media videos for stroke, a common and serious disease, remain limited. Current stroke-related video research primarily focuses on the international YouTube platform, including analyses of symptom recognition, acute stroke video information quality and social media activity23,24,25. However, comprehensive multi-platform analysis of stroke-related videos on mainstream Chinese platforms such as BiliBili, Douyin and Xiaohongshu remains insufficient. Therefore, it is necessary to evaluate and analyze the information quality and content characteristics of stroke-related videos on BiliBili, Douyin and Xiaohongshu.

Materials and methods

Search strategy

Basic searches were conducted on December 11, 2024, on BiliBili, Douyin and Xiaohongshu platforms using the standardized medical term “stroke” as the search keyword26. Although colloquial synonyms for stroke are commonly used, this study selected the standardized terminology because it has been widely adopted in previous social media health studies27,28,29, provides greater precision, and aligns with clinical guidelines30, ensuring methodological consistency and research rigor.

To ensure standardized search conditions and minimize algorithm bias, we implemented several control measures. Researchers created new accounts without viewing history to avoid personalized content recommendations. All three platforms were searched on the same day using identical procedures, with searches conducted at BiliBili (9:00–9:30 a.m.), Douyin (10:00–10:30 a.m.), and Xiaohongshu (11:00–11:30 a.m.). The 30-min intervals between searches allowed for systematic data recording while minimizing cross-platform algorithm learning effects. To further control the algorithm’s influence, we avoided clicking on videos during the search phase, used incognito browsing mode, cleared the browser cache between platforms, and maintained consistent search query formatting. All search results were immediately saved in Excel to preserve the exact ranking order. For reproducibility, we recorded search timestamps, captured screenshots of search results, documented complete video URLs with ranking positions, and noted any platform-specific interface variations.

This study selected the top 100 videos from each platform (300 videos total across three platforms) using each platform’s default search ranking function, which automatically arranges content based on factors such as relevance, popularity and recency. While each platform employs different ranking algorithms, using the default search settings reflects how ordinary users access health information on these platforms, ensuring ecological validity of the research findings. This study selected the top 100-ranked videos on each platform as samples for two reasons. First, the method established in previous YouTube health video ranking research indicates that the top 100 ranked videos have significantly higher stability than other videos27. Second, recent studies on gallstones on TikTok and atopic dermatitis on YouTube also used the top 100 videos as samples, confirming that this sample size is suitable for health content analysis on social media28,29. This approach ensures sufficient statistical power while maintaining focus on content most likely to be accessed by general users seeking stroke information.

Video screening and data collection

Data extraction from video pages and publisher homepages was completed within 48 h (December 12–13, 2024) to minimize temporal variations in engagement metrics. Each video’s characteristics and engagement data were recorded at the time of access to maintain consistency across all measurements. Exclusion criteria included duplicate videos (both within-platform and cross-platform duplicates), non-stroke content, purely commercial advertisements and non-Chinese content.

Two independent researchers (A and B) conducted the screening. Researcher A performed initial identification, while Researcher B verified selections. During initial screening, both independently excluded videos failed to meet criteria based on titles and descriptions. Subsequently, both researchers manually reviewed the initially screened videos to confirm stroke-related content. Duplicates were identified by comparing video IDs and content within platforms, and by applying cross-platform duplicate criteria based on content characteristics, including substantially similar titles, comparable duration, visually identical key content, and the same core medical information. For cross-platform duplicates, we distinguished original content from reposts by examining account creation dates, original watermarks, and content alignment with publisher expertise, then prioritized retention based on account verification status, upload chronology, video quality, and credential completeness. Non-Chinese videos were directly identified and excluded by observing the language content. Commercial advertisements were identified by product promotions or purchase links. Inter-rater reliability was assessed using Cohen’s Kappa coefficient. Disagreements were resolved through discussion or by consulting a third researcher (C).

Researchers A and B manually extracted data for all 227 videos by visiting each video page and the publisher’s homepage. They collected video duration and engagement metrics, including likes, collections, comments, shares and video sources. Sources were categorized as health professionals (clinicians, nurses, pharmacists, healthcare workers with health credentials who display qualifications or affiliated health institutions on their profile), general users (individuals without healthcare backgrounds), science communicators (educators, writers, professionals in related disciplines), news organizations (accredited media outlets), nonprofit organizations (health interest groups, patient support organizations, health education institutions) and for-profit organizations (commercial entities like pharmaceutical companies, health device manufacturers, health product sellers). Video characteristics came directly from video pages, while publisher information was based on self-reported data from official account homepages.

Quality assessment tools

This study analyzed stroke-related educational videos from two dimensions: information quality and content completeness.

For information quality assessment, this study selected the Health on the Net Code (HONCode) and the Patient Education Materials Assessment Tool (PEMAT-A/V) to assess video information quality. HONCode was chosen because its eight specific indicators enable multidimensional quality analysis of health information, allowing separate assessment of stroke videos’ authoritativeness, transparency, and reference support31,32. The HONCode framework evaluates eight principles: authority, complementarity, privacy protection, attribution, evidence support, transparency, financial disclosure and advertising policy. The specifics of the HONCode principles are presented in Table 2. Videos were classified by scoring 0–2 points as low compliance, 3–5 points as medium compliance and 6–8 points as high compliance, following similar approaches used in studies on idiopathic pulmonary fibrosis and chronic obstructive pulmonary disease33,34.

PEMAT-A/V was chosen as it’s specifically designed for audiovisual materials, making it ideal for evaluating social media video content35,36. It assesses both understandability (questions 1–13) and actionability (questions 14–17), measuring the educational effectiveness of stroke health education videos36,37. PEMAT-A/V has been validated with good reliability and validity, demonstrating strong internal consistency (α = 0.71) and inter-rater reliability38. The tool uses a binary scoring system where 0 represents the criterion not met, 1 represents the criterion met and NA represents not applicable. The final PEMAT score is calculated by dividing the total points awarded by the number of applicable items (excluding NA responses), then multiplying by 100 to obtain a percentage score. For example, if a video scores 8 points out of 10 applicable items, the PEMAT score would be (8/10) × 100 = 80%. Scores above 70% on the PEMAT indicate effective educational content, while scores below 70% suggest unsatisfactory educational effectiveness35. Detailed scoring criteria are presented in Supplementary Table S1.

To evaluate content completeness, this study divided the content into nine key sections including definition, symptoms, etiology, assessment (medical history, physical examination, laboratory tests), treatment, prognosis, complications, risk factors and prevention. Five sections (definition, symptoms, assessment, treatment and risk factors) were directly derived from the National Clinical Guidelines on Stroke published by the Royal College of Physicians, while the other four sections (etiology, prognosis, complications and prevention) were determined by the research team based on video content characteristics39. Each section used binary coding (0 = not mentioned, 1 = mentioned). The specific stroke code book is available in Supplementary Table S2.

Assessment procedure

Before the formal assessment, both evaluators (two neurologists with rich clinical experience) received standardized training on using these tools and conducted pilot testing on 10% of the sample (23 videos) to ensure a consistent understanding of the assessment criteria. Inter-rater reliability during pilot testing demonstrated excellent agreement, with all Kappa coefficients exceeding the predetermined reliability threshold of 0.840.

Two evaluators independently assessed all videos and recorded their results in separate standardized spreadsheets. For HONCode, evaluators independently assigned scores (0 = non-compliance, 1 = compliance) based on predefined criteria for each principle. For PEMAT-A/V, evaluators applied the binary scoring system according to the tool’s standard criteria. For content analysis, researchers constructed a standardized analysis framework, with two evaluators independently analyzing all videos.

To ensure assessment quality, evaluators compared results to identify discrepancies after independent assessment. When disagreements occurred and Kappa values fell below the preset reliability threshold (Kappa ≥ 0.8)40, a psychiatrist specializing in stroke reviewed the videos in question and facilitated consensus discussions. All assessments demonstrated adequate inter-rater consistency, with Kappa coefficients ranging from 0.84 to 0.87 for PEMAT-A/V, 0.82 to 0.85 for HONcode and 0.88 to 0.92 for content analysis.

Statistical analysis

Statistical analyses were performed using SPSS 29.0 with normality assessed by the Shapiro–Wilk test. For non-normally distributed metrics, pairwise comparisons utilized the Mann–Whitney U test, while overall comparisons employed the Kruskal–Wallis test. The research team compared HONCode compliance and PEMAT-A/V scores across platforms and conducted post hoc tests for significant differences. To evaluate relationships between user engagement metrics (likes, Collections, comments, shares) and information quality assessment indicators (PEMAT-A/V understandability and feasibility, HONCode scores, content richness), Spearman’s rank correlation analysis was applied. A significant level of P < 0.05 was maintained for all statistical tests in the analysis.

Ethical considerations

This study analyzes public content on BiliBili, Douyin and Xiaohongshu platforms, adhering to each platform’s service agreements and data usage regulations. The study collects and analyzes voluntarily published content without direct participant recruitment. Given this study’s nature, this study received an ethics committee exemption from the Biomedical Ethics Committee of Southwest Medical University, confirming that analysis of public social media data does not require a comprehensive ethical review. We follow ethical guidelines for social media research by removing all identifying information and presenting results in aggregate and anonymized form to protect user privacy.

Results

Video selection and screening results

After screening, 227 videos were included in the analysis, with 58 from BiliBili, 88 from Douyin and 81 from Xiaohongshu. The screening process is shown in Fig. 1.

Fig. 1
Fig. 1
Full size image

Flowchart of filtering stroke-related videos for further analysis. Flowchart depicting the screening process of videos from three social media platforms (BiliBili, Douyin, and Xiaohongshu) using the keyword “脑卒中” (stroke). The diagram shows the exclusion criteria and the final inclusion of 227 videos for data analysis.

Video characteristics and source distribution

Video characteristics varied significantly across platforms (Table 1). BiliBili videos were considerably longer and received higher engagement in collections and shares compared to Douyin and Xiaohongshu (P < 0.001).

Table 1 Comparison of video features and content sources on BiliBili, Douyin and Xiaohongshu.
Full size table

Health professionals were the predominant content creators overall (53.3%), with the highest representation on Douyin (77.3%) compared to other platforms. Conversely, general users were more prevalent on BiliBili and Xiaohongshu than on Douyin (P < 0.001). Detailed statistical comparisons are provided in Supplementary Table S3.

Information quality

The data analysis shows that most videos (72.7%) met the medium-level standards of HONCode criteria (principles 3–5), with significant differences observed among platforms (P < 0.001) (Table S4). Douyin performed best in this respect (85.2%), significantly outperforming BiliBili (63.8%) and Xiaohongshu (65.4%). Regarding PEMAT-A/V assessment, statistically significant differences were found among the three platforms for understandability (P = 0.032), with Xiaohongshu scoring lowest (median 0.73), performing worse than BiliBili (median 0.81) and Douyin (median 0.75). In terms of content completeness assessment, significant differences also existed among platforms (P < 0.001), with Xiaohongshu scoring lowest (median 2.00), inferior to BiliBili (median 3.00) and Douyin (median 3.00). Detailed pairwise statistical comparisons between platforms are provided in Table S4. Visual comparisons of these quality indicators across platforms are presented in Fig. 2.

Fig. 2
Fig. 2
Full size image

Information quality assessment across social media platforms. Comparison of information quality assessment indicators across BiliBili, Douyin, and Xiaohongshu platforms. (a) HONCode compliance levels showing the percentage of videos meeting low (0–2 principles), medium (3–5 principles), and high (6–8 principles) compliance standards. (b) PEMAT-A/V median scores for understandability and actionability dimensions across the three platforms.

Table 2 reveals significant differences among the three platforms in adherence to five key HONCode principles. These include principle 1 (health advice must come from professionals, P < 0.001), principle 2 (support patient self-management, P < 0.001), principle 3 (protect patient privacy, P = 0.003), principle 6 (provide contact information, P < 0.001) and principle 7 (identifying contributors, P < 0.001). Detailed pairwise statistical comparisons between platforms for all HONCode principles are provided in Supplementary Table S5.

Table 2 Comparison of HONCode principles compliance across platforms.
Full size table

Content analysis

Table 3 compares stroke-related video content metrics across the three platforms. Overall, treatment was mentioned most frequently (63.0%), followed by symptoms (55.1%), prevention (30.8%) and risk factors (26.9%). Assessment (15.4%) and complications (15.0%) were mentioned least frequently. For the assessment category, BiliBili (22.7%) performed significantly better than Xiaohongshu (4.9%) and Douyin also showed substantially better performance than Xiaohongshu (P = 0.004). Comprehensive statistical comparisons between platforms for all content categories are provided in Supplementary Table S6.

Table 3 Comparison of stroke-related video content indicators among BiliBili, Douyin, and Xiaohongshu.
Full size table

Correlation analysis

Table 4 shows no or minimal relationships between most information quality metrics and audience engagement measures. Overall, actionability demonstrated weak positive relationships with likes (r = 0.141, P = 0.034), favorites (r = 0.207, P = 0.002) and shares (r = 0.204, P = 0.002). On the Xiaohongshu platform, HONCODE criteria showed a weak positive correlation with favorites (r = 0.228, P = 0.041) and a weak negative relationship with comments (r = − 0.244, P = 0.028). Actionability on Xiaohongshu displayed weak positive associations with likes (r = 0.284, P = 0.010), favorites (r = 0.345, P = 0.002) and shares (r = 0.334, P = 0.002). It should be noted that while these correlations reached statistical significance, the effect sizes were generally small (r = 0.141–0.345), representing weak correlations that explain only 2–12% of the variance, indicating limited practical significance.

Table 4 Spearman correlation between audience engagement metrics and video information quality assessment (PEMAT-A/V, HQNCode, Content) across BiliBili, Douyin, and Xiaohongshu Platforms.
Full size table

Discussion

Principal findings

This study evaluated stroke-related videos across three social media platforms BiliBili, Douyin and Xiaohongshu. Videos demonstrated moderate information quality overall, with Douyin’s information quality relatively higher. Most videos lacked proper references and evidence-based support. All platforms exhibited relatively high understandability but limited actionability, with Xiaohongshu scoring lowest in understandability. Regarding content, completeness was insufficient across platforms, particularly on Xiaohongshu. Videos mainly focused on stroke treatment and symptoms, while stroke assessment and complications received less attention. Finally, findings revealed minimal correlation between information quality and user engagement for health videos on social media platforms.

Information quality of health videos across platforms

This study found that stroke-related videos on BiliBili, Douyin, and Xiaohongshu demonstrated moderate overall information quality, similar to previous findings on lymphedema and gastroesophageal reflux41,42. However, our results differ from the unsatisfactory information quality reported in the Helicobacter pylori study on Chinese social media43. This difference may stem from varying assessment methods. The Helicobacter pylori study employed the GQS assessment tool, which provides only a single overall score through five grades, potentially overlooking the multifaceted nature of health information44. In contrast, our study utilized the HONCode tool, which evaluates eight specific dimensions, allowing for a more comprehensive assessment and potentially identifying quality aspects that a single-score tool might miss31. Furthermore, comparative analysis across platforms found that stroke videos on Douyin had significantly higher information quality than those on BiliBili and Xiaohongshu. This phenomenon may stem from the combined effects of multiple factors. First, the study found that healthcare professionals created 77.3% of videos on Douyin, while this proportion was only 20.7% on BiliBili and 50.6% on Xiaohongshu. Previous research on sinusitis and gastric cancer content also indicates that health education videos produced by healthcare professionals typically demonstrate higher information quality45,46. Second, differences in platform content moderation and creator incentive policies may also affect content quality47, and Douyin’s relatively comprehensive medical content verification mechanisms may promote the production of high-quality content48. Third, algorithm design and user demographic characteristics may also contribute to these platform differences. Platform recommendation algorithms may influence the visibility of different content types, potentially affecting how health information from various sources reaches users49,50. Additionally, platforms may attract different user populations with varying demographic characteristics and health information-seeking behaviors51,52, which could influence the types of content creators who choose to engage and the overall content landscape.

Regarding HONcode principles, the three platforms exhibited common characteristics. They performed best on HONcode Principle 1 (requiring health advice in videos to come from qualified professionals unless otherwise stated) but performed relatively unsatisfactorily on Principle 4 (requiring clear references) and Principle 5 (requiring scientific evidence to support claims). This aligns with previous research findings on fall prevention, lymphangioleiomyomatosis, and vitreoretinal surgery53,54,55. The limited citation practices and scientific foundation in these videos may affect content credibility and increase the possibility of sharing less accurate health information.

Videos across platforms exhibited favorable understandability with constraints in actionability, with findings similar to laryngeal carcinoma and diabetes medication studies56,57, but showing variations compared to bladder pain syndrome research58. This variance may stem from disease characteristics. Stroke, as an acute condition with complex pathophysiology, is more challenging to address in short videos compared to chronic conditions like bladder pain syndrome that have established self-management protocols. Stroke content on Xiaohongshu demonstrated lower understandability, showing differences from findings on non-small cell lung cancer information, possibly due to stroke’s interdisciplinary nature and terminology complexity59. Given the stroke’s time-sensitive nature, this gap between understanding and actionability may particularly hinder viewers’ ability to respond effectively in emergencies where rapid intervention is critical. The PEMAT actionability scores in this study (median 0.67) reinforce this concern, indicating that while stroke videos effectively convey information, they fall below the 70% threshold for guiding viewers toward concrete actions. This actionability limitation is particularly significant for stroke care, where the transition from symptom recognition to appropriate response, such as using assessment tools like FAST, seeking emergency care, or implementing preventive behaviors, directly impacts patient outcomes and treatment timing.

This study found that stroke-related content generally lacks evidence transparency and source referencing. Previous research indicates that credible health information should maintain transparency and include reliable source references60,61; accordingly, this study recommends that video creators improve evidence transparency and provide source references for stroke-related content. Additionally, this study found that stroke videos across platforms have limitations in actionability. According to PEMAT standards, actionability refers to patients’ ability to identify specific actions from materials through clear, manageable steps35. This study suggests incorporating specific action guidelines with step-by-step instructions in videos to enhance actionability. Furthermore, this study found lower healthcare professional participation on BiliBili and Xiaohongshu compared to Douyin. Considering that previous research shows professionally created content demonstrates higher quality62, this study encourages greater healthcare professional participation, particularly on BiliBili and Xiaohongshu, to improve the overall content quality of stroke-related videos.

Content platform analysis

From a content integrity perspective, this study found that the content integrity of all three platforms was unsatisfactory. Xiaohongshu’s content integrity was lower than BiliBili and Douyin, possibly because it focuses more on personal experiences and lifestyle sharing. This positioning makes creators more inclined to present certain health information selectively rather than provide comprehensive knowledge needed for disease management. Regarding content distribution, treatment was most frequently mentioned across all three platforms, followed by symptoms, similar to previous research on anal fissures and cryptorchidism63,64. Assessment and complications were mentioned least often. Proper assessment helps patients obtain accurate diagnoses, while understanding complications is essential for informed decision-making65. Patients may underestimate disease severity or ignore symptoms requiring immediate attention66. Therefore, this study recommends that video publishers improve video content integrity, especially on Xiaohongshu, and all platforms should increase content about assessment and complications.

Information quality and engagement correlation

This study found that for stroke-related videos on Bilibili, Douyin, and Xiaohongshu platforms, most information quality indicators (HONCode, PEMAT-A/V Understandability, and Content Completeness) showed no correlation with user engagement, while only a few information quality indicators (PEMAT-A/V Actionability) showed positive correlations with user engagement. However, these significant correlations were weak in strength (r = 0.141–0.345), suggesting limited practical importance. This finding indicates that high-quality health information content does not necessarily receive more user attention and interaction, with information quality having minimal overall influence on user engagement, a finding similar to previous research on laryngeal cancer, liver cancer, and gallstone disease28,52,67. Notably, the actionability of videos on Xiaohongshu showed positive correlations with user engagement indicators. This platform difference may stem from Xiaohongshu’s unique positioning, which emphasizes content practicality and lifestyle sharing, making its users more attentive to video actionability during interaction68.

This study found that high-quality health information content does not necessarily receive more user attention and interaction across platforms, with only Xiaohongshu showing positive correlations between actionability and user engagement due to its platform positioning that emphasizes practical content. Furthermore, previous research demonstrates that visually appealing elements, storytelling techniques, and interactive elements can significantly improve health information acceptance and user engagement69,70,71. Therefore, this study recommends that video publishers adopt visual design improvements, storytelling techniques, and interactive elements to enhance user engagement across all platforms, rather than merely adjusting content according to platform characteristics.

Limitations and future directions

First, our search strategy was limited to a single keyword and did not consider other synonyms (e.g., colloquial terms), which may have affected the comprehensiveness of our results. Our approach may have missed relevant educational content using alternative terminology. Future research should implement multi-keyword search strategies that include synonyms and alternative expressions to provide more comprehensive coverage. Second, this study evaluated platforms in China, which may limit the universal applicability of the results; future research could conduct cross-linguistic and cross-cultural comparisons. Third, our video search was limited to a single date, overlooking the dynamic nature of platform content updates and temporal trends in stroke-related content evolution, which may have resulted in missing important stroke-related content that varies over time. Future research should employ multi-timepoint sampling strategies to more comprehensively capture temporal distribution characteristics and content evolution patterns on these platforms. Fourth, this study did not analyze user engagement data such as comments, which could provide additional insights into content reception. Future research could incorporate other data related to user engagement, such as user comments, to gain comprehensive insights into content reception. Fifth, platform algorithms may influence content ranking and visibility, though we used new accounts to minimize personalization effects. Future research could employ more effective methods to mitigate algorithmic influences.

Conclusion

This study evaluated stroke-related videos on BiliBili, Douyin, and Xiaohongshu platforms. The findings reveal that these videos provide moderate-quality information but generally lack scientific evidence support, with actionability and completeness requiring improvement. Content primarily focuses on treatment while neglecting assessment and complications. Douyin demonstrates higher information quality, while Xiaohongshu shows inadequate performance in understandability and completeness. This study recommends that video publishers enhance scientific evidence transparency, provide reliable references, improve video actionability by incorporating specific action guidelines with step-by-step instructions, and strengthen content completeness by increasing assessments and complications and other related aspects. Healthcare professionals should actively create health education videos across platforms, especially on BiliBili and Xiaohongshu. Video publishers can increase information acceptance by improving visual design, storytelling, and interactive elements rather than merely platform-specific adjustments.