P3.003: Assessing the Quality of ADHD Information on Tik tok

Background: Social media platforms continue to rise in popularity, ushering in a new era of using these resources as a primary information source. While apps like TikTok serve as a vast source of information on a variety of topics, including medical diagnoses and treatment, the degree to which these videos provide accurate and clinically relevant insight is uncertain. This study explores the content of videos uploaded to TikTok specifically referencing ADHD using the DISCERN scale to rate the quality of information provided and compare it across content creator type and hashtags.

Methods: We used Apify, a web scraping application, to extract TikTok videos associated with the hashtags #ADHDtreatment, #ADHDeducation, and #ADHDawareness. Three independent viewers watched 100 videos per hashtag, using a modified DISCERN scale to evaluate the reliability of each video’s content. Additionally, videos discussing specific ADHD treatments were further analyzed using Part 2 of the DISCERN scale to evaluate the reliability of treatment-related information. We used an independent t-test to compare DISCERN scores between videos created by healthcare providers and those that were not. A one-way ANOVA compared DISCERN scores across hashtags, followed by post-hoc Games-Howell tests.

Results: After excluding 67 videos due to being duplicates or non-English language, 233 videos were included in the final analysis (59 videos created by healthcare professionals and 174 videos created by non-healthcare professionals). There were 91 videos with the hashtag #ADHDtreatment, 72 videos with #ADHDawareness, and 70 videos with #ADHDeducation. Videos by healthcare professionals (M = 18.70, SD = 6.09) had significantly higher DISCERN scores compared to those by non-healthcare professionals (M = 11.68, SD = 5.27; t(231) = 8.491, p<.001). However, there was no significant difference in part 2 DISCERN scores between treatment-related videos by healthcare professionals (M = 12.59, SD = 3.45) and non-healthcare professionals (M = 11.79, SD = 4.07; t(44) = 0.70 p = 0.24), suggesting there was similar reliability in treatment-related information regardless if the video creator was a healthcare professional or not. A one-way ANOVA revealed a significant difference in the DISCERN score across the three hashtags (F(2) = 25.383, p<.001). Post-hoc analysis showed that videos tagged with #ADHDtreatment had a significantly higher DISCERN score compared to #ADHDeducation (mean difference = 3.94, p<.001) and #ADHDawareness (mean difference = 6.28, p<.001). Videos tagged with #ADHDeducation was also found to have a significantly higher DISCERN score than #ADHDawareness (mean difference = 2.33, p = .013).

Conclusion: This study found Tik Tok videos created by healthcare professionals had overall higher quality scores, but treatment-specific reliability was consistent across creators. This underscores the importance for users to critically assess credibility of online health content. Future work should explore ways to amplify reliable, evidence-based information across social media platforms.