Introduction

Mental health issues are emerging as major public health concerns in modern society, and accurate data collection is essential for devising effective policy interventions. However, because mental health is a sensitive topic, respondents may not accurately report their actual conditions owing to social desirability bias [1-3]. This bias can undermine the reliability of survey results and impede the development of effective policies. Therefore, surveys on the status and perception of mental health require methodological approaches that ensure respondent’ anonymity and encourage honest responses.

Widespread accessibility of the Internet has revolutionized global survey methodologies. In 2024, the global Internet penetration rate was 68%, and in South Korea, it exceeded 98% [4]. Web-based surveys are efficient in terms of time and cost and provide participants with anonymity, which increases the likelihood of obtaining honest responses to sensitive questions. Consequently, Internet surveys have become a vital tool in fields such as mental health, social attitudes, and political awareness.

Despite these advantages, online surveys pose challenges, particularly in terms of data quality and respondent representativeness. Selection bias, response validity, and digital accessibility disparities are significant concerns that must be addressed when interpreting online survey data. Compared with traditional paper-and-pen surveys, online surveys offer the advantages of faster and cheaper respondent recruitment and data collection. These features render them particularly useful in studies targeting large populations. For example, Internet-based surveys enable data collection across diverse cultural contexts and geographical regions unrestricted by physical boundaries, making them cost-effective.

However, selection bias is a significant issue in such surveys [5]. In online surveys, respondent pools may be limited by factors such as Internet access, technological accessibility, and digital literacy. This limitation often results in lower response rates among digitally vulnerable groups, such as older adults or low-income populations. Consequently, the representativeness of the data may be compromised, raising concerns about the reliability and generalizability of the survey’s findings [6].

Additionally, online surveys may face challenges regarding the truthfulness of responses to sensitive topics, which may vary depending on the level of anonymity provided. This issue is particularly prominent in surveys addressing sensitive subjects, such as mental health [7]. Sensitive survey items are often associated with high nonresponse rates and increased response measurement errors [8], particularly when the survey topic involves social desirability or political correctness [9]. Rather than revealing their true opinions or experiences, respondents may respond in ways they perceive as socially acceptable. This tendency is particularly evident in topics such as income levels, sexual behaviors, and religious beliefs, ultimately reducing data accuracy and validity [10].

Some studies have suggested that online surveys elicit more honest responses to sensitive questions than offline surveys do [11]. The anonymity offered by online surveys likely plays a critical role, as respondents can answer questions without concerns of being observed or judged [12]. However, these findings are not universally applicable, and the results may vary depending on the subject matter and survey question type.

Mental health surveys are gaining importance in South Korea. Social stigma and desirability biases often make obtaining accurate data for mental health research challenging. Thus, analyzing how online and offline survey methods influence mental health survey outcomes is an important research topic. Additionally, examining the effects of demographic factors such as sex, age, and occupation on mental health perceptions in South Korea is crucial for both academic advancement and practical policy recommendations.

South Korea currently operates two distinct national mental health surveillance systems: the National Mental Health Survey, which employs face-to-face interviews using the structured Korean Composite International Diagnostic Interview (K-CIDI), and the National Mental Health Literacy and Attitude Survey, which uses web-based self-report questionnaires. These systems differ not only in survey mode (interviewer-administered vs self-administered online) but also in their assessment instruments and diagnostic thresholds. Understanding how these operationally distinct systems yield different prevalence estimates and perception data is crucial for policymakers who must synthesize information from multiple sources.

This study aimed to systematically compare the mental health reporting patterns and public perceptions between these two national surveys. Rather than claiming to isolate a pure “survey mode effect,” which would require identical instruments administered across different modes, we examined how the overall survey context (encompassing mode, instrument, and setting) is associated with different reporting patterns. Additionally, we explored whether demographic factors, such as sex, age, and occupation, were associated with mental health perceptions across both surveys. Through this comparison, we sought to provide practical guidance for interpreting national mental health statistics and inform future methodological improvements in mental health surveillance.

Materials and Methods

1. Participants

1) National Mental Health Survey 2021, for offline data

The National Mental Health Survey is a mental health study of adults residing in South Korea (aged 18~79 years), conducted every five years, based on Article 10 of the Mental Health Welfare Act. Its purpose is to provide foundational data for the development of mental health policies by identifying the prevalence and risk factors for mental disorders, their sociodemographic distribution, related factors, and mental health awareness and attitudes [13]. Because this study focused on the relationship between individuals’ sociodemographic characteristics and their perceptions of and attitudes toward mental health, it analyzed data from 5,511 participants in the National Mental Health Survey from June to August 2021. This survey used face-to-face interviews conducted by trained interviewers using the K-CIDI to diagnose and analyze the presence or absence of psychiatric disorders. Additionally, supplementary instruments were used to identify perceptions, attitudes, and various related factors using additional survey items [14]. In-person interviews were conducted by visiting the selected households using tablet PCs equipped with diagnostic tools. Self-administered questionnaires were used for some tools.

2) National Mental Health Literacy and Attitude Survey 2021, for online data

We also used data from the National Mental Health Literacy and Attitudes 2021 survey conducted by the Ministry of Health and Welfare. The survey sample consisted of 2,016 citizens aged 15~69 years residing across South Korea. Participants were selected using proportional allocation based on population statistics by sex, age, region, and administrative district. The survey was conducted online from June 18 to 30, 2021. The survey gathered information on the participants’ sociodemographic characteristics, mental health literacy, and current status regarding mental health issues [15]. It employed a case vignette-based mental health perception questionnaire developed in 2021 [16].

Table 1 presents the demographic characteristics of the study participants, including age, sex, occupation, educational level, and income. Age was categorized into six groups: teenagers and adults in their 20s, 30s, 40s, 50s, and 60s or older. Occupation was classified into 13 categories based on the Korean Standard Occupational Classification system. Formal education was divided into five levels: elementary school, middle school, high school, university graduate, and graduate school or higher. Monthly income was classified into eight categories.

2. Survey questions

The two surveys used slightly different formats to assess mental health disorders. In the offline survey, structured diagnostic tools were employed, and participants were asked specific questions, including, “During your life, have you ever felt sad, empty, or depressed almost all day, every day, for more than two weeks?” (depressive disorder); “Have you ever experienced excessive or persistent worry?” (anxiety disorder); “Have you ever seriously thought about committing suicide?” (suicidal thoughts); “Have you ever been bothered by unwanted, persistent thoughts? For instance, have you had obsessive concerns about cleanliness, germs, or other intrusive thoughts?” (obsessive-compulsive disorder [OCD]); and “During your life, have you frequently been intoxicated or impaired to the extent that it interfered with school, work, or home life?” (alcohol use disorder [AUD]). Conversely, the online survey relied on simple yes/no questions designed to address the same mental health conditions. Participants were asked whether they had experienced depressive feelings lasting several days, anxiety lasting several days, suicidal thoughts, alcohol-related problems, uncontrolled obsessive thoughts, or compulsive behaviors, such as repetitive hand washing or checking.

Although the original K-CIDI items allow the derivation of both lifetime and 12-month diagnoses, and the online items reference experiences within the past 12 months, differences in recency cannot be perfectly harmonized. Therefore, we analyzed all responses as binary “ever” experience indicators, acknowledging that residual timeframe differences may remain. We acknowledge that the two surveys employed different assessment instruments. The offline survey used the K-CIDI, a structured diagnostic interview, whereas the online survey used symptom-based self-reporting questions. This methodological difference is a limitation of the present study.

3. Statistical analysis

Logistic regression analyses were conducted to investigate the differences in the lifetime prevalence rates of mental disorders between online and offline respondents. The primary aim of these analyses was to determine whether significant differences existed between the two groups while controlling for age, sex, income, and education levels as covariates. The dependent variables in the analyses were lifetime prevalence rates of mental disorders, including depression, anxiety, OCD, AUD, and suicidal thoughts. The key independent variable was the respondent group, categorized as either offline (Group 1) or online (Group 2). The regression models were controlled for covariates, including age (as a continuous variable), sex (male or female), income level, and education level (both as categorical variables).

Descriptive statistics were first calculated to summarize the demographic characteristics of the two groups and explore their distributions. Logistic regression analyses were performed in two stages. In the univariate analyses, the independent effects of each covariate and group variable on lifetime prevalence rates were assessed. In the multivariable analyses, all covariates were included in a single model to evaluate the independent contributions of the group variables while accounting for potential confounding effects. The significance of each predictor was assessed using Wald Z-tests, with p ＜.05 considered statistically significant. The regression results for each disorder were reported using estimated coefficients, standard errors (SEs), and p-values to quantify the differences between groups. The model fit was evaluated using the -2 log likelihood and Akaike Information Criterion to ensure robustness.

The analyses were conducted using Python’s “statsmodels” package for logistic regression, with data preparation performed using Pandas. Matplotlib and Seaborn were used to visualize the results. Given multiple comparisons across the five mental health outcomes and four perception variables, we applied the Benjamini-Hochberg procedure to control the false discovery rate (FDR) at 0.05. Both unadjusted and FDR-adjusted p-values were reported. Because the publicly available microdata for both surveys did not include the design weights required for population-level estimation, unweighted data were used for all analyses. Consequently, the results reflect the characteristics of the sample rather than those of the entire population, and model-based SEs were reported.

4. Ethical considerations and approvals

This study was approved by the Institutional Review Board of the National Center for Mental Health (IRB No. 116271-2024-04).

Results

The logistic regression analysis results comparing the lifetime prevalence rates of depression, anxiety, OCD, AUD, and suicidal thoughts between the two groups are presented in Table 2. These analyses were conducted to evaluate whether significant differences existed between online and offline respondents while controlling for age, sex, income, education level, and occupation. For each disorder, regression coefficients, SEs, odds ratios (ORs), and p-values were used to quantify group differences. The dependent variables were the presence of depressive or anxiety disorders, OCD, AUD, or suicidal thoughts (no=1, yes=2). The predictor variable was the group (online vs offline), with age, sex, occupation, education, and income included as control variables.

For depressive disorder, the coefficient of the group variable (offline vs online) was -1.761 (SE=0.085, p＜.001). The corresponding OR of 0.172 indicated that offline respondents had 82.8% lower odds of reporting depression than online respondents (95% CI, 0.145~0.200). Equivalently, online respondents had approximately 5.81 times higher odds (1/0.172). For anxiety disorders, the coefficient of the group variable was -1.363 (SE=0.089, p＜.001). An OR of 0.256 indicated that offline respondents had 74.4% lower odds of reporting anxiety than online respondents (equivalently, online respondents had approximately 3.91 times higher odds).

For AUD, the analysis showed that the coefficient of the group variable was 0.286 (SE=0.094, p=.002), indicating that offline respondents had significantly higher odds of reporting AUD than online respondents. The OR for the group variable was 1.331, indicating that offline respondents were 33.1% more likely to report AUD than their online counterparts. The intercept was −2.249 (SE=0.580, p＜.001), representing the baseline log odds of AUD when all predictors were at their reference levels. This pattern (higher AUD reporting in offline surveys) was the opposite of that observed for internalizing disorders.

For OCD, the coefficient of the group variable was −3.95 (SE=0.262, p＜.001), indicating that online respondents were significantly more likely to report OCD than offline respondents. An OR of 0.019 indicated that offline respondents had 98.1% lower odds of reporting OCD than online respondents (equivalently, online respondents had approximately 52.6 times higher odds).

Finally, for suicidal thoughts, the coefficient of the group variable (offline vs online) was 0.13 (SE=0.093, p=.163). Although the OR was 1.139, suggesting slightly higher odds for offline respondents, this difference was not statistically significant. The difference between groups was not statistically significant. After applying the Benjamini-Hochberg correction for multiple comparisons, all primary findings for depression, anxiety, OCD, and AUD remained statistically significant (adjusted p＜.05). The nonsignificant result for suicidal thoughts remained unchanged even after correction.

A series of linear regression analyses were conducted to examine the differences in the perceptions of individuals with mental disorders between online (Group 1) and offline respondents (Group 2). Each analysis focused on a specific aspect of public attitudes toward mental disorders, with the group (online vs offline) as the primary independent variable and age, sex, occupation, education level, and income as control variables.

The first analysis explored attitudes toward forming friendships with individuals with mental disorders. The dependent variable measured the level of agreement with the statement, “I cannot be friends with someone who has a mental disorder,” with higher values indicating more negative perceptions. The regression coefficient of the group variable (offline vs online) was 0.031 (p=.025), indicating that offline respondents demonstrated significantly more negative perceptions than online respondents. Age also showed a significant effect, suggesting that older individuals were more likely to hold negative views than younger individuals.

The second analysis addressed perceptions of hiring individuals with mental disorders. The dependent variable measured agreement with the statement, “I cannot hire someone with a mental disorder as an employee,” with higher values indicating more negative perceptions. The regression coefficient of the group variable was 0.025 (p=.032), indicating that offline respondents exhibited significantly more negative attitudes toward hiring than online respondents. Age had a significant positive effect, suggesting that compared with younger individuals, older individuals were more likely to express negative attitudes toward hiring individuals with mental disorders.

The third analysis focused on the perception of danger associated with individuals with mental disorders. Agreement with the statement “People with mental disorders are dangerous” served as the dependent variable, with higher values indicating more negative perceptions. The regression results indicated that the coefficient of the group variable was 0.022 (p=.041), suggesting that offline respondents had significantly more negative perceptions (perceived greater danger) than online respondents. Age had a significant positive effect, indicating that compared with younger individuals, older individuals were more likely to perceive individuals with mental disorders as dangerous. Other control variables, such as sex and occupation, did not consistently exhibit statistically significant effects.

The final analysis examined beliefs regarding the persistence of mental disorders over time. The dependent variable measured agreement with the statement “mental disorders persist for a lifetime,” with higher values indicating stronger agreement (stigma). The regression results showed that the coefficient of the group variable was 0.017 (p=.017), indicating that offline respondents were more likely to agree with the statement than online respondents. Age was a significant factor, suggesting that compared with younger individuals, older individuals were more likely to believe that mental disorders persist throughout their lifetimes. The linear regression findings on public perceptions of mental disorders, examining attitudes toward employment, friendships, perceived danger, and belief in lifelong persistence, are detailed in Table 3.

As a robustness check, we restricted the sample to participants aged 18~69 years (the overlapping range) and re-estimated all logistic and linear regression models. The direction, magnitude, and statistical significance of the group effects remained consistent with the main findings, indicating that the findings were not driven by differences in the age range coverage between the two surveys.

Discussion

This study compared the mental health reporting patterns of two national surveillance systems in South Korea. The findings revealed substantial differences between these systems, which may reflect a combination of the influence of survey mode effects (anonymity vs interviewer presence), differences in assessment instruments (structured diagnostic criteria vs simplified symptom questions), and their interactions. Although we may not definitively disentangle these contributing factors from the current design, the observed patterns offer important insights into both methodological refinement and policy interpretation.

Before discussing potential explanations, we emphasize that the following interpretations are necessarily speculative. Because this study compares two independently designed survey systems that differ in both survey mode and instruments, there are inherent limitations in clearly disentangling the pure effects of the survey mode from those of the survey instruments. Accordingly, the possibilities and explanatory mechanisms discussed below should not be interpreted as definitive causes of the observed results but rather as hypothetical explanations that may have contributed to the observed patterns.

The principal finding was the substantial divergence in prevalence estimates between the two surveillance systems. Respondents in the web-based survey reported markedly higher rates of internalizing disorders, specifically depression, anxiety, and OCD, than those in the interview-based survey. Several nonmutually exclusive explanations may account for this pattern. First, one possible contributing factor is reduced social pressure in anonymous web-based platforms, sometimes termed “online disinhibition,” which may facilitate a more candid disclosure of stigmatizing symptoms [17]. In face-to-face interviews, respondents may engage in impression management and underreport distress to avoid perceived judgment [12]. Second, the simplified self-reported items in the online survey may have lower diagnostic thresholds than the K-CIDI’s structured criteria, capturing a broader range of subclinical experiences. The online depression item, for instance, asked about “depressive feelings lasting several days,” whereas the K-CIDI required symptoms “almost all day, every day, for more than two weeks.” Instrument effects and mode-specific differences may operate differently across disorder types, potentially producing divergent, rather than uniform, patterns of bias. Thus, the observation of different directions of prevalence differences across conditions does not necessarily imply mode-related differences; disorder-specific measurement properties may also contribute to these patterns.

Conversely, AUD was reported at a higher rate in the interview-based survey than in the web-based survey. This reversal complicates a simple instrument-threshold explanation, arguing against a purely instrument-driven explanation. However, several of these interpretations require further investigation. First, drinking is deeply embedded in South Korean social and business culture, potentially rendering alcohol-related disclosures less stigmatizing than other psychiatric conditions in interpersonal contexts [18]. South Korean drinking culture is characterized by “hoesik,” workplace social gatherings where alcohol consumption serves as a crucial organizational socialization tool [19]. Research indicates that more than 80% of South Korean employees engage in binge drinking at least once within a 12-month period, and the “bottom-up” drinking culture, in which participants are expected to finish drinks in one shot, normalizes heavy alcohol consumption as a means of building workplace solidarity [20]. This cultural context, in which intoxication is often viewed as a sign of workplace camaraderie rather than a problem behavior, may facilitate a more open disclosure of drinking patterns during face-to-face interviews where social rapport is established. The rapport and social scaffolding of face-to-face interviews may facilitate discussions on drinking behaviors that are normatively accepted.

Second, the assessment items differed substantively between surveys. The K-CIDI alcohol module includes specific, behaviorally anchored questions about intoxication frequency (“Have you frequently been intoxicated to the extent that it interfered with work/home?”), which may prompt a concrete recall and disclosure during conversational interviews. In contrast, the online survey’s broader framing (“Have you had alcohol-related problems?”) requires respondents to self-categorize their drinking as “problematic,” a judgment that may be culturally influenced. In a society where heavy drinking and intoxication are normatively accepted and even encouraged as social bonding mechanisms [21], respondents completing the online survey may not perceive their drinking behaviors as constituting “problems,” even when those behaviors would meet diagnostic criteria in a clinical assessment. This interpretation aligns with research indicating that South Korean society maintains a tolerant attitude toward alcohol consumption, leading to many individuals remaining unaware of their drinking problems [22].

Third, disorder-specific factors may have contributed to this reverse pattern. Individuals with AUD often exhibit limited insight into the severity of their drinking problems, which may reduce self-reported prevalence in online surveys where no interviewer prompts self-reflection. Structured interviews with probing questions may facilitate a more accurate recognition of problematic drinking patterns. Conversely, the absence of interviewer guidance in online self-reports may lead to underreporting of alcohol-related symptoms that respondents do not spontaneously identify as problematic. However, we acknowledge that the AUD items also differed substantially between surveys (see Table 4). Thus, the reverse pattern, which suggests mode-specific social dynamics, does not constitute definitive evidence. Alternative explanations, such as unmeasured differences in respondents’ characteristics or differences in item interpretations, cannot be ruled out.

The two surveys yielded similar prevalence estimates for suicidal ideation, with no statistically significant between-survey differences. This null finding is interpretable considering the relatively high comparability of suicidal ideation items across surveys (see Table 4). Both items directly asked about lifetime suicidal thoughts with similar wording, unlike the more divergent items in the other conditions. The convergence of estimates for suicidal ideation suggests that when items are sufficiently similar, the two surveillance systems may yield comparable data, implying that the observed differences in other conditions may partly reflect item-level discrepancies. Alternatively, the stigma surrounding suicide may be so deeply entrenched that neither the anonymity of online surveys nor the rapport of face-to-face interviews substantially affects disclosure [9]. This interpretation aligns with South Korea’s persistently high suicide rates and strong cultural taboos regarding discussing suicidal thoughts. These competing explanations cannot be adjudicated with the current data. Future studies employing identical suicidal ideation measures across randomized survey modes would help clarify whether the observed convergence reflects true mode equivalence or coincidental measurements.

The linear regression analyses revealed that survey context was also associated with the expression of attitudes toward mental illness. Respondents in the interview-based survey expressed more stigmatizing views regarding hiring, befriending, and the perceived dangerousness of individuals with mental disorders than the web-based respondents. This pattern may reflect “public versus private” self-presentation: Individuals may feel compelled to express socially conservative or cautious attitudes when speaking to an interviewer (representing a public, accountable context) but hold or express more progressive views in the privacy of online self-completion. Alternatively, despite the statistical controls, residual differences between the two respondent populations, such as differential exposure to mental health advocacy or digital literacy correlates, may have contributed to these patterns. As with the prevalence findings, we could not isolate a pure-mode effect from other systematic differences between the surveys. However, the consistency of the patterns across multiple attitude measures suggests that the survey context meaningfully shapes expressed stigma, with implications for how public opinion data on mental health are collected and interpreted.

Our findings are broadly consistent with international research demonstrating a higher disclosure of internalizing symptoms in computer-administered and online surveys than in interviewer-administered modes [10]. A meta-analysis by Gnambs and Kaspar [10] found that sensitive behaviors were more frequently reported in self-administered modes, with effect sizes varying with topic sensitivity. However, the reversal observed for AUD in our study appears to be more context-specific and may reflect the unique cultural position of alcohol consumption in South Korean society. Cross-cultural comparative studies examining mode effects across societies with varying attitudes toward alcohol consumption would help determine whether this pattern is culturally specific or generalizable. Additionally, while Western studies have typically found that online anonymity uniformly reduces social desirability bias, our results suggest that the direction of mode effects may be disorder-specific, warranting further investigation into the psychological mechanisms underlying these differential patterns.

The substantial differences between the two national mental health surveillance systems in South Korea have important implications for data interpretation and policy planning. Policymakers who rely on these data sources should recognize that prevalence estimates are not directly comparable across systems. The interview-based K-CIDI survey likely yields more conservative, clinically aligned estimates, whereas the web-based self-report survey may capture a broader spectrum of self-perceived distress. Rather than viewing one system as superior, a pragmatic approach would leverage the complementary strengths of each. Web-based surveys, with fewer barriers to disclosure, may be valuable for detecting hidden distress in populations reluctant to seek formal evaluations. Interview-based surveys with diagnostic rigor provide data that are directly comparable to those of clinical populations and international epidemiological standards. Integrating the findings from both systems, while accounting for their methodological differences, may offer a more complete picture of the mental health of the South Korean population than either system alone.

However, caution should be exercised when translating these findings into specific policy recommendations. A higher self-reported prevalence in online surveys does not necessarily indicate more accurate detection, and without validation against clinical gold standards, we cannot determine whether web-based surveys reduce under-detection or introduce over-inclusion of nonclinical distress. Rigorous validation studies comparing survey-identified cases with structured clinical interviews are needed before recommending large-scale shifts toward online mental health screening.

1. Limitations

The most critical limitation of this study is the nonequivalence of the assessment instruments used in the two surveys. The offline National Mental Health Survey employed the K-CIDI, which is a structured diagnostic interview with specific duration, severity, and impairment criteria aligned with the Diagnostic and Statistical Manual of Mental Disorders (DSM)/International Classification of Diseases diagnostic standards. In contrast, the online survey used brief, symptom-based self-report questions with substantially lower specificity and different diagnostic thresholds (see Table 4). This fundamental methodological difference implies that the observed discrepancies in prevalence estimates cannot be unambiguously attributed to the survey mode (online vs offline anonymity).

The higher reporting of depression, anxiety, and OCD in the online survey may reflect (a) reduced social desirability bias due to the anonymity of web-based completion, (b) lower diagnostic thresholds inherent to the simplified self-report items, or (c) an interaction between both factors. Because these possibilities may not be clearly distinguished based on the current study design and findings, more sophisticated research designs, such as the simultaneous use of diagnostic tools in both online and offline modes for comparison, may be needed in the future.

The reversal observed for AUD provides suggestive evidence that mode-related social dynamics play a role beyond instrument effects; however, because the AUD items also differed between surveys, this interpretation remains tentative rather than conclusive. Additionally, the use of ORs to quantify between-group differences warrants cautious interpretation. When the prevalence exceeds 10%, ORs tend to overestimate the corresponding relative risk, and the large ORs observed for some conditions (e.g., OCD) may exaggerate the true magnitude of the difference between the survey systems.

The two surveys also employed different weighting procedures. The offline survey applied poststratification weights based on census distributions for age, sex, and region, whereas the online survey used quota sampling with panel-based adjustments. These differing weighting approaches may contribute to systematic differences in prevalence estimates independent of the actual reporting patterns. Second, the cross-sectional observational design precludes causal inferences. Respondents were not randomized to survey modes; those who participated in each national survey may have differed in unmeasured ways (e.g., help-seeking attitudes, comfort with technology, and personality traits), which confounded the comparison. Although we statistically controlled for available demographic variables, residual confounding factors likely remained. Third, the two surveys covered slightly different age ranges (offline: 18~79 years; online: 15~69 years) and showed distributional differences in education, occupation, and other characteristics. Although sensitivity analyses restricted to the overlapping age range yielded consistent results, complete population equivalence was not achievable. Fourth, we could not assess the convergent validity of either survey’s prevalence estimates against clinical gold standards (e.g., Structured Clinical Interview for DSM-5). Thus, it remains unclear whether either system yields prevalence estimates that more closely approximate population-level rates. Fifth, both datasets were collected in 2021 during the COVID-19 pandemic, which may have influenced both mental health status and survey response patterns in ways that limit generalizability to nonpandemic periods.

Considering these limitations, definitive conclusions regarding mode-related differences would require randomized experimental designs with identical instruments in different modes. We encourage future research employing randomized experimental designs with identical instruments across survey modes as well as methodological studies comparing weighting approaches to advance the understanding of mental health data collection.

In conclusion, this study documented substantial differences in mental health prevalence estimates and public perceptions between South Korea’s two national mental health surveillance systems. Web-based respondents reported higher rates of depression, anxiety, and OCD, while in-person interview-based respondents reported higher rates of AUD; suicidal ideation estimates were comparable across systems.

These differences likely reflect a combination of survey mode-related differences (anonymity vs interviewer presence), instrument effects (structured diagnostic criteria vs simplified self-reports), and their interaction, which may not be fully disentangled from observational data from independently designed surveys. The findings suggest that national mental health statistics are shaped not only by population health status but also by the methodological characteristics of the surveillance systems that generate them. For policy and practice, these results highlight the importance of methodological awareness when interpreting and comparing mental health data across sources. A multisystem approach that leverages the accessibility of web-based surveys alongside the diagnostic rigor of interview-based assessments may ultimately provide the most comprehensive picture of a population’s mental health. Future research with randomized, instrument-matched designs would help clarify these patterns.

Notes

Conflicts of interest

The authors declared no conflict of interest.

Funding

This study was supported by a clinical research grant (No. 2024-10) from the National Center for Mental Health, Republic of Korea.

References

• 1. De Leeuw ED. Data quality in mail, telephone and face to face surveys. Amsterdam: TT-Publikaties; 1992.
• 2. Dillman DA. Why choice of survey mode makes a difference. Public Health Reports. 2006;121:11-13. https://doi.org/10.1177/003335490612100106ArticlePubMedPMC
• 3. Lang FR, John D, Ludtke O, Schupp J, Wagner GG. Short assessment of the Big Five: Robust across survey methods except telephone interviewing. Behavior Research Methods. 2011;43:548-567. https://doi.org/10.3758/s13428-011-0066-zArticlePubMedPMC
• 4. The World Bank. Internet users (% of population) [Internet]. Washington, DC: The World Bank; 2023 [cited 2026 March 11]. Available from: https://data.worldbank.org/indicator/IT.NET.USER.ZS
• 5. Szolnoki G, Hoffmann D. Online, face-to-face and telephone surveys-Comparing different sampling methods in wine consumer research. Wine Economics and Policy. 2013;2:57-66. https://doi.org/10.1016/j.wep.2013.10.001Article
• 6. Schonlau M, Van Soest A, Kapteyn A, Couper M. Selection bias in web surveys and the use of propensity scores. Sociological Methods & Research. 2009;37:291-318. https://doi.org/10.1177/0049124108327128Article
• 7. Naus MJ, Philipp LM, Samsi M. From paper to pixels: A comparison of paper and computer formats in psychological assessment. Computers in Human Behavior. 2009;25:1-7. https://doi.org/10.1016/j.chb.2008.05.012Article
• 8. Krumpal I. Determinants of social desirability bias in sensitive surveys: A literature review. Quality & Quantity. 2013;47:2025-2047. https://doi.org/10.1007/s11135-011-9640-9Article
• 9. Tourangeau R, Yan T. Sensitive questions in surveys. Psychological Bulletin. 2007;133:859-883. https://doi.org/10.1037/0033-2909.133.5.859ArticlePubMed
• 10. Gnambs T, Kaspar K. Disclosure of sensitive behaviors across self-administered survey modes: A meta-analysis. Behavior Research Methods. 2015;47:1237-1259. https://doi.org/10.3758/s13428-014-0533-4ArticlePubMed
• 11. Kays K, Gathercoal K, Buhrow W. Does survey format influence self-disclosure on sensitive question items? Computers in Human Behavior. 2012;28:251-256. https://doi.org/10.1016/j.chb.2011.09.007Article
• 12. Huang HM. Do print and web surveys provide the same results? Computers in Human Behavior. 2006;22:334-350. https://doi.org/10.1016/j.chb.2004.09.012ArticlePMC
• 13. National Center for Mental Health. National mental health survey 2021. Seoul: National Center for Mental Health; 2021.
• 14. National Center for Mental Health. Microdata instructions for user. Seoul: National Center for Mental Health; 2021.
• 15. Ministry of Health and Welfare. Development of scales to measure mental health literacy and attitude in Korean people. Seoul: Ministry of Health and Welfare; 2020.
• 16. Ji HA, Kim SR, Lee MS, Park SH, Kim YS, Lee KH, et al. The comparative analysis of mental health literacy in general population: The analysis of national mental health literacy and attitude survey in 2021. Korean Journal of Psychosomatic Medicine. 2022;30:38-45. https://doi.org/10.22722/KJPM.2022.30.1.38Article
• 17. Joinson A. Social desirability, anonymity, and Internet-based questionnaires. Behavior Research Methods, Instruments, & Computers. 1999;31:433-438. https://doi.org/10.3758/BF03200723ArticlePubMed
• 18. Schomerus G, Lucht M, Holzinger A, Matschinger H, Carta MG, Angermeyer MC. The stigma of alcohol dependence compared with other mental disorders: A review of population studies. Alcohol and Alcoholism. 2011;46:105-112. https://doi.org/10.1093/alcalc/agq089ArticlePubMed
• 19. Cakar U, Kim H. Korea's drinking culture: When an organizational socialization tool threatens workplace well-being. Turkish Journal of Business Ethics. 2015;8:207-231. https://doi.org/10.12711/tjbe.2015.8.0005Article
• 20. Ko S, Sohn A. Behaviors and culture of drinking among Korean people. Iranian Journal of Public Health. 2018;47(Suppl 1):47-56.PubMedPMC
• 21. Seo S, Chun S, Newell M, Yun M. Korean public opinion on alcohol control policy: A cross-sectional International Alcohol Control study. Health Policy. 2014;118:343-353. https://doi.org/10.1016/j.healthpol.2014.10.016ArticlePubMed
• 22. Lee S, Kim JS, Jung JG, Oh MK, Chung TH, Kim J. Korean alcohol guidelines for moderate drinking based on facial flushing. Korean Journal of Family Medicine. 2019;40:204-211. https://doi.org/10.4082/kjfm.19.0059ArticlePubMedPMC

Figure & Data

References

Citations

Citations to this article as recorded by