Key PointsQuestion听
Is the shift from face-to-face to virtual mental health service delivery associated with the risk of suicide-related events?
Findings听
In this cohort study assessing 66鈥�387 data points from 16鈥�236 unique veterans, a 1% increase in the percentage of virtual mental health visits relative to the total visits was associated with a statistically significant 2.5% decrease in suicide-related events.
Meaning听
The results of this cohort study suggest that offering virtual mental health care in addition to in-person care may reduce suicide-related events.
Importance听
The rising suicide rates in the US emphasize the need for effective prevention. While telehealth has transformed access to mental health care, the impact of telehealth on suicide outcomes is unknown.
Objective听
To evaluate the association of virtual mental health services with individual-level suicide-related events (SREs).
Design, Setting, and Participants听
This retrospective cohort study using broadband access as an instrumental variable assessed a national sample of Veterans Health Administration patients who received mental health care between March 1, 2020, and December 31, 2021. Participants were recently separated (ie, discharged or released from active duty) veterans who completed their active duty service between March 1, 2019, and December 31, 2020, and who received at least 2 outpatient or inpatient diagnoses related to major depressive disorder, substance use disorder, or posttraumatic stress disorder within the year before their most recent separation date. Data were analyzed May 1 to October 31, 2023.
Exposure听
Percentage of a patient鈥檚 total mental health visits that were conducted virtually by psychiatrists, psychologists, or social workers within a calendar month.
Main Outcomes and Measures听
Binary measure indicating whether the patient had experienced an SRE (defined as a nonfatal suicide attempt, intentional self-harm, or suicide death) in a specific month and year as evaluated an instrumental variable probit model.
Results听
The sample included 66鈥�387 data points from 16鈥�236 unique recently separated veterans. Among these entries, 44鈥�766 were for male veterans (67.4%), the mean (SD) age across the sample was 32.9 (8.9) years, and the sample was representative of the US veteran population. There were 929 SREs (1.4%). Virtual mental health visits comprised a mean (SD) of 44.6% (46.1%) of all mental health visits. In instrumental variable probit analyses accounting for factors simultaneously associated with use of virtual mental health care and SRE risk, a 1% increase in the probability of virtual mental health visits was associated with a 2.5% decrease in SREs.
Conclusions and Relevance听
Findings from this cohort study using a retrospective quasi-experimental design found that an increase in virtual mental health visits relative to total visits was associated with a statistically significant decrease in SREs, suggesting that providing virtual mental health services may reduce suicide-related outcomes.
Suicide remains a significant public health concern in the US, as the suicide rate increased by 35% between 2000 and 2018.1 In 2020, an estimated 12.2 million American adults seriously contemplated suicide, 3.2 million formed a suicide plan, and 1.2 million attempted suicide.2 However, suicide is often preventable through proactive strategies, including universal interventions addressing economic and social disparities, educational campaigns for mental health awareness, and targeted health care initiatives to screen high-risk populations.3
The rise of telehealth鈥攁 system for delivering health care remotely through technology鈥攈as reshaped the landscape of mental health services and suicide prevention. Telehealth is particularly suitable for mental health care because psychiatric diagnoses and treatments rely on discussions rather than physical assessments.4 Moreover, telehealth can reduce stigma, break down geographical barriers, reduce caregiver burden, and enhance access to mental health resources.5-8 Enhancing access is vital, particularly in rural regions with limited health care professional availability.5 The COVID-19 pandemic further highlighted the appropriateness of telemedicine in maintaining access to mental health services while enforcing safety and social distancing protocols.9,10 However, challenges persist, such as technological accessibility and the seamless integration of telehealth into the established health care infrastructure.11-13
Several studies have highlighted the effectiveness of mental telehealth. Randomized clinical trials have demonstrated that telepsychiatry treatment through videoconferencing has an equivalent efficacy to in-person psychiatric treatment in terms of psychiatric test scores, treatment adherence, and patient satisfaction.14,15 However, it is important to note that much of this evidence comes from small clinical trials, and while these studies show promising results, they do not fully establish the comparative effectiveness of video vs in-person mental health care. Trials across diverse patient groups further emphasize that video-assisted psychotherapy is as effective as traditional methods for treating conditions such as posttraumatic stress disorder (PTSD),16-19 depression,20 obsessive-compulsive disorder,21 social anxiety disorder,22 schizophrenia,23 and panic disorder.24 Moreover, patients and health care professionals are generally satisfied with virtual care.25,26
Nevertheless, caution may be warranted when considering mental telehealth for individuals with suicidal ideation. While remote sessions offer opportunities for the early detection of suicidal tendencies and collaborative safety planning involving household members,27 assessing and managing emergent safety concerns remotely can be challenging. A study involving 52 mental health professionals identified several perceived challenges of telehealth, with only about one-fifth endorsing its use for high-risk patients, thus highlighting general reservations about its application in individuals with suicidal ideation or behaviors.28
Despite the extensive literature on mental telehealth, there is a gap in our understanding of its direct effect on suicide outcomes. Most studies offer insights based on specific digital interventions and self-reported outcomes.29 A national descriptive study by the US Veterans Health Administration (VHA) showed that the total number of psychiatric hospitalizations fell after patient enrollment in telehealth services between 2006 and 2010.30 However, a gap remains in experimental and quasi-experimental studies, which have been suggested to demonstrate causal relationships and can inform policy and clinical decisions. Considering the increased use of mental telehealth by individuals with suicidal ideation or behaviors, understanding causal relationships between care modality and suicide-related outcomes is paramount.31
This evaluation explores whether virtual mental health service use is associated with individual-level suicide-related events (SREs). Specifically, is the shift from face-to-face to virtual mental health service delivery associated with the risk of SREs? The VHA is an ideal setting for this study because it is the largest integrated health care system in the US, has been at the forefront of mental telehealth innovations, and hosts one of the largest mental telehealth networks in the world.32 During the past decade, the VHA has significantly expanded its virtual care offerings, leveraging platforms such as VA Video Connect, initiatives such as Tablet to Home, and establishing dedicated TeleMental Health Hubs.33 Their swift response to the COVID-19 pandemic led to a staggering 556% surge in daily virtual mental health sessions by April 2020 compared with March 2020.33,34
This cohort study was conducted as a quality improvement activity for the VHA and was deemed to not be research; thus, there was no research oversight committee involvement and this project was exempt from the requirement of obtaining informed consent. This report follows the Strengthening the Reporting of Observational Studies in Epidemiology () cohort study reporting guideline and the Transparent Reporting of Evaluations With Nonrandomized Designs () reporting guideline.
In an ideal evaluation design, we would randomize patients to various virtual care use groups to achieve the most accurate causal analysis, but doing so would be neither practical nor ethical. Therefore, we rely on observational data. However, determining the causal effects of telehealth on SREs using observational data is not possible because of the absence of randomization, the potential for reverse causality, and omitted variable bias. Reverse causality may occur if telehealth use is associated with underlying mental health needs, leading veterans to adapt their virtual mental health care use in response to changes in their mental health needs. Omitted variable bias can arise when our models exclude variables associated with both virtual care utilization and suicidal behaviors. For example, this problem may arise in patients experiencing anxiety so severe that they cannot travel for in-person clinic visits.
To address these concerns, we used a quasi-experimental instrumental variable (IV) probit model designed to address problems related to reverse causality and potential confounders when dealing with a binary outcome variable. For an instrument for virtual care to be considered valid in our IV model, it must meet 2 essential criteria. First, it needs to demonstrate relevance, indicating a strong association between exposure and the instrument, as determined through the application of the Kleibergen-Paap weak instruments test and comparison with the Stock-Yogo critical values.35 Second, the instrument should affect the outcomes of interest solely through the exposure variable, without any direct effects on the outcome, in accordance with the exclusion restriction.36 This can be both argued theoretically and supported by the falsification test included in eAppendix 4 in Supplement 1.
Our chosen instrument was the percentage of the population in a county with access to 3 or more internet providers offering broadband speeds of 100 megabits per second (Mbps) or higher. This instrument was selected based on its strong correlation with virtual care utilization, determined through an evaluation of various broadband configurations. Broadband internet access is strongly linked to virtual care because it serves as a prerequisite for virtual medical visits. However, there is no expectation that broadband internet speed and the number of available internet providers will directly influence SREs.
The cohort consisted of recently separated (ie, discharged or released from active duty) veterans with a background of behavioral health diagnoses according to the Department of Veterans Affairs (VA) Infrastructure for Clinical Intelligence (DaVINCI) data. DaVINCI provides VA investigators with access to data from the Military Health System Data Repository, allowing them to add Department of Defense (DoD) data to VA data and enhance the comprehensiveness of patients鈥� health care history. We included veterans who had completed their active duty service between March 1, 2019, and December 31, 2020, and received at least 2 outpatient diagnoses or a single inpatient diagnosis related to major depressive disorder, substance use disorder, or PTSD within the year before their most recent separation date. A recently separated cohort was chosen to establish that mental health diagnoses occurred before the measurement of our treatment variable. Using the recently separated cohort, we had access to DoD data on preexisting mental health conditions. We combined patients with depression, PTSD, and substance use disorder to address the overall impact of virtual mental health care across a range of serious mental health conditions, aligning with prior published work.37 We followed up these individuals starting March 1, 2020, for up to 13 months after separation, including the month of separation. The decision to initiate the observation period in March 2020 was driven by our focus on the period after the beginning of the COVID-19 pandemic, which saw a significant increase in telehealth services. We limited the follow-up period to no more than 13 months postseparation due to the heightened risk experienced by patients during the period closest to their separation.38
The independent variable of interest was the individual-level percentage of mental health visits provided via virtual care (mental telehealth via video conferencing) by psychiatrists, psychologists, or social workers within a specific month. To obtain these data, we used Managerial Cost Accounting stop codes from the VA Corporate Data Warehouse (CDW) to narrow our sample to outpatient mental health encounters. We excluded observations with no mental health utilization because our objective was to investigate the association of virtual mental health care utilization with SREs.
The outcome variable was a binary measure indicating whether the patient had experienced an SRE (defined as a nonfatal suicide attempt, intentional self-harm, or suicide death) during the same calendar month of the visits. The data were sourced from 2 different origins, following research indicating that data accuracy is maximized when both types are used.39,40 First, we searched for relevant nonfatal suicide attempt International Statistical Classification of Diseases, Tenth Revision codes in the CDW outpatient, inpatient, and billing tables (eAppendix 3 in Supplement 1).41 Second, we obtained surveillance data from the VA Office of Mental Health and Suicide Prevention, which included information from 3 distinct surveillance sources: the Suicide Prevention Application Network, Comprehensive Suicide Risk Evaluation, and Suicidal Behavior and Overdose Report. For this evaluation, we included all SREs from these surveillance data sources.
Individual-level covariates obtained from the CDW electronic health records included the following: age, sex, race, ethnicity, priority status, marital status, distance from closest VHA primary care facility, and rurality of the patient鈥檚 residential address location (defined by Rural Urban Commuting Area codes). Race and ethnicity were assessed because race and ethnicity are well-documented factors associated with veterans' suicide disparities.42 Race was obtained from electronic health records and grouped into the following categories: American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or Other Pacific Islander, White, more than 1 race, and unknown (including declined to answer). Ethnicity was also obtained from electronic health records that captured whether a patient identified as Hispanic or Latino, with options being Hispanic or Latino, not Hispanic or Latino, and unknown or declined.43 We also gathered information about alternative insurance coverage and income in the facility catchment area (by creating a weighted mean of values associated with the counties in which VA-enrolled veterans resided) from the VA Survey of Enrollees.44 To identify quasi-experimental variations in virtual care utilization, we obtained data on broadband availability, speed, and number of internet service providers at the county level from the Federal Communications Commission.45 To control for local COVID-19 burden, we included county-month level cumulative COVID-19 mortality counts from Johns Hopkins Coronavirus Resource Center.46
Observations with missing or incomplete data were excluded to ensure data integrity. The detailed sample creation, missingness, and the inclusion and exclusion processes are outlined in eAppendices 1 and 2 in Supplement 1.
All models incorporated the following individual-level covariates: race, ethnicity, age, sex, priority status, marital status, rurality of the patient鈥檚 residential address location (defined by Rural Urban Commuting Area codes), and distance from closest VHA primary care facility. Furthermore, we accounted for the following facility-month level covariates: alternative insurance coverage (any non-VA health care coverage, either public or private) and per capita income in the facility catchment area. We used facility fixed effects to account for time-invariant characteristics unique to each facility and applied month fixed effects to control for seasonal variations that could potentially influence the outcome of interest. To address repeated observations, we used SEs clustered at the individual level.
We estimated a first-stage IV probit in which we regressed the percentage of mental health visits that were provided via virtual care in a given month on broadband and the aforementioned individual and facility-level covariates. We included month fixed effects to control for seasonal variation and facility fixed effects to control for facility time invariant characteristics. Then, we estimated the second stage of IV probit in which we regressed having an SRE on the percentage of mental health visits that were provided via virtual care, residuals from the first stage-estimation, and all the same individual and facility-level covariates. As in the first stage, we included month and facility fixed effects. In addition to the IV probit, we also estimated a single-equation probit model with SRE as the outcome, virtual care as an independent variable of interest, and controlling for all the same individual- and facility-level covariates, as well as month and facility fixed effects.
Two sensitivity analyses were performed. The first analysis was conducted to address the concern of treatment (virtual care) and outcome (SRE) being recorded in the same period, potentially having the SRE as a reason for the virtual care visit. We introduced a lead time for the SRE outcome variable and replicated our primary model with the SRE outcome from the following month. The second analysis aimed to address the concern regarding minimal variability in mental health visit types during the early months of the pandemic. For this analysis, we replicated the primary model once more, after omitting data from the early pandemic months of March, April, and May 2020.
Similar to Feyman et al,37 we examined via Heckman modeling whether there was a potential for unobserved factors to influence both the receipt of any VHA mental health care and SREs. eAppendix 5 in Supplement 1 provides the rationale and details of this analysis. All analyses were conducted from May 1 to October 31, 2023, with Stat/MP, version 16.0 (StataCorp LLC). A 2-sided value of P鈥�<鈥�.05 was considered statistically significant.
In our sample of 66鈥�387 observations representing 16鈥�236 unique veterans (mean [SD] age, 32.9 [8.9] years; 21鈥�621 [32.6%] female and 44鈥�766 [67.4%] male; 683 [1.0%] American Indian or Alaska Native, 2271 [3.4%] Asian, 17鈥�250 [26.0%] Black, 1232 [1.9%] Native Hawaiian or Other Pacific Islander, 33鈥�538 [50.5%] White, 1488 [2.2%] more than 1 race, 9925 [15.0%] race unknown; 9602 [14.5%] Hispanic, 45鈥�897 [69.1%] non-Hispanic, and 10鈥�888 [16.4%] ethnicity unknown; and 32鈥�785 [49.4%] married), 929 of individual-month observations (1.4%) included an SRE. There was a mean (SD) of 2.2 (2.8) visits per month, with a mix of virtual, phone, and in-person visits (Table 1). Virtual mental health visits represented a mean (SD) of 44.6% (46.1%) of all mental health visits in our sample. The mean (SD) percentage of the county population with access to 3 or more internet providers offering broadband speeds of 100 Mbps or higher was 19.6% (23.0%). The mean (SD) driving distance to the nearest VHA primary care facility was 12.7 (10.8) miles.
The aforementioned demographic characteristics indicated that our sample was representative of the overall veteran population in the US. Additionally, 883 data points (1.3%) were for veterans classified under priority statuses 7 or 8 (typically assigned to higher-income veterans), and 12鈥�613 data points (19.0%) represented veterans living in rural areas. Additional descriptive statistics on virtual care use by sex, race, and ethnicity are reported in eAppendix 6 in Supplement 1.
In terms of facility-related factors, the individuals in the sample resided closest to a primary care facility, where 66.4% of enrollees had comprehensive health insurance and 41.0% reported an annual household income between $20鈥�000 and $50鈥�000. In our sample, there was a mean (SD) of 8.0 (7.9) COVID-19 fatalities for every 10鈥�000 people in a county.
In our single-equation probit models, an increase in virtual mental health visits was associated with a lower probability of suicide attempts, with a coefficient of 鈭�0.0032 (robust standard error, 0.0003; P鈥�<鈥�.001) (Table 2). The elasticity of SREs with respect to the proportion of virtual care visits was 鈭�0.41, meaning that a 1% increase in the probability of virtual mental health visits was associated with a 0.4% decrease in SREs (0.01 percentage points). Marginal effects of the single-equation probit model are reported in eAppendix 7 in Supplement 1.
The first-stage findings of the IV probit analysis illustrated the strength of our instrument, which was substantial and statistically significant. Specifically, the Cragg鈥揇onald Wald F statistic was 116.4, and the Kleibergen-Paap Wald F statistic was 40.4. Given that the critical value of the Stock-Yogo weak instrument test for 10% bias is 16.4, these F statistics indicated that the instrument was sufficiently strong to avoid weak instrument bias.
In the second stage, the coefficient associated with the independent variable of interest (percentage of virtual care) was negative and statistically significant (coefficient, 鈭�0.0161 [robust standard error, 0.0049]; P鈥�=鈥�.001) (Table 2). The elasticity of SREs with respect to the proportion of virtual care visits was 鈭�2.5, which meant that a 1% increase in the probability of virtual mental health visits was associated with a 2.5% decrease in SREs (0.07 percentage points). The IV results were larger in magnitude than those obtained through single-equation probit regression analysis. Marginal effects of the second stage results are reported in eAppendix 7 in Supplement 1.
Sensitivity analyses strengthened the validity of our primary models. Introducing a 1-month lead time for the SRE outcome variable in the model yielded statistically significant results (coefficient, 鈭�0.0179 [robust standard error, 0.0044]; P鈥�<鈥�.001) with an elasticity of 鈭�3.2. Similarly, the model that excluded data from early pandemic months was statistically significant (coefficient, 鈭�0.0178 [robust standard error, 0.0054]; P鈥�=鈥�.001) with an elasticity of 鈭�3.4. These outcomes, showing slightly larger elasticities compared with our primary model, suggest that our primary findings were on the conservative side. Full results of the sensitivity analyses can be found in eAppendices 8 and 9 in Supplement 1.
This national cohort study, using a quasi-experimental design, of virtual mental health use at the VHA between March 2020 and December 2021 found that an increase in the percentage of virtual mental health visits relative to the total visits was associated with a statistically significant decrease in SREs. This finding suggests that virtual mental health services may be protective in terms of suicide-related outcomes, corroborating previous descriptive research that reported beneficial impacts from the heightened uptake of telehealth services at the VHA.30
Our IV probit results were larger in magnitude compared with the findings obtained through a simple probit regression analysis. This result implies that the simple probit models may be underestimating the true effect. For instance, individuals at greater risk of suicide could be more inclined to use virtual mental health care, or there may be other unseen elements influencing both the propensity to opt for virtual care and the suicide risk. The consistency and statistical significance of the negative coefficients across both models indicate a reliable association between the variables studied.
There are several potential explanations for how the rise in virtual care use may contribute to a decrease in SREs. One mechanism could be the increased accessibility provided by telehealth.47 From the patient鈥檚 perspective, telehealth can cut down on travel time, enhance convenience, and lower or remove the costs linked to clinic visits.48 Another potential mechanism could be that some individuals might find virtual mental health care more comfortable. Research indicates that telehealth offers patients a sense of shared social connection with their physicians and that being in one鈥檚 home environment can enhance psychological safety.49-51 Furthermore, a significant advantage of telemental health is its potential to alleviate the stigma concerns often associated with seeking in-person care.52
Nevertheless, it is crucial to recognize that virtual mental health care does not serve all veterans uniformly. Qualitative research has noted that certain veterans may experience discomfort or have privacy worries when engaging with telehealth.53 Additional obstacles may stem from technological limitations, such as unreliable internet connections or lack of necessary technology.54 Studies also reveal significant disparities in access to telehealth video services based on race and socioeconomic status.55 Therefore, maintaining both virtual and in-person care options is essential to address these varied needs and barriers.52
This evaluation has several limitations. First, it is possible that the zip codes for veterans sourced from the DoD data may not be precise or may represent addresses where the veterans no longer live, such as their parents鈥� homes. However, this was not a major concern because only 7% of the entire sample was identified using DoD data. Second, the findings may not be generalizable to nonveteran populations because veterans differ notably from the wider US population in terms of demographics and comorbidities, as well as experiences, such as exposure to combat.56 Therefore, the implications of our findings should be considered carefully, as the unique experiences of veterans may significantly influence the outcomes. Future research should aim to include samples that are more representative of the general US population or focus on different subgroups within the general population to validate these findings in other contexts. However, given that the VHA operates one of the largest mental telehealth networks in the world, our conclusions still have significant implications for global mental telehealth guidelines and practices. Third, there remains a possibility that unmeasured confounders may influence both broadband access and the outcome of SREs. This underlying issue could potentially distort the causal interpretation of our findings. However, this concern is mitigated by the fact that simple probit results align in the same direction as the IV results, suggesting consistency in the observed associations. Fourth, these findings may not be generalizable to all veteran populations, as recently separated veterans may face distinct challenges, such as less familiarity with VHA services and greater barriers to accessing mental health care, compared with veterans with established VHA care. Fifth, the results may not be generalizable to 2019 or earlier given how the landscape of virtual care has changed as a result of the COVID-19 pandemic. Sixth, these results apply only to specialty mental health care provided by psychiatrists, psychologists, and social workers. Finally, specific subsets of the population may experience different results. Future research may benefit from conducting analyses that are stratified by different socioeconomic and racial and ethnic demographics, as well as various comorbidities. Extending the evaluation time frame beyond 2021 could provide valuable insights into whether the observed associations persist during the latter stages of the COVID-19鈥損andemic era and beyond. Despite these limitations, to our knowledge, this was the first quasi-experimental evaluation to explore the association between virtual mental health care use and SREs.
This national cohort study using a retrospective quasi-experimental design found that increased use of virtual mental health services at the VHA was associated with a significant reduction in SREs among veterans. Our results support continued expansion of mental telehealth within the VHA. Future evaluations should seek to confirm these findings among different demographic groups and extend beyond the COVID-19鈥損andemic era.
Accepted for Publication: September 12, 2024.
Published: November 5, 2024. doi:10.1001/jamanetworkopen.2024.43054
Open Access: This is an open access article distributed under the terms of the CC-BY License. 漏 2024 Tenso K et al. 糖心vlog Open.
Corresponding Author: Kertu Tenso, PhD, Department of Health Law Policy and Management, Boston University School of Public Health, 715 Albany St, Boston, MA 02118 (ktenso@bu.edu).
Author Contributions: Dr Tenso had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Tenso, Strombotne, Pizer.
Acquisition, analysis, or interpretation of data: Tenso, Strombotne, Garrido, Lum.
Drafting of the manuscript: Tenso, Strombotne.
Critical review of the manuscript for important intellectual content: All authors.
Statistical analysis: Tenso, Lum, Pizer.
Obtained funding: Pizer.
Administrative, technical, or material support: Tenso, Strombotne, Pizer.
Supervision: Strombotne, Garrido, Pizer.
Conflict of Interest Disclosures: None reported.
Funding/Support: This work was supported by grant Partnered Evaluation Initiative (PEC) 16-001 from the US Department of Veterans Affairs Quality Enhancement Research Initiative to Dr Garrido.
Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: The contents do not necessarily represent the views of the US Department of Veterans Affairs.
Data Sharing Statement: See Supplement 2.
1.Centers for Disease Control and Prevention. Facts about suicide. Accessed August 30, 2022.
2.Substance Abuse and Mental Health Services Administration. Key substance use and mental health indicators in the united states: results from the 2020 National Survey on Drug Use and Health. Accessed September 20, 2024.
3.Wasserman
听D锘�, Iosue
听M锘�, Wuestefeld
听A锘�, Carli
听V锘�. 听Adaptation of evidence-based suicide prevention strategies during and after the COVID-19 pandemic.听锘� 听World Psychiatry. 2020;19(3):294-306. doi:锘�
4.Barnett
听ML锘�, Huskamp
听HA锘�. 听Telemedicine for mental health in the United States: making progress, still a long way to go.听锘� 听Psychiatr Serv. 2020;71(2):197-198. doi:锘�
5.Bischoff
听RJ锘�, Hollist
听CS锘�, Smith
听CW锘�, Flack
听P锘�. 听Addressing the mental health needs of the rural underserved: findings from a multiple case study of a behavioral telehealth project.听锘� 听Contemp Fam Ther. 2004;26:179-198. doi:锘�
6.Ward-Ciesielski
听EF锘�, Peros
听O锘�, Conigliaro
听A锘�, Gilmore
听AK锘�. 听Perceived benefits of psychotherapy via telemedicine based on suicide risk severity.听锘� 听Gen Hosp Psychiatry. 2018;55:100-101. doi:锘�
7.De Jaegere
听E锘�, van Landschoot
听R锘�, van Heeringen
听K锘�,
听et al. 听The online treatment of suicidal ideation: a randomised controlled trial of an unguided web-based intervention.听锘� 听Behav Res Ther. 2019;119:103406. doi:锘�
8.Chen
听CK锘�, Nehrig
听N锘�, Wash
听L锘�,
听et al. 听When distance brings us closer: leveraging tele-psychotherapy to build deeper connection.听锘� 听Couns Psychol Q. 2021;34(3-4):554-567. doi:锘�
9.Bahl
听S锘�, Singh
听RP锘�, Javaid
听M锘�, Khan
听IH锘�, Vaishya
听R锘�, Suman
听R锘�. 听Telemedicine technologies for confronting COVID-19 pandemic: a review.听锘� 听Journal of Industrial Integration and Management. 2020;5(04):547-561. doi:锘�
10.Colbert
听GB锘�, Venegas-Vera
听AV锘�, Lerma
听EV锘�. 听Utility of telemedicine in the COVID-19 era.听锘� 听Rev Cardiovasc Med. 2020;21(4):583-587. doi:锘�
11.Latifi
听R锘�, Doarn
听CR锘�. 听Perspective on COVID-19: finally, telemedicine at center stage.听锘� 听Telemed J E Health. 2020;26(9):1106-1109. doi:锘�
12.Ben-Zeev
听D锘�, Davis
听KE锘�, Kaiser
听S锘�, Krzsos
听I锘�, Drake
听RE锘�. 听Mobile technologies among people with serious mental illness: opportunities for future services.听锘� 听Adm Policy Ment Health. 2013;40(4):340-343. doi:锘�
13.Wilcock
听AD锘�, Rose
听S锘�, Busch
听AB锘�,
听et al. 听Association between broadband internet availability and telemedicine use.听锘� 听JAMA Intern Med. 2019;179(11):1580-1582. doi:锘�
14.De Las Cuevas
听C锘�, Arredondo
听MT锘�, Cabrera
听MF锘�, Sulzenbacher
听H锘�, Meise
听U锘�. 听Randomized clinical trial of telepsychiatry through videoconference versus face-to-face conventional psychiatric treatment.听锘� 听Telemed J E Health. 2006;12(3):341-350. doi:锘�
15.Ruskin
听PE锘�, Silver-Aylaian
听M锘�, Kling
听MA锘�,
听et al. 听Treatment outcomes in depression: comparison of remote treatment through telepsychiatry to in-person treatment.听锘� 听Am J Psychiatry. 2004;161(8):1471-1476. doi:锘�
16.Morland
听LA锘�, Mackintosh
听MA锘�, Rosen
听CS锘�,
听et al. 听Telemedicine versus in-person delivery of cognitive processing therapy for women with posttraumatic stress disorder: a randomized noninferiority trial.听锘� 听Depress Anxiety. 2015;32(11):811-820. doi:锘�
17.Liu
听L锘�, Thorp
听SR锘�, Moreno
听L锘�,
听et al. 听Videoconferencing psychotherapy for veterans with PTSD: results from a randomized controlled non-inferiority trial.听锘� 听J Telemed Telecare. 2020;26(9):507-519. doi:锘�
18.Acierno
听R锘�, Knapp
听R锘�, Tuerk
听P锘�,
听et al. 听A non-inferiority trial of prolonged exposure for posttraumatic stress disorder: in person versus home-based telehealth.听锘� 听Behav Res Ther. 2017;89:57-65. doi:锘�
19.Germain
听V锘�, Marchand
听A锘�, Bouchard
听S锘�, Drouin
听MS锘�, Guay
听S锘�. 听Effectiveness of cognitive behavioural therapy administered by videoconference for posttraumatic stress disorder.听锘� 听Cogn Behav Ther. 2009;38(1):42-53. doi:锘�
20.Egede
听LE锘�, Acierno
听R锘�, Knapp
听RG锘�,
听et al. 听Psychotherapy for depression in older veterans via telemedicine: a randomised, open-label, non-inferiority trial.听锘� 听Lancet Psychiatry. 2015;2(8):693-701. doi:锘�
21.Goetter
听EM锘�, Herbert
听JD锘�, Forman
听EM锘�, Yuen
听EK锘�, Thomas
听JG锘�. 听An open trial of videoconference-mediated exposure and ritual prevention for obsessive-compulsive disorder.听锘� 听J Anxiety Disord. 2014;28(5):460-462. doi:锘�
22.Yuen
听EK锘�, Herbert
听JD锘�, Forman
听EM锘�,
听et al. 听Acceptance based behavior therapy for social anxiety disorder through videoconferencing.听锘� 听J Anxiety Disord. 2013;27(4):389-397. doi:锘�
23.Rotondi
听AJ锘�, Haas
听GL锘�, Anderson
听CM锘�,
听et al. 听A clinical trial to test the feasibility of a telehealth psychoeducational intervention for persons with schizophrenia and their families: intervention and 3-month findings.听锘� 听Rehabil Psychol. 2005;50(4):325-336. doi:锘�
24.Bouchard
听S锘�, Paquin
听B锘�, Payeur
听R锘�,
听et al. 听Delivering cognitive-behavior therapy for panic disorder with agoraphobia in videoconference.听锘� 听Telemed J E Health. 2004;10(1):13-25. doi:锘�
25.Hilty
听DM锘�, Ferrer
听DC锘�, Parish
听MB锘�, Johnston
听B锘�, Callahan
听EJ锘�, Yellowlees
听PM锘�. 听The effectiveness of telemental health: a 2013 review.听锘� 听Telemed J E Health. 2013;19(6):444-454. doi:锘�
26.Connolly
听SL锘�, Miller
听CJ锘�, Lindsay
听JA锘�, Bauer
听MS锘�. 听A systematic review of providers鈥� attitudes toward telemental health via videoconferencing.听锘� 听Clin Psychol (New York). 2020;27(2):e12311. doi:锘�
27.Pruitt
听LD锘�, Luxton
听DD锘�, Shore
听P锘�. 听Additional clinical benefits of home-based telemental health treatments.听锘� 听Prof Psychol Res Pr. 2014;45(5):340-346. doi:锘�
28.Gilmore
听AK锘�, Ward-Ciesielski
听EF锘�. 听Perceived risks and use of psychotherapy via telemedicine for patients at risk for suicide.听锘� 听J Telemed Telecare. 2019;25(1):59-63. doi:锘�
29.Stefanopoulou
听E锘�, Hogarth
听H锘�, Taylor
听M锘�, Russell-Haines
听K锘�, Lewis
听D锘�, Larkin
听J锘�. 听Are digital interventions effective in reducing suicidal ideation and self-harm? a systematic review.听锘� 听J Ment Health. 2020;29(2):207-216. doi:锘�
30.Godleski
听L锘�, Darkins
听A锘�, Peters
听J锘�. 听Outcomes of 98,609 U.S. Department of Veterans Affairs patients enrolled in telemental health services, 2006-2010.听锘� 听Psychiatr Serv. 2012;63(4):383-385. doi:锘�
31.Tarlow
听KR锘�, Johnson
听TA锘�, McCord
听CE锘�. 听Rural status, suicide ideation, and telemental health: Risk assessment in a clinical sample.听锘� 听J Rural Health. 2019;35(2):247-252. doi:锘�
32.Godleski
听L锘�, Nieves
听JE锘�, Darkins
听A锘�, Lehmann
听L锘�. 听VA telemental health: suicide assessment.听锘� 听Behav Sci Law. 2008;26(3):271-286. doi:锘�
33.Rosen
听CS锘�, Morland
听LA锘�, Glassman
听LH锘�,
听et al. 听Virtual mental health care in the Veterans Health Administration鈥檚 immediate response to coronavirus disease-19.听锘� 听Am Psychol. 2021;76(1):26-38. doi:锘�
34.Connolly
听SL锘�, Stolzmann
听KL锘�, Heyworth
听L锘�, Weaver
听KR锘�, Bauer
听MS锘�, Miller
听CJ锘�. 听Rapid increase in telemental health within the Department of Veterans Affairs during the COVID-19 pandemic.听锘� 听Telemed J E Health. 2021;27(4):454-458. doi:锘�
35.Stock
听JH锘�, Yogo
听M锘�. 听Testing for Weak Instruments in Linear IV Regression. National Bureau of Economic Research Cambridge; 2002. doi:锘�
36.Pizer
听SD锘�. 听Falsification testing of instrumental variables methods for comparative effectiveness research.听锘� 听Health Serv Res. 2016;51(2):790-811. doi:锘�
37.Feyman
听Y锘�, Figueroa
听SM锘�, Yuan
听Y锘�,
听et al. 听Effect of mental health staffing inputs on suicide-related events.听锘� 听Health Serv Res. 2023;58(2):375-382. doi:锘�
38.Sokol
听Y锘�, Gromatsky
听M锘�, Edwards
听ER锘�,
听et al. 听The deadly gap: understanding suicide among veterans transitioning out of the military.听锘� 听笔蝉测肠丑颈补迟谤测 Res. 2021;300:113875. doi:锘�
39.Hoffmire
听C锘�, Stephens
听B锘�, Morley
听S锘�, Thompson
听C锘�, Kemp
听J锘�, Bossarte
听RM锘�. 听VA Suicide Prevention Applications Network: a national health care system鈥揵ased suicide event tracking system.听锘� 听Public Health Rep. 2016;131(6):816-821. doi:锘�
40.Ursano
听RJ锘�, Kessler
听RC锘�, Heeringa
听SG锘�,
听et al; Army STARRS collaborators. 听Nonfatal suicidal behaviors in U.S. Army administrative records, 2004-2009: results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS).听锘� 听笔蝉测肠丑颈补迟谤测. 2015;78(1):1-21. doi:锘�
41.Hedegaard
听H锘�, Schoenbaum
听M锘�, Claassen
听C锘�, Crosby
听A锘�, Holland
听K锘�, Proescholdbell
听S锘�. 听Issues in developing a surveillance case definition for nonfatal suicide attempt and intentional self-harm using International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) coded data.听锘� 听Natl Health Stat Report. 2018;(108):1-19.
42.U.S. Department of Veterans Affairs: Office of Mental Health and Suicide Prevention. 2023 National veteran suicide prevention annual report. Accessed September 29, 2024.
43.Office of Management and Budget. 听Revisions to the standards for the classification of federal data on race and ethnicity.听锘� 听Fed Regist. 1997;62(210):58782-58790.
44.Veterans Health Administration. VA survey of veteran enrollees鈥� health and use of health care. Accessed May 5, 2023.
45.Federal Communications Commission. Broadband data collection. Accessed January 24, 2024.
46.Johns Hopkins Coronavirus Resource Center. Accessed October 3, 2021.
47.Mahtta
听D锘�, Daher
听M锘�, Lee
听MT锘�, Sayani
听S锘�, Shishehbor
听M锘�, Virani
听SS锘�. 听Promise and perils of telehealth in the current era.听锘� 听Curr Cardiol Rep. 2021;23(9):115. doi:锘�
48.Powell
听RE锘�, Henstenburg
听JM锘�, Cooper
听G锘�, Hollander
听JE锘�, Rising
听KL锘�. 听Patient perceptions of telehealth primary care video visits.听锘� 听Ann Fam Med. 2017;15(3):225-229. doi:锘�
49.Steindal
听SA锘�, Nes
听AAG锘�, Godskesen
听TE锘�,
听et al. 听Patients鈥� experiences of telehealth in palliative home care: scoping review.听锘� 听J Med Internet Res. 2020;22(5):e16218. doi:锘�
50.Duane
听JN锘�, Blanch-Hartigan
听D锘�, Sanders
听JJ锘�,
听et al. 听Environmental considerations for effective telehealth encounters: a narrative review and implications for best practice.听锘� 听Telemed J E Health. 2022;28(3):309-316. doi:锘�
51.Burmeister
听OK锘�, Ritchie
听D锘�, Devitt
听A锘�, Chia
听E锘�, Dresser
听G锘�, Roberts
听R锘�. 听The impact of telehealth technology on user perception of wellbeing and social functioning, and the implications for service providers.听锘� 听Australasian Journal of Information Systems. 2019;23:1-18. doi:
52.Olden
听M锘�, Cukor
听J锘�, Rizzo
听AS锘�, Rothbaum
听B锘�, Difede
听J锘�. 听House calls revisited: leveraging technology to overcome obstacles to veteran psychiatric care and improve treatment outcomes.听锘� 听Ann N Y Acad Sci. 2010;1208(1):133-141. doi:锘�
53.Chen
听PV锘�, Helm
听A锘�, Fletcher
听T锘�,
听et al. 听Seeing the value of video: a qualitative study on patient preference for using video in a Veteran Affairs telemental health program evaluation.听锘� 听Telemed Rep. 2021;2(1):156-162. doi:锘�
54.Chang
听JE锘�, Lai
听AY锘�, Gupta
听A锘�, Nguyen
听AM锘�, Berry
听CA锘�, Shelley
听DR锘�. 听Rapid transition to telehealth and the digital divide: implications for primary care access and equity in a post-COVID era.听锘� 听Milbank Q. 2021;99(2):340-368. doi:锘�
55.Rivera
听V锘�, Aldridge
听MD锘�, Ornstein
听K锘�, Moody
听KA锘�, Chun
听A锘�. 听Racial and socioeconomic disparities in access to telehealth.听锘� 听J Am Geriatr Soc. 2021;69(1):44-45. doi:锘�
56.Walter
听KH锘�, Levine
听JA锘�, Highfill-McRoy
听RM锘�, Navarro
听M锘�, Thomsen
听CJ锘�. 听Prevalence of posttraumatic stress disorder and psychological comorbidities among U.S. active duty service members, 2006-2013.听锘� 听J Trauma Stress. 2018;31(6):837-844. doi:锘�