Algorithm to remove inappropriate behavioral flags

IN 2022, Wisconsin enacted legislation making it a felony to assault or threaten a healthcare provider or their family. Hospitals across the country have implemented various interventions to combat increasing workplace violence (WPV). For example, Gonzales and colleagues describe using an electronic health record (EHR) behavioral flagging system.

Ferron and colleagues define behavioral flagging as a standardized approach to communicating among healthcare workers the risk of violence posed by a particular patient. Their team conducted an initial study to assess the effectiveness of behavioral flags and found a 92% mean decline in violent incidents 1 year after implementation. As noted by Gonzales and colleagues, behavioral flagging operates at the junction of delivering high-quality, equitable patient care and safeguarding clinician safety.

However, despite the good intentions of behavior flagging, Gonzales and colleagues reported that identifying aggressive or threatening behavior can be subjective and influenced by personal bias, resulting in potential error or inappropriately placed flags. Clerkin defines bias as making a judgment about something or someone; it’s more likely to occur in the context of differing perspectives, experiences, and social groups. Gopal and colleagues describe bias as both positive and negative; however, negative bias presents a particular concern within healthcare. For instance, without clear guidelines for using behavioral flags, a high risk exists for biased application.

Kopp and colleagues note the concerns behavioral flags create for patients identified as at risk for violence or posing a risk of violence. For example, patients with behavioral flags in their EHR may encounter stigmatization, limited care, criminalization of mental health or substance use disorders, and adverse impact on the provider–patient relationship.

The risk management team at the organization where we conducted our project has witnessed the significant challenges of patients with behavioral EHR flags, including care termination, outside hospitals refusing transfers, and difficulty establishing home healthcare or other outpatient services. Expired and inappropriate behavioral flags are even more concerning when no review process exists to ensure accurate use and discontinuation when applicable.

The project setting, a private midwestern nonprofit, Level 2 trauma, acute care hospital, had an inconsistent and nonfunctional process for removing behavioral flags. If a patient exhibited any Level 3 behaviors (the highest level of violence), a behavioral flag was placed in their EHR. If a staff member identified the flag as no longer appropriate or applicable, they could ask the risk management team to review it; however, a continual review process wasn’t part of the workflow. (See Violence risk levels)

To address this problem, an interprofessional project team, composed of local and systems-level risk-management personnel, led by a University of Wisconsin–Madison doctoral nursing student, developed and piloted a behavioral flag–removal algorithm to eliminate inappropriate or expired behavior flags.

The project

Through a literature review, collaboration among risk management and safety staff, discussion with an outside hospital, and a manual chart review, the project team created a preliminary algorithm using the Emancipatory Nursing Praxis (ENP) theory. As described by Walter, ENP emphasizes transforming processes to fulfill nursing’s ethical obligations and to limit risks to stigmatized or vulnerable populations, including patients with a history or risk of violent behavior.

According to Ferron and colleagues and Seeburger and colleagues, behavioral flags can serve as a proactive strategy for reducing WPV. However, they also raise ethical and procedural concerns, including the possible introduction of bias in patient–provider interactions, disparities in flag placement given their disproportionate use for historically marginalized populations, challenges such as stigmatization, policy and procedural gaps, permanence of the flag, and a need to balance safety concerns with mitigating downstream adverse effects of flags.

Due to the large number of behavioral flags present within the organization’s EHR system, the project team focused on adult patients, age 18 years and older, admitted to any unit in the hospital between July 1, 2024, and October 31, 2024. The safety and risk management staff used an EHR Epic-based data exploration tool (Slicer-Dicer) to generate a list of patients with Level 3 flags. They identified 100 charts for evaluation.

The project team then performed a manual review of the 100 charts , extracting details on each patient’s specific situation and the rationale for the behavioral flag. The team used these data to create a preliminary behavioral flag–removal algorithm. The collected de-identified data consisted of missing standard components of behavioral flags, such as date of the violent incident, factual description of the event, and attempted interventions. The team also looked for common themes in inappropriate and appropriate behavioral flags (age, diagnosis, presence of law enforcement).

Next, the local risk management and safety team piloted the preliminary algorithm using the same 100 patient charts from November 1, 2024, to December 1, 2024. The decision tree algorithm focuses on the organization’s current operational workflow and the previously determined three levels of violence.

After de-identifying the pilot data, the local risk management and safety team analyzed the results using an Excel spreadsheet. At scheduled meetings with both local and systems-level personnel, the entire team discussed the pilot results as well as recommendations for editing and feedback. Ultimately, the algorithm required only minimal changes, such as wording adjustments. (View the Behavioral flag removal algorithm )

Outcomes

After applying the preliminary algorithm to the 100 patient charts, the project team recommended 69% of the behavioral flags for editing. These flags lacked a date indicating when the incidents occurred, factual verbiage explaining reasons for applying the flag, or details regarding the interventions used. The team recommended 15% of the flags for complete removal and 5% for downgrading to a lower risk level; 11% were applied correctly, with no recommended changes.

The discrepancies and deficiencies demonstrated in these findings highlight the concerns raised by Kopp and colleagues, noting that incomplete or biased behavioral flag use can compromise the accuracy and credibility of the flagging system. The team found that two age groups (71 to 80 and 81 to 90) accounted for most of the behavioral flags reviewed (37%), with 59% of those having a documented diagnosis of dementia or neurocognitive disorder. The organization currently doesn’t use a separate, dementia-specific behavioral flag; however, such a flag would provide early identification of patients with dementia or cognitive impairment, optimizing approaches to care and patient outcomes, as demonstrated by Tan and colleagues.

The overall results of this pilot highlight the value of additional education and instruction on appropriate behavioral flag use. Specifically, personnel caring for individuals with a behavioral flag may benefit from continued education—either in person or online—focused on best practices for caring for patients with a documented diagnosis of dementia or other neurocognitive disorder. The organization also could consider adopting a dementia-specific flag. Project results support implementing the behavioral flag–removal algorithm to ensure routine flag review to ensure they remain clinically appropriate and modified or removed when necessary.

The project team disseminated the pilot results to all hospital units along with additional instructions on how to properly place a behavioral flag in the EHR. At the local level, risk management and safety personnel will use the algorithm in biannual reviews of every behavioral flag within the organization.

The team also shared the project system-wide, and work continues to implement it through other organization locations in Wisconsin with the goal of implementing it at all associated locations nationwide.

Return on investment

The project incurred no monetary costs, but team members committed more than 180 hours. However, the process aims to reduce challenges for patients accessing healthcare or other resources and decrease the cost, staff, or time required to care for those at risk of violence. Ultimately, the algorithm has the potential to create a smoother operating workflow while keeping patients and staff safe.

Potential problems

The project had limitations related to time, research gaps, and workflow specificity. It took place from June 2024 through May 2025, corresponding with the DNP student’s course sequence, limiting time for data collection. Because EHR access was available only on-site, staff working on the project faced time constraints and scheduling conflicts.

The paucity of pre-existing research on behavioral flags also limited the project. Future research can help assess the specific downstream effects of inappropriate flags. A need also exists for a long-term analysis of removal algorithm use. In addition, no universal definition of WPV violence exists, making its application within an algorithm difficult. Because this algorithm was developed and piloted by a single organization, it’s specific to the organization’s operational processes and workflows, limiting its transferability to other organizations or settings.

Recommendations

The organization’s commitment to reducing WPV and its leaders’ interest helped support the project. The professional knowledge of the interdisciplinary risk management team, previous work on violence risk, and the team’s integration with administration also ensured project success. Other organizations seeking to implement a similar process and algorithm should work closely with their risk management team or the team designated to implement the algorithm as part of their workflow.

Implications for nursing

Guided by the ENP, this project implemented an algorithm for behavioral flag removal and improved education around flag use, with the intent of addressing bias, health disparities, and inequities, with the underlying assumption that equitable care remains a fundamental right of patients and a professional responsibility of healthcare workers. Although, no universal strategy for preventing WPV exists, Ferron and colleagues note that EHR behavioral flagging has proven helpful in decreasing violent incidents.

Creating and implementing a behavioral flag–removal algorithm presents one strategy for reducing WPV while also ensuring accurate use and discontinuation of flags to avoid adverse outcomes for those identified as at risk of violence. Ferron and colleagues emphasize that flagging isn’t a standalone, one-time practice; it requires ongoing review and revision as part of a comprehensive WPV strategic plan.

All implemented healthcare interventions, including behavioral flagging, require continual guideline and workflow review. Behavioral flagging is a standardized way to communicate the risk of violence to healthcare workers. However, as Kopp and colleagues explained, negative bias, patient stigmatization, and inaccurate use of interventions may emerge when an organization doesn’t address process gaps. A sustainable behavioral flag–removal algorithm can help address this workflow gap while also allowing organizations to provide a safe work environment for patients and staff.

Eva Schill is a nurse practitioner at UW Health University Hospital in Madison, Wisconsin. Susan Luginbuhl is an RN project manager for SSM Health in Madison, Wisconsin. Jessica Leiberg is the dean of the School of Nursing at Concordia University in Mequon, Wisconsin.

American Nurse Journal. 2026; 21(5). Doi: 10.51256/ANJ052654

References

Clerkin C. Beyond Bias: Move from Awareness to Action. Greensboro, NC: Center for Creative Leadership; 2021.

Ferron EM, Kosny A, Tonima S. Workplace violence prevention: Flagging practices and challenges in hospitals. Workplace Health Saf. 2022;70(3):126-35. doi:10.1177/21650799211016903

Gonzales RE, Seeburger EF, Friedman AB, Agarwal AK. Patient perceptions of behavioral flags in the emergency department: A qualitative analysis. Acad Emerg Med. 2024;31(7):640-8. doi:10.1111/acem.14887

Gopal DP, Chetty U, O’Donnell P, Gajria C, Blackadder-Weinstein J. Implicit bias in healthcare: Clinical practice, research and decision making. Future Healthc J. 2021;8(1):40-8. doi:10.7861/fhj.2020-0233

Kopp Z, Kryzhanovskaya I, Garcia ME. Lower the flag: A qualitative analysis of behavioral alerts in the electronic health record. Health Equity. 2024;8(1):762-9. doi:10.1089/heq.2024.0089

Seeburger EF, Gonzales R, South EC, Friedman AB, Agarwal AK. Qualitative perspectives of emergency nurses on electronic health record behavioral flags to promote workplace safety. JAMA Netw Open. 2023;6(4):e239057. doi:10.1001/jamanetworkopen.2023.9057

Tan ZS, Qureshi N, Roberts P, et al. Alerting providers to hospitalized persons with dementia using the electronic health record. J Am Geriatr Soc. 2024;72(3):822-7. doi:10.1111/jgs.18673

Walter RR. Emancipatory nursing praxis: A theory of social justice in nursing. ANS Adv Nurs Sci. 2017;40(3):223-41. doi:10.1097/ANS.0000000000000157

Key words: workplace violence, behavioral flag, implicit bias, electronic health records