AzTrials
AzTrials

HIV/AIDSNCT03896776

A Pragmatic Trial of Two Strategies for Implementing an Effective eHealth HIV Prevention Program (Keep It Up! 3.0)

A study testing Keep It Up! 3.0 in people with HIV/AIDS and Gonorrhea, run by Northwestern University at 1 site in United States.

Completed

Finished. The study has ended and is not enrolling. Results may be posted on the official record.

Official record
Search all trials

Trial summary

This study evaluates and compares two implementation strategies of an online HIV prevention intervention: Strategy 1 in which community-based organizations apply, and are selected, for funding to deliver Keep It Up! through current HIV testing programs; and Strategy 2 which is a "direct-to-consumer" model where… Read more about this study

Who is it for?
Ages 18 to 34 years, men onlyHealthy volunteers can take part
Conditions the study is for
HIV/AIDS, Gonorrhea, Chlamydia
What is being tested?
Keep It Up! 3.0
Could I receive a placebo?
No, everyone receives an active treatment
How many people
2,249 participants
Where
1 siteUnited States
Study dates
9 Oct 2019to 31 Mar 2023
Phase
Interventional
Are there results?
Yes, posted in the registrySummary in the doctors' view

Key requirements

Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.

  • Ages 18 to 34 years
  • Men only
  • Healthy volunteers accepted
  • Adult

Who may be able to join

  • recently received an HIV negative test result OR self-report negative or unknown HIV status
  • speaks/reads English
  • has an active email address

Who cannot join

  • has not been on PrEP and adherent for 6 months prior

Trial locations and contacts

1 site in 1 country.

United States1 site
  • Northwestern University

    Chicago, Illinois, 60611

Other trials in United States

About this study

This study evaluates and compares two implementation strategies of an online HIV prevention intervention: Strategy 1 in which community-based organizations apply, and are selected, for funding to deliver Keep It Up! through current HIV testing programs; and Strategy 2 which is a "direct-to-consumer" model where centralized staff at Northwestern University recruit participants nationally through online advertising campaigns and manage engagement.

Read the detailed description

Substantial effort has gone into developing and testing the efficacy of eHealth HIV prevention interventions, but these interventions will not affect the spread of the epidemic unless implemented effectively to reach the most at-risk populations. Despite emerging evidence that eHealth approaches can be highly effective in reducing HIV risk or improving care engagement, implementation requirements do not fit within established strategies for public health scale up, leaving it an open question as to how to bring these programs into practice. To date, little to no implementation science has examined strategies to effectively scale up eHealth HIV prevention programs.

The overarching goal of this study is to address this need by developing and evaluating novel strategies for implementing eHealth HIV prevention programs. Keep It Up! (KIU!) is an online program proven to reduce HIV risk among diverse young men who have sex with men (YMSM). KIU! was designed for YMSM who just tested HIV negative to keep it up and maintain negative status by reducing risk and enacting protections. Strong evidence of effectiveness on behavioral (decreased condomless sex) and biomedical outcomes (reduced STI incidence) makes KIU! an ideal eHealth intervention for implementation research. Lessons learned from KIU! implementation will set the pathway for implementing the many HIV eHealth programs currently undergoing efficacy testing.

Implementation strategies need to be evaluated using rigorous research designs. Through formative research with community-based organizations (CBOs), health departments, the CDC, and researchers, the investigators have identified two pragmatic implementation strategies that will be evaluated and compared:

In Strategy 1, CBOs apply for, and are selected for, funding to deliver KIU! through current HIV testing programs. Because CBOs do not have the capacity to host the technology, KIU! is hosted centrally at Northwestern and deployed by local CBO staff. Training and coaching of CBO staff on how to use the technology and integrate it into routine HIV testing is provided through capacity building assistance.

Strategy 2 is a "direct-to-consumer" (DTC) model where centralized staff at Northwestern primarily recruit participants nationally through online advertising campaigns and manage participants' engagement. HIV/STI testing kits are shipped directly to YMSM, and upon documentation of an HIV negative test result, participants are delivered KIU! This DTC model has been widely adopted in technology dissemination but it is innovative in terms of implementation of HIV prevention programs.

The investigators will accomplish the goal to inform eHealth intervention implementation with two specific aims:

Aim 1: Compare two implementation strategies using a quasi-experimental trial.

The investigators selected a type III hybrid implementation-effectiveness design, which prioritizes empirical comparison of implementation strategies while also collecting evidence of effectiveness. KIU will be delivered using Strategy 1 (CBOs) in 22 counties for two years; simultaneously, the investigators will deliver KIU nationally using Strategy 2 (DTC). YMSM participants in the DTC strategy will be matched to CBO strategy participants based on post-hoc propensity scores. Implementation and effectiveness outcome data will be collected from YMSM participants, CBOs, and the technology vendor. The primary outcomes are public health impact (defined as reach X effectiveness) and cost per infection averted, a key component of CDC decision making about which HIV prevention programs will be supported. There continues to be equipoise for the trial (i.e., either strategy could be superior). Because implementation success can be defined differently for different stakeholders, the investigators will also continue to collect and report as secondary outcomes multiple metrics of reach, effectiveness, and implementation.

Aim 2: Examine adoption characteristics that explain variability in implementation outcomes.

There is remarkable variability in implementation success when CBOs adopt evidence-based interventions. The investigators will seek to explain variability in implementation success across counties by conducting mixed-methods research on the domains from the consolidated framework for implementation research (CFIR), such as county characteristics, adaptations, support from organization leadership, and approach to planning adoption. Data on CFIR characteristics will be collected through administrative data, surveys, and telephone interviews with key stakeholders.

In addition to these 2 specific aims, the investigators will explore sustainment of KIU! at the completion of the study. CBOs will be provided with materials to facilitate applying for external funding to continue to provide KIU! after study completion, including an Impact Tool to estimate local impact and costs. The investigators will also examine factors that predict applying for funding and ongoing sustainment. For Strategy 2, through consultation with the CDC, capacity building assistance providers, health departments, and small businesses involved in Health 2.0 initiatives, the investigators will examine models for ongoing sustainment of the DTC model.

Being tested
Keep It Up! 3.0
Medical subject headings
Acquired Immunodeficiency Syndrome, Gonorrhea, Chlamydia Infections

What participants receive

What participants receive. In a randomised study each person is assigned to one group by chance.

  • Keep It Up! 3.0Behavioural

    Also called KIU! 3.0

    KIU! is an online HIV prevention intervention developed for high-risk young men who have sex with men (YMSM) who recently tested HIV negative. Content was developed in collaboration with YMSM-serving CBOs and subjected to usability testing with diverse YMSM. The Information-Motivation-Behavioral Skills model and eLearning principles guided development of highly interactive, engaging, and culturally relevant health messages. KIU! involves 7 modules completed across 3 sessions, totaling ~1 hour of main content, plus 2 booster sessions at 6- and 12-week follow-ups. Each module is based on a setting or situation relevant to YMSM (e.g., connecting to the gay community and meeting guys through apps), with developmentally appropriate behavior change content embedded. KIU! uses diverse delivery methods (e.g. videos, animation, games) to address HIV knowledge gaps, motivate safer behaviors, teach behavioral skills, and instill self-efficacy for preventive behaviors.

Study groups in NCT03896776
Study groupRoleWhat they receive
Community Based Organization (CBO) DeliveryExperimentalCBOs will be selected through a Request for Proposal (RFP) process. The RFP will describe research and KIU! delivery activities to be conducted as part of the study and allowable budget to carry out the activities. The RFP will ask CBOs to describe past experience providing HIV services for YMSM. Selected CBOs will recruit participants into the intervention and encourage participants to complete each session of intervention content at baseline, 6 Week Follow-up, and 12 Week Follow-up. Participants in the CBO delivery arm will receive baseline HIV and STI testing at a CBO. Participants will receive an at-home STI test kit at 12 Week Follow-up if testing is not provided at their CBO site.
Receives: Behavioral: Keep It Up! 3.0
Direct to Consumer (DTC) DeliveryExperimentalIn the DTC arm, participants will be recruited online via paid social media advertising (e.g., Facebook, Instagram). Advertisements will target placements by age, gender, sexual orientation, racial background, "likes" that are relevant to YMSM (e.g., local LGBT organizations, "out" celebrities), and location (i.e., target county). Dating/sex-seeking apps (e.g., Grindr) will also be used to recruit YMSM, using a similar advertising approach as social media. These online recruitment strategies will be supplemented by referrals from local organizations and participant registries, and snowball recruitment. Study staff at Northwestern University will manage this recruitment process and encourage participants to complete each session of intervention content at baseline, 6 Week Follow-up, and 12 Week Follow-up. Participants in the DTC delivery arm will receive at-home HIV and STI test kits at baseline. Participants will receive an at-home STI test kit at 12 Week Follow-up.
Receives: Behavioral: Keep It Up! 3.0

What is being measured

The primary outcome is the main question the study is designed to answer.

  1. Public Health Impact

    Public Health Impact (PHI) is defined as the number of HIV infections averted.

    HIV infections averted were calculated as the difference between baseline and 12-week extrapolated HIV incidence rates (IRs) per 100 person-years. Given the incidence of HIV among YMSM in the US, very large sample sizes would be required to directly estimate HIV infections averted, so HIV IRs were extrapolated for each study arm-timepoint using a model predictive of HIV incidence published by Mulick and Murray (2020) that uses rectal gonorrhea infection as a determinant.

    Measured: 12 Weeks

Timeline

From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.

  1. First posted1 Apr 2019
  2. Study start9 Oct 2019
  3. Primary completion31 Mar 2023
  4. Study completion31 Mar 2023

Resources

More details

For doctors and researchers: how the study is designed, who runs it, and its registry IDs.

Study design
Study type
Interventional
Phase
Phase N/A
Primary purpose
Prevention
Allocation
Randomised
Intervention model
Parallel groups
Masking
None (open label)
Enrolment
2,249 participants (actual)
IDs, dates and publications
ClinicalTrials.gov ID
NCT03896776
Sponsor's protocol ID
R01MH118213
Status last verified
Jun 2026
First submitted
28 Mar 2019
Last updated
28 Oct 2026
Data monitoring committee
Yes
FDA-regulated drug
No
FDA-regulated device
No
Individual data sharing
Yes: The PI agrees to develop a de-identified database, codebook, and mechanism by which IPD could be shared with other investigators upon approval of the PI. Data will be available for request approximately 6 months after completion of the project. Interested investigators will be asked to complete a standardized request form stating the specific aims of the analysis, the analytic plans, resources the requestors have to carry out the project, the proposed timeline, and distribution goals (manuscripts and/or grant application). The PI will review these requests to determine whether the proposed analyses constitute an innovative and significant exploration of the data, whether the proposed team has sufficient resources to undertake the request, how data will be protected/managed, and whether there are sufficient resources to honor the request. If any of these issues are problematic, the PI will attempt to negotiate a fair resolution with the interested parties and/or with NIH program staff.

Publications

  • Zamantakis A, Zapata JP, Smith JD, Danielson EC, Macapagal K, Saber R, Li D, Benbow N, Mustanski B. Incentivizing Prevention: Community Based Organizations' Perceptions on Financial Incentives for Recruitment and Retention of Participants in Keep it Up! (KIU!), an eHealth HIV Prevention Intervention. AIDS Behav. 2026 Apr;30(4):1239-1251. doi: 10.1007/s10461-025-04963-w. Epub 2025 Nov 21.

    PubMed 41266922
  • Zamantakis A, Danielson E, Rudd E, Zapata JP, Benbow N, Saber R, Knapp AA, Mustanski B. Community-Based Organizations' Approaches to Recruitment and Retention for a Digital HIV Prevention Intervention for Young Men Who Have Sex With Men: A Mixed Methods Study. J Med Internet Res. 2025 Nov 3;27:e63199. doi: 10.2196/63199.

    PubMed 41183372
  • Mustanski B, Benbow N, Macapagal K, Li D, Madkins K, Saber R, Linas B, Smith JD, Brown CH, Munroe S, Reddy S, Schackman BR, Swann G, Janulis P, Zamantakis A, Zapata JP. Comparing Implementation and Effectiveness Outcomes for Two Implementation Strategies of the Keep It Up! Digital HIV Prevention Program: A Type 3 Hybrid Effectiveness-Implementation Trial. AIDS Behav. 2025 Dec;29(12):4030-4042. doi: 10.1007/s10461-025-04838-0. Epub 2025 Aug 19.

    PubMed 40826259
  • Zamantakis A, Zapata JP, Greenawalt I, Knapp AA, Benbow N, Mustanski B. Barriers and Facilitators to Implementing Keep It Up!, A Digital Health Intervention, in Community-Based Organizations. AIDS Behav. 2024 Dec;28(12):3944-3955. doi: 10.1007/s10461-024-04525-6. Epub 2024 Oct 10.

    PubMed 39387999
  • Li DH, Zamantakis A, Zapata JP, Danielson EC, Saber R, Benbow N, Smith JD, Swann G, Macapagal K, Mustanski B. A mixed-methods approach to assessing implementers' readiness to adopt digital health interventions (RADHI). Implement Sci Commun. 2024 Aug 27;5(1):91. doi: 10.1186/s43058-024-00628-2.

    PubMed 39192345

Questions about this trial

What is this clinical trial studying?

A study testing Keep It Up! 3.0 in people with HIV/AIDS and Gonorrhea, run by Northwestern University at 1 site in United States.

The full registered description is in the About this study section and the official record.

Who may be eligible to take part?

The sponsor lists 3 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.

Is this clinical trial currently recruiting?

The record says: finished, as of the sponsor's last update on 28 Oct 2026. Status can change between updates, so confirm with the study contact before making plans.

Where is the study taking place?

The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.

What are the possible benefits and risks?

They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.

Are results available for this study?

Yes. Results are posted in the registry (first posted 28 Oct 2026). A summary is in the For doctors view, and the full results are on ClinicalTrials.gov.

How can a doctor refer a patient or contact the research team?

The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.

What does the primary outcome measure?

Public Health Impact (time frame: 12 Weeks). It is what the study was designed to measure, not a finding.

Where can I read the original study information?

On the ClinicalTrials.gov record for NCT03896776, which also lists 5 linked publications.

Does a listing mean the treatment is approved or proven to work?

No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.

How often is this information updated?

AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 28 Oct 2026.

Free account

Log in to keep searching

You’ve used your 3 free searches. AzTrials is free: log in or create a free account to keep searching clinical trials and save the ones you want to follow.

Create a free accountLog in