Eligibility criteria

Background

HemOnc.org has grown organically over time. From the beginning, the goal of the website has been to capture treatment details and the supporting literature for the systemic antineoplastic treatments used in the daily practice of hematology/oncology. As such, we initially drew our source material primarily from clinical practice guidelines, in particular focusing on treatments with a "top-level" recommendation (e.g., category 1 evidence in the NCCN Guidelines). As we have increasingly observed the standard of care evolve over time (e.g., immunotherapy was still in its infancy when the website was launched in 2011), we have developed an increasingly systematic approach to the identification and inclusion of regimens and their supporting literature. This is reflected in the first two of our four overarching goals:

• Create a database of all approved systemic antineoplastic therapy agents and supportive medications used in the field of hematology/oncology.
• Create a database of all standard-of-care systemic antineoplastic therapy regimens and references to primary literature (PubMed and direct links to the abstracts articles).

Information about the processes we use to identify clinical studies and maintain core website functions are described on pages across the site, such as eligibility criteria and sources. In order to centralize this information for quick reference, we have created a Standard Operating Procedure (SOP) describing the QC/QA processes for the website and for the HemOnc Ontology.

In the following sections, we describe our current approach to curating and selecting literature for inclusion on HemOnc.org. This information is current as of April 2024.

Defining the Standard of Care (SOC)

One of the main goals of HemOnc.org is creating a database of all standard of care systemic antineoplastic therapy regimens. This is difficult because there is no universally accepted definition of standard of care beyond the legal definition. From Wikipedia, the legal definition of the medical standard of care is: "the level at which an ordinary, prudent professional with the same training and experience in good standing in a same or similar community would practice under the same or similar circumstances." We currently employ four separate definitions that meet the threshold of standard of care:

1. The control arm of a phase 3 randomized controlled trial (RCT), including non-randomized parts of phase 3 RCTs. By implication, this means that all phase 3 RCTs with a control arm or a non-randomized part must eventually be included on the website. See prioritization below for more details.
2. The experimental arm(s) of a phase 3 RCT that provide(s) the community standard of evidence (P-value less than 0.05 and/or a metric such as hazard ratio that does not cross the boundary of 1.00) of superior efficacy for an intermediate surrogate endpoint (e.g., PFS) or a strong endpoint (e.g., OS)
3. A non-randomized study that is either:
   1. The basis for a regulatory agency approval (e.g., the US Food and Drug Administration [FDA])
   2. Recommended as a top-level regimen by ASCO, ESMO, or the NCCN
4. Any study (including case series and retrospective studies) that is specifically recommended by a member of our Editorial Board. All section editors of the Editorial Board with direct oversight of disease-specific pages are board-eligible or board-certified physicians (the "ordinary, prudent professional" standard).

Note: It should be assumed that if a regimen is present on a main cancer subtype page, it has met one or more of these conditions. On the contrary, regimens on the historical pages presumably no longer meet these conditions, even if they were met at one time in the past. In the future, we may consider more explicitly labeling regimens with which of these conditions were met for inclusion on the page.

Randomized trials

RCTs that have two control arms

These are generally non-inferiority trials. Regardless of the outcome, based on criteria #1 above, the study reference will be added to both control arms. If the control arm regimens are missing, they will be built and added to the website.

RCTs with one control arm and one or more experimental arms

Control arm(s)

• Regardless of whether the trial is "positive" or "negative", the study reference will be added to the control arm regimen. If the control arm regimen is missing, it will be built and added to the website.

Experimental arm(s)

• These will be added if the experimental arm has a statistically superior, non-inferior, or equivalent primary outcome, defined as p-value less than or equal to 0.10 or a metric such as hazard ratio (HR) that does not exceed 1.05 for superiority studies or the predefined boundary for non-inferiority/equivalence studies (see levels of evidence)
  • Following the community standard, superior experimental arms with p-value less than or equal to 0.05 or a metric such as HR that does not exceed 1.00 will be added to the main cancer subtype page, whereas those with p-value between 0.05 and 0.10 or a metric such as HR between 1.00 and 1.05 will be added to the historical page counterpart.
• These will be generally not be added if the experimental arm has a statistically nonsignificant primary outcome, defined as p-value greater than 0.10 (see levels of evidence)
  • Exception 1: if a secondary outcome is the basis of a regulatory agency approval, we will add the regimen
  • Exception 2: if a member of the Editorial Board requests that the experimental arm be added, we will add the regimen
• Note: If the experimental arm(s) include a regimen with at least one investigational drug, the study reference will be added on the experimental drug's page, not as a detailed regimen on a disease-specific page. If and when the drug is approved by any regulatory agency, the information will be transferred from the experimental drug's page to the relevant disease-specific page(s).

RCTs with experimental arms only

This situation is very uncommon with Phase 3 trials, but relatively common with Randomized Phase 2 trials; it is a major reason why we relatively de-prioritize Randomized Phase 2 trials (see below). If the study is "negative" (i.e., the primary outcome is statistically nonsignificant), the study reference is not added to HemOnc.org. If the study is "positive", the study reference is added to the regimen which had the superior finding. If the regimen is missing, it will be built and added to the website.

Exceptions

The main exception to these guidelines is the case where a control arm is a non-systemic therapy such as surgery (given the focus of the website, trials where the experimental arm is a non-systemic therapy are typically excluded during the screening process). Example: a phase 3 RCT with a non-systemic control arm and a systemic experimental arm does not meet its primary endpoint (p-value greater than 0.10). In this case, the study reference will not be added to HemOnc.org.

Non-randomized or retrospective studies

Regimens consisting solely of approved drugs

• There is no strict cutoff for efficacy, although the study must meet criteria #3 or #4, above. General guidance for the types of studies that may meet this criteria:
• The studied regimen should be described by the authors as equivalent or better than current standard-of-care, using such terms as "promising", "new standard-of-care", "at least as good", etc.
• If comparative efficacy to historic controls is reported, the new regimen should have a statistically superior outcome, defined as p-value less than or equal to 0.10 (see levels of evidence)
• The study should have at least 20 participants

Regimens that include an experimental drug

Note that if such a regimen is deemed eligible for inclusion, it will be added as a reference on the experimental drug's page, not as a detailed regimen on a disease-specific page. If and when the drug is approved, the information is transferred from the experimental drug's page to the relevant disease-specific page(s).

• The study should meet its predetermined primary efficacy endpoint
• It should be clear that the regimen will go on to be tested in phase III trials
  • Exception: if the non-randomized study is the basis of an FDA approval, this is not required
• With few exceptions, the study should have at least 20 participants
  • If a phase I or Ib study, this should be at least 20 participants receiving the maximum tolerated dose (MTD)

Real-world data (RWD) (new for 2024)

In 2024, we will be initiating a pilot for inclusion of regimens and regimen variants that are only observed in real-world data (RWD). We are still ironing out the exact criteria in collaboration with several RWD partners, but here are some preliminary ideas:

• The regimen should have been observed in RWD at least 100 times (i.e., 100 unique patients have received the regimen)
• The regimen should have been observed in RWD at at least 3 different institutions, reasonably separated geographically
• The regimen should only consist of approved drugs

Screening for trials

In order to identify new regimens and study references for inclusion on HemOnc.org, we undertake several parallel screening methods. These are described in further detail here:

1. PubMed search for ("Phase 3" OR "Phase III") AND "neoplasms"[MeSH Terms] (episodic; last performed in 2020)
   1. 2020 review: A full search back to the earliest occurrence of these terms, which is 1964, was performed on 2020-11-26.
      1. There were 52,215 non-duplicated results.
      2. Of these, 49,395 were not already present on HemOnc.org
      3. Of these, 38,106 remain after removing references with the following words/phrases in their titles: "review", "design", "rationale", "retrospective", "compassionate", "subgroup", "-graphy", "morphine", "sugammadex", "motility", "psoria-", "-itis", "multiple sclerosis", "asthma", "allergen", "hypertension", "heart disease", "lipid", "cardiovascular", "diabetes", "contraceptive", "imaging", "pain", "opioid", "hot flashes", "HCV", "HBV", "infection", "viral". These were manually reviewed.
2. PubMed email alert for "Clinical Trial, Phase III "[Publication Type] AND "Neoplasms"[Mesh] (subscription; ongoing per PubMed alerting criteria)
3. Review of the eTOC of the following "top-tier" general medical and hematology/oncology journals (subscription; ongoing per journal publication schedules):
   1. JAMA
   2. The Lancet
   3. The New England Journal of Medicine
   4. Annals of Oncology Official journal of the European Society for Medical Oncology (ESMO)
   5. Blood Official journal of the American Society of Hematology (ASH)
   6. JAMA Oncology
   7. Journal of Clinical Oncology Official journal of the American Society of Clinical Oncology (ASCO)
   8. The Lancet Haematology
   9. The Lancet Oncology
4. Time permitting, review of the eTOC of other hematology/oncology journals, conference proceedings, and e-mail alerts (see sources)
5. Review of all freely available Cochrane Library Reviews under the topics "Cancer" and "Blood disorders" (episodic; last completed: October 2018)
6. Review of all ASCO, ESMO, and NCCN clinical practice guidelines (ongoing)
7. Review of all studies cited on the FDA drug label ("package insert") section 14 (CLINICAL STUDIES) for all antineoplastic agents (ongoing for new approvals and new indications; review of all available existing labels completed: September 2019)
8. Queries to all Editorial Board members (quarterly)

Prioritization

Because HemOnc.org is a voluntary effort driven by the contributors, it is not possible to add every study describing every treatment regimen, in anything close to real time. What follows is the prioritization list for adding new regimens to HemOnc.org:

1. Regimens described in registration studies (FDA has priority; other approval agencies are considered) whether they are randomized or not. However, if a product is approved and labeled based on both randomized and non-randomized studies, the priority will be to add the randomized studies.
2. Clinically relevant regimens, evaluated in fully enrolling phase 3 RCTs (including trials not specified as phase 3 but having a statistical power of 90% or greater), published in top-tier journals
   1. Control arms of RCTs reported by the paper describing these regimens, if they are described in sufficient detail
   2. Experimental arms of RCTs reported by the paper describing these regimens, if they have statistically superior findings
3. Clinically relevant regimens, evaluated in fully enrolling phase 3 RCTs (including trials not specified as phase 3 but having a statistical power of 90% or greater), published in other journals (see Sources)
   1. Control arms of RCTs reported by the paper describing these regimens, if they are described in sufficient detail
   2. Experimental arms of RCTs reported by the paper describing these regimens, if they have statistically superior findings
4. Clinically relevant regimens, evaluated in randomized phase 2 RCTs or incompletely enrolled phase 3 RCTs (e.g., those closed early due to poor accrual), published in top-tier journals
   1. Control arms of RCTs reported by the paper describing these regimens, if they are described in sufficient detail
   2. Experimental arms of RCTs reported by the paper describing these regimens, if they have statistically superior findings
5. Clinically relevant regimens, evaluated in randomized phase 2 RCTs or incompletely enrolled phase 3 RCTs (e.g., those closed early due to poor accrual), published in other journals (see Sources)
   1. Control arms of RCTs reported by the paper describing these regimens, if they are described in sufficient detail
   2. Experimental arms of RCTs reported by the paper describing these regimens, if they have statistically superior findings
6. Clinically relevant regimens, evaluated in non-randomized trials, published in top-tier journals
7. Clinically relevant regimens, evaluated in non-randomized trials, published in other journals (see Sources)
8. Clinically relevant regimens, evaluated in case series, published in top-tier journals
9. Clinically relevant regimens, evaluated in case series, published in other journals (see Sources)
10. Clinically relevant regimens, evaluated in retrospective studies, published in top-tier journals
11. Clinically relevant regimens, evaluated in retrospective studies, published in other journals (see Sources)
12. Clinically relevant regimens, evaluated in fully enrolling phase 3 RCTs (including trials not specified as phase 3 but having a statistical power of 90% or greater), published in conference proceedings (see Sources)
    1. Experimental arms of RCTs reported by the abstract describing these regimens, if they have statistically superior findings
13. Clinically relevant regimens, evaluated in randomized phase 2 RCTs or incompletely enrolled phase 3 RCTs (e.g., those closed early due to poor accrual), published in conference proceedings (see Sources)
    1. Experimental arms of RCTs reported by the abstract describing these regimens, if they have statistically superior findings
14. Clinically relevant regimens, evaluated in non-randomized trials, published in conference proceedings (see Sources)
15. Clinically relevant regimens, evaluated in case series, published in conference proceedings (see Sources)
16. Clinically relevant regimens, evaluated in retrospective studies, published in conference proceedings (see Sources)

Disclaimer

Note: these eligibility criteria, which pertain mainly to treatment regimens, are guidelines that we try to follow rigorously. However, there are exceptions. For example, if a contributor or a member of our Editorial Board requests that a certain regimen be added, we will strive to do this even if the criteria are not strictly met. See the content tutorial and sources pages for more information. As outlined in our disclaimer, inclusion of a regimen on HemOnc.org is not an endorsement of its efficacy or appropriateness for use in any given clinical situation.