Print Page   |   Contact Us   |   Your Cart   |   Report Abuse   |   Sign In   |   Apply for NAQC Membership
Site Search
Sign up for NAQC membership today!

Receive a monthly issue of Connections!
Implementation Guide - Reach

NAQC-RECOMMENDED QUALITY STANDARD:

CALCULATING TREATMENT REACH

 

IMPLEMENTATION GUIDE

 

Table of Contents

 

Purpose

Implementation Timeline

Key Definitions

Measures Related to Reach

Evidence-based Treatment

Target Population

Treatment Reach

Data Sources

Behavioral Risk Factor Surveillance Survey (BRFSS)

Tobacco Use Supplement to the Current Population Survey (TUS-CPS)

Canadian Tobacco Use Monitoring Survey (CTUMS)

Canadian Community Health Survey (CCHS)

Critical Recommendations from the NAQC Issue Paper

References

Calculating Treatment Reach Worksheet

 

 


Purpose

Tobacco users have access to quitlines throughout the United States, Canada, Mexico, and 30 other countries. Yet there has been no standard way to assess and report on quitline reach. In addition, there have not been consistent definitions for the terms "reach” and "utilization.”

 

The North American Quitline Consortium (NAQC) quality standard measurement for reach of quitlines is intended to provide a consistent mechanism for both defining and measuring one type of reach: Treatment Reach, or the proportion of tobacco users that received evidence-based services. This standard definition will result in a more conservative estimate of reach of quitlines than using call volume or other measures but will also allow for a more consistent measure of reach to be used for comparisons between quitlines.

 

A consistently used standard calculation for Treatment Reach provides quitlines with at least one measure that will be calculated the same way by all quitlines, and it can serve as a comparison not only to other quitlines but also to other measures of reach. If quitlines find it useful to do so, they should continue to monitor and report on other measures of reach, such as call volume, which can be used to justify additional allocation of resources to ensure appropriate service delivery.

 

This document is intended to provide a quick and user-friendly reference to Key Definitions, a listing of the Critical Recommendations from the full Issue Paper, and a calculation worksheet. Descriptions of some of the primary Data Sources for population estimates of tobacco use prevalence, and how to access them, are also included.

 

Back to Top

Implementation Timeline

Quitlines are encouraged to begin using the new standard definition and calculation as soon as possible. NAQC’s Issue Paper, "Measuring Reach of Quitline Programs,” was finalized in the spring of 2009 and serves as the primary reference guide for calculating reach (available at http://www.naquitline.org/?page=qiiissuepapers). Implementation technical assistance materials (including this Implementation Guide) were released in the fall of 2009.

 

NAQC’s 2009 annual survey of quitlines was the first NAQC survey to collect utilization data according to the new standard definition of Treatment Reach. Implementation technical assistance will continue throughout 2010. To request technical assistance or to ask questions about using the Treatment Reach calculation, contact NAQC at naqc@naquitline.org or 602 279-2719.

 

Back to Top

 

Key Definitions 

Measures Related to Reach

Quitlines may find it useful to measure and report on several types of reach, including those listed here. A brief definition, as well as some of the pros and cons of using each measure, is included.

 

Definition

Pros

Cons

Awareness: Proportion of tobacco users who know about quitline services or who report using the quitline. May be a useful promotion-related measure.

Measured with population surveys, which are representative of the total population.

Self-report may be inaccurate and does not indicate willingness to use the service.

Call volume: Number of calls that come to the quitline service number, whether answered, hang-ups, or left a voice mail message. Can be used to make the case for additional resources to be allocated to service delivery, especially if many calls that come in cannot be fielded or served appropriately.

Easy to obtain from the phone system. It is an indication of interest in the service and is independent of service capacity.

There is no way to determine repeat callers. Are 100 people each calling once, or is one person calling 100 times? It is not a measure of service delivery.

Intake: Completed Minimal Data Set (MDS) questions, including how they heard about the service. Allows quitlines to track specific promotional efforts.

All quitlines currently collect intake information.

Many callers do not intend to receive counseling (e.g., proxies, those who want materials only, and those who have brief questions).

Intended service: Enrolled in service, eligible, and agreed to participate. Indicates how well the quitline "sells” evidence-based services.

Easy to measure for those who have a formal registration process.

Conversion from intended to received service can vary across quitlines and time.

Received evidence-based service: Participated in any treatment informed by the evidence base, e.g., telephone counseling or medications. Reinforces high standards for quitlines by counting as reached only callers receiving evidence-based treatment.

Relevant, meaningful, available, and reliable measure.

Conservative measure and therefore will be lower than other calculations of reach used in the past.

 

Back to Top

 

Evidence-based Treatment 

For the purposes of calculating quit rates for quitlines, evidence-based services include the provision of any amount of counseling or medications. Even a small amount of counseling has been shown to be effective in a clinical setting. While a clinical setting is not identical to a quitline setting, it is reasonable to assume that the same types of relationships exist between length of total contact time and outcomes for quitlines as for counseling in a clinical setting.

 

The 2008 Clinical Practice Guideline divides counseling into several categories based on total amount of person-to-person contact time in a clinical setting, from 1-3 minutes to more than 300 minutes (Fiore et al., 2009). (See Table 6.9 on page 85 of the Guideline for details.)

 

While any contact time significantly increased abstinence rates over those produced by no contact, there was a clear correlation between increased contact time and higher abstinence rates up to 90 minutes. There was no evidence that abstinence rates continued to increase beyond 90 minutes of total contact time (Fiore et al., 2009). For quitlines, time spent conducting intake or other assessment procedures should not be included in the calculation of the number of minutes of counseling. If quitlines cannot identify exactly when counseling starts, they should identify an average number of minutes for noncounseling activities (e.g., intake, assessment), and subtract that from the total number of minutes of interaction to obtain the number of minutes of counseling. Estimates are fine, especially if identifying an exact number would be burdensome for a quitline.

 

While it is not critical to be able to identify the exact number of minutes of counseling each caller received, quitlines are encouraged to consider how the total number of counseling minutes provided fits into their goals and available resources. Quitlines may want to examine the relationship between quit rates and total number of minutes of counseling in order to help answer the question of whether the differences in efficacy found in the literature for clinical settings also hold for telephone quitlines.

 

Medications counted as evidence-based treatment include nicotine replacement therapy (NRT), bupropion, and varenicline.

 

As new treatments amass sufficient proof of their efficacy, the list of evidence-based treatments will grow. For example, while the evidence base for Web-based services is still developing, a recent meta-analysis concluded that there is sufficient clinical evidence to support the use of Web- and computer-based smoking cessation programs for adult smokers (Myung, 2009). Additional research is needed, especially to determine what elements of Web-based services are necessary to be effective. NAQC will review the literature and revise the definition of "evidence-based services” that should be counted in the calculation of quit rates on a regular basis.

 

Back to Top

 

Target Population

The target population for a quitline is the population that the quitline aims to serve. Some quitlines serve all adult tobacco users in a given state or province, while some serve only uninsured tobacco users. The total target population is the denominator of the reach calculation.

 

U.S. quitlines serving all adult tobacco users can use the Behavioral Risk Factor Survillance System (BRFSS) or Tobacco Use Supplement – Current Population Survey (TUS-CPS) to identify estimates of population size for their target population.

 

Canadian quitlines can use the Canadian Tobacco Use Monitoring Survey (CTUMS) or Canadian Community Health Survey (CCHS) to identify the total target population. (See Data Sources section below.) Quitlines serving a subset of all adult tobacco users (e.g., under- or uninsured tobacco users) should use data sources that provide relevant estimates of population size for each quitline.

 

Quitlines interested in calculating reach for subpopulations of callers, especially priority populations, should use data sources that provide tobacco use prevalence estimates for those populations. For assistance in locating appropriate prevalence estimates, contact NAQC at naqc@naquitline.org or 602 279-2719.

 

Back to Top

Treatment Reach

 
Number of tobacco users who received quitline evidence-based treatment
___________________________________________________
Total target population
          =   TREATMENT
    REACH
 

 

Data Sources  

Behavioral Risk Factor Surveillance Survey (BRFSS)

The Behavioral Risk Factor Surveillance Survey is the world’s largest, on-going telephone health survey system, tracking health conditions and risk behaviors in the United States yearly since 1984. Currently, only cigarette smoking is assessed, but the 2009 survey is scheduled to add questions about smokeless tobacco use. Annual data are generally available about 6 months into the following year (e.g., 2008 BRFSS data were released at the end of May 2009).

 

BRFSS data are available by accessing the STATE system at http://apps.nccd.cdc.gov/StateSystem.

 

Back to Top

Tobacco Use Supplement to the Current Population Survey (TUS-CPS)

The Tobacco Use Supplement to the Current Population Survey is a National Cancer Institute-sponsored survey of tobacco use that has been administered as part of the U.S. Census Bureau’s Current Population Survey in 1992-1993, 1995-1996, 1998-1999, 2000, 2001-2002, 2003, and 2006-2007. The Centers for Disease Control and Prevention has been a co-sponsor with NCI since 2001-02.

 

The TUS-CPS is a key source of national and state level data on smoking and other tobacco use in the U.S. household population because it uses a large, nationally representative sample that contains information on about 240,000 individuals within a given survey period.

 

The TUS-CPS has been translated into Spanish, Chinese, Khmer, Korean, and Vietnamese. Information about the translated questionnaires, including reports on the review and pretesting of the translations, and copies of the questionnaires in all available languages are available at http://riskfactor.cancer.gov/studies/tus-cps/translation/.

 

Results for 2006-2007 are available at http://riskfactor.cancer.gov/studies/tus-cps/results/data0607/.

 

TUS-CPS data are also available by accessing the STATE system at http://apps.nccd.cdc.gov/StateSystem.

 

Back to Top

Canadian Tobacco Use Monitoring Survey (CTUMS)

The Canadian Tobacco Use Monitoring Survey has been conducted for Health Canada since 1999 and provides data on tobacco use and related issues. The primary objective of the survey is to track changes in smoking status, especially for populations most at risk, such as the 15- to 24-year-olds. The survey allows Health Canada to estimate smoking prevalence by province, sex, and age groups on a semi-annual basis. The annual sample for CTUMS is approximately 20,000. All questions are related to smoking.

 

CTUMS 2008 summary results are available at http://www.hc-sc.gc.ca/hc-ps/tobac-tabac/research-recherche/stat/_ctums-esutc_2008/ann-eng.php. Table 2 shows smoking prevalence estimates by province, age group (15-19, 20-24, 25-44, and 45+), and sex.

 

Smoking prevalence estimates are also available for each province for the 18+ adult population. For updated figures, contact naqc@naquitline.org.

   

Back to Top

Canadian Community Health Survey (CCHS)

The Canadian Community Health Survey is a cross-sectional survey that collects information related to health status, health care utilization, and health determinants for the Canadian population. It relies upon a large sample of respondents and is designed to provide reliable estimates at the health region level. The annual sample size is approximately 65,000. Smoking questions are asked in the context of a wide range of health-related behaviors.

 

Since 2007, CCHS data are collected yearly instead of every two years.

 

CCHS asks about both cigarette smoking and use of other tobacco products, but summary tables generally report on cigarette use and current smoking status only.

 

While the differences between CTUMS and CCHS may influence the smoking estimates produced at a single point in time, the trends produced by the two surveys have been very consistent over time. Because of the larger sample size, CCHS may provide better estimates of prevalence for health regions and communities. Overall, provincial data is consistent between CTUMS and CCHS. Quitlines should choose a single data source (CTUMS or CCHS) and use that source consistently.

 

Most results are reported including all those 12 and older. Some tables include age breakdowns of 15 and older, 20 and older, 35 and older, etc. Summary tables are available at http://cansim2.statcan.gc.ca/cgi-win/cnsmcgi.exe?LANG=e&ResultTemplate=CST&CORCMD=GETEXT&CORTYP=1&CORRELTYP=5&CORID=3226.

 

Back to Top


 

Critical Recommendations from the NAQC Issue Paper

The full Issue Paper, "Measuring Reach of Quitline Programs,” is available at http://www.naquitline.org/?page=qiiissuepapersor by contacting naqc@naquitline.org.

 

1.      Identify the quitline program and timeframe of interest.

1.1.   Define the quitline and timeframe of interest. For example, you might be interested in assessing the reach of a state quitline during fiscal year 2007.

1.1.1.      Most reach calculations will be for a 12-month period, but reach can also be calculated over a multiple-year period.

1.1.2.      Quitlines are encouraged to work with their service providers to obtain quitline data for the time period of interest.

 

Back to Top

 

2.      Identify the target population. What is the target population that the quitline is asked to reach? (This will be used to determine both the numerator and the denominator.)

2.1.   Most quitlines provide service to all tobacco users rather than only to cigarette smokers. Therefore, the target population should be all tobacco users.

2.1.1.      If population survey data are only available for cigarette users, restrict the numerator to cigarette users to calculate Treatment Reach. Note: The TUS-CPS and CCHS surveys provide data about tobacco use other than cigarettes; the BRFSS before 2009 asked only about cigarette use. The BRFSS will begin asking about use of smokeless tobacco in 2009. (See Data Sources section for more information.)

2.2.   Treatment Reach can be calculated for any age group, but the target population should be adults unless otherwise specified.

2.2.1.      Canada’s CTUMS survey (and census) summary tables report on ages 15 and older, and 25 and older, but not 18 and older. Estimates for the 18+ adult population are available by contacting NAQC at naqc@naquitline.org. Treatment Reach can be calculated using any age limit as long as the same age limits are placed on both the numerator and the denominator. Whichever age limit is used should be noted when Treatment Reach is reported. For more information on the CTUMS, see the Data Sources section.

2.2.2.      Comparisons between the United States and Canada should be made with great caution because of the difference between the age limits used for reach calculations.

2.2.3.      A definition of the target population should be included when reporting reach.

 

Back to Top

 

3.      Identify which quitline services are evidence-based. What services does the quitline provide, and how does the quitline identify the receipt of a quitline service?

3.1.   Identify the evidence-based services provided by the quitline. Research supports telephone counseling (proactive and reactive, single or multiple) and pharmacotherapy as evidence-based services, but ongoing research will likely expand the list.

3.1.1.      Until further notice, the start of at least one counseling session or the provision of NRT or other medications (e.g., bupropion or varenicline) through the quitline will be considered as receipt of evidence-based treatment. Quitlines are encouraged, but not required, to identify the number of minutes of counseling each caller receives. Intake and other noncounseling interactions should not be counted in the number of minutes of counseling. All callers receiving any amount of counseling or cessation medication through the quitline should be included in the overall treatment-reach calculation.

3.1.2.      NAQC will review the literature on a regular basis and update the list of evidence-based treatments as well as the definitions of each type of treatment based on that review.

3.2.   Verify that the quitline records the type of service each client receives in a way that differentiates evidence-based treatment from other services.

3.2.1.      Some quitlines do not register callers for services. This makes identification of callers who received counseling difficult. NAQC will continue to work with these quitlines to find mechanisms to identify those who receive counseling.

3.2.2.      Quitlines are encouraged to modify their data collection systems as necessary and as resources allow in order to easily identify callers who received evidence-based services.

 

 

4.      Obtain the numerator. How many people in the target population received evidence-based service?

4.1.   Clients who receive evidence-based treatment should be included in the measurement of Treatment Reach regardless of how they were referred into the program. Include fax referral and "click to call” (i.e., Web-based referral mechanisms that result in a call from the quitline to the tobacco user) in the measure of Treatment Reach if the clients go on to receive one of the evidence-based treatments.  

4.1.1.      There is no need to report on the reach for each mode of entry to the quitline. Treatment Reach can be reported as a single number.

4.1.2.      Tracking how callers were referred to the quitline may be useful to assess the impact of various promotional and outreach efforts.

4.2.   Count each client only once in the measure of Treatment Reach, regardless of the number of services they receive in the time period.

4.2.1.      Quitlines should set an "anchor date” or "anchor event” as a marker for who is included in the treatment-reach calculation. Ideally, the marker would be the date of receipt of the first counseling call, but quitlines may not be able to easily identify this date. The simplest anchor date is the date of first contact with a caller, although not all first calls result in receipt of evidence-based services. As long as the marker is applied consistently, it does not matter what the marker is. If a caller has more than one quit attempt in one year, that caller should only be counted once for that year.

4.2.2.      If quitlines cannot easily identify the start of a new quit attempt, or for quitlines that do not have a mechanism in place for identifying repeat callers, the question "Have you called the quitline in the past year?” should be used as a means of identifying repeat callers. Those who reply "yes” to that question any time in 2009 should not be counted in the treatment-reach calculation for 2009 because it can be assumed that they received treatment in 2008 and were counted as having been served in 2008. This will exclude callers whose initial call in 2008 might have been for information only and who are calling back for counseling in 2009, and, therefore, will underestimate the number of callers receiving treatment in 2009. Because of this limitation, quitlines are encouraged to develop mechanisms for easily identifying repeat callers to avoid this potential undercounting for the treatment-reach calculation.

4.2.3.      Examples (The examples below all assume a January 1 – December 31 reporting period; examples will vary based on a quitline’s reporting period or fiscal year.)

4.2.3.1.A client who calls in January 2008 for an initial counseling call, relapses, and calls back in November 2008 to receive multiple counseling calls and NRT would be counted only once for the treatment-reach calculation for 2008.

4.2.3.2.A client who calls in December 2008 and continues treatment for that quit attempt into 2009 would ideally be counted only once, in 2008. Receipt of the first counseling call would serve as the marker for who should be included in the treatment-reach calculation.

4.2.3.3.A client who calls for a quit attempt in November 2008, receives treatment, relapses, and calls back for a new quit attempt in June 2009 should be counted once in 2008 and once in 2009.

4.3.   In addition to Treatment Reach, record and report other measures related to reach that are useful.

4.3.1.      Many measures related to reach may be useful for quitlines to track and report (see Key Definitions above).

4.3.2.      Reporting these additional measures are not critical or necessary for the calculation of Treatment Reach.

4.3.3.      Quitlines should be as clear and explicit as possible when reporting on other measures of reach, including the definition that is being used for each measure.

 

Back to Top

 

5.      Identify the proper survey to use for the denominator. What survey is available to obtain the best estimate of the size of the target population?

5.1.   To estimate the size of the target population, select a population survey that is relevant and covers the geographic area and time of interest.

5.1.1.      Opt for a survey that is likely to provide an accurate estimate because of the sound sampling scheme, large sample size, and high participation rate (especially for special populations).

5.1.2.      If there is no survey estimate of all tobacco users, select a survey that estimates the number of smokers in the population and restrict the numerator to smokers who received evidence-based treatment.

5.1.3.      It may be difficult to identify a survey that provides a sound sampling scheme, large sample size, and high participation rate for smaller populations (e.g., priority populations). NAQC will be identifying the available tobacco use prevalence estimates for smaller populations and mechanisms for using existing data to generate these estimates. For assistance with determining the best source of tobacco use prevalence data for your state or province, contact naqc@naquitline.org.

5.2.   To assess Canadian provincial quitlines, consider using the CTUMS.

5.2.1.      CTUMS 2008 results are available at http://www.hc-sc.gc.ca/hc-ps/tobac-tabac/research-recherche/stat/_ctums-esutc_2008/ann-eng.php. Provincial smoking estimates are located in Table 2. Smoking estimates for the 18+ adult population are available by contacting NAQC at naqc@naquitline.org.

5.2.2.      There may be other surveys that provide better estimates to compare reach over time for a given province.

5.3.   To assess a U.S. state quitline, consider using the BRFSS once the survey includes questions about all tobacco use, starting in 2009.

5.3.1.      BRFSS data is available by accessing the STATE system at http://apps.nccd.cdc.gov/StateSystem.

5.3.2.      There may be other surveys that provide better estimates, such as the Adult Tobacco Survey, to compare reach over time for a given state.

5.4.   Comparison of reach rates

5.4.1.      To compare between states or provinces, NAQC recommends using the same survey to calculate the size of the target population.

5.4.2.      If the target populations of two quitlines differ, comparing treatment-reach numbers is not recommended. For this reason, the target population and the population survey used to estimate that population should be reported with Treatment Reach.

5.4.3.      To compare quitlines in different countries, use the best survey for each country and note in the reporting how the surveys differed, which may limit the comparability of treatment-reach rates.

 

Back to Top

 

6.      Obtain the denominator. Select the proper estimate from the chosen survey.

 

Back to Top

 

7.      Divide the numerator by the denominator to obtain the quitline’s Treatment Reach.

7.1.   To report on reach to subgroups of the general population, assess the proportion of quitline participants receiving evidence-based services from a particular subgroup relative to their proportion of tobacco users in the general population.

7.1.1.      Because estimates of subgroups are likely to be based on small sample sizes, work with a statistician if you want to compare reach across these populations. A statistician can help you obtain a confidence interval for each calculation, which will indicate how accurate the estimate is.

7.1.2.      To assess Treatment Reach for several quitlines in a state or province, consider collaborating with the various service providers to determine the number of the target population who receive evidence-based treatment through any of the services.

7.1.2.1.It may not be possible to de-duplicate the callers using multiple services to arrive at a single treatment-reach estimate for a state or province where there are multiple quitlines in operation. This should be noted any time combined Treatment Reach is reported.

  

Back to Top


 

References

 

Fiore, M. C., Jaén, C. R., Baker, T. B., et al. (2009). Treating tobacco use and dependence: 2008 update. Quick reference guide for clinicians. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service.

 

Myung, S. K., McDonnell, D. D., Kazinets, G., Seo, H. G., Moskowitz, J.M. (2009). Effects of web- and computer-based smoking cessation programs: meta-analysis of randomized controlled trials. Archives of Internal Medicine, 169(13), 1194.

Back to Top

 

Calculating Treatment Reach Worksheet

 

Task

Example from Oklahoma

Enter calculations/ notes

Step 1

Identify the program and timeframe (generally a 12-month period).

Oklahoma state quitline during fiscal year 2007

 

Step 2

Identify target population. (Who is the quitline trying to serve?)

Adult (18 and older) smokers

 

Step 3

Identify which quitline services offered are evidence-based(any telephone counseling or NRT).

Telephone counseling and quitting medications to eligible callers

 

Step 4

Obtain the numerator: How many of the target population (step 2) received evidence-based treatment(step 3) in the identified timeframe (step 1).

The number of adult smokers who received counseling or quitting medications between 7/1/06 to 6/30/07 is 15,793.

 

Step 4a

De-duplicate the callers identified in step 4 to ensure, to the degree possible, that each individual is counted only once for the time period in question.

Oklahoma’s quitline system identifies repeat callers through a combination of methods. Repeat callers are linked to their previous call history in the database. The number calculated in Step 4 (15,793) is already de-duplicated.

 

Step 5

Identify the proper survey to use for the denominator (i.e., BRFSS, CTUMS, TUS-CPS, ATS). Make sure the survey is measuring the same population as identified in step 2. You may need to adjust the target population in step 2 based on what survey data are available. Note: If your quitline serves a large number of smokeless tobacco users, you may want to select TUS-CPS or other population-based survey that includes a measure of smokeless tobacco use.

The 2007 BRFSS was selected because the data are more recent. Because BRFSS before 2009 only provides data on smokers (not other tobacco users), the numerator in step 4 was restricted to cigarette smokers.

 

Step 6

Obtain the denominator from the survey identified in step 5.

BRFSS-estimated number of smokers in Oklahoma in 2007 is 699,120.

 

Step 7

Divide the de-duplicated numerator from step 4a by the denominator from step 6 for Treatment Reach.

15793 divided by 699120 equals 0.0226, or 2.26%.

 

  

Back to Top

 

Sign In


Latest News

Membership 11 years!.

    3219 E. Camelback Road, #416, Phoenix, AZ 85018 | Ph: 800.398.5489 | Fax: 800.398.5489 | email:naqc@naquitline.org