BackgroundCompletion axillary lymph node dissection is recommended for patients with metastases to the sentinel lymph node (SLN) in breast cancer although nonsentinel lymph nodes (NSLN) are often negative for tumor. Online nomograms are available to predict risk of NSLN disease.
ObjectiveTo compare the accuracy of the Memorial Sloan-Kettering Cancer Center (MSKCC) nomogram (using 9 variables) with the Stanford nomogram (using 3 variables) in predicting NSLN metastasis.
SettingA single academic center.
PatientsProspectively maintained database of patients with breast cancer who underwent SLN biopsy from October 1, 1999, through January 31, 2008.
MethodsRisk of NSLN metastasis was calculated using each nomogram's online calculator. Results from the axillary lymph node dissection were reviewed for positive NSLNs. Nomograms were evaluated using the area under the receiver operating characteristic curve, false-negative rates, positive predictive value, and calibration plot.
Main Outcome MeasuresNomogram scores and axillary lymph node dissection results.
ResultsOf 579 patients who underwent SLN biopsy, 179 (30.9%) had a positive SLN. For 123 patients who underwent axillary lymph node dissection, the area under the curve for the MSKCC and Stanford nomograms was 0.72 and 0.70, respectively. False-negative rates for nomogram values of 10% or less were low (4.1% for the MSKCC and 7.8% for the Stanford). The positive predictive value for nomogram probabilities of 80% or greater was higher for MSKCC than for Stanford (90.9% vs 61.8%). The Stanford nomogram performed more accurately in low-risk patients with isolated tumor cells or micrometastatic SLN disease; however, the MSKCC nomogram more accurately predicted NSLN outcomes across the entire study population.
ConclusionAlthough the MSKCC and Stanford nomograms performed similarly on the basis of the area under the curve, the MSKCC nomogram was consistently more reliable in predicting actual NSLN outcomes.
Axillary lymph node status is the most clinically significant factor in the management of invasive breast cancer.1-3 In the 1990s, clinical guidelines replaced axillary lymph node dissection (ALND) with sentinel lymph node (SLN) biopsy for assessing regional lymph node involvement in patients with early-stage breast cancer and clinically negative nodes.2,4 This practice reduces the morbidity associated with ALND in those patients who have a negative SLN and who are therefore at low risk for additional nodal disease.
Completion ALND is the current standard of care for patients with a positive SLN. However, there is ongoing scrutiny regarding the need for completion ALND in every patient with a positive SLN. Following ALND, only 35% to 50% of patients are found to have metastases within nonsentinel lymph nodes (NSLNs), and axillary recurrence following systemic treatment is low.5-8 This translates into additional surgery and increased risk of morbidity, possibly without additional therapeutic benefit, for a significant subset of patients. A recent randomized trial of SLN biopsy alone vs completion ALND for SLN-positive patients with early-stage breast cancer found no difference in 5-year survival.9 Indeed, the idea that ALND may not affect overall survival was first suggested by the National Surgical Adjuvant Breast and Bowel Project B-04 trial published nearly 35 years ago.10
Accordingly, several mathematical models have been developed to help predict whether tumor has metastasized to NSLNs when the SLN is positive. The most widely used models include 4 nomograms (Memorial Sloan-Kettering Cancer Center [MSKCC], Mayo, Cambridge, and Stanford) and 3 scoring systems (Tenon, M. D. Anderson, and Saidi).6,7,11-15 These models use an array of primary tumor characteristics (tumor size, histologic grade, lymphovascular involvement, and hormonal receptors) and nodal factors (number of positive SLNs, detection method, and size of metastases) to calculate the risk of NSLN metastasis. Only 2 of these models, the Stanford and MSKCC nomograms, are available as online calculators. Online availability allows easy, unrestricted access by physicians at any institution worldwide, and the calculator tool functions to generate results quickly, facilitating its use in patient counseling and tumor board discussions. The purpose of our study was to compare the accuracy and clinical utility of these 2 online calculators in predicting NSLN metastasis in SLN-positive breast cancer patients during a 10-year period at our institution.
We conducted a review of a prospectively maintained database of all patients who underwent SLN biopsy for early-stage breast cancer (defined as T1-T3 and clinically node negative) at Oregon Health & Science University in Portland, Oregon, from October 1, 1999, through January 31, 2008. Approval from the institutional internal review board was obtained before data collection. Our inclusion criteria were as follows: presence of an invasive breast cancer (T1-T3), a clinically negative axilla, SLN biopsy with tumor metastasis in at least 1 node, performance of a completion ALND, and documentation of all variables as required by each nomogram. Patients who did not meet these criteria were excluded from the study. Risk of NSLN metastasis was calculated for every patient using the online calculator for each nomogram.
Surgical technique and pathology review
Our methods for SLN biopsy and pathology review have previously been described in detail.16 In brief, all SLNs were identified using a combination of technetium Tc 99 sulfur colloid tracer and isosulfan or methylene blue dye. Sentinel lymph nodes were immediately examined by frozen section and interpreted by a surgical pathologist. If the frozen section was positive, an axillary dissection was completed in the same procedure.
We re-reviewed the pathologic characteristics for all positive SLNs to confirm size of tumor and method of detection. The size of metastatic tumor deposits within all nodes was categorized according to the American Joint Committee on Cancer StagingManual (7th ed): isolated tumor cells (ITC) were defined as tumor deposit of 0.2 mm or less (pN0i+), micrometastases (MI) were defined as tumor deposit of more than 0.2 mm but not more than 2 mm (pN1mi), and macrometastases were defined as tumor deposit of more than 2 mm (pN1).17 Maximal size of SLN tumor deposit was used for nomogram calculations.
Mskcc breast cancer nomogram
Calculations were made using the MSKCC “Additional Nodal Metastasis” online calculator (). The nomogram provides a risk estimate of NSLN metastasis based on 9 histopathologic variables: frozen section analysis (yes/no), primary tumor size (in centimeters), primary tumor type (ductal/lobular) and nuclear grade (1-3), number of positive SLNs, method of SLN detection (routine hematoxylin and eosin, serial hematoxylin and eosin, or immunohistochemistry), number of negative SLNs, lymphovascular invasion (yes/no), multifocality (yes/no), and estrogen receptor positivity (yes/no).7 A tumor is considered estrogen-receptor positive if at least 10% of cells stain positive within the specimen. In our study, the Nottingham Histologic Score was used in place of nuclear grade, and lymphovascular invasion was scored as negative when not stated in the pathology report.
Calculations were also made using the “Stanford Online Calculator” for NSLN metastasis (). This nomogram requires 3 histopathologic variables: primary tumor size (in centimeters), size of SLN metastasis (ITC, MI, and macrometastases), and angiolymphatic invasion (yes/no).6 Similar to MSKCC, the Stanford nomogram provides a predicted probability of having positive NSLNs.
The performance of each nomogram was evaluated by drawing a receiver operating characteristic (ROC) curve and calculating the area under the curve (AUC). The AUC reflects a test's ability to discriminate between diseased and nondiseased patients.5 An AUC of 1 indicates perfect concordance (100% sensitivity and 100% specificity), whereas an AUC of 0.5 indicates a result equal to chance. The 95% confidence interval (CI) for each AUC was determined using a 2000-replicate bootstrap procedure.
The nomograms were further assessed using false-negative rate, positive predictive value, and calibration plots. The false-negative rate (ie, the probability that the nomogram predicts there is no disease when there actually is disease) for nomogram values of 10% or less was used to assess the ability of each nomogram to accurately identify patients at very low risk for NSLN metastasis. The positive predictive value (ie, the probability that the patient has disease given a positive test result) for nomogram values of 80% or greater was used to assess the ability of each nomogram to accurately identify patients at very high risk for NSLN metastasis. The accuracy of each nomogram was determined by calibration plot, which compares predicted (nomogram) probabilities to the observed incidence of NSLN metastasis. Predicted probabilities were represented in quintiles to increase homogeneity within groups while minimizing the size of the 95% CIs. Finally, we tested both nomograms in a subset of patients who had only ITC or MI disease in the SLN. Previous validation studies have shown that nomograms often underestimate or overestimate NSLN metastasis in this low-risk population.5,18,19 Statistical analysis was performed using R, version 2.12.1 (R Foundation for Statistical Computing, Vienna, Austria).20
From October 1, 1999, through January 31, 2008, there were 579 SLN biopsies for invasive breast cancer performed at our institution, of which 179 (30.9%) had at least 1 positive SLN. Of those, 123 patients met our inclusion criteria. Fifty-six patients were excluded: 26 did not undergo completion ALND and 30 were missing a required variable for the nomograms. The mean number of SLNs identified was 2.3 (range, 1-6), and the mean number of nodes removed during axillary dissection was 15 (range, 8-40). Most women who underwent axillary dissection had invasive ductal carcinoma (87.8%) and an SLN with macrometastatic disease (73.2%). At least 1 positive NSLN was found in 52 patients (42.2%), and ALND resulted in upstaging of 25 patients (20.3%). The descriptive tumor and nodal characteristics of our population as used in the MSKCC and Stanford nomograms are listed in the Table.
The ROC curves for the MSKCC and Stanford nomograms are shown in Figure 1. The AUC values were 0.72 and 0.70, respectively. There were no statistically significant differences between the curves given that the 95% CIs overlap. False-negative rates for nomogram values of 10% or less were low for each nomogram: 4.1% for the MSKCC (95% CI, 0%-9.2%) and 7.8% for the Stanford (95% CI, 1.7%-14.4%). The positive predictive value for nomogram probabilities of 80% or greater was higher for MSKCC than Stanford: 90.9% for the MSKCC (95% CI, 70.0%-100.0%) and 61.8% for the Stanford (95% CI, 45.7%-73.6%). On a calibration plot (Figure 2), predicted rates of NSLN metastasis were closer to observed rates for MSKCC than for Stanford.
We also tested the performance of each nomogram in a subset of low-risk patients with SLN metastasis size of 2.0 mm or less (ITC and MI). Thirty-three patients (26.8%) were identified from our original study group as having ITC or MI as the largest tumor deposit within the SLN. Of these patients, 11 (33.3%) had NSLN metastases (3 patients with ITC disease and 8 patients with MI disease). The ROC curve for this subanalysis is shown in Figure 3. The MSKCC nomogram produced an AUC of 0.75 and the Stanford nomogram produced an AUC of 0.70. The difference was not statistically significant. On calibration plot, the Stanford nomogram was closer to the ideal predictor, indicating better accuracy in this subgroup.
Adequately assessing nodal status is essential to accurate staging and treatment planning in early-stage breast cancer.1,3,8,21 Completion ALND, which has been the standard of care for SLN-positive patients, has the advantage of more accurate staging, focused treatment planning, and possibly a lower rate of recurrence.8 However, NSLNs are often negative for tumor, and the risk of morbidity associated with a completion ALND is not insignificant. Following a completion ALND, 10% to 15% of patients will develop permanent lymphedema in the adjacent arm.22,23 Additional complications include wound infection, seroma, neurologic damage, arm weakness, and decreased upper extremity range of motion.22 Recently, Giuliano et al9 reported results from the American College of Surgeons Oncology Group Z0011 study, a randomized, multicenter trial that showed that ALND did not significantly affect overall or disease-free survival at 5 years for SLN-positive patients with T1 or T2 breast cancer when treated with lumpectomy, adjuvant systemic therapy, and tangential-field whole-breast radiation. Thus, physicians must balance understaging with overtreating the clinically negative axilla.
Accordingly, several studies have attempted to identify tumor and nodal characteristics that can predict the presence of positive NSLNs. These studies have shown primary tumor size and SLN metastasis size to be most closely associated with NSLN involvement.7,11,24,25 Compared with other nomograms that have been developed to predict NSLN metastasis, the online availability of the MSKCC and Stanford nomograms makes them widely accessible and clinically applicable. For this reason, we focused our validation on these 2 nomograms.
The MSKCC nomogram, published in 2003 by Van Zee and colleagues,7 was the first nomogram of its kind to predict NSLN metastasis following a positive SLN biopsy. When applied prospectively to a group of 373 patients, the model produced an AUC of 0.77. Subsequent validation studies have shown an AUC ranging from 0.58 to 0.84.5,14,18,19,25,26 In 2008, a modified nomogram using only 3 histopathologic variables was developed at Stanford by Kohrt and colleagues.6 The nomogram was developed using a data set of 171 patients, in which most (96.7%) had an SLN with ITC or MI disease. The nomogram was validated against the MSKCC nomogram using an independent data set of 77 patients with predominantly macrometastases disease. The AUC was 0.74 and 0.62 for the Stanford and MSKCC nomograms, respectively. Subsequent validation studies of the Stanford nomogram have demonstrated an AUC of 0.64 to 0.73.5,25,26
In our population of 123 SLN-positive patients, the nomograms had similar AUCs (0.72 for the MSKCC and 0.70 for the Stanford) with no statistically significant difference. Previous studies have relied on the AUC alone to validate nomograms; however, we further assessed their performance using the false-negative rate, the positive predictive value, and calibration plots. The nomograms performed similarly when identifying patients at low risk for NSLN metastases, but the MSKCC nomogram demonstrated an increased ability to correctly predict patients at high risk for NSLN disease. This difference in performance was more clearly seen in the calibration plots (Figure 2), which showed that across all nomogram values, the MSKCC-predicted probabilities were closer to the observed rate of NSLN metastasis.
Identification of ITC and MI disease in the SLN has become increasingly common with the use of serial sectioning and immunohistochemistry on pathologic evaluation. The clinical significance and management of these findings remain unclear. The reported incidence of NSLN metastasis in patients with ITC ranges from 4.7% to 16%6,27,28 and in patients with MI disease from 12% to 42%.6,28 As mentioned previously, the Stanford nomogram was developed around a population of patients with mostly ITC and MI disease, and unlike the MSKCC nomogram, the size of the SLN metastasis factors into its calculations. When we tested the accuracy of the nomograms in a subset of patients with only ITC or MI disease, the Stanford nomogram closely matched the observed incidence of NSLN involvement (Figure 4).
In light of the American College of Surgeons Oncology Group Z0011 trial, some physicians have questioned the continued need for nomograms in clinical decision making. On the contrary, we believe that need for nomograms will increase in the future because there will be a growing cohort of SLN-positive early-stage breast cancer patients with undissected axillae. Among these patients, nomograms can be used to provide an estimated risk of residual axillary disease and, potentially, of axillary relapse. Such a cohort has already been the subject of a preliminary report from MSKCC published in 2004 by one of our coauthors (A.M.N.). In fact, this analysis8 suggested a higher axillary relapse rate among patients not undergoing ALND.
Furthermore, the findings from the American College of Surgeons Oncology Group Z0011 trial are only applicable to a limited number of breast cancer patients (patients with T1-T2 primary tumors and a positive SLN who undergo lumpectomy, adjuvant systemic therapy, and tangential-field whole-breast radiation). The trial did not address patients with early-stage disease who elect mastectomy, patients undergoing neoadjuvant chemotherapy who have clinically negative axillae but a positive SLN, or patients in whom axillary radiation is being considered. For these patients, nomograms are still applicable because axillary dissection remains the standard of care.
There were some limitations to our study. First, because a nomogram could not be applied unless all histopathologic variables were known, 30 patients were excluded from the study. Most commonly missing was SLN metastasis size, which is a variable required by the Stanford nomogram. Although the MSKCC nomogram requires more variables, this was not a limiting factor in our study population. Another limitation was our relatively small population size, particularly affecting the subanalysis of patients with ITC and MI disease. Furthermore, as the use of nomograms in our multidisciplinary breast cancer conference increased, some patients with very small nodal tumor deposits did not undergo ALND dissection and thus were not included in the study. In fact, use of nomograms has been shown to reduce the number of ALND.29 An interesting future study would be the correlation of nomogram probabilities with clinical outcomes in patients treated with SLN biopsy alone.
In conclusion, our study is unique in that it offers a more comprehensive validation of the MSKCC and Stanford nomograms, thereby elucidating the clinical utility of each nomogram. Despite similar AUC values between the nomograms, the MSKCC model more accurately identified patients at high risk for NSLN metastasis; therefore, it may be more useful when considering a completion ALND. Conversely, the Stanford nomogram more accurately predicted NSLN involvement in a subset of patients with ITC and MI disease and should be considered for use in these patients. However, when calibrated across all nomogram values, the MSKCC nomogram was consistently more reliable in predicting actual NSLN pathologic outcomes in our study population.
Correspondence: Crystal J. Hessman, MD, Department of Surgery, Oregon Health & Science University, 3181 SW Sam Jackson Park Rd, Mailstop Code L223, Portland, OR 97239 (hessmanc@ohsu.edu).
Accepted for Publication: May 3, 2011.
Author Contributions:Study concept and design: Hessman, Naik, Troxell, and Vetto. Acquisition of data: Hessman, Kearney, Jensen, Troxell, and Vetto. Analysis and interpretation of data: Hessman, Kearney, Jensen, Diggs, and Vetto. Drafting of the manuscript: Hessman and Vetto. Critical revision of the manuscript for important intellectual content: Hessman, Naik, Kearney, Jensen, Diggs, Troxell, and Vetto. Statistical analysis: Hessman and Diggs. Administrative, technical, and material support: Hessman and Kearney. Study supervision: Naik, Troxell, and Vetto.
Financial Disclosure: None reported.
Previous Presentation: This paper was presented at the 82nd Annual Meeting of the Pacific Coast Surgical Association; February 20, 2011; Scottsdale, Arizona, and is published after peer review and revision.
1.AlbertiniJJ, LymanGH, CoxC,
et al.Lymphatic mapping and sentinel node biopsy in the patient with breast cancer.Ѵ.1996;276(22):1818-18228946902
2.GiulianoAE, KirganDM, GuentherJM, MortonDL.Lymphatic mapping and sentinel lymphadenectomy for breast cancer.Ann Surg.1994;220(3):391-4018092905
3.PuglieseMS, KaramAK, HsuM,
et al.Predictors of completion axillary lymph node dissection in patients with immunohistochemical metastases to the sentinel lymph node in breast cancer.Ann Surg Oncol.2010;17(4):1063-106820033325
4.KragDN, WeaverDL, AlexJC, FairbankJT.Surgical resection and radiolocalization of the sentinel lymph node in breast cancer using a gamma probe.Surg Oncol.1993;2(6):335-3408130940
5.CoutantC, OlivierC, LambaudieE,
et al.Comparison of models to predict nonsentinel lymph node status in breast cancer patients with metastatic sentinel lymph nodes: a prospective multicenter study.J Clin Oncol.2009;27(17):2800-280819349546
6.KohrtHE, OlshenRA, BermasHR,
et al;Bay Area SLN Study.New models and online calculator for predicting non-sentinel lymph node status in sentinel lymph node positive breast cancer patients.BMC Cancer.2008;8:6618315887
7.Van ZeeKJ, ManassehDM, BevilacquaJL,
et al.A nomogram for predicting the likelihood of additional nodal metastases in breast cancer patients with a positive sentinel node biopsy.Ann Surg Oncol.2003;10(10):1140-115114654469
8.NaikAM, FeyJ, GemignaniM,
et al.The risk of axillary relapse after sentinel lymph node biopsy for breast cancer is comparable with that of axillary lymph node dissection: a follow-up study of 4008 procedures.Ann Surg.2004;240(3):462-47115319717
9.GiulianoAE, HuntKK, BallmanKV,
et al.Axillary dissection vs no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial.Ѵ.2011;305(6):569-57521304082
10.FisherB, JeongJH, AndersonS, BryantJ, FisherER, WolmarkN.Twenty-five-year follow-up of a randomized trial comparing radical mastectomy, total mastectomy, and total mastectomy followed by irradiation.N Engl J Med.2002;347(8):567-57512192016
11.BarrangerE, CoutantC, FlahaultA, DelpechY, DaraiE, UzanS.An axilla scoring system to predict non-sentinel lymph node status in breast cancer patients with sentinel lymph node involvement.Breast Cancer Res Treat.2005;91(2):113-11915868438
12.DegnimAC, ReynoldsC, PantvaidyaG,
et al.Nonsentinel node metastasis in breast cancer patients: assessment of an existing and a new predictive nomogram.Am J Surg.2005;190(4):543-55016164917
13.HwangRF, KrishnamurthyS, HuntKK,
et al.Clinicopathologic factors predicting involvement of nonsentinel axillary nodes in women with breast cancer.Ann Surg Oncol.2003;10(3):248-25412679309
14.PalA, ProvenzanoE, DuffySW, PinderSE, PurushothamAD.A model for predicting non-sentinel lymph node metastatic disease when the sentinel lymph node is positive.Br J Surg.2008;95(3):302-30917876750
15.SaidiRF, DudrickPS, RemineSG, MittalVK.Nonsentinel lymph node status after positive sentinel lymph node biopsy in early breast cancer.Am Surg.2004;70(2):101-10515011910
16.JensenAJ, NaikAM, PommierRF, VettoJT, TroxellML.Factors influencing accuracy of axillary sentinel lymph node frozen section for breast cancer.Am J Surg.2010;199(5):629-63520466107
17.Edge SB, Byrd DR, Compton CC, Fritz AG, Greene FL, Trotti A, eds. AJCC Cancer Staging Manual. 7th ed. New York, NY: Springer; 2009
18.AlranS, De RyckeY, FourchotteV,
et al;Institut Curie Breast Cancer Study Group.Validation and limitations of use of a breast cancer nomogram predicting the likelihood of non-sentinel node involvement after positive sentinel node biopsy.Ann Surg Oncol.2007;14(8):2195-220117294071
19.D’EreditàG, TroiloVL, GiardinaC,
et al.Sentinel lymph node micrometastasis and risk of non-sentinel lymph node metastasis: validation of two breast cancer nomograms.Clin Breast Cancer.2010;10(6):445-45121147687
20.Team RDCR.A Language and Environment for Statistical Computing.2.12.1 ed. Vienna, Austria: R Foundation for Statistical Computing; 2010
21.VeronesiU, PaganelliG, GalimbertiV,
et al.Sentinel-node biopsy to avoid axillary dissection in breast cancer with clinically negative lymph-nodes.Lancet.1997;349(9069):1864-18679217757
22.PurushothamAD, UpponiS, KlevesathMB,
et al.Morbidity after sentinel lymph node biopsy in primary breast cancer: results from a randomized controlled trial.J Clin Oncol.2005;23(19):4312-432115994144
23.SchrenkP, RiegerR, ShamiyehA, WayandW.Morbidity following sentinel lymph node biopsy versus axillary lymph node dissection for patients with breast carcinoma.Cancer.2000;88(3):608-61410649254
24.ChuKU, TurnerRR, HansenNM, BrennanMB, BilchikA, GiulianoAE.Do all patients with sentinel node metastasis from breast carcinoma need complete axillary node dissection?Ann Surg.1999;229(4):536-54110203087
25.MoghaddamY, FalzonM, FulfordL, WilliamsNR, KeshtgarMR.Comparison of three mathematical models for predicting the risk of additional axillary nodal metastases after positive sentinel lymph node biopsy in early breast cancer.Br J Surg.2010;97(11):1646-165220641049
26.GurAS, UnalB, OzbekU,
et al;Turkish Federation of Breast Disease Associations Protocol MF08-01 investigators.Validation of breast cancer nomograms for predicting the non-sentinel lymph node metastases after a positive sentinel lymph node biopsy in a multi-center study.Eur J Surg Oncol.2010;36(1):30-3519535217
27.CalhounKE, HansenNM, TurnerRR, GiulianoAE.Nonsentinel node metastases in breast cancer patients with isolated tumor cells in the sentinel node: implications for completion axillary node dissection.Am J Surg.2005;190(4):588-59116164927
28.CyrA, GillandersWE, AftRL, EberleinTJ, GaoF, MargenthalerJA.Micrometastatic disease and isolated tumor cells as a predictor for additional breast cancer axillary metastatic burden.Ann Surg Oncol.2010;17:(suppl 3)
303-31120853051
29.ParkJ, FeyJV, NaikAM, BorgenPI, Van ZeeKJ, CodyHSIII.A declining rate of completion axillary dissection in sentinel lymph node-positive breast cancer patients is associated with the use of a multivariate nomogram.Ann Surg.2007;245(3):462-46817435554