INTRODUCTION
The incidence of squamous cell carcinoma (SCC) has been increasing worldwide. Of the various treatment options, Mohs micrographic surgery (MMS) achieves the highest cure rates and lowest recurrence rates compared to any other therapy. Because the peripheral margins are oriented in a single plane of sectioning, MMS offers the possibility to examine 100% of the surgical margins. The Mohs surgeon personally controls the processing, orientation, and interpretation of the histological sections. As such, continued quality assurance is important to reduce errors.
Cutaneous SCC is a malignant neoplasm of epidermal keratinocytes that exists on a continuum between the mild atypia of early actinic keratosis (AK) and the disordered architecture, nuclear pleomorphism, and full thickness atypia that characterize squamous carcinoma in situ (SCCIS).1 At the University of Virginia Mohs Micrographic Surgery and Dermatologic Oncology Fellowship program, the fellow is exposed to interpretations from four separate fellowship-trained/board certified Mohs surgeons who see "borderline" histological patterns that could be interpreted differently as AK, SCCIS, or AK with focal SCCIS, etc. While there exist inter-observer differences, general concordance amongst Mohs surgeons in distinguishing AK from SCCIS is paramount when evaluating the need to take another Mohs stage.
There have been a limited number of retrospective studies comparing the concordance rates of Mohs surgeons to dermatopathologists in the interpretation of MMS cryosections.2-6 Most of the studies only evaluate presence or absence of tumors examined by specialists usually at a single institution. One recent study examined the histologic grade of invasive SCC amongst six raters.7 Despite the importance of quality assurance measures in maintaining consistent accurate results, there is a lack of validated criteria to assess degree of concordance amongst Mohs surgeons in the interpretation of in situ carcinoma.
Objective
The primary aim of this study was to assess interrater concordance amongst Mohs surgeons in distinguishing actinic keratosis from squamous cell carcinoma in situ when evaluating Mohs frozen sections and the histologic criteria(s) used in making the distinction. The secondary objective was to identify management decisions associated with the selected diagnosis.
Cutaneous SCC is a malignant neoplasm of epidermal keratinocytes that exists on a continuum between the mild atypia of early actinic keratosis (AK) and the disordered architecture, nuclear pleomorphism, and full thickness atypia that characterize squamous carcinoma in situ (SCCIS).1 At the University of Virginia Mohs Micrographic Surgery and Dermatologic Oncology Fellowship program, the fellow is exposed to interpretations from four separate fellowship-trained/board certified Mohs surgeons who see "borderline" histological patterns that could be interpreted differently as AK, SCCIS, or AK with focal SCCIS, etc. While there exist inter-observer differences, general concordance amongst Mohs surgeons in distinguishing AK from SCCIS is paramount when evaluating the need to take another Mohs stage.
There have been a limited number of retrospective studies comparing the concordance rates of Mohs surgeons to dermatopathologists in the interpretation of MMS cryosections.2-6 Most of the studies only evaluate presence or absence of tumors examined by specialists usually at a single institution. One recent study examined the histologic grade of invasive SCC amongst six raters.7 Despite the importance of quality assurance measures in maintaining consistent accurate results, there is a lack of validated criteria to assess degree of concordance amongst Mohs surgeons in the interpretation of in situ carcinoma.
Objective
The primary aim of this study was to assess interrater concordance amongst Mohs surgeons in distinguishing actinic keratosis from squamous cell carcinoma in situ when evaluating Mohs frozen sections and the histologic criteria(s) used in making the distinction. The secondary objective was to identify management decisions associated with the selected diagnosis.
MATERIALS AND METHODS
Study Design
The study was approved by the University of Virginia Institutional Review Board, which waived written consent because all data were deidentified.
Twenty-five hematoxylin and eosin-stained slides were obtained from cases performed by Mohs surgeons at the University of Virginia, Department of Dermatology from September to November 2021. Locations included the ear, nose, eyelid conjunctiva, cheek, forehead, scalp, and pretibial. The sectioned slides were photographed with Olympus BX43F microscope, converted to jpeg image files, edited accordingly, and organized into a list of 17 final cases. Eight cases were excluded due to inadequate exposure, poor quality, or insufficient magnification. The 17 digital cases and corresponding questionnaire were compiled into an electronic survey (Google Forms) and distributed to the members of the American College of Mohs Surgery (ACMS).
Statistical Analysis
When evaluating interrater concordance, the original diagnosis was considered the standard for purposes of comparative analysis. Percent agreement was calculated as the number of agreement scores divided by the total number of scores. The kappa statistic was used to test interrater reliability as it represents the extent to which the data collected are correct representations of the proportion of agreement beyond that expected by chance. Fleiss κ specifically allows calculation of κ for 3 or more raters with ranges from -1 to 1, with 0 depicting agreement expected by random chance, 1 with perfec agreement, and -1 with perfect disagreement.8 The McHugh κ interpretation was primarily used to analyze the results in the discussion section.8 Minitab version 20 was the statistical software used in this study.
The study was approved by the University of Virginia Institutional Review Board, which waived written consent because all data were deidentified.
Twenty-five hematoxylin and eosin-stained slides were obtained from cases performed by Mohs surgeons at the University of Virginia, Department of Dermatology from September to November 2021. Locations included the ear, nose, eyelid conjunctiva, cheek, forehead, scalp, and pretibial. The sectioned slides were photographed with Olympus BX43F microscope, converted to jpeg image files, edited accordingly, and organized into a list of 17 final cases. Eight cases were excluded due to inadequate exposure, poor quality, or insufficient magnification. The 17 digital cases and corresponding questionnaire were compiled into an electronic survey (Google Forms) and distributed to the members of the American College of Mohs Surgery (ACMS).
Statistical Analysis
When evaluating interrater concordance, the original diagnosis was considered the standard for purposes of comparative analysis. Percent agreement was calculated as the number of agreement scores divided by the total number of scores. The kappa statistic was used to test interrater reliability as it represents the extent to which the data collected are correct representations of the proportion of agreement beyond that expected by chance. Fleiss κ specifically allows calculation of κ for 3 or more raters with ranges from -1 to 1, with 0 depicting agreement expected by random chance, 1 with perfec agreement, and -1 with perfect disagreement.8 The McHugh κ interpretation was primarily used to analyze the results in the discussion section.8 Minitab version 20 was the statistical software used in this study.
RESULTS
The survey was distributed by the ACMS over a period of two months, and a total of 67 complete responses were obtained. The respondents consisted of 55 Mohs surgeons, 6 dually trained Mohs surgeons and dermatopathologists, and 6 fellows-in-training. Of the 17 cases, nine cases were originally diagnosed as SCCIS, six cases as AK, one case was noted as SCCIS with superficially invasive dermal component, and one case was AK with follicular extension. Out of 17 cases, the average number of correct responses was 9.2 (95% CI 8.65-9.75), with a range of 2-15.
Within the SCCIS category, 61.9% of raters agreed with the diagnosis of SCCIS, followed by 16.9% who selected AK with focal SCCIS, 13.8% who selected AK, and 7.5% who selected others (Table 1). The most common diagnosis in the other subcategory included AK with follicular/adnexal involvement, Bowen's disease, superficially invasive SCC, and tangential sectioning.
Within the AK category, 56.7% or raters agreed with the diagnosis, followed by 23.1% who selected SCCIS, 17.7% who selected AK with focal SCCIS, and 2.3% who selected others (Table 1). The most common diagnosis in the other subcategory included Bowenoid AK, chronic sun damage, and seborrheic keratosis.
Within the SCCIS category, 61.9% of raters agreed with the diagnosis of SCCIS, followed by 16.9% who selected AK with focal SCCIS, 13.8% who selected AK, and 7.5% who selected others (Table 1). The most common diagnosis in the other subcategory included AK with follicular/adnexal involvement, Bowen's disease, superficially invasive SCC, and tangential sectioning.
Within the AK category, 56.7% or raters agreed with the diagnosis, followed by 23.1% who selected SCCIS, 17.7% who selected AK with focal SCCIS, and 2.3% who selected others (Table 1). The most common diagnosis in the other subcategory included Bowenoid AK, chronic sun damage, and seborrheic keratosis.
When assessing agreement between all the raters without consideration of the standard, the Fleiss Kappa was 0.23 for AK, 0.15 for AK with focal SCCIS, and 0.21 for SCCIS, with an overall κ of 0.19 (Table 2). When assessing agreement between all the raters vs the standard, the Fleiss κ was 0.34 for AK, 0.37 for SCCIS, with an overall κ of 0.26.
Histologic features most commonly considered in the diagnosis of SCCIS and AK are summarized in Table 3. After diagnosis was made, raters were asked to select the most appropriate management plan. For SCCIS and AK with focal SCCIS, most raters selected to take another Mohs stage/layer (79.9%) and (77.2%) respectively. For AK most raters selected not to take another Mohs stage/layer (67.9%).
DISCUSSION
According to the 2012 Mohs appropriate use criteria, Mohs surgery is considered appropriate in the treatment of SCCIS for most locations except area L (trunk and extremities).9 On the other hand Mohs surgery is considered inappropriate in the treatment of AK with focal SCCIS of all types. The selection process was based on evidence review and analysis of surgical and disease outcomes related to appropriateness of Mohs micrographic surgery in the treatment of cutaneous cancers at the time of initial biopsy diagnosis. It is unclear if similar criteria apply to subsequent Mohs frozen sections. While most Mohs surgeons would select to take another stage if SCCIS was present at the surgical margins, borderline histological patterns such as AK with focal SCCIS, Bowenoid AK, or AK with follicular extension are met with variable interpretation and management.
In this current study, raters were first tasked with determining the degree of atypia at the surgical margins and selecting one diagnosis out of four possible options: AK, AK with focal SCCIS, SCCIS, and others. Under the SCCIS category, at least 78.8% of raters recognized the presence of SCCIS, albeit 16.9% of raters ultimately selected AK with focal SCCIS. Compared to the standard, the overall concordance was slight to minimal for SCCIS (κ=0.37). Because none of the standards selected AK with focal SCCIS kappa was not calculated for this subgroup. Nevertheless, the management decided to take another Mohs stage/layer appears similar for both the SCCIS raters (79.9%) and AK with focal SCCIS raters (77.2%). Even though AK with focal SCCIS on initial biopsy appears inappropriate for Mohs surgery, residual SCCIS on Mohs histological sections, even focal in nature, would prompt the surgeon to take an additional layer to achieve negative margins in majority of responses.
Determining what constitutes SCCIS from borderline cases is difficult as no definitive histologic criteria are available.10,11 Our results suggest that certain features such as atypia that involves the full thickness of epidermis or that invades adnexal structures, abnormal maturation, and nuclear pleomorphism (ie, hyperchromatic, bizarre nuclei) help distinguish cancers that should be classified as SCCIS with malignant potential from AK (Figure 1). Atypia that extends into hair follicles is often resistant to field therapy and have a greater potential to progress to invasive SCC, if not surgically excised.12,13 In our study one case was read as AK with follicular extension, but only 46.3% recognized some form of extensive atypia beyond that associated with simple AK. Thus only 38.8% of the raters elected to take another Mohs stage.
When examining the AK category, a little over half of the raters agreed with the diagnosis (57.7%), while 40.8% observed presence of some component of SCCIS. Compared to the standard, the overall concordance was slight to minimal for AK (κ=0.34). One explanation could be that certain areas may contain tangential sectioning or freeze artifacts that may have led some to upstage an AK to SCCIS. One rater commented that additional sections
In this current study, raters were first tasked with determining the degree of atypia at the surgical margins and selecting one diagnosis out of four possible options: AK, AK with focal SCCIS, SCCIS, and others. Under the SCCIS category, at least 78.8% of raters recognized the presence of SCCIS, albeit 16.9% of raters ultimately selected AK with focal SCCIS. Compared to the standard, the overall concordance was slight to minimal for SCCIS (κ=0.37). Because none of the standards selected AK with focal SCCIS kappa was not calculated for this subgroup. Nevertheless, the management decided to take another Mohs stage/layer appears similar for both the SCCIS raters (79.9%) and AK with focal SCCIS raters (77.2%). Even though AK with focal SCCIS on initial biopsy appears inappropriate for Mohs surgery, residual SCCIS on Mohs histological sections, even focal in nature, would prompt the surgeon to take an additional layer to achieve negative margins in majority of responses.
Determining what constitutes SCCIS from borderline cases is difficult as no definitive histologic criteria are available.10,11 Our results suggest that certain features such as atypia that involves the full thickness of epidermis or that invades adnexal structures, abnormal maturation, and nuclear pleomorphism (ie, hyperchromatic, bizarre nuclei) help distinguish cancers that should be classified as SCCIS with malignant potential from AK (Figure 1). Atypia that extends into hair follicles is often resistant to field therapy and have a greater potential to progress to invasive SCC, if not surgically excised.12,13 In our study one case was read as AK with follicular extension, but only 46.3% recognized some form of extensive atypia beyond that associated with simple AK. Thus only 38.8% of the raters elected to take another Mohs stage.
When examining the AK category, a little over half of the raters agreed with the diagnosis (57.7%), while 40.8% observed presence of some component of SCCIS. Compared to the standard, the overall concordance was slight to minimal for AK (κ=0.34). One explanation could be that certain areas may contain tangential sectioning or freeze artifacts that may have led some to upstage an AK to SCCIS. One rater commented that additional sections
into the block would have been helpful to distinguish true in situ carcinoma from oblique sectioning. Overall, the raters who observed actinic keratosis were more likely to report features of atypia confined to the lower epidermis, confluent parakeratosis, and presence of solar elastosis (Figure 2).
In terms of management, 67.9% of raters selected not to take another Mohs stage, and instead selected field therapy (44.8%), curettage (23.9%), cryotherapy (20.9%), or clinical monitoring (19.4%).
Our study results suggest that there exists notable variability among Mohs surgeons who interpret the spectrum of in situ carcinoma differently. While some variables such as size and tumor presence are more easily measured, histological differentiation can be quite subjective. Ultimately high levels of interrater variability may adversely affect patient care due to "over" or "under" calling, leading to inappropriate additional surgery or false negative margins respectively. In situ carcinoma, if present at the surgical margin and not recognized and excised, may lead to cancer recurrence. Conversely, upstaging an actinic keratosis as in situ SCC may lead to unnecessary surgery that is not medically appropriate.
As a potential source of error, residency and fellowship programs are expected to implement training to reduce the amount of variability in how trainees view and interpret Mohs frozen slides and record appropriately on the Mohs map. In addition, Mohs surgeons in practice are expected to report the degree of agreement (interrater reliability) among their peers at least annually. Resources to improve consensus building can consist of implementation of a Mohs histopathological image bank, participation in the recently administered Mohs Dermatologic Surgery exam, and continued self-driven learning through regular CME modules.
LIMITATION
Both percent agreement and kappa have strengths and limitations. The percent statistic is easily calculated and directly interpretable, however, it does not take into account the possibility that raters guessed on scores. It thus may overestimate the true agreement among raters. The kappa was designed to incorporate the possibility of guessing, but the assumptions it makes about rater independence and other factors are not well supported, and thus it may lower the estimate of agreement.
The McHugh κ interpretation was primarily used in our study, which is a more stringent scale than that published by Landis and Koch.14 As such, the κ scores may be an underestimate of true agreement if there is overlap among categories. This was observed between raters who selected SCCIS and AK with focal SCCIS, as their subsequent management decision was nearly identical.
This study was also limited by the digitally formatted nature of the interpretation of Mohs frozen histological slides. Because raters were only given a fixed area of the actual histological slides to review, the inability to scan the entirety of the section may lead to an erroneous conclusion that an area is focal SCCIS in nature, when in fact the SCCIS extends beyond what is viewable.
The McHugh κ interpretation was primarily used in our study, which is a more stringent scale than that published by Landis and Koch.14 As such, the κ scores may be an underestimate of true agreement if there is overlap among categories. This was observed between raters who selected SCCIS and AK with focal SCCIS, as their subsequent management decision was nearly identical.
This study was also limited by the digitally formatted nature of the interpretation of Mohs frozen histological slides. Because raters were only given a fixed area of the actual histological slides to review, the inability to scan the entirety of the section may lead to an erroneous conclusion that an area is focal SCCIS in nature, when in fact the SCCIS extends beyond what is viewable.
CONCLUSION
In summary, the percent agreement of raters in the ability to
distinguish AK from SCCIS was moderate, while the kappa was
low to minimal. It may make sense to rely more on the percent
agreement over the kappa statistic if raters are well trained and
little guessing is likely to exist. Nevertheless, results collected
from this study showed that variability exists in interpretation
of in situ carcinoma on histological frozen sections. Ongoing
learning and consensus building among Mohs surgeons and
trainees can aid in quality patient care, even if there may not be
agreement on every case.
DISCLOSURES
None of the authors have any conflicts of interest to report.
REFERENCES
1. Ackerman AB, Mones JM. Solar (actinic) keratosis is squamous cell carcinoma. Br J Dermatol. 2006;155(1):9-22.
2. Grabski WJ, Salasche SJ, et al. Interpretation of Mohs micrographic frozen section: A peer review comparison study. J Am Acad Dermatol.. 1989;20(4):670-674.
3. Chia JC, Abi Daoud MS, Williamson TS, Kurwa HA. Mohs Micrographic Surgery Dermatopathology Concordance in Canada: A Single-Institution Experience. J Cutan Med Surg. 2019;23(1):20-28.
4. Semkova K, Mallipeddi R, Robson A, Palamaras I. Mohs micrographic surgery concordance between Mohs surgeons and dermatopathologists. Dermatol Surg. 2013;39(11):1648-1652.
5. Mariwalla K, Aasi SZ, Glusac EJ, Leffell DJ. Mohs micrographic surgery histopathology concordance. J Am Acad Dermatol. 2009;60(1):94-98.
6. Kesty K, Sangueza OP, Leshin B, Albertini JG. Mohs micrographic surgery and dermatopathology concordance; An analysis of 14221 Mohs cases over 17 years. J Am Acad Dermatol. 2017;13:S0190-9622(17):32814-1.
7. Prezzano JC, Scott GA, Lambert SF, et al. Concordance of squamous cell carcinoma histologic grading among dermatopathologists and Mohs surgeons. Dermatol Surg. 2021;47(11):1433-1437.
8. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22:276-282.
9. Connolly SM, Baker DR, Coldiron BM, et al. AAD/ACMS/ASDSA/ASMS 2012 appropriate use criteria for Mohs micrographic surgery: A report of the American Academy of Dermatology, American College of Mohs Surgery, American Society for Dermatologic Surgery Association, and the American Society for Mohs Surgery. J Am Acad Dermatol. 2012;67(4):531-550.
10. Jagdeo J, Weinstock MA, et al. Reliability of the histopathologic diagnosis of keratinocyte carcinomas. J Am Acad Dermatol. 2007;57(2):279-284.
11. Cockerell CJ. Histopathology of incipient intraepidermal squamous cell carcinoma ("actinic keratosis"). J Am Acad Dermatol. 2000;42(1 Pt 2):11-17.
12. Tanaka R, Tanese K, Zhu Y, et al. Follicular extension of atypical keratinocytes predicts the resistance of actinic keratosis to topical imiquimod treatment: A single-center retrospective analysis. J Dermatol. 2021;48(8):1262-1267.
13. Fernandez-Figueras MT, Saenz-Sarda X, et al. The depth of follicular extension in actinic keratosis correlates with the depth of invasion in squamous cell carcinoma: implication for clinical treatment. J Eur Acad Dermol Venerol. 2018;32(10):1657-1661.
14. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1997;33:159-174.
2. Grabski WJ, Salasche SJ, et al. Interpretation of Mohs micrographic frozen section: A peer review comparison study. J Am Acad Dermatol.. 1989;20(4):670-674.
3. Chia JC, Abi Daoud MS, Williamson TS, Kurwa HA. Mohs Micrographic Surgery Dermatopathology Concordance in Canada: A Single-Institution Experience. J Cutan Med Surg. 2019;23(1):20-28.
4. Semkova K, Mallipeddi R, Robson A, Palamaras I. Mohs micrographic surgery concordance between Mohs surgeons and dermatopathologists. Dermatol Surg. 2013;39(11):1648-1652.
5. Mariwalla K, Aasi SZ, Glusac EJ, Leffell DJ. Mohs micrographic surgery histopathology concordance. J Am Acad Dermatol. 2009;60(1):94-98.
6. Kesty K, Sangueza OP, Leshin B, Albertini JG. Mohs micrographic surgery and dermatopathology concordance; An analysis of 14221 Mohs cases over 17 years. J Am Acad Dermatol. 2017;13:S0190-9622(17):32814-1.
7. Prezzano JC, Scott GA, Lambert SF, et al. Concordance of squamous cell carcinoma histologic grading among dermatopathologists and Mohs surgeons. Dermatol Surg. 2021;47(11):1433-1437.
8. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22:276-282.
9. Connolly SM, Baker DR, Coldiron BM, et al. AAD/ACMS/ASDSA/ASMS 2012 appropriate use criteria for Mohs micrographic surgery: A report of the American Academy of Dermatology, American College of Mohs Surgery, American Society for Dermatologic Surgery Association, and the American Society for Mohs Surgery. J Am Acad Dermatol. 2012;67(4):531-550.
10. Jagdeo J, Weinstock MA, et al. Reliability of the histopathologic diagnosis of keratinocyte carcinomas. J Am Acad Dermatol. 2007;57(2):279-284.
11. Cockerell CJ. Histopathology of incipient intraepidermal squamous cell carcinoma ("actinic keratosis"). J Am Acad Dermatol. 2000;42(1 Pt 2):11-17.
12. Tanaka R, Tanese K, Zhu Y, et al. Follicular extension of atypical keratinocytes predicts the resistance of actinic keratosis to topical imiquimod treatment: A single-center retrospective analysis. J Dermatol. 2021;48(8):1262-1267.
13. Fernandez-Figueras MT, Saenz-Sarda X, et al. The depth of follicular extension in actinic keratosis correlates with the depth of invasion in squamous cell carcinoma: implication for clinical treatment. J Eur Acad Dermol Venerol. 2018;32(10):1657-1661.
14. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1997;33:159-174.
AUTHOR CORRESPONDENCE
Tian Hao Zhu MD
E-mail:hzhu678@gmail.com
