The future is now: pediatric neuropsychological presurgical epilepsy evaluation in the age of COVID‐19 – Nussbaum – 2021 – Epileptic Disorders
Objective. The objective of this brief report is to review an assessment paradigm for conducting virtual neuropsychological pre-surgical evaluations in the context of the COVID-19 pandemic.
Methods. A multidisciplinary epilepsy team at a Level 4 epilepsy center within a large children’s academic medical center convened to discuss the challenges and possible solutions for Phase II evaluations for pediatric patients with pharmacoresistant epilepsy during the COVID-19 pandemic. The neuropsychologists explored evidence-based methods of virtual evaluation and developed a systematic decision-making process for youth requiring a Phase II evaluation.
Results. We propose models of assessment which prioritize teleneuropsychology when possible to reduce the risk of infection: (1) evaluation with directly administered tests through a completely virtual format; (2) virtual/in-person hybrid evaluation; and (3) clinical observation/interview in a virtual format supplemented by survey data. These models are illustrated by three cases.
Significance. Using virtual assessment models, the team was able to meet the urgent patient care needs and collect useful data while minimizing the risk of virus spread. The paradigms presented may be useful examples for other multidisciplinary surgical teams interested in incorporating teleneuropsychology into their practices.
Neuropsychological evaluation is an essential component of the multidisciplinary presurgical assessment for epilepsy patients 1-4.The purpose of this investigation is to share proposed models for presurgical neuropsychological evaluations via virtual testing. We describe three case examples of youth living with intractable epilepsy who were amid a Phase II surgical work up at a Level 4 epilepsy center when social distancing policies were implemented due to the COVID-19 pandemic between April and June of 2020.
The neuropsychological evaluation serves to characterize functioning in the context of seizure activity to identify cognitive risks associated with proposed surgery and provide baseline data to compare postsurgical functional outcomes. At the outset of the COVID-19 outbreak, most “elective” surgeries were temporarily halted, raising concerns about the risks of postponing epilepsy surgery that could lessen seizure burden and improve quality of life. While a neurologist or nurse practitioner visit may take an hour or less, neuropsychological evaluations typically take three to six hours of in-person contact in a small enclosed space. The epilepsy team recognized the need to create a method to accurately gather critical neuropsychological data while adhering to emerging safety protocols. To minimize infection risk to patients, family members, and health care workers posed by in-person evaluations, the team prioritized virtual testing whenever possible and supplemented with in-person testing as necessary.
We reviewed existing literature on teleneuropsychology (TNP) from prior to the COVID-19 pandemic 5-7, as well as research and practical information from the International Neuropsychological Society 8 and National Academy of Neuropsychology 9. Dr. Lana Harder also led a question-and-answer session with our group based on her research and experience using TNP with pediatric medical populations. As first author of the only published pediatric TNP study 10, she shared her expertise gained through virtual evaluations carried out with 58 participants experiencing pediatric demyelinating disorders (see table 1 for information shared). In addition to Dr. Harder’s guidance on test selection and testing format, we reviewed materials suitable for virtual testing via several testing platforms (see tables 1, 2 for rationale and a description of measures used). The cases presented below illustrate the specifics of how the evaluation process was operationalized for virtual and in-person settings.
Guidance shared by Dr. Harder
( Age 6 as the lower limit of successful completion of virtual assessment with a clinical population
( Having an adult nearby for technical support
( Suggestions for test selection (see table 2)
Considerations for virtual assessment measures
Did not require manipulatives (e.g., Wechsler Block Design, D-KEFS Tower), specialized equipment (e.g., Grooved Pegboard test), specific subject response booklets (e.g., Wechsler Processing Speed subtests, WJ-IV Calculations, Beery VMI-6 and Motor Coordination, D-KEFS Trails), or millisecond-level timing (Conners’ CPT)
Ability to present virtually (see table 2; some measures we typically use were not available for online administration [e.g., D-KEFS, NEPSY-II])
Finding appropriate substitutions conducive to virtual assessment (e.g., NEPSY-II Fingertip Tapping instead of Grooved Pegboard)
Relying more heavily on behavioral observations and parent report in lieu of formal attention testing (e.g., Conners CPT, NEPSY Auditory Attention / Response Set)
Notable virtual session challenges
More effortful to engage attention and establish rapport
Less sensitive to observation of body language/eye contact
Inability to directly manage behavioral difficulties
Less control over testing environment to minimize distractions
Parent report of difficulty focusing on the feedback session because their children were distracting
Minor frustration initiating teleneuropsychology connections
Screen sharing had to be adjusted at times to ensure the full visual stimulus was visible
Transitions between measures were not as smooth as in person (e.g., stop sharing, open new window, reshare, adjust as needed)
One patient chose to use their cell phone as a hotspot because of unstable wifi which could lead to additional cellular data cost
D-KEFS: Delis-Kaplan Executive Function System; WJ-IV: Woodcock-Johnson IV Tests of Achievement; Beery VMI-6: Beery Visual Motor Integration, Sixth Edition; Conners’ CPT: Conners Continuous Performance Test; NEPSY-II: A Developmental NEuroPSYchological Assessment, Second Edition.
Measures used in virtual administration.
WJ-IV Letter Word Identification
WJ-IV Passage Comprehension
WJ-IV Applied Problems
KTEA-3 Brief: Letter & Word Recognition
KTEA-3 Brief: Reading Comprehension
KTEA-3 Brief: Math Concepts and Applications
WISC-V/WAIS-IV Digit Span Forwards, Backwards, Sequencing
D-KEFS Verbal Fluency, Category Switching
Lateral Dominance Test
NEPSY-II Fingertip Tapping
WISC-V/WAIS-IV Matrix Reasoning
WISC-V/WAIS-IV Visual Puzzles
Beery-Buktenica Visual Perception
JP, CG, KS
D-KEFS Verbal Fluency
TOMAL-2 Memory for Stories
WISC-V/WAIS-IV Digit Span Forward
Rey-Osterrieth Complex Figure Test
TOMAL-2 Facial Memory
ABAS-3: Adaptive Behavior Assessment System, Third Edition; BASC-3: Behavior Assessment System for Children, Third Edition; BRIEF-2: Behavior Rating Inventory of Executive Function, Second Edition; CVLT-C/CVLT-III: California Verbal Learning Test Children’s Version/Third Edition; D-KEFS: Delis-Kaplan Executive Function System; EVT-3: Expressive Vocabulary Test, Third Edition; KTEA-3: Kaufman Test of Educational Achievement, Third Edition; NEPSY-II: A Developmental NEuroPSYchological Assessment, Second Edition; TOMAL-2: Test of Memory and Learning, Second Edition; WAIS-IV: Wechsler Adult Intelligence Scale, Fourth Edition; WISC-V: Wechsler Intelligence Scale for Children, Fifth Edition; WJ-IV: Woodcock-Johnson IV Tests of Achievement; WPS: Western Psychological Services, Administration Procedures: SPS: Stimulus Presented on Screen; SRA: Stimulus Read Aloud; RRP: Responses Recorded on Protocol; RSS: Responses Screenshotted for Scoring; ARO: Administered Remotely Online.
In order to minimize in-person testing, we developed a process to estimate the viability of virtual testing and determine the most appropriate model for evaluation. In consultation with Dr. Harder, we determined that patients should be developmentally and behaviorally functioning at least at the seven-year-old level in order to collect valid data through virtually administered performance-based measures. Her guidance was based on experience, successfully completing virtual assessment with clinical research participants as young as six. At this time, the rationale for our seven-year-old cut-off level is primarily based on practical and clinical considerations rather than strictly empirical evidence given the paucity of specific research in this area. While a number of measures with norms starting at age six are available, such as the WISC-V, and Dr. Harder reported successfully completing virtual evaluations down to age six, we chose to take a cautious approach and selected the seven-year-old level as the lower age and developmental limit. The neuropsychologist reviewed records and had a virtual intake consultation with the parent/guardian and patient to estimate if the functional criterion was met. Following this screening process, 15 virtual evaluations were attempted with one instance of non-compliance requiring discontinuation of the virtual assessment. Useful clinical data were gathered successfully for the remainder of patients using a variety of neuropsychological measures with examples given in table 2. Additional decision factors are illustrated in figure 1.
Decision algorithm for determining teleneuropsychology assessment type.
For patients who met criteria for a TNP assessment, the team ensured the patient had access to the minimum technology necessary to conduct the assessment, including secure, reliable high-speed internet access, and a device with a screen with a diagonal measurement of at least 9.75” and video conferencing capabilities (see https://iopc.online/teleneuropsychology for comprehensive guidelines). We used the HIPAA compliant Zoom video conferencing service to conduct evaluations. Before each virtual visit, clinic staff helped families download and practice using Zoom. For the testing session, we requested that patients be situated in a distraction-free room with the necessary technology in place. Parents were instructed to remain nearby and available by phone throughout the testing session for assistance with technical support and seizure safety.
All TNP was conducted with the caveat that patients may have to come to the office for in-person testing if information gathered via TNP was insufficient for any reason. Families also completed a short interview with the neuropsychologist after the case concluded in order to gather feedback on their experience with TNP.
Three cases are presented to showcase the utility and flexibility of TNP evaluation for pre-surgical planning during the COVID-19 pandemic. The cases represent three assessment models: virtual evaluation with tests administered to the patient; virtual/in-person hybrid; and virtual evaluation through clinical observation/interview and questionnaire data. No issues with device use or connectivity were reported in the cases presented below.
A description of testing sessions and assessment tools used in these cases are presented in tables 1, 2 and 3. The measures selected for TNP assessments were all well-validated tools that we commonly use in our in-person practice. Tools were selected based on the guidelines set forth by the Inter Organizational Practice Committee 11, as well as for their ease of adaptability to TNP, and the availability of virtual stimulus materials from the test publishers. The only measure we found problematic was the ‘Repetitions’ subtest of NEPSY-II Finger Tapping because of difficulty visually counting the patients’ responses. Subsequently, we only included the ‘Sequences’ subtest, which we found we could reliably administer and score.
Description of testing sessions.
Testing session description
Virtual session: 1.25-hour morning session; 40 minute lunch break; 2.25 hour afternoon session. Brief breaks for casual conversation throughout morning and afternoon sessions.
Day 1 virtual session: 2.5 hours with intermittent brief breaks for casual conversation with no formal breaks needed, though they were offered. Concluded when patient fatigue became evident.
Day 2 virtual session: 35 minutes.
Day 3 in person session: 2.5 hours with intermittent brief breaks for casual conversation with no formal breaks needed, though they were offered.
Virtual session: Approximately 1-hour session for parent interview and patient observation; interaction facilitated by child’s mother.
Model 1: Virtual evaluation with test administration
JP is a 10-year-old right-handed female with focal epilepsy. She participated well and was able to complete all virtual testing independently after her mother assisted in setting up the session. Cognitive functioning ranged from average to low-average range. Neuropsychological findings were indicative of anterior network dysfunction with subtle lateralization to the right anterior region; relatively concordant with EEG data showing extremely frequent right temporo-parieto-central spikes.
JP and her mother expressed general satisfaction with the process. The mother was appreciative of the safety of the virtual appointment and the financial and time savings, since she did not have to travel over an hour and was able to work from home that day.
Model 2: Virtual/in-person hybrid evaluation
CG is a 17-year-old, right-handed female with focal intractable epilepsy. She is bilingual (Spanish and English) with English as her dominant language. While she participated well in virtual testing, the need for additional in-person testing was discussed during peer supervision and was ultimately recommended given the virtual data, her history of bilingual language development, atypical language representation identified on MEG, and right-hemisphere seizure focus. Evaluation in-person allowed for a more comprehensive assessment using graphomotor, attention, visual memory and processing speed measures not available at that time in a virtual format. Also, in-person testing was largely concordant with scores obtained via virtual testing, suggesting virtual testing produced a valid assessment of her current functioning. Results of both virtual and in-person testing were then interpreted together, indicating anterior system involvement with inconclusive lateralization of dysfunction.
CG reported that she was mostly satisfied with the TNP evaluation, though she felt the experience was better in-person. She reported it was easier to pay attention and connect with the examiner in-person. Alternatively, her mother reported that she preferred TNP testing to in-person as it lowered the financial burden of the visit (the family lives 3.5 hours away by car and had to miss a day of work for the appointment).
Model 3: Clinical observation/interview & parent surveys
KS is a 26-month-old female with a history of intractable epilepsy with seizure onset at age five months. To obtain a baseline for KS’s developmental level, KS’s mother completed developmental and behavioral rating forms and KS was observed via a virtual neurobehavioral status exam. KS’s mother endorsed delays across developmental areas on rating scales. In the neurobehavioral exam, we observed examples of KS’s gross and fine motor development and communication, which were largely consistent with the parent report. The data were sufficient to understand KS’s developmental functioning and to establish a baseline with which to follow her development.
KS’s mother expressed satisfaction with the TNP session. Although she found that her attention was challenged at times due to managing KS and her siblings, she felt the necessary information was communicated. She identified the main advantages of TNP as limiting exposure to illness, less travel and flexibility/convenience (particularly given the unpredictability of KS’s seizures).
For many years, neuropsychologists have grappled with the question of remote evaluation. The prospect has historically been met with healthy skepticism due to issues including technical concerns, patient rapport, and validity and reliability of non-standard test administration. Moreover, reimbursement for this type of evaluation was limited or non-existent, which has disincentivized the incorporation of virtual assessment practices. Others saw the potential to extend services to those who found it difficult or impossible to attend in-person evaluation due to various challenges including transportation, physical health, and family or personal circumstances. In the current time of COVID-19, concerns about infection risk have created an imperative for neuropsychologists and multidisciplinary teams to seriously consider TNP as a safe alternative or supplement to in-person evaluation.
Prior to the current global health crisis, research on TNP was sporadic, and the current body of evidence remains sparse. Extant research on TNP indicates scores obtained via virtual assessment in adult populations are consistent with in-person administration 6, 7, 12. To the authors’ knowledge, however, there is only one publication on the efficacy of virtual assessment in the pediatric neuropsychology arena 10.
Amid the COVID-19 pandemic, we must balance the risk of infection with the likelihood of obtaining data that will significantly contribute to surgical decision making for an individual patient. Our primary decision factors include the potential to lateralize or localize dysfunction, the likelihood of obtaining data concordant with seizure foci, and the ability to determine functional adequacy (figure 1), as these considerations are at the heart of neuropsychological utility for assessing risk to postoperative functioning. Patients with strong evidence of global impairment or other challenges to obtaining the above information were provided with alternative evaluations to establish a baseline of cognitive functioning and ensure continued care.
Moving forward, we must weigh the advantages of remote evaluation against the limitations of this methodology. Compared to in-person testing, TNP is safer, easier to schedule, does not require transportation, and can be less anxiety-provoking for some patients (e.g., those who are more comfortable testing in a familiar environment or wish to avoid in-person contact). However, limitations must also be acknowledged. Videoconference administration is not yet standardized and there is limited evidence for how the format affects test scores and interpretation, especially in a pediatric population. Additional challenges we encountered are described in table 1.
In addition to validity considerations, large socioeconomic disparities exist regarding access to and familiarity with necessary technologies, which raises concerns around equitable care. We have done our best to have direct conversations with patients and families about the benefits, limitations and equity concerns related to TNP and ensure that families are able to choose the options with which they feel most comfortable while maintaining the highest standards of patient care.
The future was thrust upon us by a global pandemic, and clearly the airplane we have been building while flying at 30,000 feet is still under construction. The cases we have presented illustrate an early attempt to gather useful information for pre-surgical planning in a safe, flexible, sensitive and patient-centered manner, while recognizing we have much more work to accomplish in TNP.
True or False: A typical neuropsychological evaluation takes about as long as a routine medical visit.
True or False: Teleneuropsychological evaluations were in use before COVID-19.
True or False: Teleneuropsychological evaluations cannot be used with special populations such as individuals with cognitive challenges, bilingual populations or very young children.
Note: Reading the manuscript provides an answer to all questions. Correct answers may be accessed on the website, www.epilepticdisorders.com, under the section “The EpiCentre”.
Summary didactic slides are available on the www.epilepticdisorders.com website.
Acknowledgements and disclosures
We would like to acknowledge the contributions of the multidisciplinary epilepsy team members at Dell Children Medical Center to the development of this model of care. Also, we would like to thank Lana Harder, PhD, ABPP, Associate Professor of Psychiatry and Neurology and Neurotherapeutics at the University of Texas Southwestern, for her invaluable input on the efficacious use of teleneuropsychology with a pediatric population.
None of the authors have any conflit of interest to declare.
1Wilson S., Baxendale S., Barr W., Hamed S., Lanfitt J., Samson S. et al.: Indications and expectations for neuropsychological assessment in routine epilepsy care: Report of the ILAE Neuropsychology Task Force, Diagnostic Methods Commission, 2013-2017. Epilepsia, 56 (5) 2015, 674– 681.
2Vogt V.L., Äikiä M., del Barrio A., Boon P., Borbély C., Bran E. et al.: Current standards of neuropsychological assessment in epilepsy surgery centers across Europe. Epilepsia, 58 (3) 2017, 343– 355.
3Morrison C.E., MacAllister W.S., Barr W.B.: Neuropsychology within a tertiary care epilepsy center. Arch Clin Neuropsychol, 33 (3) 2018, 354– 364.
4Baxendale S., Wilson S.J., Baker G.A., Barr W., Helmstaedter C., Hermann B.P. et al.: Indications and expectations for neuropsychological assessment in epilepsy surgery in children and adults. Epileptic Disord, 21 (3) 2019, 221– 234.
5Cullum M.C., Hynan L.S., Grosch M., Parikh M., Weiner M.F.: Teleneuropsychology: evidence for video teleconference-based neuropsychological assessment. J Int Neuropsychol Soc, 20 (10) 2014, 1028– 1033.
6Muehlhausen W., Doll H., Quadri N., Fordham B., O’Donohoe P., Dogar N. et al.: Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health Qual Life Outcomes, 13 (1) 2015, 167.
7Brearly T.W., Shura R.D., Martindale S.L., Lazowski R.A., Luxton D.D., Shenal B.V. et al.: Neuropsychological test administration by videoconference: A systematic review and meta-analysis. Neuropsychol Rev, 27 (2) 2017, 174– 186.
8Stolwyk R, Hammers DB, Harder L, Cullum CM. Teleneuropsychology (telenp) in response to covid-19: Practical guidelines to balancing validity concerns with clinical need. 2020. https://www.the-ins.org/webinars/.
9 National Academy of Neuropsychology. Telehealth Resources. Telehealth Resources, 2020. https://nanonline.org/nan/Professional_Resources/Telehealth_Resources/NAN/_ProfessionalResources/Telehealth_Resources.aspx?hkey=7414d46d-46f2-4cd2-a4a6-29096996f802.
10Harder L., Hernandez A., Hague C., Neumann J., McCreary M., Cullum C.M., Greenberg B.: Home-based pediatric teleneuropsychology: A validation study. Arch Clin Neuropsychol, 35 (8) 2020, 1266– 1275.
11Bilder R.M., Postal K.S., Barisa M., Aase D.M., Cullum C.M., Gillaspy S.R. et al.: InterOrganizational practice committee recommendations/guidance for teleneuropsychology (TeleNP) in response to the COVID-19 pandemic. Clin Neuropsychol, 0 (0) 2020, 1– 21.
12Ball C.J., Scott N., McLaren P.M., Watson J.P.: Preliminary evaluation of a low-cost videoconferencing (LCVC) system for remote cognitive testing of adult psychiatric patients. Br J Clin Psychol, 32 (3) 1993, 303– 307.