Verbal Fluency Tests: Normative Data Stratified by Age and Education in an Istanbul Sample

Yasemin Sohtorik İlkmen1, Ezgi Soncu Büyükişcan2

1Bogazici University, Department of Psychology, Istanbul, Turkey
2Yeditepe University, Department of Psychology, Istanbul, Turkey

Keywords: Executive functions, verbal fluency, phonemic fluency, semantic fluency, normative data

Abstract

Objective: This study aimed to provide updated normative data for one of the widely used neuropsychological assessment tools, namely verbal fluency tests, and examine their reliability and validity.

Materials and Methods: To establish normative data stratified by gender, age and education, a total of 1,431 healthy participants were administered phonemic and semantic verbal fluency tests. Concurrent validity, internal consistency and test-retest reliability analyses were conducted. The effect of gender, age and education was tested through stepwise linear regression analysis.

Results: The results revealed good internal consistency and acceptable level of test-retest reliability for verbal fluency tests. While gender did not have any effect on test performance, the combined contribution of age and education on phonemic and semantic fluency performance were 37% and 25%, respectively. Verbal fluency performance decreased as a function of age, whereas it increased as a function of education.

Conclusion: The normative data for phonemic and semantic fluency performance between the ages 18 and 89 stratified by four education levels was presented.

Introduction

Tests that measure phonemic (lexical) and semantic (categorical) fluency skills expressed with the concept of verbal fluency are frequently used tools in clinical neuropsychology evaluations. These neuropsychological tests are indicated in neurological disorders such as Alzheimer’s type dementia and Parkinson’s disease (PD), as well as psychiatric disorders such as schizophrenia and depression (1,2,3,4,5,6). For this reason, they have an important place in clinical evaluation, differential diagnosis and research. In addition, they are preferred because they are easy to apply, take a short time, and are based on verbal language skills rather than writing and reading (7,8).

Tools used to measure phonemic fluency require words that begin with a letter of the alphabet to be spoken within the given time. In English-speaking countries, these letters are F, A, and S, and in our country, they are K, A, and S (7,9,10). The letter sets used in the verbal fluency tests are chosen according to the frequency of the words starting with these letters in that language. Namely, a path from easy to difficult is followed, with more words starting with the first letter in the set, fewer words starting with the second letter, and fewer words starting with the third letter than the other two (7). Since there were not many words starting with the letter F in Turkish, the letter K was chosen instead of F in the Turkish adaptation study, and the letters A and S were left as they were (10). While measuring semantic fluency, all words that come to mind from a category are asked to be said within the given time limits. The most used categories are animals and the things that can be bought in a supermarket (11).

The verbal fluency tests basically evaluate executive functions and verbal skills (1,12). Semantic fluency is more affected by verbal skills (12). However, in a more recent study, it was found that both verbal fluency tests were mainly based on language skills rather than executive functions, and especially on language processing (13). In a longitudinal study with twins, factor analysis was performed using a total of 6 verbal fluency tests (F, A, S for phonemic fluency; categories of animal names, male names, and consecutive categories of fruit/furniture names for semantic fluency) (14). As a result, a general fluency factor, and a factor explaining only semantic fluency were found. Those researchers suggested that this general factor could be explained by vocabulary from verbal skills, and by updating and inhibiting from executive functions. In addition, it has been argued that the factor representing semantic fluency can be explained by the access speed of episodic memory or lexicon [memory store that includes all information about words (e.g., phonological representations of words, spelling and articulation, grammatical structures and meanings)] (15). The finding that verbal fluency tests evaluate different cognitive skills in addition to the common skills they measure gains importance in clinical diagnosis and research.

Both neuropsychological studies and neuroimaging studies indicate that phonemic and semantic fluency tests are partially related to different brain regions and show susceptibility to different brain injuries. In general, verbal fluency tests are among the functions of both left frontal and temporal brain regions, but it has been found that phonemic fluency is predominantly sensitive to frontal region functions and that semantic fluency is predominantly sensitive to temporal region functions (1,16,17). In a more recent study, it was found that the basal ganglia were activated in both verbal fluency performances, while the superior temporal lobe was more dominantly activated during phonemic fluency, while the medial temporal regions were activated during semantic fluency (18). When clinical studies are examined, it has been determined that lesions in the frontal brain regions impair phonetic fluency performance, while damages in the left temporal region affect semantic fluency negatively (19).

When neurological diseases are examined, it has been revealed that semantic fluency is impaired prior and more, especially in dementia syndromes associated with Alzheimer’s disease (AD) (6). Disturbances in semantic fluency can be distinctive even in mild cognitive impairment seen in the early stages of AD, therefore it is important in early diagnosis (20). It has also been argued that the differences between phonemic and semantic fluency performances are helpful in diagnosing patients with AD who have not yet shown clinical symptoms (8). In a longitudinal study, semantic fluency performance was found to be effective in determining PD that would develop 3.5 years later, but the same result was not found for phonemic fluency (3). On the other hand, it was determined that phonemic fluency performance was impaired in patients with Huntington’s disease, even without clinical symptoms (21). In another study, semantic fluency was able to distinguish between a group with a very high risk of psychosis and a non-psychotic group (22). It has been shown that perseverations (repeated words by the participant) in verbal fluency tests can also help in diagnosis, and that perseverations seen in individuals without clinical diagnosis in the semantic fluency test (animal category) determine the cognitive impairments that will occur later (23). These findings support the necessity of calculating separate scores and determining norm values for both phonemic and semantic fluency test performances.

Öktem translated the verbal fluency tests, which were widely used in our country, into Turkish, and Tumaç (10) conducted the study that created the first norm values. These norms were obtained from 180 healthy individuals participating in the study and were stratified according to three age strata (15-28, 32-45 and 50-75) and three education levels (low education group: Literate, primary school graduate and secondary school dropout; secondary education group: Secondary school, high school and associate degree graduates; higher education group: Undergraduate and postgraduate education).

This study aimed to determine the verbal fluency performance of healthy individuals between the ages of 18-89 living in Istanbul with different education levels, based on the necessity of renewing the norm values over time and determining them according to narrower age and education levels over a larger sample. In both international and national studies, it was found that age and education significantly affected both phonemic and semantic fluency performance (10,24,25). In order to interpret the different performance levels between age and education groups in a meaningful way, it is necessary to compare the test results with the norms obtained from healthy individuals. Evaluation of neuropsychological impairement can be made on the basis of both norms and the individual’s previous performance (7). It is known that performance in executive functions decreases with aging (26). In order to distinguish the neurological effects of normal aging from abnormal skill loss, norm values obtained from appropriate age groups are needed (7). Based on this, it was aimed to stratify the performances of participants from different age groups according to their education level. This study also aimed to determine the test-retest reliability and internal consistency of verbal fluency tests.

Material and Methods

Sample

Yeditepe University Ethics Committee approval was obtained before starting the study (number: 75078252-050.01-0494). All volunteers gave signed consent to participate in the study. The data presented here are part of an extensive neuropsychological testing norm development study. The research data were obtained from a total of 1,431 healthy individuals between the years 2016- 2019 using the convenience sampling method. Participants were reached through acquaintances and their referrals. The mean age of the sample was 48.48 years [standard deviation (SD): 19.81], and the mean education period was 11.92 years (SD: 4.81). Four participants had no formal education but were literate; other participants received at least 1 year of formal training. The female/ male ratio was 704/727.

In order to evaluate the test-retest reliability of verbal fluency tests, a group of 61 volunteers was retested a minimum of 3 and a maximum of 5 months after the initial assessment. The mean age of this group, which consisted of 30 women and 31 men, was 42.61 years (SD: 16.79), and the mean education period was 11.15 years (SD: 4.66). The mean mini mental state exam (MMSE) score of this group was 27.93 (SD: 1.42).

Inclusion and Exclusion Criteria

Those with a neurological and/or psychiatric diagnosis, those who used drugs that might affect the speed of thinking (drugs from antipsychotic, benzodiazepine and barbiturate groups), and those who used alcohol and/or drugs were not included in the study. In addition, those who scored below 24 on the MMSE [MMSE (27) for the educated and MMSE (28) for the uneducated], those who scored above 16 points on the Beck depression inventory (BDI) (29), and those who scored above 13 points on the geriatric depression scale (GDS) (30) were not included in the study. The cut-off value of 24 for the MMSE is the mild dementia threshold that researchers recommend for individuals in the Turkish population (27). The cut-off values for both depression scales were determined based on the criteria specified in the Turkish adaptation studies. Accordingly, it was stated that individuals with a score of 16 or more on the BDI might have a moderate or higher level of depression, and those with a score above 13 on the GDS might have a definitive diagnosis of depression (29,30). For this reason, the data of the participants who scored above these values were not included in the research. Participation was on a voluntary basis and no awards were given to the participants.

Scales Used

Verbal Fluency Tests: The first version of verbal fluency tests was developed by Thurstone (31). This test, which was defined as “controlled association”, was used to produce words that were similar in meaning to the given word (31). The most common form of the test, of which different versions have been created over the years, includes saying words and producing animal names using the letters F-A-S in English (32). The Turkish norms of the tests, the Turkish version of which was created by Öktem for the first time, were used in the study conducted by Tumaç (10).

The adaptation of verbal fluency tests used in this study consists of two parts, phonemic fluency and semantic fluency. In phonemic fluency, participants are asked to produce words starting with the letters K, A and S, respectively, and the words spoken for 60 seconds for each letter are recorded by the tester. The time is kept with a stopwatch. In this task, the participants are instructed not to say words including human name, place name, and words produced with number and inflectional suffix. The total score is the number of correct words that fit the criteria. Error and perseveration numbers are also calculated. In semantic fluency, the person is asked to say all the animal names that come to mind within 60 seconds. In semantic fluency tests, the total number of correct words, errors and perseveration numbers are calculated. For test applications, undergraduate students of Yeditepe University Psychology Department were trained by the first author and collected data with regular supervision during the data collection period. The education of the students continued until they mastered the test instructions, test termination and return rules, and scoring criteria. A total of 71 students contributed to the research. Appointments were made with the participants and the tests were administered individually. The majority of testing was conducted at the participant's residence, where this was not possible, in a location preferred by the participant (for example; in a local library study room) where confidentiality could be maintained. A comprehensive neuropsychological battery was administered to the participants. The order of administration of the tests reported in this article was as follows: Demographic data form, BDI (or GDS), MMSE, and verbal fluency tests.

Statistical Analysis

All data were analyzed with SPSS 23.0 program. The sample was divided into four (primary school graduates and below, 6-12 years of education, minimum 1 year university education, at least 1 year postgraduate education) education strata, and into eight strata according to age (18-19, 20-29, 30-39, 40-49, 50-59, 60- 69, 70-79 and 80-89 years old) and mean and standard deviation values of both total test scores and error and perseveration numbers were calculated for each subgroup.

The relationship between gender, age and education level, which were socio-demographic variables, and verbal fluency performance was examined by multiple linear regression analysis using stepwise selection method.

In order to determine the internal consistency of phonemic fluency, the Cronbach alpha coefficient was calculated. The relationship between the test-retest scores and the verbal fluency tests, which would show coincidence validity, were examined by using Pearson correlation analysis. Bonferroni correction was applied in multiple correlation analyses (p<0.01). In all other analyses, the threshold of significance was determined as p<0.05.

Results

Socio-demographic characteristics and mean MMSE scores of each subgroup are given in Table 1.

Phonemic (Lexical) Fluency Norms

In Table 2, the mean, standard deviations and the 25th, 50th (median) and 75th percentile values of the total number of valid words produced in the phonemic fluency category starting with the letters K, A and S for each group separated by age and education level, are given. The mean error and perseveration values of the phonemic fluency test, along with their standard deviations, are given in Table 3.


Phonemic fluency was found to be negatively correlated with age [r (1429): -0.301, p<0.001], and positively correlated with education level [r (1426): 0.555, p<0.001] (Figures 1, 2). The relationshipbetween gender and phonemic fluency performance was analyzed using the independent sample t-test. The result of the test showed that gender had no significant effect on fluency performance [t (1426): -1.082, p=0.279]. When the relationship between sociodemographic variables and phonemic fluency score was analyzed by using stepwise linear regression analysis; it was found that education year (β: 0.532, p<0.001) and age (β: -0.251, p<0.001) statistically significantly predicted the phonemic fluency score and explained 37% of the variance [F (2, 1425): 419.354, p<0.001].

Figure 1. The relationship between age and phonemic fluency

Figure 2. The relationship between education and phonemic fluency

In order to determine the internal consistency between letters in the phonemic fluency test, Cronbach’s alpha values were calculated using the total number of words produced for each of the 3 letters, and the alpha coefficient was found to be 0.90, and the standardized alpha coefficient to be 0.91. This showed that the internal consistency of the test was quite high. In order to measure the test-retest reliability, the Pearson correlation coefficient was calculated for the scores of 60 participants who were tested with a minimum interval of 3, a maximum of 5 months (mean: 3.85, SD: 0.40), and the value obtained showed that the reliability level of the test was high [r (58): 0.801, p<0.001].

Semantic (Categorical) Fluency Norms

The norm data of the semantic fluency test evaluated with the animal category are shown in Table 4. The mean error and perseveration values of the semantic fluency test, along with their standard deviations, are given in Table 5.


Pearson correlation analysis findings showed that age, one of the socio-demographic variables, was negatively and significantly correlated with semantic fluency [r (1429): -0.352, p<0.001] (Figure 3). A positive and significant relationship was found between education and semantic fluency [r (1429): -0.385, p<0.001] (Figure 4). Semantic fluency performance did not differ by gender [t (1429): 1.130, p=0.259].

Figure 3. The relationship between age and semantic fluency

Figure 4. The relationship between education and semantic fluency

By means of stepwise linear analysis, the relationship of age, gender and education year with the semantic fluency score was examined. Accordingly, education year (β: 0.355, p<0.001) and age (β: -0.319, p<0.001) were found to significantly predict semantic fluency scores and both variables explained 25% of the variance [F ( 2, 1428): 236.156, p<0.001].

In order to examine the test-retest reliability of the semantic fluency test, the relationship between the scores of both applications of 61 participants, who were retested minimum 3, maximum 5 months (mean: 3.84, SD: 0.42) after the first application, was examined by using Pearson correlation analysis and the reliability of the test was found to be at an acceptable level [r (59): 0.64, p<0.001].

Finally, in order to evaluate the concurrent validity of both tests, the correlation between semantic and phonemic fluency scores was examined. Results showed that the semantic fluency score showed moderate to strong correlations with the total phonemic fluency score [r (1423): 0.629, p<0.001] and the number of words produced separately for the letters K [r (1424): 0.612, p<0.001], A [r (1425): 0.547, p<0.001], and S [r (1426): 0.568, p<0.001].

Discussion

In this study, the norm values of verbal fluency tests calculated over an Istanbul sample were presented. Thus, the current norms of these neuropsychological tests, which were frequently used and helpful in diagnosis in the field of clinical neuropsychology, were established. In previously published data, age ranges were kept wide and norms were calculated using wide ranges of education groups (10). On the other hand, in this study, the age groups were formed by choosing narrower ranges to include the groups aged 80 and over. As such, it is more compatible with the age groups used in intelligence tests and executive function tests in the literature. For example, the adult age groups of the Delis-Kaplan executive functions test battery, which is widely used in the United States, exactly match the age groups used in this study (33). The age ranges used to establish the norms in the Weschsler intelligence scale for adults III are narrower, but they are similar (34). The age groups used for the standardization study of the Öktem verbal memory processes test, which is used in neuropsychological assessments in Turkey, are similar to the age groups in this study (15-39, 40-49, 50-59, 60-69, 70-79 and 80+) (35). Very similar age groups were used in the norm study of the Stroop test anchor form developed by Emek Savaş et al. (36) (18-29, 30-39, 40-59, 60-69 and 70 years and above). Similarly, it is aimed that the norm values created by using narrower ranges for education levels better represent these groups. In this way, the norm values of verbal fluency tests, which had an important place in neurological diagnosis, were considerably improved. Verbal fluency tests help specialists not only in making a diagnosis, but also in determining the risk of developing dementia in undiagnosed and asymptomatic individuals (37).

The study also examined how gender, age and education affected verbal fluency performance. While gender did not have a significant effect on verbal fluency scores, age and education explained 37% of phonemic fluency scores and 25% of semantic fluency scores. While a decrease was observed in both phonemic and semantic fluency performance with increasing age, both fluency scores increased as education increased. When the effects of age and education were examined separately, it was observed that education explained a larger variance for both phonemic and semantic fluency. The effect of education on phonemic and semantic fluency was also found in Tumaç’s (10) study. However, in the same study, the effect of age was not shown for semantic fluency performance (10). The effects of age and education on verbal fluency performance were also determined in studies conducted in both English and other languages (24,25,38).

The analyses showed that the validity and reliability values of the verbal fluency tests were strong for this sample. It was observed that the test-retest interval varied in different neuropsychological tests. For example, this interval, which was 3-4 weeks for some tests, increased to 12 months in other tests (35,39). In this study, the test-retest interval varied between 3 and 5 months, and it was observed that the reliability coefficients obtained were sufficient. In addition, the test-retest sample was representative of the entire sample in terms of demographic characteristics. Based on the results of the correlation analyses between the tests, it can be concluded that the phonemic and semantic fluency tests measure both similar and different cognitive functions. When both the findings of neuroimaging studies involving healthy participants and the performance of individuals with neurological diagnosis were examined, it was seen that phonemic and semantic fluency were associated with different brain regions (40,41). Similar findings reveal the importance of evaluating both fluency performances in neuropsychological assessments.

Conclusion

The main limitation of the study was that the entire sample was selected from Istanbul, the participants could not represent individuals living in different regions of Turkey, and as a result, the norms could not be generalized. However, among the words given by the research participants, the words used in different regional dialects were accepted as correct, and regional differences were tried to be represented in this way. Considering both the cosmopolitan nature of Istanbul, the high number of samples and the diversity of subgroups, it was predicted that the findings would be much more beneficial for clinicians and researchers compared to the old norms.

Ethics Committee Approval

Yeditepe University Ethics Committee approval was obtained before starting the study (number: 75078252-050.01-0494).

Peer Review

Externally peer-reviewed.

Author Contributions

Concept: Y.S.İ., Design: Y.S.İ., Data Collection or Processing: Y.S.İ., Analysis or Interpretation: E.S.B., Literature Search: Y.S.İ., E.S.B., Writing: Y.S.İ., E.S.B.

Conflict of Interest

No conflict of interest was declared by the authors.

Financial Disclosure

The authors declared that this study received no financial support.

References

  1. Henry JD, Crawford JR, Phillips LH. Verbal fluency performance in dementia of the Alzheimer’s type: a meta-analysis. Neuropsychologia 2004;42:1112-1122.
  2. Clark LJ, Gatz M, Zheng L, et al. Longitudinal verbal fluency in normal aging, preclinical, and prevalent Alzheimer’s disease. Am J Alzheimers Dis Other Demen 2009;24:461-468.
  3. Williams-Gray CH, Evans JR, Goris A, et al. The distinct cognitive syndromes of Parkinson’s disease: 5 year follow-up of the CamPaIGN cohort. Brain 2009;132:2958-2969.
  4. Pereira JB, Junqué C, Martí MJ, et al. Structural brain correlates of verbal fluency in Parkinson’s disease. Neuroreport 2009;20:741-744.
  5. Krukow P, Harciarek M, Morylowska-Topolska J, Karakuła-Juchnowicz H, Jonak K. Ineffective initiation contributes to deficient verbal non-verbal fluency in patients with schizophrenia. Cogn Neuropsychiatry 2017;22:391-406.
  6. Henry J, Crawford JR. A meta-analytic review of verbal fluency deficits in depression. J Clin Exp Neuropsychol 2005;27:78-101.
  7. Lezak MD. Neuropsychological Assessment. 2nd ed. New York, NY: Oxford University Press, 1983.
  8. Vaughan RM, Coen RF, Kennny R, Lawlor BA. Semantic and phonemic verbal fluency discrepancy in mild cognitive impairment: potential predictor of progression to Alzheimer’s disease. J Am Geriatr Soc 2018;66:755-759.
  9. Straus E, Sherman EMS, Spreen O. A compendium of neuropsychological tests: Administration, norms, and commentary, 3rd ed. New York, Oxford University Press, 2006.
  10. Tumaç A. Normal deneklerde, frontal hasarlara duyarlı bazı testlerde performansa yaş ve eğitimin etkisi (Yüksek Lisans Tezi). İstanbul: İstanbul Üniversitesi; 1997.
  11. Lezak MD, Howieson DB, Loring DW. Neuropsychological assessment. 4th ed. New York, NY: Oxford University Press, 2004.
  12. Shao Z, Janse E, Visser K, Meyer AS. What do verbal fluency tasks measure? Predictors of verbal fluency performance in older adults. Front Psychol 2014;5:772.
  13. Whiteside DM, Kealey T, Semla M, et al. Verbal fluency: language or executive functions measure? Appl Neuropsychol Adult 2016;23:29-34.
  14. Gustavson DE, Panizzon MS, Elman JA, et al. Genetic and environmental influences on verbal fluency in middle age: a longitudinal twin study. Behav Genet 2018;48:361-373.
  15. Karakaş S. Prof. Dr. Sirel Karakaş Psikoloji Sözlüğü: Bilgisayar Programı ve Veritabanı, 2017. Accessed date: 18 December 2021, Available from: www. psikolojisozlugu.com (version: 5.1.0/2021).
  16. Birn RM, Kenworthy L, Case L, et al. Neural systems supporting lexical search guided by letter and semantic category cues: a self-paced overt response fMRI study of verbal fluency. Neuroimage 2010;49:1099-1107.
  17. Costafreda SG, Fu CHY, Lee L, et al. A systematic review and quantitative appraisal of fMRI studies on verbal fluency: role of the left inferior frontal gyrus. Hum Brain Mapp 2006;27:799-810.
  18. Chouiter L, Holmberg J, Manuel AL, et al. Partly segregated corticosubcortical pathways support phonologic and semantic verbal fluency: a lesion study. Neuroscience 2016;329:275-283.
  19. Henry JD, Crawford JR. A meta-analytic review of verbal fluency performance in patients with traumatic brain injury. Neuropsychology 2004;18:621-628.
  20. Monsch AU, Bondi MW, Butters N, et al. Comparisons of verbal fluency tasks in the detection of dementia of the Alzheimer type. Arch Neurol 1992;49:1253-1258.
  21. Larsson MU, Almkvist O, Luszcz MA, Wahlin TB. Phonemic fluency deficits in asymptomatic gene carriers for Huntington’s disease. Neuropsychology 2008;22:596-605.
  22. Magaud E, Kebir O, Gut A, et al. Altered semantic but not phonological verbal fluency in young help-seeking individuals with ultra high risk of psychosis. Schizophr Res 2010;123:53-58.
  23. Pakhomov SVS, Eberly LE, Knopman DS. Recurrent perseverations on semantic verbal fluency tasks as an early marker of cognitive impairment. J Clin Exp Neuropsychol 2018;40:832-840.
  24. Cavaco S, Golçanves A, Pinto C, et al. Semantic fluency and phonemic fluency: regression-based norms for the Portuguese population. Arch Clin Neuropsychol 2013;28:262-271.
  25. Tombaugh TN, Kozak J, Rees L. Normative data stratified by age and education for two measures of verbal fluency: FAS and animal naming. Arch Clin Neuropsychol 1999;14:167-177.
  26. Diamond A. Executive functions. Annu Rev Psychol 2013; 64:135-168.
  27. Güngen C, Ertan T, Eker E, Yaşar R, Engin F. Reliability and validity of the standardized mini mental state examination in the diagnosis of mild dementia in Turkish population. Turk Psikiyatri Derg 2002;13:273-281.
  28. Ertan T, Eker E, Güngen C, et al. The standardised Mini-Mental Examination for illiterate Turkish elderly population. Paper presented at 2nd International Symposium on Neurophysiological and Neurophysiological Assessment of Mental and Behavioral Disorders, Kirazlıyayla, Bursa, Turkey, 1999. p. 28- 30.
  29. Hisli N. Beck depresyon envanterinin geçerliği üzerine bir çalışma (A study for the validity of the Beck depression inventory). Psikoloji Dergisi 1988;6:118-122.
  30. Ertan T, Eker E, Sar V. Reliability and validity of the geriatric depression scale in Turkish elderly population. Arch Neuropsychiatry 1997;34:62-71.
  31. Thurstone LL. Primary mental abilities. Chicago, The University of Chicogo Press, 1943:26.
  32. Lezak MD. Executive functions and motor performance. Neuropsychological Assessment. New York, NY: Oxford University Press, 1995:650-685.
  33. Swanson J. The Delis-Kaplan executive function system. Canadian Journal of School Psychology 2005;20:117-128.
  34. Ryan JJ, Sattler JM, Lopez SJ. Age effects on Wechsler Adult Intelligence Scale-III subtests. Arch Clin Neuropsychol 2000;15:311-317.
  35. Öktem ÖT. Öktem sözel bellek süreçleri testi. (Öktem-SBST) el kitabı. Ankara: Türk Psikologlar Derneği Yayınları, 2011.
  36. Emek Savaş DD, Yerlikaya D, Yener GG, Öktem ÖT. Validity, Reliability and Normative Data of the Stroop Test Çapa Version. Turk Psikiyatri Derg 2020;31:9-21.
  37. Palmer K, Bäckman L, Winblad B, Fratiglioni L. Detection of Alzheimer’s disease and dementia in the preclinical phase: population based cohort study. BMJ 2003;326:245.
  38. Zimmermann N, de Mattos Pimante Parente AM, Joanette Y, Fonseca RP. Unconstrained, phonemic and semantic verbal fluency: age and education effects, norms and discrepancies. Psicol Reflex Crit 2014;27:55-63.
  39. Karakaş S. Bilnot bataryası el kitabı: nöropsikolojik testler için araştırma ve geliştirme çalışmaları. Ankara: Dizayn Ofset, 2004.
  40. Tupak SM, Badewien M, Dresler T, et al. Differential prefrontal and frontotemporal oxygenation patterns during phonemic and semantic verbal fluency. Neuropsychologia 2012;50:1565-1569.
  41. Murphy KJ, Rich JB, Troyer AK. Verbal fluency patterns in amnestic mild cognitive impairment are characteristic of Alzheimer’s type dementia. J Int Neuropsychol Soc 2006;12:570-574.