ABSTRACT

OBJECTIVES: to elaborate and analyze the psychometric properties of a questionnaire to assess among pregnant women's knowledge about toxoplasmosis.
METHODS: methodological study elaborating an instrument conducted in the north of Minas Gerais in 2019. The steps followed were: 1. Elaboration of the questionnaire items based on medical literature available in indexed databases. 2. Content validation. 3. Apparent validation. 4. Construct validation with hypothesis testing after the application of the questionnaire and comparison of the mean scores using the Mann-Whitney U test. 5. Reliability analysis with internal consistency analysis and test-retest.
RESULTS: the final instrument consisted of 26 items. Content validation reached 90% agreement among experts. The hypothesis test found a significant difference among the scores of the groups evaluated (p<0.001). Cronbach's alpha found a value of 0.84 and the test-retest showed an intra-class correlation index of 0.78 (p<0.001).
CONCLUSIONS: the instrument elaborated proved to be valid and reliable and could be used to assess among pregnant women's knowledge about toxoplasmosis and, therefore, guiding to the educational and preventive measures.

Keywords: Psychometrics, Validation studies, Toxoplasmosis, Health education

RESUMO

OBJETIVOS: elaborar e analisar as propriedades psicométricasde um questionáriopara avaliação do conhecimento sobre toxoplasmose entre gestantes.
MÉTODOS: estudo metodológico de elaboração de instrumento, realizado no norte de Minas Gerais, em 2019. O estudo seguiu os seguintes passos: 1. Elaboração dos itens a do questionário, com base na literatura médica disponível em bases indexadas. 2. Validação de conteúdo. 3. Validação aparente. 4. Validação de constructo, com teste de hipóteses, após aplicação do questionário e comparação de médias dos escores por meio do teste U de Mann-Whitney. 5. Análise de confiabilidade, com análise da consistência interna e teste-reteste.
RESULTADOS: o instrumento final foi composto por 26 itens. Na validação de conteúdo alcançou 90% de concordância entre os experts. O teste de hipótese registrou diferença significativa entre os escores dos grupos avaliados (p<0,001). O alfa de Cronbach apresentou valor de 0,84 e o teste-reteste apresentou índice de correlação intra-classe de 0,78 (p<0,001).
CONCLUSÃO: o instrumento elaborado apresentou-se válido e confiável e poderá ser utilizado para avaliação de conhecimento sobre toxoplasmose entre gestantes, norteando medidas educativas e preventivas.

Palavras-chave: Psicometria, Estudos de validação, Toxoplasmose, Educação em saúde

Introduction

Toxoplasmosis is parasitic disease that affects neglected populations found almost worldwide, infecting a significant number of individuals, with a wide range of seroprevalence.¹ Literature analysis revealed a global prevalence of 33.8% for the latent disease in pregnant women in middle and low income countries. In South America, the levels of latent prevalence found in pregnant women were above 50%.²

Humans are intermediate hosts for the parasite, presenting asymptomatic or non-specific clinical condition often mistaken for other diseases¹. The disease becomes relevant when affecting pregnant women due to the potential risk for transmission to the fetus as well as for permanent and very severe damages, such as neurological, visual, hearing impairment, and among others.³ The risk of contagion is directly related to the gestational age, exceeding 70% of chance of fetal contamination if acute infection occurs after 37 weeks of gestation.⁴

Studies have indicated that pregnant women are often unaware of the risk factors for acquiring the disease. In the absence of effective immunoprevention, health education associated with behavioral measures during prenatal care has been pointed out as a low-cost and feasible possibility of preventing contagion, the parasitosis and its adverse outcomes.^3,5-7

In Europe, some authors have suggested positive effect of primary prevention of toxoplasmosis as a result of increased knowledge of the patients about the exposure to risk factors for the disease, evidenced by the decrease in acute infection during pregnancy^7,8. These same studies have highlighted the need for research on primary health with more rigorous methodological designs in order to effectively establish the relevance of primary prophylaxis for the disease.^3,8,9

Given the paucity of more extensive studies on toxoplasmosis prophylaxis and the need for greater methodological rigor in research on this subject, this study aimed to elaborate and analyze the psychometric properties of a questionnaire to assess pregnant women’s knowledge about toxoplasmosis prophylaxis.


Methods

This is a methodological study for the elaboration and analysis of psychometric properties of a questionnaire. The proposed instrument addresses the knowledge about toxoplasmosis and prophylaxis measures focused on pregnant women.

The steps followed were: 1. Elaboration of the items to be researched based on the medical literature available in indexed databases. 2. Content validation. 3. Apparent validation. 4. Construct validation with discriminatory analysis. 5. Reliability analysis with analysis of internal consistency and test-retest.

Figure 1 summarizes the steps followed.


 



In the first step, the items to be researched were elaborated based on the medical literature available in indexed databases.

In order to formulate the questions, the subject was researched with emphasis on the following topics: prophylaxis of toxoplasmosis, prevention of gestational toxoplasmosis, prevention of congenital toxoplasmosis and education on toxoplasmosis. The following indexed medical literature databases were searched: Medical Literature Analysis and Retrieval System Online (MEDLINE), the US National Library of Medicine (PubMed), and the Scientific Electronic Library Online (SCIELO) in Portuguese and English.

In addition to scientific papers, protocols and manuals of conduct as well as other materials available for health professionals were also researched. The search was conducted from June to July 2018, for papers in Portuguese, English, Spanish and French.^3,4,6,8-15

After selection and synthesis of the material found, 30 short and objective questions were drafted in order to meet the objective set. Two dimensions were defined for the instrument: prevention and knowledge about toxoplasma infection.

In the second step, the content validation was carried out with the instrument sent via electronic mail to 11 professionals with PhD degrees and relevant publications in the study area in different States in Brazil as well as professionals currently active with degrees in the areas of medicine, nursing, infectiology, veterinary and biology. Of those, five professionals participated in the content analysis and semantic structure of the questionnaire suggesting, reviewing and assessing the relevance of each item using a Likert scale (very relevant, relevant, little relevant and irrelevant) as well as the intelligibility of each item (yes or no).

After the analysis of the experts, the instrument was reformulated and the items that reached a content validation index of at least 90% (i.e., items considered very relevant and relevant by at least four of the evaluators) were kept. Some items were rewritten according to the evaluators’ suggestions and appraisals.

In the third step, the apparent validation was carried out and after the consolidation of the new version, the questionnaire was sent again to 9 specialists active in the areas of infectious diseases, obstetrics and pediatrics to be assessed whether the items were clear and understandable. Questions with more than 70% of agreement among the specialists were kept and the questionnaire was applied again to 15 specialist physicians (pediatricians, clinicians, and obstetricians) active in primary health care services. All items were considered clear and easy to understand by the professionals.

Before proceeding with the application of the questionnaire to the next step, about 10% of the questions were randomly selected to be turned into false statements. All questionnaire items followed Likert scale response options, where the respondents were able to register their level of agreement or not with the assertions (“I’m sure it’s correct”, “I think it’s correct”, “I think it’s wrong”, “I’m sure it’s wrong” and “I don’t know if it’s right or wrong”).

A pilot test was also conducted with 30 pregnant women in two Family Health Strategies to assess the clarity, understanding and time required to complete the questionnaire.

In the fourth step, the construct validation was carried out from the hypothesis test. The pregnant women were selected from the public healthcare system (at five units of the Family Health Strategy randomly chosen, patients assisted at the high-risk service of a reference hospital, patients assisted at high-risk outpatient clinics of the city and state networks of women’s health) and from the private healthcare system (at private obstetrician’s offices and two ultrasound services for pregnant women). Data were collected from March to October 2019.

The pregnant women were approached at the healthcare service sites, the research was briefly explained to them and the questionnaires, without any item that would allow their identification, were handed. The average time required to complete the questionnaire was 15 to 20 minutes. Once completed, the questionnaires were handed to the secretaries and stored in appropriate boxes at each of the sites mentioned above. They were periodically collected and checked. Incomplete questionnaires were excluded.

In order to calculate the sample size, the collection of at least 10 respondents for each valid question of the questionnaire was estimated, as recommended by the experts in the area^16-18 For the hypothesis test, the instrument was also applied to 15 specialist physicians (pediatricians or gynecologists/obstetricians).

All responses (from the pregnant women and specialists) were turned into scores and the questions received values as follows: (0) when the option chosen was “I don’t know if it’s right or wrong” or when there was agreement for false questions or disagreement for true questions, and (+1) for questions appropriately marked as correct (“I’m sure it’s correct” and “I think it’s correct”) or appropriately marked as incorrect (“I’m sure it’s wrong” or “I think it’s wrong”).

The hypothesis test aimed to verify whether the instrument was able to discriminate between the specialists (with higher scores expected) and the pregnant women (with lower scores expected). The mean scores were calculated for each group of respondents and compared using the Mann Whitney U-test, with a significance level of 5%.

In order to avoid selection bias, the questionnaires were collected in different healthcare services for pregnant women, both in the public and private systems. The services assisting a larger number of patients were chosen and the patients were randomly approached. Twelve sites were defined: one city polyclinic, three private practices, two fetal ultrasound services (private and public), five Family Health Strategy units, two high-risk outpatient clinics (one city service and one state service) and an outpatient clinic for pregnant women linked to the largest maternity hospital in the city.

In the fifth step, the reliability of the instrument was measured based on the internal consistency analysis of the instrument through the item-total correlation and the calculation of Cronbach’s alpha, assuming a value above 0.7 to define satisfactory reliability of the instrument.^17,19,20,21

In addition, the reliability of the instrument was also measured through test-retest and calculation of the intraclass correlation coefficient, considering the mean scores of a sample of respondents at two different times, with a maximum interval of 10 to 15 days.

All statistical analyses were performed using SPSS software, version 22.0 for Windows.

All ethical requirements for this study were met. The research project was evaluated and approved by the Research Ethics Committee of the State University of Montes Claros under No. 2,341,969. All participants consented to the study and signed the informed consent form.


Results

Literature analysis resulted in the elaboration of 30 items. After being evaluated by the group of experts, the questionnaire ended up with28 questions. Some questions were rewritten, and some terms were changed according to the suggestions presented. In the apparent validation step, the questions were kept and the agreement among the experts was over 90%.

In the construct validation step, the instrument was applied to 355 pregnant women and 15 specialists; 20 questionnaires applied to the pregnant women were excluded for being partially answered. In the analysis of the answers, two questions presented a percentage of correct answers above 90% in the two groups analyzed and were excluded (considered very easy). The exclusion of questions correctly answered by less than 10% of the respondents (considered very difficult) was also expected, but no questions reached such rate.

The main characteristics of the pregnant women who answered the questionnaire are shown in Table 1. About 11% of the respondents were adolescents, over 50% graduated from high-school, 43% were primiparous and nearly 70% had their prenatal care through the Brazilian Public Health System (SUS).


 



Table 2 shows the comparison among the respondents’ mean scores, according to some characteristics. There was a statistically significant difference between the scores of the pregnant women and the specialists (p<0.001) and no difference was found for other characteristics of the group.


 



The internal consistency analysis, assessed by the Cronbach’s alpha, showed a result of 0.84 for the final instrument with 26 questions. After the retest, the respondents’ mean scores were compared, and the Intraclass Correlation Index was 0.78 (p<0.001).

The final version of the instrument, with emphasis on false and true items, and the percentage of correct answers for each item by the group of pregnant women, is shown in Table 3.


 



Discussion

The present study enabled the elaboration and analysis of the validity and reliability of an instrument to research the knowledge about toxoplasmosis and prevention practices during pregnancy among pregnant women. Measuring instruments are very relevant in health-related research because they cover clinical practice and health assessment. Reliability and validity are considered the main measures of such instruments.^16,20

Although the literature records the importance of pregnant women’s awareness of toxoplasmosis in order to take self-protection measures, no studies presenting valid and reliable instruments for measuring such knowledge were found. Non-validated questionnaires elaborated from observational studies were used in similar studies.^5,10-12

The content validation step was particularly important in the elaboration of the questionnaire, since the suggestions, criticism and evaluation of the intelligibility of the statements made the instrument more robust with a clearer and more objective definition of the items to be investigated, and relied on the expertise of the evaluators, allowing adjustments of semantic aspects as well as conceptual equivalence. Two questions were removed from the questionnaire in this step as they addressed very specialized content that would be important if the knowledge of health professionals were being evaluated. The questions were: Q28. There are specific drugs to treat children infected with toxoplasma and Q29. Treatment for toxoplasmosis in children lasts an average of one year and yet, they could have long-term health problems associated with the disease.

The apparent validation step, with the application of the questionnaire to a small group of specialists, allowed a further refinement of the objectives of each statement, the structure of the topics and relevance of the questions. Agreement above 90% among the evaluators dispensed with the need for new structural changes in the questionnaire, reinforcing that the items elaborated met the criteria of relevance, practical pertinence and consonance with the literature under study.²²

Assessment instruments in the health area should consider the objectivity and intelligibility of the items, as well as the number of items to be evaluated. Although the ideal number of items is controversial in the literature, the conciseness of the instrument makes the application fast, simple, and therefore feasible.^22,23 The steps followed in the present study were meant to address this aspect.

Two questions considered very easy (correct answers above 90% both in the specialists’ group and the pregnant women’s group) were removed in the construct validation step. The questions were: Q1: Toxoplasmosis is an infection caused by a parasite that, in our region, is known as “cat disease” and Q6: Eating fruit and raw vegetables without washing them previously is a way of acquiring toxoplasmosis. Questions considered too easy address aspects regarded as common sense, being unsuitable for knowledge assessment studies.

Construct validation, using the hypothesis test, indicated that the instrument is capable of adequately distinguishing the most knowledgeable individuals. The analysis of the independent samples showed that the knowledge of the specialists about toxoplasmosis and preventive measures is superior to that of the pregnant women, corroborating the latent feature of the construct, which proposed to evaluate knowledge. Other variables tested showed no differences between the scores. It is important to point out that there was no statistically significant difference among the scores of pregnant women with elementary education and those with high school or higher education, and one can speculate that health knowledge is a determining factor for better results in terms of knowledge and prophylaxis for the disease.

The final instrument showed a satisfactory level of internal consistency as assessed by Cronbach’s alpha. The reliability of the instrument is related to the consistency of the results, with no large oscillations among repeated measures that would reflect the presence of measurement errors.²⁰ The coefficient considers the homogeneity of the items of the scale and has the advantage of requiring only one application of the instrument. It is the most used method to measure reliability, when reliability is understood as an internal consistency of the scale indicators, i.e., the scale indicators highly interrelated must measure the same latent construct.^22,24

Regarding the reproducibility of the instrument, the test-retest showed that the instrument has good temporal stability, considering the Intraclass Correlation Coefficient, which presented a satisfactory score agreement value among the respondents after a period of 10-15 days.

When analyzing specifically the results of the responses given by the pregnant women, it is observed that there are essential knowledge gaps with the potential to interfere with the prevention of toxoplasmosis such as the intake of untreated water or undercooked meat, questions that over 30% of them did not regard as a risk factor. There are studies showing similar ignorance of pregnant women before educational interventions.^10-12

This study has found that more than 90% of the pregnant women associate the transmission of toxoplasmosis with the presence of cats, but they ignore the role of other vertebrate animals as intermediate hosts.

Questions related to consumption of unpasteurized and unboiled milk and freezing and defrosting food obtained less than 50% correct answers among pregnant women and even among specialists. This result highlights that knowledge about primary prevention is also deficient in the medical environment, requiring further guidance on the subject in order to improve health education standards, since they are opinion makers and can intervene in the habits of their patients.⁷

A significant number of the pregnant women interviewed knows that if they acquire the disease during pregnancy, they can contaminate their babies, but they are unaware of the symptoms of toxoplasmosis when they become infected. Approximately 50% of the pregnant women identify the possible sequelae of the disease correctly, although they ignore the fact that over 70% of infected babies are asymptomatic at birth, a finding that is in line with a study on the evaluation of an educational approach to toxoplasmosis.¹⁰

The limitations of this study are due to the fact that this questionnaire was applied in a single region of the country (in the North of Minas Gerais State), where the cultural issues and customs of the population are rather peculiar. However, the questionnaire represents the starting point for other studies. However, studies evaluating the knowledge of pregnant women about toxoplasmosis did not have a valid and reliable reference.

It is worth adding that educational materials can be elaborated from the results obtained through reliable instruments in the health area. These materials are intended to facilitate the professionals’ work, mainly in the early education intervention of the health service users as well as their families, healthcare team guidance and policy interventions required to improve the population’s health a. Therefore, the content of such educational materials in the context of toxoplasmosis should be based on studies that use the elaborated, validated, and reliable questionnaire.

The questionnaire evaluated has shown satisfactory psychometric properties, proving to be a valid and reliable instrument to assess pregnant women’s knowledge about toxoplasmosis. It should be considered as a useful instrument to identify knowledge gaps and can be used as a basis for the development of prevention and education strategies for pregnant women from different regions of the country..


References

1. Ministério da Saúde (BR). Protocolo de notificação e investigação: “Toxoplasmose gestacional e congênita”. Secretaria de Vigilância em Saúde. Departamento de Vigilância das Doenças Transmissíveis. 1^st ed. Brasília (DF): Ministério da Saúde; 2018; [access in 2020 Mai 3]. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/protocolo_notificacao_investigacao_toxoplasmose_gestacional_congenita.pdf

2. Rostami A, Riahi SM, Gamble HR, Fakhri Y, Nourollahpour Shiadeh M, Daneshm M, et al. Global prevalence of latent toxoplasmosis in pregnant women: a systematic review and meta-analysis. Clin Microbiol Infect. 2020 Jun; 26 (6): 673-83.

3. Di Mario S, Basevi V, Gagliotti C, Spettoli D, Gori G, D’Amico R, et al. Prenatal education for congenital toxoplasmosis. Cochrane Database Syst Rev. 2015 Oct; 2015 (10): CD006171.

4. Maldonado YA, Read JS; Committee on Infectious Diseases. Diagnosis, Treatment, and Prevention of Congenital Toxoplasmosis in the United States. Pediatrics. 2017 Feb; 139 (2): e20163860.

5. Smereka J, Szarpak L, Ruetzler K, Schacham Y, Smereka A, Dabrowski M, et al. A multicenter survey on toxoplasmosis knowledge among pregnant women in Poland (the TOWER study). BMC Pregnancy Childbirth. 2018 Oct; 18 (1): 389.

6. Moura FL, Goulart PRM, Moura APP, Souza TS, Fonseca ABM, Amendoeira MRR. Fatores associados ao conhecimento sobre a toxoplasmose entre gestantes atendidas na rede pública de saúde do município de Niterói, Rio de Janeiro, 2013-2015. Epidemiol Serv Saúde. 2016; 25 (3): 655-61.

7. Lopes-Mori FMR, Breganó RM, Copabiango JD, Inoue IT, Reiche EMV, Morimoto HK, Casella AMB, Bittencourt LHFB, Freire RL, Navarro IT. Programas de controle da toxoplasmose congênita. Rev Assoc Med Bras. 2011 Oct; 57 (5): 594-9.

8. Pawlowski ZS, Gromadecka-Sutkiewicz M, Skommer J, Paul M, Rokossowski H, Suchocka E. Impact of health education on knowledge and prevention behavior for congenital toxoplasmosis: the experience in Pozna, Poland. Health Educ Res. 2001 Aug; 16 (4): 493-502.

9. Gollub EL, Leroy V, Gilbert R, Chêne G, Wallon M, European Toxoprevention Study Group (EUROTOXO). Effectiveness of health education on Toxoplasma-related knowledge, behaviour, and risk of seroconversion in pregnancy. Eur J Obstet Gynecol Reprod Biol. 2008 Feb; 136 (2): 137-45.

10. Dabritz HA, Conrad PA. Evaluation of an educational handout on knowledge about toxoplasmosis. Sci Med. 2010 [access in 2021 Jul 26]; 20 (1): 51-8. Available from: https://revistaseletronicas.pucrs.br/ojs/index.php/scientiamedica/article/view/5890/5057

11. Branco BHM, Araújo SM, Guilherme ALF. Prevenção primária da toxoplasmose: conhecimento e atitudes de profissionais de saúde e gestantes do serviço público de Maringá, estado do Paraná. Sci Med. 2012 [access in 2021 Jul 26]; 22 (4): 185-90. Available from: https://revistaseletronicas.pucrs.br/ojs/index.php/scientiamedica/article/view/11718/8615

12. Jones JL, Dargelas V, Roberts J, Press C, Remington JS, Montoya JG. Risk factors for Toxoplasma gondii infection in the United States. Clin Infect Dis. 2009 Sep; 49 (6): 878-84.

13. Hill DE, Benedetto SMC, Coss C, Mccrary JL, Fournet VM, Dubey JP; Effects of Time and Temperature on the Viability of Toxoplasma gondii Tissue Cysts in Enhanced Pork Loin. J Food Prot. Aug 2006; 69 (8): 1961-5.

14. Jones JL, Dubey JP. Waterborne toxoplasmosis – Recent developments. Exp Parasitol. 2010 Jan; 124 (1): 10-25.

15. Moura L, Bahia-Oliveira LMG, Wada MY, Jones JL, Tuboi SH, Carmo EH, et al. Waterborne Toxoplasmosis, Brazil, from Field to Gene. Emerg Infect Dis. 2006 Feb; 12 (2): 326-9.

16. Cunha MC, Neto OPA, Stackfleth R. Principais métodos de avaliação psicométrica da validade de Instrumentos de medida. Rev Aten Saúde São Caetano do Sul. 2016; 14 (47): 75-83.

17. Nunes CH S, Primi R. Impacto do tamanho da amostra na calibração de itens e estimativa de escores por teoria de resposta ao item. Aval Psicol. 2005 Nov; 4 (2): 141-53.

18. Scatena LM, Wysocki AD, Beraldo AA, Magnabosco GT, Brunello MEF, Netto AR, et al. Validação e confiabilidade: instrumento para avaliação de serviços que tratam tuberculose.Rev Saúde Pública. 2015; 49: 7.

19. Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciênc Saúde Coletiva. 2011; 16 (7): 3061-8.

20. Souza AC, Alexandre NMC, Guirardello EB. Propriedades psicométricas na avaliação de instrumentos: avaliação da confiabilidade e da validade. Epidemiol Serv Saúde. 2017; 26 (3): 649-59.

21. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Rev. 2010 May; 19 (4): 539-49.

22. Coluci MZO, Alexandre NMC, Milani D. Construção de instrumentos de medida na área de saúde. Ciênc Saúde Coletiva. 2015; 20 (3): 925-36.

23. Leite SS, Áfio ACE, Carvalho LV, Silva JMS, Almeida PC, Pagliuca LMF. Construção e Validação de Instrumento de Validação de Conteúdo Educativo em Saúde. Rev Bras Enferm. 2018; 71 (Suppl. 4): 1635-41.

24. De Bem AB, Lanzer EA, Tambosi Filho E, Sanchez OP, Bernardi Júnior P. Validade e Confiabilidade de instrument de avaliação da docência sob a ótica dos modelos de equação estrutural. Avaliação (Campinas). 2011; 16 (2): 375-401.

Received on August 3, 2020
Final version presented on November 10, 2021
Approved on March 25, 2022

Authors’ contribution
Soares JAS and Caldeira AP participated in the design and planning of the study, as well as data collection and analysis, literature review, writing of the initial version and final review and approval of the final version of the manuscript. Sousa RG participated in the literature review, database typing and approval of the final version of the manuscript. Alves BBS, Silveira AAD, Lima CFQ participated in the literature review and data collection and final approval of the manuscript.

The authors declare no conflict of interest