Preparedness of community-based organisations in biohazard: reliability and validity of an assessment tool

Fatemeh Rezaei,1 Mohammad R Maracy,2 Mohammad H Yarmohammadian,3 Ali Ardalan,4 Mahmood Keyvanara5

ABSTRACT The purpose of this study was to develop a tool for community-based health organisations (CBHOs) to evaluate the preparedness in biohazards concerning epidemics or bioterrorism. We searched concepts on partnerships of CBHOs with health systems in guidelines of the Centers for Disease Control and Prevention and literature. Then, we validated the researcher-made tool by face validity, content validity, exploratory factor analysis(EFA), confirmatory factor analysis (CFA) and criterion validity. Data were collected by sending the tool to 620 CBHOs serving under supervision of Iran’s ministry of health. Opinions of health professionals and stakeholders in CBHOs were used to assess face and content validity.Factor loads in EFA were based on three-factor structure that verified by CFA. We used SPSS V.18 and Mplus7 software for statistical analysis. About 105 health-based CBHOs participated. After conducting face validity and calculating content validity ratio and content validity index,we reached 54 items in the field of planning, training and infrastructure. We conducted construct validity using 105 CBHOs. Three items exchanged between the fields according to factor loads in EFA, and CFA verified the model fit as Comparative Fit Index, Tucker-Lewis index and root mean square error of approximation were 0.921, 0918 and 0.052, respectively. The Cronbach’s of the whole tool was 0.944. Spearman correlation coefficient confirmed criterion validity as coefficient was 0.736. Planning,training and infrastructure fields are the most important aspects of preparedness in health-based CBHOs. Applying the new assessment tool in future studies will show the weaknesses and capabilities of health-based CBHOs in biohazard and clear necessary intervention actions for health authorities.

INTRODUCTION

Community-based organisations (CBOs) are indispensable partners in health systems.1In disasters and management of epidemics,some governments cannot provide an adequate response with limited resources.2These organisations have the necessary knowledge about community culture, structure and resources, and facilitate access to deprived and marginalised communities in times of need.34Besides, Federal Emergency Management Agency and Centers for Disease Control and Prevention (CDC) have encouraged coordination with volunteer organisations whose goal is to support and coordinate with government agencies in disasters.15

Contrary to these requirements, there is limited evidence regarding the readiness of these organisations for bioterrorism or epidemic/pandemic.6For instance, in order to prevent the spread of Zika virus, organisations are expected to have very high communication skills to warn high-risk groups. In a study by Zhi et al; they emphasised the need to educate the staff of faith-based organisations especially in exercises and drills.7Eventually, Clawson et al examined the community health centres’ preparedness in the case of bioterrorism. They demonstrated that less than half of the centres possessed bioterrorism preparedness in their plan and only one-third among them included bioterrorism preparedness in their written policies.8

Frequent religious mass gatherings and food serving in customs gathering exposure Iran in the various potentiality of outbreaks and bioterrorism.9-11Besides, limited access to resources in epidemic seasons in developing countries like Iran would worsen the situation.12Although Iran has an extensive community-based primary healthcare (PHC)network that raise community awareness of local risk profiles and aid community, a population-based study revealed that overall community awareness and preparedness for even routine disasters is low.13-15Iran’s PHC network has taken some advocacy and training programmes focusing on community partnership.1314CBOs can equip public health officials with information about vulnerable groups and how to meet their particular needs.16However, Iran still lacks an assessment tool evaluating the preparedness of CBOs in biohazards.

According to the above evidence, the necessity of formulating specific criteria in the context of preparedness for CBOs is obvious. Therefore, this study aimed to develop a comprehensive tool to evaluate the level of CBHOs' preparedness in times of epidemics or bioterrorism thereby identifying the prerequisites of CBHOs’ participation for government decision makers.

METHOD

Participants and sampling method

The study population was 620 CBHOs in the country. Based on Cochrane’s formula,17approximately 100 samples should be included in the study. Receiving a compiled list of CBOs from vice-chancellery for social affairs, the tool was disseminated in CBHOs’ social networks. Additionally, we sent the participation appeal through contact channels five times for each CBHOs. These are organisations that, according to experts from the Centers for Disease Control at deputy of health, possess the eligibility and capabilities necessary to work with the health system as an assisting or cooperating agencies during biohazards.Inclusion criteria were: CBHOs that served more than 50 clients and provide services to clients at the time of the study with a registered office to carry out their duties. The identity of the person filling the questionnaire and their organisation remained disclosed in the data collection forms; instead, a code was assigned at the time of data entry.

Instrument

Face validity, content validity, exploratory factor analysis(EFA), confirmatory factor analysis (CFA) and criterion validity were used to validate this researcher-made tool.Assessing the face validity, health professionals in disasters/emergencies and some stakeholders from CBHOs read questions of the instrument to examine the level of difficulty, the degree of mismatches and to check the ambiguity of phrases and meanings of words,18thereby making appropriate changes in Persian based on their comments. The qualitative content validity determined by the grammar, the proper use of words, the importance of questions, the ordering of questions and the time required to complete the toolkit were all taken into consideration.1920For quantitative content validity,content validity ratio (CVR) was used to ensure that the most important content was chosen (necessity of question), and the content validity index (CVI) was used to ensure using the best way to measure content.2122CVR is used to ensure that the most relevant and correct content is selected.2123In quantitative content validity, after collecting expert opinions, if the CVR based on Lawshe’s table was more than 62%24and the CVI based on the Davis study was over 80%,22the necessity, relevance, transparency and simplicity of the questions were acceptable.

We used EFA to determine the factors loads based on three factors extracted as the assumption of this study.CFA, also, was used to verify the factor structure and the hypothesis that a relationship between observed variables and their underlying latent constructs exists.25

In order to assess the concurrent criterion validity, the‘organisational preparedness checklist for a major earthquake or other large-scale disaster events’ questionnaire developed by Austin et al was applied.26The questionnaire consists of 30 phrases in four periods that include‘Last year’, ‘Between 1 year to 3 years’, ‘Over the past 3 years’ and ‘Never’.

Data collection

The researcher-made tool was designed to determine the preparedness level of CBHOs in biohazards. After studying the guidelines of Office of the Assistant Secretary for Preparedness and Response (ASPR)27-29along with other information sources, the research team eliminated duplicates/repetition of initial items pool.

The researchers extracted all concepts promoting the partnership and cooperation of CBOs with the health system under the first set of capabilities named ‘community preparedness’ in the Public Health Preparedness capabilities guideline. Then, fundamental concepts of the tools that are available from the literature were integrated into the primary tool to evaluate CBHOs’ preparedness.

Statistical analysis

We evaluated the construct validity with EFA in SPSS V.18 software and CFA in Mplus7 software (Muthén and Muthén, Los Angeles, California, USA). Fitting indexes included Comparative Fit Index (CFI), Tucker-Lewis index comparative fit index (TLI), root mean square error of approximation (RMSEA) and weighted root mean square residual. Cronbach’s α was used to measure the internal consistency.

RESULTS

Model assumption

At first, we obtained 115 items. There were 56 items in the planning field, 21 items in the field of education and 38 items under infrastructure. For each item, a range of four options was considered, including ‘not done’, ‘due to review’, ‘planned but not implemented’ and ‘completely implemented’.

Face and content validity

In face validity, four items in the planning field, one item in the field of training and two items in the infrastructure field were omitted. After reviewing the questions,10 healthcare professionals who specialised in the field of disasters and emergencies inspected the qualitative content validity. Following this, based on the comments,certain corrections were made regarding the wording and language used in the questionnaire. In quantitative content validity, based on the CVR and CVI indicators,21 items in the planning field, 10 items in the field of training and 24 items in the infrastructure field were deleted. Finally, 54 items remained for entry into the construct validity stage.

EFA and CFA

About 105 CBHOs volunteered to participate in the study. The Kaiser-Meyer-Olkin test for sampling adequacy was 0.773. Next, to determine if the correlation matrix had a significant difference with zero and factor analysis was justifiable or not, the Bartlett test of sphericity was performed, which turned out to be 4293 χ2and p＜0.001.These values indicated that the factor analysis is justifiable based on the correlation matrix, and the items can be used for factor analysis. In this study, limiting the extracting factors by referring to the ‘community preparedness’principle of the CDC guideline in the field of public health emergencies27and applying varimax rotation, EFA with three factors were performed.

We considered an inflection point of 0.3 as the minimum factor load needed to maintain the items. Thus, 54 items without deletion were entered into the CFA stage. Two questions of the planning field and one question of the infrastructure field were exchanged based on the rotation matrix and the loadings factor, which were conceptually meaningful. The questions 30 in the table 1 and 12 and 13 in table 2 were those exchanged questions. All 11 items in the training field remained in the factor structure according to factor loads (table 3). The three-factor model confirmed by deleting item 13 in the substructure field based on CFA findings (table 2). Table 4 shows CFA fitting indexes.

Reliability analysis

The Cronbach’s α in the planning, training and infrastructure field were 0.938, 0.916 and 0.889, respectively.The Cronbach’s α of the whole tool was 0.944. Scores showed that the reliability of the tool is acceptable.

Concurrent criterion validity

Thirty-four CBHOs completed the Austin26questionnaire, and the Spearman correlation coefficient was 0.736 that mean the criterion validity of the tool is acceptable.

DISCUSSION

In this study, a tool was developed to assess the preparedness of CBHOs during biohazards. The researcher-made tool consisted of planning, training and infrastructure fields based on previous studies.78273031After achieving face and content validity, CFA was used for the threefactor model fit. EFA was also used to compare the items replaced in the proposed model.

Some studies noted that items with a factor load above 0.7 and even 0.4 were acceptable.3233Therefore, in the CFA, a single item with factor load less than 0.4 was deleted. table 4 shows that the RMSEA value of the model is 0.052. According to previous studies, if the RMSEA is between 0.05 and 0.08, the model is acceptable.34The index of CFI and TLI is more than 0.9, which shows that the three-factor model is acceptable.35Hence, in this study, we used CFA to compare the model fit of our tool with the proposed structure of The HHS Office of the ASPR. The office has divided the functions of community preparedness into three categories as planning, training and skills, equipment and technology.27However, in this research, the research team consented to use the term‘infrastructure’ instead of ‘equipment and technology’.

According to a cross-sectional study, the most challenging aspect in implementing the 15 capabilities of the CDC’s guideline is training and planning, and 18%of failing is related to infrastructure field.3637Evidence from a qualitative study shows that planning and training fields are the most significant challenges faced by health workers in response to Hurricane Sandy.38Another study showed ‘“community preparedness”’ as the common standard in both the accreditation standards developed by Public Health Accreditation Board(PHAB) and the 15 capabilities of CDC’s guideline.Therefore, strengthening all organisations in community-based preparedness can improve both accreditation standards and follow CDC guideline. Also, the fields of training, planning and infrastructure are common in both the guideline and the accreditation standards.39In this regard, the categorisation of CBHOs’ preparation activities in the three areas in our research is consistent with the study.

Importance of community partnerships in disasters has motivated researchers to develop various assessment tools for evaluating CBHOs’ preparedness. Glik et al and Clawson et al developed tools as an instrument monitoring collaboration between local health departments and CBOs.840However, they focused more on the duties of health departments in engaging CBOs. Austin et al26showed that CBOs’ preparedness in earthquake needs seven clusters of assessment including internal training, external response, response capabilities, information collection and distribution to staff, preparation,building protection and supplies. Moreover, Baezconde et al assessed preparedness of non-governmental organisations (NGOs) considering social and structural needs.Socially, NGOs have high ‘social will’ but little ‘community readiness’ to participate in emergencies. Structurally, NGOs’ linkage to voluntary organisations and public health departments lack enough coordination.41An assessment tool with biohazard approach for CBO’preparedness is rare in literature, and we have tried to fill this gap.

The CBHOs in Iran conduct weekly meetings at the health departments of medical universities wherein their fields of cooperation are identified. The representatives of CBHOs participate at these meetings, and health authorities discuss fields that health systems lack an adequate budget or cannot intervene due to legislation. Thus, CBHOs depending on their capabilities would offer their cooperation with the health system.The aforementioned meetings will be held more actively in times of disasters.4243

images/BZ_59_217_194_1136_3072.png

Table 4 Factor loadings of confirmatory factor analysis

The evaluation of these CBHOs in Iran in terms of their capacities, capabilities, strengths and weaknesses in the field of planning, training and infrastructure can be achieved using this assessment tool. Therefore, the government departments that licensed CBHOs could plan out the relevant training needed before biohazards and be aware of their capacities to use them in times of disasters or empower capacities for future actions. Moreover, biological hazard is a threatening disaster in Iran owing to the various cultural and religious mass gathering posing a high risk of occurrence,namely, the epidemic of influenza, hepatitis B and D,various types of haemorrhagic fevers and brucellosis.In response to these epidemics, Iran has used many strategies to cope with them. These include recruiting a surveillance system with mandatory reports of particular disease according to the guidelines of the ministry of health, training health personnel to prevent transmission of the disease in the community especially the high-risk group, using mass media to persuade community involvement in preventing transmission,controlling the vectors by using pesticides, educating people with high-risk job due to exposure to the disease source, preventing high-risk people in participating in Hajj (pilgrimage), mandatory vaccination if participating in certain religious occasions and educating hygiene habits like washing hands and using a mask in very crowded places. CBHOs participate in these actives to serve their covered population.44-48This tool can help them to assess their preparedness in biohazard and recognise the need of enhancing their capacity.

LIMITATIONS

It was not possible to check the reliability through testretest due to the low participation of CBHOs in the research. We will use some qualitative or mixed method studies to verify this tool in future. In comparison with the ASPR guidelines, our findings might reflect some potential item limitations regarding to ambiguous translation and wording sentences that would make difficult to answer for participants.

CONCLUSION

We collected data on CBHOs through the ministry of health. We used creditable mathematic calculation based on mature model and revised by experts’ opinion.Besides, this tool is used to measure the preparedness of CBHOs and their ability to participate in biohazard and identify their weaknesses. The tool could aid better understanding of the training and skills required for CBHOs to participate during hazards. Furthermore,CBOs can use this tool to participate in drills and practices. Finally, the preparedness tool can help CBOs improve their planning, training and infrastructure.The authors will verify credibility and extend its usability to improve its quality continuously.

AcknowledgementsThe authors would like to thank the members of the departments of health in disasters and emergencies in the medical university of Isfahan and Tehran for contributing helpful comments on developing a validated assessment tool.

ContributorsFR and MK contributed to the conception and design of the work; FR and MRM participated in data acquisition, analysis and interpretation. MHY and AA revised the work critically for important intellectual content.

FundingThe author(s) received no financial support for the research, authorship and/or publication of this article.

Competing interestsNone declared.

Patient consent for publicationNot required.

Provenance and peer reviewNot commissioned; externally peer reviewed.

Open accessThis is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially,and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http:// creativecommons. org/ licenses/ by- nc/ 4. 0/.

ORCID iDFatemeh Rezaei http:// orcid. org/ 0000- 0002- 2275- 7501