The National Federation of Voluntary Bodies Providing Services to People with Intellectual Disabilities is the national umbrella organisation for voluntary/non-statutory agencies that provide direct services to people with intellectual disability in the Republic of Ireland. In 2009, as part of the National Federation’s Research Strategy 2008-2013, an audit was undertaken of all the research abstracts that had been submitted for inclusion on the National Federation’s website (www.fedvol.ie). The database, which is available free to the public, is a collection of intellectual disability-related research conducted within member organisations since 2000- Submission to the database is on a voluntary basis and at the time of the study it contained 98 article abstracts.
Aim of the audit
The aim of the audit which was conducted in 2009 was to
- Determine the performance of research activity in member organisations,
- Measure the types and prevalence of research methodologies, designs and data collection methods in this field of research,
- Help identify standards and develop quality indicators for research in methodologies relevant to intellectual disability research in Ireland.
The audit’s findings are relevant to researchers, National Federation member organisations, research department heads and researchers in services and any other groups involved in dissemination of research or carrying out research in intellectual disability. The findings have implications for practice and research, as well as describing the types of research taking place in services across the 62 member organisations.
There were two parts to the audit—the first part examined aspects of research design and methodology of members’ research, including;
- Data collection methods,
- Design types and other descriptors
- Quality indicators
- Research topics under investigation.
All the data gathered for part one of the audit was reported through a voluntary, self-reported, abstract template for the database. Submissions to the database had been active since 2000. Several calls for submissions to the database were also made during 2009. All abstracts submitted to the National Federation database since 2000 were included in the audit. This included a total of 98 abstracts. The 98 templates were analysed quantitatively using Microsoft Excel and SPSS, and qualitatively to extract common themes and underlying trends.
Part 2 of the audit was to ascertain the coverage and generalisability of the database to Irish intellectual disability research by comparison with other research databases available online. This part of the study tells us that the National Federation database represents 28% of Irish intellectual disability research available through big online databases such as IDAAL (Intellectual Disability and Allied Libraries) or other university online library systems. The database also contains a lot of project information and references not found elsewhere online (61%), indicating the value of this resource as a source of ‘grey’ or unpublished literature available to people through www.fedvol.ie.
What the audit tells us about intellectual disability research in Ireland
A popular trend in Irish intellectual disability research is the use of surveys, scales or questionnaires, both for qualitative and quantitative research designs. Few studies took account of research methods which account for and report non-verbal communication of participants, such as body language etc.
Another interesting finding was that qualitative and quantitative designs were used both separately and as combined research methodologies.
Other methods commonly employed in research studies included
- cross.sectional designs—giving descriptions and statistics for certain groups,
- evaluations—evaluating the effectiveness of training, workshops or interventions, and
- reviews—or secondary data analysis and reviews of the literature.
In particular, the audit revealed that Irish intellectual disability research in the database has had a large interest in the topic of staff and/ or service development. This could complement the large transitions that have been seen in service provision styles in more recent years.
Research can be labelled as either a quest to identify trends and correlations or to test and refine ideas that are already established, i.e. inductive or deductive logics, respectively. This audit highlights that the majority of studies recorded on the database used inductive designs. This identifies the database’s intellectual disability research as being explorative, attempting to look for observations and patterns. It also highlights the less.used design that validates findings and refines theory that has been developed. There were few experimental designs reported and this was indicated by the low use of control groups in the abstracts submitted to the database research.
RELIABILITY AND VALIDITY
The audit also examined whether researchers had established reliability or validity in their research studies. In 28% of studies this was mentioned, but there were 69% of abstracts where this was not referred to. A considerable proportion of the studies reported triangulative methods. This gives us an indication of quality and robustness of research methodology employed. This was not mentioned directly in the abstracts, but was referred to more often than reliability or validity indicators.
Finally, the use of validated measures in research studies was examined. This was a further indication of quality and reliability, but was hard to ascertain in all studies. The lack of information in this regard makes it difficult to ascertain a study’s reliability and validity, which is an important factor when reviewing and reading research. It is vital that the information relating to reliability and validity is clear when reading research abstracts, thereby allowing the reader to give weight to the study (or not).
THE PURPOSE OF RESEARCH IN INTELLECTUAL DISABILITY MEMBER ORGANISATIONS AS RECORDED BY THE DATABASE
The audit also sought to determine the reasons why research was conducted in member organisations by examining common themes recurring among the 98 abstracts.
What were the popular areas of study?
Social Life Development
|Service/Staff Development & training||34||27.6%|
|Employment & recruitment of staff||3||2.4%|
|Mental Health & Psychiatric Disorders||6||4.9%|
|Family support & family research||9||7.3%|
|Total||123 (98 articles)||100%|
Table 1 shows that ‘Service/ Staff Development and Training’ (at 28%) received the most attention of all the research areas in the database of returned abstracts. The themes were identified by using the keywords first to identify some recurring themes that could be used to group each key word. When each theme had been initially picked, the abstracts were read to see if the themes could be identified by the main keywords in the abstracts. The final list of themes was decided after the two rounds of analysis. For instance, ‘service/ staff development and training’ would identify staff training, new services evaluation, creation of guidelines that inform best practice or provide guidelines for staff, etc.
HOW THE DATA WAS COLLECTED
31% of the research studies gathered data directly from people with intellectual disability, rather than from staff or family alone. And 10% of the research used inclusive methods in the research design. This tells us that people with an intellectual disability have their voices heard directly or indirectly in some of the research. It appears, however, that the research is in the main designed for those involved in supporting people with an intellectual disability, i.e. staff, policy developers, family, etc.
GENERALISABILITY TO IRISH INTELLECTUAL DISABILITY RESEARCH
The fact that 38 of the database studies are available online indicates that a large section of the studies are not publically available. Google, IDAAL, NIHS and UL databases do not have many of the studies referenced that are available in the National Federation’s database. This tells us that the database is a valuable resource of research in conjunction with other sources and that the database has real value for the Irish research community. The database has a large collection of unpublished ‘grey’ literature available. From the audit we also found that 38% of the database studies are published in peer-reviewed journals.
Recommendations for quality research
The following recommendations arose from the audit of the National Federation Research Database.
- Researchers should identify, in their abstract, the reliability and validity measures used, in order to grab people’s attention, and draw attention to quality results.
- Researchers need to disseminate their research findings more widely. This would help to inform practice and policy. It would also improve access to knowledge for all intellectual disability stakeholders. There should be support to encourage researchers to achieve this. Researchers should be encouraged to publish results of studies in peer-reviewed journals.
- The question remains, how does research conducted inform practice? Does the research conducted get assimilated into the policies and practice of the host organisations? Further research is needed to determine the outcomes of these research studies and their applicability to practice.
- For the National Federation, we need to measure the quality of submissions to the database.
A checklist that could help standardise Irish abstracts in intellectual disability research to contain quality standards—such as brief mentions of reliability testing, the research design used, sampling methods, report generalisability, etc.—could be a useful way of developing the research submitted. These standards could make reading abstracts a better way of judging an article’s value in informing good practice.
From the findings of this audit, a number of questions arose, which were provoked, but not answered, by it. First of all, from the trends in the research submitted, we can say that the methods used, such as questionnaires, are cheaper, faster and easier to compare than some other methods, for instance participant observation. However, there are problems sometimes using these methods with people with intellectual disability. Some of the popular methods used do not take account of the accessibility of the research for the participants. These data collection methods can also reduce the descriptive power of the data collected and even reduce its reliability.
Secondly, do the lack of experimental design and the extent of inductive design have any effects on the quality of intellectual disability research? By not using experimental designs to verify interventions, can researchers be sure of best practice? Or is this a sign of how fast research designs are changing, indicating how research is focused on new methods, rather than acting as a buffer for established methods? Should there be a promotion of experimental design in this area of research? Or is there a difficulty in applying experimental designs to real.life settings, particularly with vulnerable persons such as those with an intellectual disability?
Other questions which arose from the audit were:
- Is research designed to be read by people with an intellectual disability?
- Are easy-to-read versions available?
- Is research mostly done in services and organisations or in colleges and learning institutions?
- Is the research being carried out because it is requested or needed in the organisation to inform practice, or as fulfilment for a university or college course?
The final question was what level of dissemination of the research takes place and how likely is the researcher to bring the research to peer review? If our members’ research were more widely-published in peer-reviewed journals, it could enhance the value of the research being conducted for the wider Irish and international research community.