4.1 Field Research
Whilst answering the research question and research hypothesis would answer the main objectives of the study, secondary objectives included establishing a cross-section of views on whether it was a good idea to incentivise collaboration and the level of increased effectiveness that collaboration might be expected to bring to the New Zealand charitable sector.
The research aimed also to establish causality by distinguishing between true incentives (i.e. an incitement, provocation, ‘spur’”) that caused the desired collaboration in contrast to aspects that were features of successful collaborations but not necessarily instigators in the creation of the collaboration.
The research methodology also needed to provide data and support analysis that would enable generalisation and thereby prompt further related research topics and areas of application.
4.2 Key Issues in Field Research Methodology
A number of key issues were identified that influenced research method, design and techniques.
Large number of questions - The literature review identified 19 incentives that could potentially be used to encourage collaboration in the New Zealand charitable sector. These needed to be individually explored in depth.
Complexity – Many of the identified incentives were complex at a conceptual or practical application level. Some incentives were quite similar and required detailed explanation of their distinctions.
Open-ended questions – Since many of the incentive ideas proposed were anticipated to be relatively novel to respondents in the New Zealand charitable sector, it was important that open questions be used to add depth to answers and assumptions.
Sensitive questions – For the incentive ideas that had already been trialled in New Zealand, it was hoped that, in some cases, foundations and charities might be prepared to supplement their information with examples from their experience. Since this was likely to draw upon negative as well as positive cases, it was important that the research methodology provided an environment where the information could be safely shared.
Differing perspectives – In addition to understanding how likely charities were to positively respond to the proposed incentives it was important to assess the opinions of the foundations and trusts who would apply the incentives. This was due to the fact that many had direct experience of the results of use of incentives and some could be considered key respondents in the area since they had already commissioned research on the topic, reviewed international case studies and reports or considered the topic deeply as part of their strategy and social change ‘theory of change’.
Sampling of the charitable sector – Over 25,000 charities of highly diverse characteristics in terms of their size, geographical coverage, areas of activity, number and type of beneficiaries and range of approaches.
Research bias – Since the author of this report is a chief executive of a community foundation and therefore both a fundraiser and a grantmaker who already worked closely with charities and other foundations there was the potential of bias from both interviewer and interviewee.
Question sequencing – If the issue of whether collaboration should be incentivised was considered prior to examination of the actual incentives it was likely to limit the depth of consideration of the individual incentives.
4.3 Field Research Method
In light of the issues outlined above and because the study would be drawing on intuition, opinions, imagination and judgment of respondents, it was determined that an exploratory, qualitative approach would be most appropriate, although consideration was given to a large quantitative survey following the initial qualitative work to provide statistical support for the key incentives identified as most effective. (After full assessment was made of the time required to interview a strong sample of charities and foundations on a qualitative basis, it was decided that this additional quantitative work should be reserved for further research work).
4.4 Field Research Techniques
Consideration was given to using focus groups or structured interviews or semi-structured interviews.
Focus groups were discounted due to the difficulty in presenting the incentives in an easy publicly expressible format (Jankowicz, 2005) and lack of control over question sequencing. It was also decided that individual interviews would allow more diversity of opinions that could be triangulated in the analysis phase.
The approach that was finally used had elements of both structured and semi-structured interviews, since it had a clear and ordered set of questions (particularly the ones centred on the 19 incentives under investigation) but also allowed for open discussion, development of themes, linking of issues and ideas in a relatively free form way particularly towards the end of the interview.
4.5 Field Research Questions
Using the research question, hypotheses and results of the literature search as an anchor, a structure for the interview was designed that took the respondent through a step-by-step qualitative research process;
- The unprompted views of the respondents on possible incentives were gathered.
- All incentives (including any new ones suggested by respondents) were ranked for potential effectiveness.
- Incentives identified as high potential were explored in depth to build a fuller picture including possible explanations, examples, strengths, weaknesses, other comments and linkages to other incentives.
- Broader discussion on the value and effectiveness of collaboration and its incentivisation.
This structure provided a natural and intuitive flow which zeroed in on incentives that might be effective in the New Zealand charitable sector and then explored the ramification of that in an expansive way. A single structure and set of questions was used for foundations and charities to assist in making direct comparisons between the opinions of the ‘funders’ and the ‘funded’ on the subject.
In order to convert the potential incentives into a respondent-friendly format that made them more accessible and understandable, a single page ‘Post-It’ display was created which had brief descriptions of the incentives and grouped them by type (see Appendix A – Incentives Grouped by Type). Word choice was checked to ensure that it did not introduce bias.
4.6 Respondent Sampling
‘Purposeful sampling’ was used to indentify senior respondents from both foundation and charity sectors.
In the case of the foundations, the small number of large philanthropic funders in New Zealand meant that eight interviews with senior executives in turn meant that the eight organisations interviewed represented over 15% of the philanthropic funding in the country in an average year (excluding the Lotteries Grants Board). The sample included the gaming trusts and statutory trusts. Interviews were also held with representatives from private family foundations including four that had recently jointly commissioned a major study on collaboration. (Although a range of philanthropic funders was interviewed, the generic term ‘foundations’ has been used to identify their comments in the Results and Recommendations sections of this report).
In the case of the charitable sector the nine interviews included Chief Executives and senior managers of charities of varying sizes, differing geographical coverage and a range of activities and beneficiaries in order to provide a diversity of perspective. (Similarly to foundations, although a range of not-for-profits were interviewed the generic term ‘charities’ has been used in the Results and Conclusions and Recommendations sections).
In the case of both philanthropic and charitable sector, respondents had already had a current relationship with the report author. Whilst this ensured that they were prequalified for the potential quality of their input, it also did mean that there was the potential of bias. In order to mitigate this risk of bias, additional respondents were recruited from the relevant ‘peak bodies’ – Philanthropy New Zealand for the foundations and the Charities Commission for the charities. Bias was further minimised through the consistent use of a standardised interview guide.
In total, 19 interviews were conducted. The interviewees were guaranteed anonymity to encourage openness on sensitive issues.
4.7 Interview Pilot
Prior to the pilot interview, an analysis grid and data recording template were developed to assist with the capture, review and analysis of interview information.
The pilot interview was conducted with the trustee of a major trust and recorded on tape. The interview experience was discussed with the interviewee at the end of the interview. The analysis grid and recording template were subsequently filled in.
In response to the pilot interview experience, minor amendments were made to the interview process. These included clearer introductory comments and the introduction of definitions for both ‘incentive’ and ‘collaboration’ to assist with understanding. A comments section was added to the strengths and weaknesses questions for the highest rating incentives.