Delphi Study

Description:

The Delphi group approach is a technique for gathering data that is similar to focus groups. Its value is that unlike focus groups, Delphi groups do not have to physically meet. The Delphi technique is a method of generating ideas and facilitating consensus among individuals who have special knowledge to share, but who are not always in contact with each other. A Delphi study carefully selects individuals who have knowledge necessary to analyse a specific problem.

Most often, a Delphi study is conducted through the mail, by telephone, and sometimes by personal interviews. However, this technique can also be used with faxes and email. Initially, the participants do not interact with each other. Through the efforts of one facilitator, who serves as a clearinghouse, the panellists see and react to each others ideas. Through a series of surveys, they share and generate new ideas based on an emerging consensus among the panel members.1

Nehiley says ‘the Delphi technique is an innovative way to involve busy experts and specialists who may not be able to come together to brainstorm, but who nevertheless need to interact with each other to generate new ideas’. Using email, one central contact person (who may be conducting research) will send questions and background information to individuals who have been selected on the basis of the relevance of their expertise. These people will reply, stating their thoughts on the topic. The researcher or facilitator will then compile these ideas to develop a concrete proposal, set of guidelines, or wording for an agreement, and will send this out again for comment. The process is continued until agreement on the wording or process or action to be taken has been reached.

Objectives:

A Delphi study aims to engage a large number of experts and/or stakeholders in a process of coming to agreement without necessitating their leaving their usual domain. This usually involves circulating documents or options papers by email or post so that all comments and suggestions can be noted.

Outcomes:

The Delphi study process should lead to an agreed set of guidelines and/or recommendations that includes the input of all relevant areas of expertise, regardless of how geographically far-flung this network might be.

Uses/strengths:

  • Allows sharing of ideas and consensus decision making by a large number of stakeholders who are geographically distanced.
  • Can be used when the issue is complex.
  • Works well to produce a consensus decision.
  • Provides a transparent and democratic technique.
  • Can deal with quite technical issues.
  • Offers convenience to participants, as they can contribute from their own office or home.

Special considerations/weaknesses:

  • The process can be expensive to run.
  • Large amounts of data need to be assessed and distributed.
  • Takes time for the organisers (can run for several months).
  • Participant commitment may falter if the process takes too long or they have other commitments.

Resources required :

  • Staffing
  • Relevant communication media
  • Relevant technical information needs to be made available to participants

Can be used for:

  • Engage community
  • Communicate an issue
  • Build alliances, consensus

Number of people required to help organise:

  • Large (> 12 people)
  • Medium (2-12 people)
  • Individual

Audience size:

  • Large (> 30)
  • Medium (10-30)
  • Small (<10)

Time required:

  • Medium (6 weeks - 6 months)

Skill level/support required:

  • Medium (Computer & other expertise)

Cost:

  • Low (< AUD$1,000)

Participation level:

  • High (Stakeholders participate in decision)

Innovation level:

  • Medium (Some new elements)

Method:

The following steps are necessary to conduct an effective Delphi study2 (McElreath 2001):

  1. Identify a panel of experts or specialists by soliciting nominations from specialists or individuals appropriate to serve on the Delphi panel. Cooperation and participation is improved significantly when prospective panellists are told how they were nominated by their peers. The panellists’ primary qualification should be their specialist knowledge. This knowledge can be gained through experience (e.g. readers of a certain publication) or specialist knowledge (e.g. safety engineers). Another key qualification is that panellists be willing to share their information (e.g. non-competitors). The terms of reference of the study need to be described to the panellists at this time.
  2. Invite an appropriate number of panellists to participate – 30-50 individuals should be members of the final panel. This is large enough to see patterns in responses, but not so large as to overwhelm the facilitator or researcher, who must sift through all of the responses individually. The invitation should explain what is expected from each panel member in terms of time and effort to complete each wave of the Delphi study.
  3. Prepare and distribute the initial survey instrument. The initial survey may contain open-ended probes or specific closed-ended questions, depending on the focus of the research.
  4. Receive and analyse the first responses. Compile the responses by question, with only minor editing as necessary for clarity and consistency. If open-ended questions were used extensively, then it may be necessary to analyse and present the first set of responses within an appropriate theoretical framework, typology, or outline.
  5. Prepare and distribute the second survey instrument. Most often panellists are asked, with this second wave enquiry, to clarify and rank order survey items suggested during the first wave. When the panellists receive the second survey instrument, it will be the first time they will have seen the responses of the other panel members. It is often appropriate at this time to ask for additional ideas, clarifications, and elaborations based on the initial survey responses.
  6. Receive and analyse the second lot of responses (second wave of data). If the initial questions were open-ended and the second wave asked for clarifications and elaborations, the analysis of the second wave of data can be very challenging because it requires numerous subjective decisions about rewording and revising the initial responses. Care should be exerted to include all of the new ideas and suggestions, for the main purpose of the Delphi study is to generate new ideas.
  7. Prepare and distribute the third survey instrument. Most often, panellists are asked, with the third wave, to rank order and clarify the new set of revised survey items.
  8. Receive and analyse the third wave of data. Often by this stage, the analysis is less subjective and judgmental, and more quantitative and objective.
  9. Repeat the process with additional waves, if necessary. For example, sometimes certain priority items are selected for more in-depth treatment by the Delphi panellists, who may be asked to propose answers to questions or short-range strategies for long-range goals, and so on.
  10. Prepare and distribute a final report to panel members. One of the motivations for participating in a Delphi panel, particularly for specialists, is to learn firsthand, before others, what the results of the Delphi study are.
  11. The final report is acted upon according to the initial terms of reference.

References:

  • Department of Public Health (Flinders University) & South Australian Community Health Research Unit (2000) Improving health services through consumer participation: a resource guide for organisations, Commonwealth Department of Health and Aged Care, Canberra
  • McElreath, M. (2001) Managing Systematic and Ethical Public Relations Programs, cited in Nehiley, JM (2001) ‘How to conduct a Delphi study’ [no details available]
  • Nehiley, JM (2001) ‘How to conduct a Delphi study’ [no details available]
  • http://www.participateinhealth.org.au/how/practical_tools.htm

__________

1 Nehiley, JM (2001) ‘How to conduct a Delphi study’ [no details available]

2McElreath, M (2001) Managing systematic and ethical public relations programs, cited in Nehiley, JM. (2001) ‘How to conduct a Delphi study’ [no details available]