Wednesday September 9, 2009 10:00

CompInfoFuture Homework 3: The Delphi Method

Posted by chrishota

1. If your question was one of those that the class used the Delphi method on, find the median and the range of the middle 50% of the responses. Using a graphics editor of your choice (even paint works for this), make a graph that is analogous or similar to the one in the lecture notes, showing the total range, middle 50% range, and median. If your question was not discussed in class, then (1) figure out a way of saying the question that will hopefully work well when we apply the Delphi method to it in class, and (2) explain why you designed the question the way you did.

  • My question was not asked in class, but it would be: “How long will Homo sapiens sapiens be the dominant species on planet Earth?” I ask this question because it is disruptive: not many people regularly think about the long-term survivability of the species, and maybe they should. I’ve recently been following The Long Now lecture series, which is all about different experts’ take on how to survive the next 10,000 years.
  • I designed this question because we only rose to dominance on the planet a short while ago (200,000 years ago on a planet with 4,540,000,000 years of history).  Also, there’s a pretty hard limit to the end date, and that is the end date of the planet itself, some 7,500,000,000 billion years distant (according to estimates).

2. Read up on the Delphi method on the Web (or the library). Explain how the process that we went through in class differs from the process as described in the sources you found.

  • From Rowe and Wright (1999): The Delphi technique as a forecasting tool: issues and analysis. International Journal of Forecasting, Volume 15, Issue 4, October 1999: “The Delphi method is a systematic, interactive forecasting method which relies on a panel of independent experts. The carefully selected experts answer questionnaires in two or more rounds. After each round, a facilitator provides an anonymous summary of the experts’ forecasts from the previous round as well as the reasons they provided for their judgments. Thus, experts are encouraged to revise their earlier answers in light of the replies of other members of their panel. It is believed that during this process the range of the answers will decrease and the group will converge towards the “correct” answer. Finally, the process is stopped after a pre-defined stop criterion (e.g. number of rounds, achievement of consensus, stability of results) and the mean or median scores of the final rounds determine the results.”
  • The process we performed in class differs from the “official” Delphi method in a few respects. First, I don’t believe we classmates meet the standard for “independent experts” who were “carefully selected.” Second, our predictions and reasoning were not anonymous; we were able to interact with the person providing the reasoning. We also chose (I think arbitrarily) the stop condition of 2 rounds of voting. Perhaps with more rounds, our answers would’ve converged even more.

3. Based on what you can find about the Delphi method, what shortcomings, risks, or other weaknesses do you see for the process that we followed in class? Also do you see a way to fix some of these?

  • The process we followed in class has plenty of shortcomings. Usually, anonymity of the participants is maintained, so that the “bandwagon effect” or “halo effect“ is minimized. Additionally, using the process in a class is not the best idea: the Delphi process must be interactive; it really only works if the participants are willing to participate, and are truly experts in the field being forecasted.
  • Narrowing the topics being forecasted would help improve the accuracy of our predictions; perhaps a suitable topic would be “walking at UALR”—something we all have to deal with. Also, if it were compulsory to make feedback on the suggestions, anonymously, there might be some better discussion and possibly convergence on a single prediction.

Comments are closed.