Each research topic outlined in the previous section should be evaluated to judge its merits for implementation. This evaluation can be based on a 3- point rating scale.
A group of experts may be employed to judge the merit of the topic, which will rank the proposed item based on this 3-point scaling. Each expert in his judgment will assign a score from 1 through 3 for each criterion.
A research topic receiving the highest score will be considered for implementation.
The scaling process is enumerated below in table;
|Criteria for Evaluating a Priority Research Topic|
|1. Not very relevant|||
|2. Somewhat relevant|||
|3. Very relevant|||
|Avoidance of duplication|
|1. Sufficient information already available|||
|2. Some information available but significant issues not covered|||
|3. No sound information available to base problem-solving|||
|1. Not feasible considering available resources and research environment|||
|2. Feasible considering available resources and research environment|||
|3. Very feasible considering available resources and research environment|||
|1. Not acceptable to high-level policymakers|||
|2. More or less acceptable|||
|3. Fully acceptable|||
|1. No chance of recommendations being implemented|||
|2. Some chance of recommendations being implemented|||
|3. Good chance of recommendations being implemented|||
|The urgency of data needed|
|1. Information not urgently needed|||
|2. Information could be used right away but a delay of some months would be acceptable|||
|3. Data very urgently needed for decision-making|||
|1. Maj or ethical problems|||
|2. Minor ethical problems|||
|3. No ethical problems|||
Suppose ten experts or expert groups were requested to judge the suitability of a research topic for its funding. They were also given a set of scores to be assigned to each criterion.
The scores are shown in the accompanying table, along with the total and average scores computed from the assigned scores:
|Table 10.2: Results of Research Topic Evaluation|
|Avoidance of duplication||2||6||2||20||2.0|
|Urgency of data||2||2||6||24||2.4|
|(2.5+=25H0), (25’=lxl+2×3+3×6), (2.1″=149-70)|
It appears from the scores that the proposed topic is very relevant (which received the highest score, 25). The next higher score is due to the urgency of data needed (24).
This score indicates that we need relevant data on an urgent basis. It may, however, be difficult to obtain approval of the policymakers since this criterion received the lowest score (19)
Nevertheless, if the implementing authority decides a threshold score for each criterion (say 18, in advance), then the proposed topic may be given approval for execution. When compared with other areas of research, then the total score (here 149) may be taken into consideration for the selection of the research topic.
When the number of experts is exceedingly high, an average may be computed for ease of comprehension simply by dividing the weighted sums in the last column by the number of experts (in this example, 10).
The overall average is 2.1, which is obtained by dividing the total score by 10×7=70.
If five such research areas or topics are to be evaluated, one with the highest score will qualify to be included in the list of propriety research, for which proposals may be invited.
Note that the maximum score that could be achieved is 210 when all the ten evaluators assign score 3 to each of the criteria, while the minimum is 70 when all of them assign score 1 to each of the requirements.
If, based on the evaluation, the research topic is considered to be worth funding, the fund giving agency may decide to contact out the study and call for submission of proposals from the firms or researchers.