Evaluating Proposed Research Topic

Each research topic outlined in the previous section should be evaluated to judge its merits for implementation. This evaluation can be based on a 3-point rating scale.

A group of experts may be employed to judge the merit of the topic, which will rank the proposed item based on this 3-point scaling. Each expert, in his judgment, will assign a score from 1 through 3 for each criterion.

A research topic receiving the highest score will be considered for implementation. The scaling process is enumerated below in the table;

Criteria for Evaluating a Priority Research Topic
Criteria/opinions	Scores
Relevance
1. Not very relevant	[1]
2. Somewhat relevant	[2]
3. Very relevant	[3]
Avoidance of duplication
1. Sufficient information already available	[1]
2. Some information available, but significant issues not covered	[2]
3. No sound information available to base problem-solving	[3]
Feasibility
1. Not feasible considering available resources and research environment	[1]
2. Feasible considering available resources and research environment	[2]
3. Very feasible considering available resources and research environment	[3]
Political acceptability
1. Not acceptable to high-level policymakers	[1]
2. More or less acceptable	[2]
3. Fully acceptable	[3]
Applicability
1. No chance of recommendations being implemented	[1]
2. Some chance of recommendations being implemented	[2]
3. Good chance of recommendations being implemented	[3]
The urgency of data needed
1. Information not urgently needed	[1]
2. Information could be used right away but a delay of some months would be acceptable	[2]
3. Data very urgently needed for decision-making	[3]
Ethical acceptability
1. Maj or ethical problems	[1]
2. Minor ethical problems	[2]
3. No ethical problems	[3]

Example

Suppose ten experts or expert groups were requested to judge the suitability of a research topic for its funding. They were also given a set of scores to be assigned to each criterion.

The scores are shown in the accompanying table, along with the total and average scores computed from the assigned scores:

Table 10.2: Results of Research Topic Evaluation
	Scores			Weighted
Criteria	1	2	3	Sum	Average
Relevance	1	3	6	25′	2.5+
Avoidance of duplication	2	6	2	20	2.0
Feasibility	4	1	5	21	2.1
Political acceptability	3	5	2	19	1.9
Applicability	4	1	5	21	2.1
Urgency of data	2	2	6	24	2.4
Ethical acceptability	3	5	2	19	1.9
Total	–	–	–	149	2.1″
(2.5+=25H0), (25’=lxl+2×3+3×6), (2.1″=149-70)

The scores show that the proposed topic is very relevant (which received the highest score, 25). The next higher score is due to the urgency of data needed (24).

This score indicates that we need relevant data on an urgent basis. It may, however, be difficult to obtain approval of the policymakers since this criterion received the lowest score (19)

Nevertheless, if the implementing authority decides a threshold score for each criterion (say 18, in advance), then the proposed topic may be approved for execution. When compared with other areas of research, then the total score (here 149) may be taken into consideration for the selection of the research topic.

When the number of experts is exceedingly high, an average may be computed for ease of comprehension simply by dividing the weighted sums in the last column by the number of experts (in this example, 10).

The overall average is 2.1, which is obtained by dividing the total score by 10×7=70.

Suppose five such research areas or topics are to be evaluated. In that case, one with the highest score will qualify to be included in the list of propriety research, for which proposals may be invited.

Note that the maximum score that could be achieved is 210 when all the ten evaluators assign a score of 3 to each criterion, while the minimum is 70 when all of them assign a score 1 to each requirement.

If, based on the evaluation, the research topic is considered to be worth funding, the fund-giving agency may decide to contact the study and call for the submission of proposals from the firms or researchers.