Non-Experimental and Experimental Research: Research Study Design

A research design, also called study design, is the plan and structure specifying the methods and procedures for collecting and analyzing data with the ultimate goal of answering research questions and meeting the objectives of the study.

By plan, we mean the overall scheme or program of research, a plan that describes how, when, and where the data are to be collected and analyzed.

By structure, we mean the conceptual framework used to specify the relationships among the study variables and answering the research questions. The nature and objectives of a study determine, to a large extent, the type of research design to be employed to conduct a study.

The design of a study defines the study type (e.g., descriptive, correlational, pre-experimental, truly experimental, or quasi-experimental), research problem, hypothesis, data collection methods, and analysis plan.

Specifically, a research design will be aimed at answering broadly the following questions: What technique or techniques will be used to gather data? This question raises the issue of whether a survey, an experiment, or any other method or methods will be employed to conduct a study.

Although there is no simple classification of research design that covers the variations found in practice, yet several classifications of the research designs are possible depending on the type of studies adopted.

From the standpoint of research strategies, two broad classifications of research designs are infrequent use.

These are non-experimental designs and experimental designs.

In this post, we will discuss the whole design strategy about two types of study: experimental study and non-experimental study.

Research Study Design: Non-Experimental and Experimental Research

Non-Experimental Study

A non-experimental study is one in which the researcher just describes and analyzes researchable problems without any manipulation or intervention of the situations.

Non-experimental studies include, among others, the following types of studies:

  • Exploratory studies.
  • Descriptive studies.
  • Causal studies.

Exploratory Studies

An exploratory study is a small-scale study of a relatively short duration, which is undertaken when little is known about a situation or problem. An exploratory study helps a researcher to

  • Diagnose a problem;
  • Search for alternatives;
  • Discover new ideas;
  • Develop and sharpen his concepts more clearly;
  • Establish priority among several alternatives;
  • Identify variables of interest;
  • Set research questions and objectives;
  • Formulate hypotheses;
  • Develop an operational definition of variables;
  • Improve his final research design.

The exploratory study helps to save time and money. If the problem appears not as important at first sight, a research project may be abandoned at the initial stage.

Exploratory study progressively narrows down the scope of the research topic, and transforms the undefined problems into defined ones, incorporating specific research objectives.

An exploratory study comes to an end when the researcher is fully convinced that he has established the major dimension of the research, and no additional research is needed to conduct the larger study.

Example#1

Tribal people in Bangladesh have some peculiar and uncommon background characteristics that distinguish them from the rest of the population.

Sporadic information tends to suggest that they have a large family size, low age at marriage, and high mortality.

Because of the fact that the researchers in the past did not have easy access to this population, the above conjectures could not be examined scientifically.

A small scale study was planned to examine these in the face of these peculiarities to launch a large scale survey.

The planned study is explorative, which will help the researchers to formulate objectives and hypotheses, keeping in view the above peculiarities regarding their distinct demographic characteristics.

Example#2

The government-supplied oral contraceptive pills (OCP) and condoms are provided free of charge through government field workers and clinics.

Even then, why a significant proportion of women in the lowest quintile prefer to purchase contraceptives produced by Social Marketing Company (SMC) rather than availing free services, is very critical and valuable information for SMC. To explore this, the SMC wants to conduct a study.

They intend to prepare a profile of the SMC brand contraceptive users of the poorest and poor quintile to gather their perception of SMC product and their views on the govt, supplied contraceptives.

We cite one more example from business research (Saleh, 1995)

Example#3

Entrepreneurship nowadays has become a focal point in the business community in both government and non-government agencies, and also in business education in Bangladesh.

This has led to a greatly accelerated effort among the researchers to undertake studies on the growth and development of entrepreneurship and small business.

Although there has been quite a good number of studies focusing on the characteristics of the entrepreneurs, yet there is still a great deal of mystery about the women in business and queries remain about their entrepreneurial characteristics, motivation, and success in business.

An exploratory study, therefore, is planned to be conducted with the sole objective of identifying the motivation of the women entrepreneurs for business and assessing their entrepreneurial skills.

The exploratory study offers an opportunity to obtain insights into the problem through four major ways:

  • Analyzing any existing documents or studies. This is secondary data analysis;
  • Sharing experiences with knowledgeable individuals. This is an experience survey;
  • Investigating the situations informally. This falls under a pilot study;
  • Conducting a case study. The case may be an individual or a group of individuals;
  • Designing a focus group.

Secondary Data Analysis

Electronic data processing dates back to about 50 years, and the large-scale collection and analysis of social science data are not much older.

In recent years, the data processing cost has considerably decreased owing to the availability of the increased facilities. The most emerging barrier that stands now is the cost of data collection for any scientific study. This is seriously limiting the research endeavor of the students and professionals.

More and more researchers, however, are overcoming this cost obstacle by engaging in secondary analysis-building research projects around re-analyzing data originally collected by someone else for another purpose.

Secondary data, sometimes also called historical data, are data previously collected and assembled for some project other than one at hand. Studies based on secondary data do not need access to respondents or subjects.

The process thus enables you to avoid the cost of data collection by producing a new set of findings out of old data.

In recent years, survey data are increasingly likely candidates for secondary analysis because of the volume of such data and because of their availability in an inexpensive and well-organized form.

The Bangladesh Bureau of Statistics (BBS), ICDDR’B, and National Institute of Population Research and Training (NIPORT) are the major sources of data for secondary analysis.

Scientists of various disciplines and students are taking advantage of this abundant database for their research.

The chief advantage of secondary data analysis is that data for such studies are almost always less expensive to collect than acquiring primary data.

Also, secondary analysis can be completed relatively more quickly since it involves less time in the collection procedure. These data are very often available on soft copies.

Studies based on secondary data can help you to explore and decide what further research needs to be done. It further contributes to enrich your research proposal with specific references and citations.

Analysis of available records may often be the only way to obtain quantitative data about the past.

As more and more survey data accumulate, trend studies comparing responses to similar survey questions asked over many years become more practical and valuable for testing or developing theory.

Secondary analysis can often be the basis for an important pilot study.

Before embarking on an extensive and costly study, researchers may use secondary analysis of past research to assess the soundness of their research design, to pretest the plausibility of their hypotheses, and to determine the strengths and weaknesses of formerly used indicators and question wordings.

More importantly, a secondary study may be used as a sole basis for a research study, since in many research situations, one cannot conduct primary research because of physical, legal, or cost constraints.

The most important limitation of secondary analysis is that the information may not meet your specific needs. The most common problems are that.

  • The data may be outdated;
  • There may be variations in the operational definition of terms;
  • The units of measurement may be different;
  • The research design and sampling design may not be known or may be inappropriate;
  • There may be no codebook available for re-analyzing the data.

Experience Survey

Although the objectives of an exploratory study may be accomplished with both qualitative and quantitative techniques, yet it relies more heavily on the qualitative techniques.

When studies based on secondary data become difficult, researchers may well profit by seeking information from persons experienced in the area of study, tapping into their collective memories and experiences. A survey, which involves such persons, is referred to as an experience survey.

In essence, they are the key informants (KI) with abundant experience in their area, and the interview with them is known as the key informant interview (K1I).

The purpose of surveying such experts and seeking their opinions is to help sharpen the research problems and clarifying concepts rather than develop conclusive evidence. The outcome of an experience survey may result in a new hypothesis, discarding the old one, or may give information about the practicality of doing the study.

Sharing experiences with the experts may indicate whether certain facilities are available or not, what factors need to be controlled, and who is supposed to cooperate in the study.

An experienced survey is usually informal and involves a small number of people who have been carefully selected. The investigating format to be used in the survey should be flexible enough so that we can explore various avenues that emerge during the interview.

Case Study

A case study is an exploratory social research methodology aimed at intensively investigating one or a few situations identical to the researcher’s problem situations.

Rather than using random samples and following a rigid protocol (strict set of rules) to examine a limited number of variables, case study methods involve an in-depth, longitudinal examination of a single instance or event: a case.

They provide a systematic way of looking at events, collecting data, analyzing data, and reporting the results.

As a result, the researcher may gain a sharpened understanding of why the instance happened as it did, and what might become important to look at more extensively in future research.

Case studies lend themselves to both generating and testing hypotheses.

When selecting a case for the case study, researchers often use information-oriented sampling, as opposed to random sampling. This is because the average case is often not the richest in the information.

Extreme or atypical cases reveal more information because they activate more basic mechanisms and more actors in the situation studied.

Since a case study places emphasis on detail, it provides valuable insight for problem-solving, evaluation, and strategy. In the case of studies, researchers are not trying to establish a representative probability sample, and no attempt is made to meet the minimum design requirements.

Despite these limitations, case studies have a significant scientific role.

A single, well-designed case study can provide a major challenge to theory and provide a source of new hypotheses and direction of research.

Pilot Study

A pilot study, when discussed in the context of an exploratory study, refers to a small-scale research study that uses sampling but does not apply the rigorous standard. A pilot study generates primary data, usually for qualitative analysis.

This feature distinguishes a pilot study from secondary data analysis, which gathers background information. A pilot study is a tentative study using relatively unstructured interviews of a handful of respondents or subjects who are similar to those who will be the target of the later survey.

A pilot study is often compared to a theatrical dress rehearsal, which, before a final theatre is staged. These studies are intended to allow the researchers to try out various possibilities before deciding which ones to adopt.

A pilot study nearly always results in a considerable improvement to the survey documents leading to a general increase in the efficiency of the research design.

Such studies can often stimulate new lines of inquiry, prompted by the researchers or unsolicited responses of the respondents or subjects.

They can also suggest new types of data that should be collected, point out and resolve ambiguities in the way that questions are being asked indicate modifications needed in the order of topic covered and help to eliminate fruitless lines of inquiry.

A well- planned pilot study offers an opportunity to the researchers based on its results, as to whether the main study is still worth to carry out or not. Any investigator, who contemplates an exclusive survey, should conduct the pilot study as an opportunity to discover and correct mistakes before they become serious and incurable.

Focus Group Discussion

A focus group discussion (FGD) is a way of reducing the amount of time and personnel required for conducting and analyzing in-depth study and yet getting detailed qualitative information by interviewing a panel of a relatively large number of respondents.

The focus group interview has become so popular that many researchers now consider it to be the only exploratory research tool.

Typically, the panel is made up of 6 to 10 respondents.

The panel is led by a trained moderator or facilitator who meets for 90 minutes to 2 hours at a designated time. The moderator uses group dynamics principles to focus or guide the group in an exchange of ideas, feelings, and experiences on a single topic.

The moderator introduces the topic by using a general discussion guide and encourages the group members to discuss the subjects among themselves.

In ideal situations, the group discussion will proceed uninterrupted.

It is often rewarding, depending on the subject matter of discussion, to form separate groups for different subsets of the population. This type of homogeneous grouping tends to promote more intense and freer discussions.

As an alternative to the face-to-face focus group discussion, it may also be conducted by telephone. This is particularly effective when;

  • It is difficult to reach the target group, particularly when the members of the group represent experts, professional, high-level executives, etc.
  • When the target group members are rarely found, and
  • When the group members are so sensitive that anonymity is warranted.

The primary advantage of the focus group interview as an exploratory research tool is its relative flexibility and its ability to quickly and inexpensively understand the core issue of the topic, especially when compared with the rigidity of a formal study.

However, because they are qualitative devices with limitations of sampling accuracy, results from focus group discussions should not be considered a replacement for quantitative analyses.

Descriptive Studies

Descriptive studies are those used to describe the characteristics of a population or phenomena.

The objective of descriptive study is to focus on ‘who,’ ‘what,’ ‘when’ and ‘how’ questions. The simplest descriptive study aims at

  • Describing phenomena or characteristics associated with a population by univariate questions;
  • Estimating the proportions of a population that have the characteristics outlined above, and
  • Discovering association (but not causation) among different variables.

Descriptive studies may be carried out on a small or a large scale. Such a study often may be completed within a few months or weeks or even within a few hours.

When its findings pertain to a smaller population and are of short duration, we may call it a descriptive case study or micro-study, and by nature, it is an explorative type study.

However, if one wishes to test whether the findings pertain to a larger population, a more extensive study has to be designed.

Some common variants of a descriptive study.

  • Cross-sectional study.
  • Longitudinal study.
  • Trend study.
  • Panel study.
  • Baseline study.
  • Impact Assessment study.
  • Feasibility study.

Cross-sectional study

It is a single unrepeated descriptive study aimed at studying a cross-section of the population at a single point in time.

By cross-section, we mean a broad sampling of persons of different ages, different educational levels, and different religions, and so on.

The cross-sectional study is sometimes referred to as the ‘snapshot approach’ because although the single study can provide a momentary representative portrait of a population, it cannot trace the process of changes.

A national census is a good example of a cross-sectional study. A population census provides enormous descriptive information on such cross-sectional characteristics as age, sex, religion, ethnicity, occupational composition, household structure, and the like.

These characteristics can be expressed in absolute terms (e.g., number of illiterate persons) or proportion (e.g., percentage of illiterate persons).

One might attempt to examine also the relationship between household structure and occupational composition by a simple cross-tabulation of these variables.

A cross-tabulation of the level of education and occupation may also reveal a close association. A census is considered a macro study since its unit of analysis is a large aggregate of persons covering a large geographical area.

A descriptive study may go much beyond the simple relationship, as we perceive above.

Such studies are more complex and involve studying inter-relationships of a large number of factors suggesting a multivariate analysis.

Such a descriptive study might indicate causal relationships between the dependent variables and the independent variables. This may ultimately suggest useful hypotheses.

Longitudinal study

A descriptive study may also be longitudinal. The difference between a cross-sectional study and a longitudinal study is in the way they deal with time.

Longitudinal studies are repeated over an extended period to measure the rate and degree of change occurring in patterns of response.

Trend study

One type of longitudinal study is the trend study, which consists of several successive surveys, each based on a different sample of subjects.

Such a study involves studying the same topic (for example, attitude towards the use of traditional methods of contraception) by re-interviewing over some time, but with no attempt to re-interview the same respondents each time.

Gallup polls are conducted in this way, and comparisons of the results of several different polls can be quite useful for analyzing trends.

Panel study

The panel study is also a longitudinal study and designed specifically to minimize the effects of repeated sampling error as encountered in trend study.

In a panel study, a sample or a panel is chosen, and the same group of respondents is re-surveyed at selected intervals. Thus the later responses of any subject or the sample as a whole can be directly compared to responses given at an earlier time.

Baseline study

A baseline study is a research in which data on pre-project socio­economic and business aspects are generated to facilitate the assessment of the future impact of project intervention.

A baseline survey is conducted in the absence of available published data on various socio­economic and business aspects.

Impact Assessment study

The research, which is undertaken to measure the quantitative benefits derived out of project intervention and qualitative changes that occurred due to intervention, is known as an impact assessment research. This type of research also provides information for identifying the negative impact of the project.

Assessment research is said to primarily involve characterizations-objective description, while evaluation research is said to involve characterizations and appraisals-determinations of merit and/or worth.

Feasibility study

This type of study is undertaken before starting any business enterprise or any business-related project to assess the technical, economic, market and financial viability of the project.

The issue of whether the project is socially desirable and environmentally acceptable is also taken into consideration.

Causal Studies

A causal study, also called explanatory or analytical study, attempts to establish causes or risk factors for certain problems.

Our concern in causal studies is to examine how one variable ‘affects,’ or is ‘responsible for’ changes in another variable. The first variable is the independent variable, and the latter one is the dependent variable.

While no one can ever be certain that variable A (say) causes variable B (say), one can gather some evidence that increases the belief that A leads to B.

Examine the following queries and try to guess if there is any association between A and B:

  • Is there a predicted co-variation between A and B? Do we find that A and B occur in the way we hypothesized? When A does not occur, is there also an absence of B? Or when there is less of A, does one find more or less of B? When such conditions of covariance exist, it is an indication of a possible causal connection between A and B.
  • Is the time order of events moving in the hypothesized direction? Does A occur before B? If we find that B occurs before A, we can have little confidence that?! causes.
  • Is it possible to eliminate other possible causes of B? Can we determine that C, D, and E do not co-vary with B in a way that suggests possible causal connections?

Let’s discuss three types of causal study:

  • Comparative studies,
  • Case-control studies, and
  • Cohort studies.

Comparative study

This is a study that has its main focus on comparing as well as describing groups.

In a study of malnutrition, for example, the researcher will not only describe the prevalence of malnutrition, but by comparing malnourished or well-nourished children, he will try to determine which socio-economic behavior and other independent variables have contributed to malnutrition.

In analyzing the results of a comparative study, the researcher must watch out confounding or intervening variables that may have a distorting effect on the true relationship between the dependent and independent variables.

Case-control study

A case-control study is a retrospective study that looks back in time to find the relative risk between a specific exposure (e.g., second-hand tobacco smoke) and an outcome (e.g., cancer).

The investigator compares one group of people with the problem, the cases, with another group without a problem, or who did not experience the event called a control group or comparison group.

The goal is to figure out the relationship between risk factors and disease or outcome and estimate the odds of an individual getting a disease or experiencing an event.

Case-control studies have four main steps:

  1. The study begins by enrolling people who already have a certain disease or outcome.
  2. A second control group of similar size is sampled, preferably from a population identical in every way except that they don’t have the disease or condition being studied. They should not be selected because of their exposure status.
  3. People are asked about their exposure to risk.
  4. Finally, an odds ratio is calculated.

In an epidemiological study, we find the exposure of each subject to the possible causative factor and see if this differs between the two groups. We cite an example here.

Doll and Hill (1950) carried out a case-control study into the etiology of lung cancer. Twenty London hospitals notified all patients admitted with carcinoma of the lung, the cases.

An interviewer visited the hospital to interview the cases, and at the same time, selected a patient with a diagnosis other than cancer, of the same sex, and within the same 5-year age group as the case, in the same hospital, at the same time, as control.

The accompanying table shows the relationship between smoking and lung cancer for these patients. A smoker was anyone who had smoked as much as one cigarette a day for as much as one year.

It appears that cases were more likely than controls to smoke cigarettes. Doll and Hill concluded that smoking is an important factor in developing carcinoma of the lung.

Case-control study-example

The case-control study is an attractive method of investigation because of its relative speed and cheapness compared to other approaches. However, there are difficulties in the selection of the cases, the selection of the controls, and obtaining the data. The matching of cases and controls has to be done with care.

There are difficulties, too in interpreting the results of a case-control study.

One is that case-control study is often retrospective, that is, we are starting with the present disease state, e.g., lung cancer, and relating it to the past, e.g., history of smoking. We may rely on the unreliable memories of the

subjects. This may lead both to random error among cases and controls and systematic recall bias, where one group, usually the cases, recalls events better than the others.

Cohort study

In a cohort study, also called a prospective study, we take a group of people, the cohort, and observe whether they have the suspected causal factor.

We then follow them over time and observe whether they develop the disease. This is a prospective study, as we start with the possible cause and see whether this leads to the disease in the future.

It is also longitudinal, meaning that subjects are studied at more than one time. A cohort study usually takes a long time, as we must wait for the future event to occur. It involves keeping track of large numbers of people, sometimes for many years.

Often it becomes necessary to include a large number of people in the sample to ensure that sufficient numbers will develop the disease to enable comparisons to be made between those with and without the factor.

A study may start with one large cohort.

After the cohort is selected, the researcher may then determine who is exposed to the risk factor (e.g., smoking) and who is not and follow the two groups over time to determine whether the study group develops a higher prevalence of lung cancer than the control group.

If it is not possible to select a cohort and divide it into a study group and a control group, two cohorts may be chosen, one in which the risk factor is present (study group) and one in which it is absent (control group).

In all other respects, the two groups should be as alike as possible.

The control group should be selected at the same time as the study group, and both should be followed with the same intensity.

Cohort studies are the only sure way to establish causal relationships.

However, they take a fairly long time than the case-control studies and are labor-intensive and, therefore, expensive.

The major problem is usually related to the identification of all cases in a study population, especially if the problem has a low incidence. The other problem is the problem of ‘censoring’ due to the inability to follow up with all persons included in the study over several years because of population movements or death.

The major difference between a case-control study and a cohort study is that in a case-control study, we select by problem status and look back to see what, in the past, might have caused the problem.

By contrast, in a cohort study, we wait to see whether the problem develops. The following diagrams represent the two types of study.

case control study and cohort study

The example below distinguishes the cohort study from a case-control study and a comparative study.

Example#1

Suppose we anticipate a causal relationship between the use of a certain water source and the incidence of diarrhea among children under- 5 years of age in a village with different water sources.

You can select a group of children under-5 years and check at regular intervals (e.g., every 2 weeks) whether the children have had diarrhea and how serious it was. Children using the suspected source and those using other sources of water supply will be compared with the incidence of diarrhea.

This example illustrates a cohort study.

Or

You may compare children who present themselves at a health center with diarrhea (cases) during a particular period with children presenting themselves of other complaints of roughly the same severity, for example with acute respiratory infections (controls) during the same time and determine which source of drinking water they had used.

This example illustrates a case-control study.

Or

You could interview mothers to determine how often their children have had diarrhea during, for example, the past month, obtain information on their sources of drinking water, and compare the source of drinking water of children who did and did not have diarrhea.

This is a comparative, also called a cross-sectional comparative study.

Experimental Study

An experimental study is one in which the researcher manipulates the situation and measures the outcome of his manipulation. This is in contrast with a correlational study, which has very little control over the research environment.

The experimental study exercises considerable control over the environment. This control over the research process allows the experimenter to attempt to establish causation rather than mere correlation. Thus the establishment of causation is the usual goal of the experiment.

The classical experimental study has three characteristics:

  • Manipulation;
  • Control and
  • Randomization.

Individuals in an experimental study are randomly selected and allocated to at least two groups. One group is subject to intervention (or manipulation or test stimulus), while the other group(s) is not.

In a true experimental study, the experimenter can measure the values of the dependent variable both before administering the stimulus (the pretest) and after administering it (the posttest).

The difference between these scores gives a rough indication of the effect of the causal variable. The group, to which the test stimulus or manipulation is administered, is called the experimental group. The group that does not receive the test stimulus is called the control group.

We further elaborate on the terms control and randomization.

By control, we mean, all factors except the independent variable must be held constant and not confounded with another variable (extraneous variable), that is not part of the study.

By randomization, we mean that the researcher takes care to randomly assign subjects to the control and experimental groups, meaning that each subject is given an equal chance of being assigned to either group.

The goal of all selection procedures for experimental and control groups is to make the groups as similar as possible in terms of the dependent variable and thus necessarily in terms of all factors affecting it.

Therefore, pretest scores for the experimental and control groups will ideally be identical or similar before the introduction of the test stimulus. Randomization does not necessarily ensure that pretest scores for the two groups will be identical.

However, it should ensure that whatever differences do remain, are random, by which we mean differences are chance outcomes.

The basic logic of experimentation is quite simple. The experimenter begins with a causal hypothesis, which states that one variable (the independent variable) causes changes in a second variable (the effect or dependent variable).

The next step is to;

  1. measure the dependent variable (pretest);
  2. introduce the independent variable to the situation or change its level if it is already present, and
  3. measure the dependent variable (posttest) to see whether there has been any resultant change in its value.

One important question that arises in an experimental study is: how does one separate the portion of a total change in pretest and posttest scores, which is caused by extraneous factors from the portion that is caused by the test stimuli?

This cannot be done with a. single group of subjects but can be accomplished with two groups if certain assumptions can be made.

The assumptions are

  • The subjects in the two groups (experimental and control) are identical in their characteristics;
  • The pretest plus any extraneous factors that affect one group will also affect the second group to the same degree.

The first assumption implies that the average pretest scores should be identical. In contrast, assumption 2 ascertains that the difference between pretest and posttest scores that is caused by extraneous factors is the same in each group.

If these assumptions hold, one pretests both groups but administers the causal stimulus to the experimental group.

The control group should show a change in the dependent variable that is attributable only to the extraneous variation. In contrast, the dependent variable in the experimental group should show a larger change, caused by extraneous variation plus the test stimulus.

By subtracting the extraneous change (change in the control group) from the total change in the experimental group, one can estimate the amount of change due to the causal stimulus.

A second and most straightforward approach to controlling assignment error is matching. It is a procedure for the assignment of subjects to groups; it ensures that each group of respondents is matched based on pertinent characteristics.

Suppose an experiment is to be conducted to examine if the mother’s education has any effect on nutritional knowledge. It is apprehended that age is a factor that might influence knowledge.

To control by matching, we need to be sure that the age distribution of the mothers is the same in all groups.

Although matching assures that the subjects in each group are similar to the matched characteristics, the researcher can never be sure that the subjects have been matched on all characteristics that could be important to the experiment.

The disadvantage of matching is that any subject who does not have a matching partner on all relevant characteristics cannot be assigned to either group and thus cannot be used in the experiment.

Self-selection is another thorny problem in selecting a control group.

People who choose to enter a program are likely to be different from those who do not, and the prior differences (in interest, aspiration, values, initiative, etc.) make post-program comparisons between ‘served’ and ‘unserved’ groups risky.

Self-selection problems can sometimes be overcome if the subjects of both experimental and control groups are selected from volunteers. Such selection can be thought of as a true experiment if the volunteers are randomly assigned to either group.

We emphasize here that randomization is the basic method by which equivalence between experimental and control groups is ensured. Experimental and control groups must be established so that they are equal.

It is best to assign subjects either to experimental or to control groups at random. If the assignments are made randomly, each group should receive its fair share of different factors.

Matching and control are useful, but they do not account for all unknowns. They are the supplemental ways of improving the quality of measurement, reducing the extraneous noise in the measurement.

 

Advantages and Disadvantages of Experimental Study

Experimental research is seen as true research by many scientists. Yet there are many advantages and disadvantages of experimental research. The advantages and disadvantages of any research are usually subjective as one cannot claim that advantage in one experiment will also be an advantage in another experiment.

Some people feel that human input is a disadvantage in these studies as humans do always have their thoughts and can manipulate the results.

There is also another thought that testing on humans is also a disadvantage as you cannot tell whether their answers or reactions are true or a show for the experiment.

We summarize below a few important advantages and disadvantages of an experimental study.

Advantages of Experimental Study

  1. In experimental studies, the researcher has control and ability to change the experiment if the answers are inconclusive. This allows for less time-wasting in experiments;
  2. In experimental studies, contamination from extraneous variables can be controlled more effectively than in other designs. This helps the researcher to isolate experimental variables and evaluate their impact over time;
  3. The cost and convenience of experimentation is much less compared to other methods;
  4. The experiment provides the opportunity for studying changes over time through repeated measurements. This replication leads to the discovery of an average effect of the independent variable across people, situations and times;

Disadvantages of Experimental Study

  1. Most social sciences research is conducted under an artificial environment. This is perhaps the main problem with using experimentation in social sciences where sufficient control is impossible in natural settings.
  2. It is sometimes impossible to control all the extraneous variables.
  3. With a large group of subjects, it is difficult to control the environment.
  4. Generalization from a non-probability sample can pose problems despite random assignment.
  5. Experimentation is most effectively targeted at problems of the present or immediate future. Experimental studies of the past are not feasible and predictions are not possible in experimental studies.
  6. Intervention and control that are the two important elements in experiments are sometimes hard to achieve when ethical issues are involved.
  7. Scientist manipulates values, as a result of which they may not be making a completely objective experiment.
  8. People can be influenced by what they see around them and may give answers that they think the researcher wants to hear rather than how they think and feel on a subject.

Properties of a Good Research Design

We enumerate below a few desirable properties of good research design. These are as follows:

  • A good research design is ethical research design.
  • A good research design is one that is capable of obtaining the most reliable and valid data
  • A good research design is one that is capable of measuring any odd events in any circumstances
  • A good research design is one that helps an investigator avoid making mistaken conclusions.
  • A good research design is one that can adequately control the various threats of validity, both internal and external.

Guidelines for Selecting a Good Research Design

The researchers often encounter problems in selecting an appropriate research design. Here are some guidelines that one can follow to choose a research design for his or her study.

  • Whenever possible try to create experimental and control groups by assigning cases randomly from a single population study group.
  • When a random assignment is not possible, try to find a comparison group that is nearly equivalent to the experimental group as possible.
  • When neither a randomly assigned control group nor a similar comparison group is available, try to use a time-series design that can provide information on trends before and after a program intervention (X)
  • If a time series design cannot be used, as a minimum and before a program starts, try to obtain baseline (pretest) information that can be compared against post-program information (a pretest-posttest design).
  • If baseline (pretest) information is unavailable, be aware that we will be limited in the type of analysis we can conduct. We should consider using multivariate analytic techniques.
  • Always keep in mind the issue of validity. Are the measurements true?. Do they do what they are supposed to do? Are there possible threats to validity (history, selection testing, maturation, mortality, or instrumentation) that might explain the results?

In all cases, the experimenter must take into consideration the ethical, practical, administrative, and technical issues.

More 'Research Design' Posts ⁄
Related Posts ⁄