A case study of methods to minimize Research Bias

A case study of methods to minimize Research Bias

Muhammad Shahid Imran (Research Scholar, Mphil)

University of Central Punjab

dj.shaa@yahoo.com

Dr Zakria Zakar, Professor

University of the Punjab

mzzakir@yahoo.com

Introduction

The term bias in research is by no means straightforward in meaning. Usually it refers to systematic error or deviation from actual results due to different intervening and mediating variables mostly affecting the objectivity of research. It can also be conscious or unconscious impact of researcher’s personal likeness or failure to interpret results which may lead to erroneous conclusions. 

Researchers see bias as big obstacle in achieving credibility and accuracy in research. As influence of personal or ideological beliefs on their methods to achieve a certain results can endanger principle of impartiality. (Research Integrity, Scientific Misconduct, 2014)

While conducting an empirical social science research, a researcher may encounter bias at any of the many stages involved in research procedure, from topic selection to interpretation and generalization of results. (Henry, 1995)

Social Science is believed as field which is not governed by any sacred values and which gives us scientific method to explain social behavior, regardless of our political beliefs, race, gender, and class. However, personal beliefs of social scientists and many other factors could affect the decision of what to study or interpretations, sometimes unintentionally, as a research bias. ( The Situationist Staff, 2011)

So social scientist and other researchers community need methodological safeguards against these biases. (Hammersley & Gomm, 1997)

In this article we will try to explore some epistemological positions regarding different methods used to minimize personal, social and ideological biases in social science research and review associated concepts.  It is important to note that we cannot accurately divide biases in personal, social or ideological categories, however writer tried to outline nearest category for different type of biases.

As a research community, we take methodological safeguard against these biases seriously, because a research driven more by ideological or theological gaze than valid insight is destined to insignificance, and will not stand to empirical replication causing scientific peers  to reject that paper for publication. (Hammersley, Taking sides in social research: Essays on partisanship and bias, 2000)

Personal Bias

Bias in topic selection

Social research mostly an Interpretive research. So bias begins from  influences of researcher’s interests. The beliefs of researchers affects not only its choice of methodology, research design and instrumentation but also selection of research topic. (Mehra, 2002)

For Example,

A researcher may have his own likeness or dislikeness about America and its war expeditions. He may select topics like ‘American values building democratic values in Pakistani society’ in case of likeness and ‘impact of America’s misguided clash of civilizations on Pakistani society’ in case of dislikeness.

To eliminate this kind of bias researcher should try to select topics which could benefit his subject or field of study by setting his interests and likeness aside.

Bias in sampling

Selection bias may also be introduced while sampling as sample may be not true representative of population. For example, Researcher may interested in finding the affect of social media use on youth and it may selected universities to collect data from and come up with a sample from these universities but it may not be true representative as may be majority of youth was not attending university education and may be social media affect on university youth varies from the youth in playground or streets. (Conway & Lance, 2010)

Sampling sometimes encountered with self selection bias. (Collier & Mahoney, 1996)

For example,

A researcher wanted to conduct a research on news consumption patterns among Lahore audience and it used a telephone machine to conduct survey interviews. It is possible that individuals who are highly motivated to respond, resulting in overrepresentation, and individuals that are indifferent or not interested are less likely to respond and may have created self bias.

So we try to use large sample size applying random Independent sampling techniques so that every unit of population may have equal chances of selection in research sample.

Measurement Bias

Measurement bias may result due to poorly measuring which should actually be measured. (Cohn, 1997)

In measurements, researcher could also introduce statistical bias by selecting optimal actions for solving research questions.

For example.

Researcher want to know mortality rate in Lahore in specific time period. In survey. Poorly trained researcher may ask about the deaths taken place before that specific time period. The results of this results will not be actual reflection of research findings as it will lead to overvalue of the mortality rate because deaths which should not be included are included.

To omit this kind of bias researcher should properly trained in gathering results and measurement and there should be control group study before conducting actual research. (London School of Hygiene and Tropical Medicine, 2009)

Research method

Qualitative research may involve in-depth analysis, focus group or case study and sometime observation of respondent or researcher in closed and controlled environment. The results of qualitative studies may not be generalized due to small sample size.  (Monique, Hutter, & Bailey, 2010)

Qualitative research base on researcher’s understanding of particular phenomena like we do study in Phenomenology which deals with study of social occurrence from first person point of view. (Shuttleworth)

For example,

Researcher wanting to know how domestic violence affect a child’s behavior. He may collect the data by interviewing parents, teachers and observing particular child in case study, however he cannot generalize these results to other children due to change in their background, type of violence and other intervening factors.

As we use qualitative research mostly when we do not know much about the topic and have no guess about its possible findings, however, it is advised to use quantitative methods and to transform topics into more statistical terms which can be measurable through quantitative methods to avoid generalization problem and other biases. Like Filipe R. Campante, David H. Yanagizawa-Drott did in their article Does Religion Affect Economic Growth and Happiness? Evidence from Ramadan, for National Bureau Of Economic Research, in which they transform qualitative variables like happiness etc into measurable quantitative variables.

Writing tune bias

Researcher should use objective language avoiding subjective language by presenting results as they are in most fair way possible. A social scientist should avoid making judgmental statements like  saying something  gorgeous,  splendid or terrifying. Emotional phrases should also be eliminated from research articles, avoiding exaggeration like using very, really or most words to stress upon some point.

Researcher should avoid  first-person and second-person pronouns and use third person prounouns to discuss research results. For example researcher should avoid writing I think, I suppose which may predict a biased opinion.  (Leffler)

Ignorance Bias

Sometime researcher may not know which test should be applied to particular data, showing  insufficient knowledge of researcher which may cause ignorance bias.  (Mullane & Williams, 2013)

Researcher can avoid ignorance bias by properly studying and following similar studies and methods in research field.

Preference bias

When data has been collected, bias may be introduced as researcher interpret the data. It can be in term of control or omitted variables affecting research results.(King, 2010) Also researcher may prefer one results over other for his personal interests and choice.  (Wilholt, 2015)

When researcher failed to measure possible trait then there occurs method bias which has severe empirical consequences, mostly resulted due to lack of construct validity. (Jones, 2009)

For example,

A researcher want to know how many time a person take parts in social activities in his locality. So rather counting actually the number of occurrences he uses a questionnaire and ask respondent to fill that for his survey. So measuring self reporting rather than actual usage may not always alter results but have its effects researchers’ construct.

Convenience bias

Objectivity is an important part of any social science research and being impartial could result into wrong findings. As social science research is majorly based on people’s respond, the most important part is to reduce researcher’s personal bias.  Personal bias occurs when while collecting data researcher eyes on his/her convenience. (Reginiel, 2013)

For example,

A social scientist want to research people’s response about banned books in Pakistan, so rather collecting a random sample  he may just interview anyone gets his way. This research will not do any good and cannot be generalized as responses are not representative of population under study.

Bias elimination by testing External and Internal Validity of Research

Last but not the least, social science experts suggest that during research a researcher must verify that it’s research is valid externally and internally. This will also help minimizing bias in research. (Wimmer & Dominik, 1987)

For a research to be internally valid a researcher must control over research conditions to avoid extraneous variables to result into incorrect outcome.

For example,

Researcher come to an conclusion that children who are heavy viewers perform low at their study grades, testing it on Internal validity scales we may come to know that it is not only TV viewing which affects school grades but factors like IQ level, parents help and time at leisure activities could also affect grades. So we have to watch out these variables in a most possible controlled way.

On the other hand external validity is how easily and how well we can generalize research results.

Subject selection, instrumentation, and experimental conditions badly affect the research results making them to subject to that particular sample tested through research and cannot be generalized. Using random and heterogeneous samples and replicating the research study several time may solve this external validity issue so that results could be generalized over large population.

Ideological Bias

Research can also be affected by existing beliefs of researcher and respondents, their religious, political and ethnical stances on specific research topics result in to ideological biases in research. (Ziegler, 2009)

For example,

Researcher  wanted to conduct a research about masses perception of moratorium on capital punishment or Death penalty in Pakistan and researcher has inclination in favour of death penalty. Researcher is likely to introduce ideological by explaining topic in a way most favourable to researcher’s own set of beliefs. Like in the said research he may explain that previous research had found the capital punishment deters crime in society. After hearing participants were more likely to answer against any possibility of moratorium on execution.

It is also noted that respondents are also more sceptical of research findings which can go against their own attitude or their political ideology about that particular issue. Also when researcher exclude possible alternatives in options given in survey questionnaire it is being prey to ideological bias. (Vembunarayanan, 2013)

If they feel something could be against their set beliefs they will be supercritical of that topic and will try to find minor flaws, magnifying them into big reasons for responses in a particular way suited to their set values and ideology.

For example,

A researcher trying to explore effects of Turkish dramas on Pakistani Culture will treat that influence as a threat and may design all questions in a way which shows them as a biggest threat Pakistan currently facing.

Political and religious affiliations and preferences also influence researchers’  attitude and research design. Mostly media researchers may caught to these biases due to number of reasons. Sometime, politicians and religious bodies fund studies to get favourable results.

For example,

A researcher who is also a member of x political party going to conduct a research on voters’ behavior towards developmental work. He may introduce bias by mixing sympathetic or positive points or by quoting doubtful data about development works to respondents to get positive results.

To avoid this kind of bias researcher should rely on trusted data sets verified from multiple sources and by remaining objective in interpretations.  

For example,

Suppose a researcher believes that women should not work outside home and he is going to research on abuse at work places. If he sticks to his that very thought of keeping women inside home he will try to prove that outer environment is highly dangerous for women, instead of sticking to data. So researcher should stick to what data shows and try not to manipulate results.

Social Bias

Omission Bias

A type of social bias is omission bias. It is when researcher omits some of social classes from its research deliberately.

For example,

A researcher is going to study violations of minority human rights in Pakistan and exclude some minority groups from his data by studying only one marginally better minority. The results will be misleading.  Using only males, only whites or other ethnic groups could also be dangerous for generalization of results.

Procedural bias

Another social bias is Procedural bias, which occurs when researcher or research institution put undue pressure on respondent to complete the response in short time.

For example,

A research professor wants to study attitude of students towards social science subject and he asked them to fill a questionnaire quickly when they have ten minutes break after three hour long lecture, they will try to fill the questionnaire in rush without reading questions properly.

Social desirability bias

Another type of bias in social bias category is Social desirability bias occurs when survey respondents answers in a way suited to social norms and widely acceptable and is pressurized by societal taboos. (Norris)

For example,

A researcher is conducting research on domestic violence and asks men how many times they happen to beat their wives over domestic works, as beating or domestic violence is not considered good among society so respondent will try to under report this by selecting less number than actual. Similarly respondent may over-report about something which can build his image. Other types of such researchers may include sexual behavior, drug use and other in which respondent have to self report their behavior or action. 

In this kind of bias respondents try to draw favourable image of themselves. (Mortel, 2008)

Other methods to eliminate bias from research includes proper training of researcher, encouraging alternative hypothesis, enhancing feedback on results, and replication of study. However, not every method suitable for every topic, it may work for one but not for others. (MacCoun, Richard, & Goldman, 1998)

References

The Situationist Staff. (2011, March). Ideological Bias in Social Psychology? Retrieved July 2015, from https://thesituationist.wordpress.com/2011/03/02/ideological-bias-in-social-psychology/

Cohn, D. A. (1997). Minimizing Statistical Bias with Queries. In Advances in Neural Information Processing Systems (p. 417). MIT Press.

Collier, D., & Mahoney, J. (1996). Research Note Insights and Pitfalls: Selection Bias in Qualitative Research. World Politics , 1 (49), pp. 56-91.

Conway, J. M., & Lance, C. E. (2010). What reviewers should expect from authors regarding common method bias in organizational research. Journal of Business and Psychology (25), 325-334.

Hammersley, M. (2000). Taking sides in social research: Essays on partisanship and bias. Psychology Press.

Hammersley, M., & Gomm, R. (1997). Bias in social research. Sociological Research Online .

Henry, S. J. (1995). American legal realism and empirical social science. Univ of North Carolina Press.

Jones, A. B. (2009, September). Minimizing Method Bias Through Programmatic Research. MIS Quarterly .

King, G. (2010). A hard unsolved problem? Post-treatment bias in big social science questions. In Hard Problems in Social Science Symposium. Harvard University.

Leffler, S. (n.d.). Objective Writing Tips: Keeping Your Research Paper Free of Bias. Retrieved July 22, 2015, from Write content solution: http://www.write.com/objective-writing-tips-keeping-your-research-paper-free-of-bias/

London School of Hygiene and Tropical Medicine. (2009). London School of Hygiene and Tropical Medicine. Retrieved 2015, from The use of epidemiological tools in conflict-affected populations: open-access educational resources for policy-makers: http://conflict.lshtm.ac.uk/page_43.htm

MacCoun, R. J., & Paletz, S. (2009). Citizens’ Perceptions of Ideological Bias in Research. Political Psychology, Vol. 30, No. 1, , 30 (1).

MacCoun, R. J., Richard, & Goldman, R. (1998). Biases in the Interpretation and Use Of Research results. Annual review psychology .

Mehra, B. (2002). Bias in qualitative research: Voices from an online classroom. The Qualitative Report , 1-19.

Monique, H., Hutter, I., & Bailey, A. (2010). Qualitative research methods. Sage.

Mortel, T. F. (2008). Faking it: social desirability response bias in Self report research. ePublications@SCU (40).

Mullane, K., & Williams, M. (2013, September 17). Bias in research: the rule rather than the exception? Retrieved July 12, 2015, from Editors' Update 17 Sep 2013 /: http://editorsupdate.elsevier.com/issue-40-september-2013/bias-in-research-the-rule-rather-than-the-exception

Norris, N. (n.d.). Error, Bias and Validity in Qualitative Research . Norwich, United Kingdom: University of East Anglia.

Reginiel, P. (2013, August 12). How to reduce research bias in social research. Retrieved from Simply educate me: http://simplyeducate.me/2013/08/12/how-to-reduce-researcher-bias-in-social-research/

Research Integrity, Scientific Misconduct. (2014, August 11). How to Deal with Bias in Academic Research? Retrieved from Enago Blog: http://www.enago.com/blog/bias-in-academic-research/#ixzz3gZ7nRMQq

Shuttleworth, M. (n.d.). Research Bias. Retrieved July 2, 2015, from Explorable: https://explorable.com/research-bias

Vembunarayanan, J. (2013, July). Ideological Bias. Retrieved July 2015, from Seeking Wisdom: https://janav.wordpress.com/2013/07/25/ideological-bias/

Wilholt, T. (2015, August). Bias and values in scientific research . Studies in History and Philosophy of Science Part A , 52.

Wimmer, R. D., & Dominik, J. R. (1987). Mass Media Research: An introduction. Wadsworth Publishing Company.

Ziegler, T. (2009, February). Pure Science: Seeing ideological bias in research findings. Retrieved July 2015, from Cricky: http://blogs.crikey.com.au/purepoison/2009/02/24/pure-science-seeing-ideological-bias-in-research-findings/