Research Design, Strategies, and Methods
Research design refers to the plan, strategies and data collection methods used in a research project. Understanding this will help with assessing research literature, as well as with procuring a research project externally.
Have a Question or Urgent Need?
Have a look at the list of questions below to see if something there approximates where you’re at, and then click on the link in each for help:
- We need help with monitoring. (link to a tip sheet on where we start by identifying what it is they are looking to monitor, for whom, etc.)
- We would like to know more about documentation protocols such as, pedagogical documentation, interview protocols, etc.
- Exploring research methodologies
- Working with qualitative data
- Survey design & effective question design
- School and Board Improvement Planning – evidence (SIPSA, BIPSA)
- School Effectiveness Framework (SEF)
- School Improvement Learning Cycle (SILC)
- Research articles, literature reviews, jurisdictional scans
What is evidence?
The term ‘evidence’ includes, of course, research and evaluation findings, but also data and information, and professional expertise and experience. Consider this definition from Canadian Health Services Research Foundation:
Evidence is the information that comes closest to the facts of the matter. The form it takes depends on context. The findings of high quality, methodologically appropriate research are the most accurate evidence. Because research is often incomplete and sometimes contradictory or unavailable, other kinds of information are necessary supplements to, or stand-ins for, research. The evidence base for a decision is the multiple forms of evidence combined to balance rigour with expedience – while privileging the former over the latter.
This definition captures the essence of evidence, while acknowledging that research evidence competes with a variety of other factors, including personal attitudes, public expectations, political biases, resource constraints and conflicting information (Burns & Schuller, 2007).
Types of Evidence
Evidence comes in many forms:
- systematic reviews
- empirical studies
- stakeholder feedback
- policy & program evaluations
- expert knowledge / practitioner knowledge
- data analysis
- costings of policy options
- public opinion
- pilot studies & case studies
We would also invite you to consider the following definitions developed by Daniel Perkins (2010):
- Good practice – ‘we’ve done it, we like it, and it feels like we make an impact’;
- Promising approaches – some positive findings but the evaluations are not consistent or rigorous enough to be sure;
- Research-based – the program or practice is based on sound theory informed by a growing body of empirical research;
- Evidence-based – the program or practice has been rigorously evaluated and has consistently been shown to work. (Perkins (2010) in Nutley (2013))
The Need for Evidence-Based Policy
There are many advantages of using research evidence in the policy-making process. In a governmental report on evidence-based policy in practice (Campbell et al., 2007), the authors conclude that policies and programs that are based on evidence:
- Are more likely to be better informed, more effective and less expensive than they otherwise might have been;
- Contribute to improved accountability about how public money was spent, and can inform risk assessment in order to avoid policy failure;
- Give policy makers and ministers confidence to defend their decisions in parliament, and to the public;
- Help to ensure cost effectiveness by identifying what works and what doesn’t, so that only programs and policies that are shown to improve outcomes for students continue to be funded; and
- Give a more detailed understanding of the situation, which in turn could help shape the direction of policy in ways that were not evident prior to the research being collected (Campbell et al., 2007, p. 17).
Another way to think about the need for evidence in programs and policies is to consider the following questions:
- How can we be most effective in our work?
- How can we maximize the likelihood of achieving our desired results?
- How are we addressing our challenges?
- How can we make programs more effective?
- What strategies are most likely to increase equity in our schools?
Ontario's Commitment to Evidence-Based Policy & Practice
The Ontario Ministry of Education is committed to developing and implementing policies, programs, and practices that are evidence-based, research-informed, and connected to provincial education goals. The ministry’s Research and Evaluation Strategy is the cornerstone of efforts to support the use of evidence and data in education decision-making. The strategy was designed to build research and evaluation capacity within the ministry, as well as to develop strategic partnerships with the wider research community. Key elements of the strategy include:
- applying research and evaluation to support evidence-based policy and practice;
- building individual and organizational capacity to access, use and conduct research;
- fostering research collaboration through networking and partnerships between and among ministry staff, researchers and educators across Ontario;
- communicating information about existing and new research activities and findings; and
- contributing to the growing body of research knowledge about educational policies, programs and practices.
The ministry has come a long way in its goal towards being evidence-based and there has been increased emphasis on the use of research to inform policy and practice in many ministry initiatives. Some examples include: The Literacy and Numeracy Secretariat’s What Works series, Monograph series, and Webcast series; Student Success/Learning to 18’s Student Engagement Strategy; the School Effectiveness Framework; the Ontario Leadership Strategy; Managing Information for Student Achievement (MISA); the Vendor of Record (VOR) for Education Research and Evaluation Services that has four categories: research, evaluation, educator research capacity building, and knowledge mobilization; and the Knowledge Network for Applied Education Research (KNAER).
Facilitators to Evidence-Based Policy
The following factors will help connect evidence to policy and practice (Davies, UK, 2004):
- Integrating Research into Professional Competence – Building capacity for accessing and critically appraising research will lead to more use.
- Ownership of the Evidence – Evidence doesn’t just belong to researchers; policy makers and practitioners can own it, too.
- Getting Appropriate “Buy-In” – Senior officials can commit to own and champion evidence that supports good practice.
- Shared Notions of Evidence – Researchers and policy makers must work together to fully understand the questions to be answered, and to agree on the most appropriate evidence to solve their policy problems.
- Incentives to Use Evidence – Practitioners need incentives (e.g., targets) to use evidence and to do things that have been shown to be effective.
- Availability of Sound Evidence – High quality reviews of what we already know will help sort out what’s sound and conclusive, and point to the gaps in the evidence base.
- If you are looking for detailed information on how to cite references as you are doing literature searches, visit the Berkeley Library resource, which also contains a link to Purdue Library’s APA Guide. Also, from UC Santa Barbara library.
- And remember that if you are using Zotero as a citation manager, it can take care of APA for you.
- APA Guide: Publication manual of the American Psychological Association / American Psychological Association.: Washington, D.C. : American Psychological Association, 2010. (Book)
- And also this resource: Mastering APA style: student’s workbook and training guide / American Psychological Association.: Washington, DC : American Psychological Association, 2010. (Book)
- Checklist: Criteria for Assessing Research (KNAER)
- Clarifying the Issue and Determining the Scope of a Project
- Creating Online Surveys Tip Sheet
- University of Surrey: Focus Groups (Dr. Anita Gibbs)
- Tracking and Synthesizing Evidence Work Sheet
- Writing Effective Survey Questions Tip Sheet
- Association of Educational Researchers of Ontario (AERO)
- Canadian Society for the Study of Education (CSSE)
- Canadian Educational Researchers’ Association (CERA)
- American Educational Research Association (AERA)
- Centre for the Use of Research and Evidence in Education (CUREE)
- Teacher Professional Learning and Development, Best Evidence Synthesis (BES) (New Zealand)