Planning a Program
Evaluation for Bishop Murray High School
Timeline
Responsibilities
-Budget
|
|
|
|
|
|
|
|
|
|
|
Describe program-logic model<!
Define purpose
-Determine use/users
-Determine key questions
-Select indicators
Determine design
|
|
|
|
Identify sources
Select method(s)
Pilot test
Set schedule
Determine sample
|
|
|
Process data
Analyze
Interpret data
What did you learn?
What are the limitations?
|
|
Share findings and lessons learned
Use in decision making
Determine next steps
|
|
|
Who
should be involved? How might they be engaged?
The Stakeholders are
the teachers, administration and students. The teachers and administration will
be involved by completing a survey and being interviewed. Several will also be
involved in providing input when analyzing the information collected. Several
former and current students will also be interviewed. As well, there are many programs that involve using stakeholders from around the city, therefore, stakeholders also consist of the people who work the teachers and students of Bishop Murray.
What
are you going to evaluate? Describe
program (logic model). What is the purpose of the evaluation?
I am going to
evaluate the graduation success rates of Bishop Murray’s First Nations Student
Population. The goal is to determine if the programming offered is helping to
improve the rates as is intended.
Who
will use the evaluation? How will they use it?
Who/users
|
How
will they use the information?
|
Administration
|
To
lead the staff to modify/adapt programs for the current needs.
|
Teachers
|
To
help determine the supports that are needed for the students which could
include further training
|
Superintendents
|
To
provide the training and personnel needed
To
support possible changes to the existing building
|
What questions will the evaluation seek to answer?
Does
the current programming work for First Nations Students? How is the programming
implemented? Are activities delivered as intended? Are students being reached
as intended? How can we improve graduation success rates? What other
programming could be added or taken away?
What information do you
need to answer the questions?
What
I wish to know
|
Indicators
– How will I know it?
|
What
programs have been most successful for First Nations Students and why?
|
Questionnaires,
surveys, interviews from teachers, former and current students, research of
successful programs
|
What
are the graduation rates of First Nations students?
|
Statistics,
charts on file
|
What
programs are least successful and why?
|
Questionnaires,
surveys, interviews
|
How
can we include the successful programs within Bishop Murray’s current
building?
|
Results
of least effective/most effective, re-evaluation of building, possible
movement of school to a new facility
|
When is the evaluation needed?
The
evaluation is needed by June of 2013 so that new programming can be put in
place before the September 2013/2014 school year.
What evaluation design will you use?
What sources of information will you use?
Existing information:
|
Past
statistics of graduation rates for First Nations Students.
|
People:
|
Long
term staff members, students (former and current)
|
Pictorial records and observations:
|
Anecdotal
notes, Observations of programs, Surveys, Interviews
|
What
data collection method(s) will you use?
X
|
Survey
|
X
|
Document review
|
X
|
Interview
|
|
Testimonials
|
X
|
Observation
|
X
|
Expert panel
|
|
Group techniques
|
|
Simulated problems or situations
|
|
Case study
|
|
Journal, log, diary
|
|
Tests
|
|
Unobtrusive measures
|
|
Photos, videos
|
|
Other (list)
|
Instrumentation: What is needed to record
the information?
I will use ‘Survey Monkey’ to create the survey, a
digital recorder and transcriber for the interviews, a log to hold my anecdotal
notes and observations, and recording sheets to track common themes of
graduation rates and programs.
When
will you collect data for each method you’ve chosen?
Method
|
Before
program
|
During
program
|
Immediately
after
|
Later
|
Statistics on First Nations Success Rates at Bishop
Murray
|
|
X
|
|
|
Literature
Review of Effective Programming for First Nations Students
|
|
X
|
|
|
Survey
|
|
X
|
|
|
Interview
|
|
X
|
|
|
Will a sample
be used?
No
|
|
|
Yes
|
X
|
If
yes, describe the procedure you will use.
|
I will meet with one or more staff members to discuss the programming and use the information to help guide the questions for the surveys and interviews. Over several meetings, these questions will be modified with assistance from these teachers.
|
Pilot
testing: when, where, how?
The survey and interview questions will be shared ahead of time so that teachers can provide their input. Editing and new questions will be made based on these conversations at an agreed upon meeting place.
How will the
data be analyzed?
The link to the survey will
be sent to the staff via e-mail but the survey for the current students will be
administered by me. I will also perform the interviews and digitally record
them; a transcriber will be hired to compile the conversation. Next, I will chart
common themes from the data provided. The statistics will also be evaluated and
charted.
How will the
information be interpreted—by whom?
Once the data has been
compiled, I will meet with a small group of experienced teachers from Bishop
Murray to evaluate the results. All input will be considered. Suggestions for
improvement will be made.
What did you
learn? What are the limitations?
From the evaluation, I will
learn whether or not the current programming at Bishop Murray is making a
difference in the graduation rates of First Nations students. I will also learn
what is making a difference and what is not.
The limitations are that I,
myself have worked within a similar structured program so there could be
biases. The people who are involved in the evaluation all have an invested
interest in the program, so looking to an outside evaluator to assist in the
process could proved to be beneficial.
How will the
evaluation be communicated and shared?
To whom
|
When/where/how to present
|
Superintendent
|
Written
summary of results with charts, including recommendations – email - June 2013
|
Administration
|
Written
summary of results with charts, including recommendations – email June 2013
|
Staff Members
|
Informal
Meeting with summary chart – June 2013
|
|
|
Next steps?
The next steps would
include future planning for the 2013/2014 school year.
X
|
Human subject’s protection
|
|
Utility
|
|
Management chart
|
|
Feasibility
|
X
|
Timeline
|
|
Propriety
|
X
|
Responsibilities
|
|
Accuracy
|
X
|
Budget
|
|
|
All participants will be given a pseudonym; their
confidentiality will be protected.
The timeline for the evaluation is from January 2013 till
it’s completion in June 2013.
Responsibilities for the evaluation fall on the evaluator,
but the sharing of information will come from all stakeholders.