Share |

Research

Teacher Evaluation 2009
Principal Evaluation 2009
Summary of Preliminary Findings 2009/10

Tools for Life® (TFL) Evaluation: Teacher Survey Data – June 2009

In late 2008, kidsLINK was awarded a three-year Ontario Trillium Foundation grant to undertake evaluation of the Tools for Life Relationship-building Solutions program.  The purpose was to provide kidsLINK with systematic and scientifically-grounded evidence on the merit, worth, effectiveness and usefulness of the Tools for Life curriculum in Junior and Senior Kindergarten, Grades 1, 2 and 3 classrooms, schools, and family settings in Waterloo Region.  The evaluation is being conducted by an independent third-party evaluator.

Our pilot evaluation was conducted in those schools that were introduced to Tools for Life for the first time in the 2008-09 year.  The attached report for teachers is the first data release in a series of evaluations that are to be conducted with principals, vice-principals, teachers and parents from 2009 to 2011.

Summary Report

The following are key findings stemming from preliminary Tools for Life Evaluation Teacher Survey data. If you have any questions please do not hesitate to contact the principal investigator, Sara White, at sarajane.white@hotmail.com.

I. Demographics

  • 29 teachers of JK to Grade 3 classrooms implementing TFL for the first time in 2008-2009, completed teacher surveys
  • 85.71 per cent of classrooms have received all lessons
  • 96 per cent of teachers use TFL curriculum in their classrooms
  • 96 per cent of teachers use TFL resources in their classrooms
  • Teachers surveyed averaged 13 years experience as a full-time licensed teacher, six years teaching in their current school and four years teaching their current grade in this school.

II. Details – Professional usefulness of TFL curriculum

  • The following table depicts particular details on teachers’ perceived professional usefulness of TFL curriculum.
Perceived Professional Usefulness Factor Percentage of
Teachers Who
Agree or
Strongly Agree		 Number
of Valid
Teacher
Responses
 TFL has improved the quality of teaching I provide 87.50 24
 TFL curriculum has been advantageous for my job 95.83 24
 TFL curriculum is compatible with classwide learning objectives 95.83 23
 Results from using the TFL curriculum are apparent to me 95.65 23
 I can explain why using the TFL curriculum is beneficial for teachers in my school  95.83 23
 TFL curriculum provides a common language for desired school behaviours 100 25
 TFL resources are useful for my teaching activities 100 24
 It has been easy to implement the TFL curriculum into my daily teaching routine  95.83 24
 TFL curriculum is helpful in providing feedback to parents 91.30 23
 School policies/procedures fit well with the TFL curriculum 100 24

III. Sustainability

  • 95.83 per cent (n=24) of teachers stated that they will likely continue to apply the TFL curriculum in their teaching activities.
  • 88.46 per cent of teachers evaluated the TFL curriculum as somewhat to extremely worthy for their classroom.
  • 88.89 per cent (n=27) of teachers agreed that, if fully implemented in their school, the TFL curriculum would make a significant change to student outcomes.

IV. Students demonstrating TFL components in the classroom setting during the past month

Figure 1. Percentage of teachers who have observed students, during the past month, demonstrating appropriate use of TFL

curriculum components by frequency type

V. Qualitative data

Teacher 2: The TFL curriculum was very much in line with the strategies that I had been using in my classroom over the past 11 years.

Teacher 3: The children have enjoyed the lessons so fa,r although we have not yet completed them all. My students are beginning to use the language in this program to solve social situations.

Teacher 4: It has been an awesome experience! It helps give appropriate vocabulary and steps to solving problems in my class. We refer to the posters often and students have been implementing the tools regularly.

Teacher 5: My students looked forward to seeing their TFL instructor each visit. They have responded very well to using the tools they were taught– often using the TFL language. They seem happier, too.

Teacher 9: The overall program expanded on some great areas using conflict resolution. I did observe my students were bored when they had to sit too long – needs to be interactive each session where they can practice.

Teacher 11: Gave children confidence and ownership over their feelings. I like the common language. Works well with almost all types of children.

Tools for Life® (TFL) Evaluation: Principal Survey Data - June 2009

Summary report

The following are key findings stemming from preliminary Tools for Life Evaluation: Principal Survey data. If you have any questions, please do not hesitate to contact the Principal Investigator, Sara White at sarajane.white@hotmail.com.

I. Demographics

  • Eight principals working in schools implementing TFL for the first time in 2008-2009, completed TFL principal surveys.
  • principals surveyed averagd three years as the principal of their current school, 3.5 years worked as a principal in schools other than their current school, and nearly 15 years worked as a full-time licensed teacher.
  • 42.86 percent of principals stated that they use components of TFL curriculum in their daily activities.
  • Principals stated that an average of 35.75 percent of teachers in grades other than JK to Grade 3 and 18.63 percent of school staff had been oriented to the TFL curriculum.
  • 85.71 percent of principals stated that classroom management was a minor to moderate problem in JK to Grade 3 classrooms.

II. Details – Professional usefulness of TFL curriculum

The following table depicts particular details on principals’ perceived professional usefulness of TFL curriculum.

Perceived Professional Usefulness Factor Percentage of principals
who agree or strongly agree
 TFL curriculum has been advantageous for my job 100 per cent  100
 TFL curriculum is compatible with school-wide learning objectives  100
 Results from using the TFL curriculum are apparent to me  85.71
 I can explain why using the TFL curriculum is beneficial for teachers in my school  100
 TFL curriculum provides a common language for desired school behaviours  100
 TFL resources are useful for my daily activities  85.71
 It has been easy to implement the TFL curriculum into my daily routine  71.43
 School policies/procedures have been modified to reflect the TFL curriculum  57.14
  School policies/procedures fit well with the TFL curriculum  71.43

III. Educational and supportive processes

  • 85.71 percent of principals stated that learning to use the TFL curriculum was easy.
  • 57.14 percent of principals stated that they did not have enough time to learn about the TFL curriculum before it was implemented.
  • 100 percent of principals stated that they felt supported by JK-Grade 3 teachers in their efforts to implement the TFL curriculum.
  • 57.14 percent of principals stated that they felt supported by Grade 4 or higher teachers in their efforts to implement the TFL curriculum.
  • 14.29 percent of principals stated that they felt well supported by school staff (excluding teachers) in their efforts to implement the TFL curriculum.
  • 85.17 percent of principals stated that they felt well prepared to carry out the TFL curriculum with the existing resources available to them.
  • 14.29 per cent of principals stated that they were not able to carry out the key concepts contained in the TFL curriculum school-wide.

IV. Sustainability

  • 100 percent of principals stated that they will likely continue to apply the TFL curriculum in their daily activities.
  • 100 percent of principals stated that, if fully implemented, the TFL curriculum would make a significant change to student outcomes.

V. Qualitative Data

Principal 1:  It has been a great supplement to what we have been trying to do with our school culture. As always, more time would be most beneficial.

Principal 2:  The program has been a welcomed addition to our classes eventhough we began late in implementation. Due to constraints on my time, I have not been able to internalize themes and concepts but I know teachers have. Once I am fully in-serviced, I will expand use.

Principal 3:  I am appreciative of the program but as an administrator need to find the time to become more knowledgeable and informed about the program in order to use the language in the office as well.

Principal 4: Good common language to use across the school. Buy-in from parents is important.

Principal 5:  We are in our infancy (year one) of implementing the TFL program… I would expect stronger/positive results as we continue to move forward with the curriculum over the next few years.

Summary of Preliminary Findings – 2009/2010*

In the 2009-2010 school year, 13 Waterloo Region elementary schools participated in the Tools for Life (TFL) Evaluation Project. Teachers, parents/caregivers and principals from 8 schools received training and support in the implementation of TFL in classrooms and schools. Five schools did not receive any training or information on TFL or its curriculum. In order to test the impact of the TFL curriculum, parents/caregivers, teachers and principals were surveyed prior to TFL curriculum implementation (September 2009) and again following the implementation of TFL in exposed schools (April/May 2010). Teachers, parents/caregivers and principals not exposed to TFL curriculum were also surveyed at Time 1 (September 2009) and Time 2 (April/May 2010). Statistical tests assessing the equality of means show that, at Time 1, classrooms scheduled for exposure to the TFL curriculum were not significantly different from classrooms that would not be exposed to the TFL curriculum.

Who Participated

Teachers of participating Junior Kindergarten through to and including Grade 3 classrooms and principals in participating schools were asked to complete teacher/principal surveys at Time 1 and Time 2. Parents/Guardians of students in participating classrooms were also asked to complete parent/guardian surveys at each data collection interval.

Total Number of Valid Surveys Received Per Data Collection Interval by Respondent Type

Respondent
 Type		 Time 1
(September 2009)		 Time 2
(April/May 2010)
Parent/Guardian 137 91
Principal 13 9
Teacher 89 39

 

 

 

 

 
Matched-Pairs Analysis – Classrooms

Individual teacher survey data completed at Time 1 was matched with this same teacher’s survey data at Time 2, resulting in a total of 28 matched pairs for TFL curriculum exposed classrooms and 9 matched pairs for non-exposed classrooms (total 74 teacher surveys). Overall, when Time 2 teacher survey findings are compared to their matched-pair baseline levels, classrooms exposed to the TFL curriculum show statistically significant improvements in classroom management factors, teacher efficacy, student peer relations and student use of appropriate social skills.

Classroom Management

From Time 1 to Time 2, teachers in classrooms exposed to the TFL curriculum experienced consistent improvements large enough to not be attributable to chance, in the following classroom management factors: successfulness of joint activities during instructional time, teacher satisfaction with the effectiveness of their classroom management strategies and teacher efficacy. TFL implementation classrooms also experienced a significant reduction in the percentage of instructional time spent dealing with classroom management demands and a reduction in the degree to which classroom management was a problem in the classroom.

Comparatively, teachers in classrooms not exposed to the TFL curriculum experienced significant change in two classroom management measures solely. From Time 1 to Time 2, teachers in non-TFL exposed classrooms experienced a statistically significant increase in delays during instructional time due to classroom management demands and an increase in the percentage of instructional time spent dealing with classroom management demands.

Teacher Checklist of Peer Relations

When compared to their baseline levels, classrooms exposed to the TFL curriculum experienced statistically significant improvements across all Teacher Checklist of Peer Relations (Tschannen-Moran & Hoy, 2001) measures. No significant differences from Time 1 to Time 2 were found in classrooms not exposed to the TFL curriculum.

Demonstrating Appropriate Social Skills in Classroom Settings

Students in the TFL curriculum-exposed classrooms significantly improved, from Time 1 to Time 2, across all measures capturing student use of appropriate social skills in the classroom setting. Specifically, TFL-exposed students demonstrated statistically significant increases in their appropriate use of: listening skills, listening behaviour, identification of feelings, recognition of feelings in others, recognition of own body clues, matching feelings with problems, calming down and relationship problem-solving tools. Classrooms not exposed to the TFL curriculum showed no significant improvement in these social skills from Time 1 to Time 2, except for a statistically significant improvement in students identifying feelings.

Teacher’s Perceived Usefulness, Worthiness and Effectiveness of TFL Curriculum

Following implementation of the TFL curriculum in their classrooms, 73 percent of teachers stated that the TFL curriculum was advantageous for their job. Ninety-six percent of teachers found the curriculum compatible with classwide learning objectives and 93 percent of teachers recognized TFL as a useful tool for classroom management activities. Eighty-nine percent of teachers agreed that the TFL curriculum provides a common language for desired school behaviours while 78 percent found TFL Curriculum useful for teaching activities. Eighty-two percent of teachers found the TFL curriculum easy to implement into their daily classroom routines and 89 percent of teachers had adapted the TFL curriculum to fit the needs of their class.

Ninety-three percent of teachers exposed to the TFL curriculum felt supported by their principal in their efforts to implement the TFL Curriculum. Ninety-three percent of teachers also felt that the TFL curriculum fit well with school policies and procedures. Eighty-nine percent of teachers rated the TFL curriculum as worthy for continued use in their classrooms while 77 percent of teachers agreed that, if fully implemented in their school, the TFL curriculum would make a significant change to student outcomes. Seventy-three percent of teachers exposed to the TFL curriculum have integrated TFL concepts into student learning objectives.

Parents/Guardians

Parents/guardians of children in classrooms exposed to the TFL curriculum experienced statistically significant improvements in: the degree they were informed about school’s goals and achievements, their inclusion on planning committees, the degree to which school staff responded to the needs of parents, the availability of regular meetings to keep informed about the school, and the presence of organized community groups meeting regularly to discuss school issues. No significant improvements in these community engagement measures were experienced by parents/guardians of children in classrooms not exposed to the TFL curriculum.

*Significance difference at the minimum level of .05

Source Cited:

Tschannen-Moran, M., & Hoy, A. (2001). Teacher efficacy: capturing an elusive construct. Teaching and Teacher Education, 17, 783-805.