September 2011

SEGway
An update from SEG Measurement
News and information about educational research and assessment
September 2011
In This Issue
Evaluating Teachers Based on Student Test Results
Assessment and Research Challenges that are "Too Hot to Handle"
Featured Project: Developing Probes for RTI
SEG at Upcoming Conferences

Evaluating Teacher Performance Based on Student Test Results:  Data Management Challenges

States and districts are increasingly using student test results to evaluate teachers.  While the measurement and statistical challenges associated with using student test results for this purpose have been well-documented, the data management choices made in preparing for analyses can have an even greater effect on the results. 

There are many decisions/challenges to be made in preparing student and teacher data for analysis.  Here we highlight three of those challenges to alert those considering using student test results for teacher evaluation to the importance of such decisions.   

  1. Student Mobility-  Not all students remain at one school for their academic career.  In fact, we have seen cases where a student appears in four different locations in one year!  Annual mobility is as high as 50% in some areas.  You will need to make decisions about the length of time students need to be in a given location or with a specific teacher to consider them associated with that location and teacher.   
  2. Associating Teachers with Students-  Students typically have more than one teacher;  this is particularly true of the upper grades.  But even at the lower grade levels, students may have a regular classroom teacher, a special education teacher and one or more teachers addressing specific issues. In preparing the data, you will need to decide whether a single teacher (perhaps the primary classroom teacher) will be held accountable for a given student's results, whether all teachers working with the student will be held accountable or if some subset of teachers working with the student based on established criteria will be held accountable.
  3. Missing data- For a wide range of reasons, some of the desired data will no doubt be missing. This may be the result of deficits in record keeping, failure to retain data for students when they move teachers, grades, schools, etc. or decisions made about criteria for retaining data.  The absence of data can have a significant impact on the overall statistics as well as the particular teachers and students for which data is missing.  Rules for handling missing data need to be established at the outset to handle these issues.  

These are only a subset of the data management decisions faced in using student test results to evaluate teachers.  SEG would be happy to discuss your unique challenges. 

If you are considering using student data to evaluate teachers, please contact us to help you plan and execute this research. 
Greetings!

Welcome back! Schools are back in session and Publishers and Technology Providers are again working with schools to help address the array of challenges they face.

This issue of SEGway examines several topics that should help you in your quest to build better assessments and obtain evidence of product effectiveness.

Many states and local education agencies are considering using student test data to evaluate teachers.  In our technical corner, Cathy Mikulas, SEG's Director of Research Services, discusses several important issues in data management that can affect results. 

Also in this issue, we introduce you to SEG's "Too Hot to Handle Campaign."  SEG has launched an educational campaign to highlight the array of assessment and research issues faced by educational publishers and technology providers that are just.... "too hot to handle."

Our featured project this month looks at the development of Assessment Probes to measure student progress as part of Response to Intervention (RTI) Programs.

SEG will be attending several conferences and events in this Fall.  We look forward to seeing you there.  Please let us know if you will be attending and if you would like to meet with us.

As always, please feel free to learn more about us at www.segmeasurement.com.

Sincerely,

Melissa Garza
Editor
 

SEG Launches Fall "Too Hot to Handle Campaign"

From common core standards to growth measurement, educational publishers and technology providers are wrestling with a greater array of issues than at any time in the recent past.   SEG has launched its "Too Hot to Handle"  campaign to highlight the hot issues that publishers and other educational providers are contending with.

The campaign will include a variety of giveaways and special publications--from tweets to white papers--highlighting the notion that educational publishers are dealing with assessment and research issues that are too hot too handle--important challenges that require the expertise and assistance of SEG Measurement.

Look for many undercooked puns, half-baked metaphors, and simmering similes as we assault you with assessment and research issues that are....too hot to handle.    

 

Featured Study
Project: Developing Probes for Use in Evaluating Student Progress for RTI
 

More and more schools are implementing Response to Intervention Programs to better address the individualized needs of students.  In part, these programs offer a rapid cycle of instruction, measurement of student performance based on the instruction, and re-tailoring of instruction to address the needs identified.

  

This active cycle of instruction and assessment calls for frequent assessment that can be delivered and acted upon quickly.  Continuing their legacy of providing high quality assessments to meet school assessment needs, Curriculum Advantage asked SEG to assist in developing assessments that could be used effectively as part of RTI.

  

SEG worked with Curriculum Advantage to develop a series of assessments in both reading and mathematics for use in evaluating the skill levels of students as part of RTI.  At each grade level and for each content area, SEG developed about 10 short assessments that could be used to assess students frequently and quickly.  

 

The assessments were field tested to evaluate their technical characteristics and to establish their equivalence.  The assessments at each grade level were statistically equated to ensure that the test results had a common meaning regardless of when they were administered or which form was administered. 

 

 We were pleased to have the opportunity to partner with Curriculum Advantage in completing this important assessment work. We welcome the opportunity to  help you meet your assessment challenges.  Please contact us or visit us at www.segmeasurement.com to learn more.   

 

 SEG at Upcoming Conferences