SEGway September 2016
Research and Assessment News from SEG Measurement
|
|
|
31 Pheasant Run
New Hope Pennsylvania 18938
800.254.7670
267.759.0617
|
|
|
|
Greetings!
We are looking forward to attending the EdNET networking conference later this month in Dallas. EdNET is the industry's premier networking event and we look forward to attending this conference each year. It is our number one opportunity to reconnect with our colleagues and to make new friends. We always walk away with a better understanding of the education marketplace.
Again, I would like to thank Anne Wujcik, editor of EdNET Insight, for recognizing the value of SEG Measurement's SEGway newsletter. We would like to return the praise and let you all know that SEG Measurement relies on the annual EdNET conference and the EdNET Insight newsletter as a critical source of news in education.
"In the April issue of SEGway, Scott Elliot offered insights on the different types of evidence that can be gathered to provide proof of product effectiveness, provided a quick review of test validation and answered some basic questions about conducting an efficacy study. Scott follows this pattern in each monthly issue and I always come away having learned (or been reminded) of something useful."
We encourage you to learn more about our work in assessment and efficacy research by connecting with us at conferences. We also look forward to continuing the discussion with you through the newsletter, on Twitter(@segmeasure) and on LinkedIn (SEG Measurement).
Sincerely,
Scott Elliot
SEG Measurement
|
|
|
 |
Get Results Quickly! Conduct a Compressed Study For Rapid Results!
Be ready for ESSA or a Marketing Campaign within three to six months
Do you have evidence that your product works? Your customers want to know.
With an SEG Measurement compressed fall 2016 or spring 2017 three to six month effectiveness study, you will be able to: 
-
Have results available within a few short months
-
Provide tangible proof that your product increases student achievement
-
Have your product qualify and be ready for school buyers to purchase under ESSA requirements
-
Assist students with higher learning outcomes
-
Pilot your product, discover any issues and make needed improvements prior to launch
-
Measure student and teacher engagement and get answers for your clients
Conducting a shorter rather than year-long efficacy study can be a very cost effective way to obtain proof of effectiveness. A short fall or spring study of your educational program can be conducted at a lower cost than studies conducted during the full school year. And, the results can be used for marketing.
Join our growing list of clients at SEG Measurement that have benefited from a shorter-term efficacy study. Increase your customer base with more sales, while helping students gain critical knowledge and skills. Be ESSA ready!
|
|
Wishing You Had Conducted a Study This Year?
You May Have the Data Needed to Evaluate Effectiveness of Your Product
|
Do you have districts that used your product diligently throughout last school year? Do you wish you could formally evaluate whether those students learned more than comparable students who did not use your product? Well, we have good news; it is not too late to evaluate the effectiveness of use of your product.
In order to evaluate the effectiveness of your product used during last school year, you will need the following:
-
Product usage data for the classes you would like to include in the analysis
-
Identification of comparable classes or schools in the same or similar district who did not use your product
-
Ability to obtain state or district test data
The performance of students using your product can be compared to performance of comparable students who did not use your product while controlling for any initial differences, using a sound research design. In addition to the quantitative evaluation, we can also collect feedback from the teachers who have used your product during the year.
Conducting this type of post hoc evaluation of product effectiveness would allow you to have effectiveness data now rather than waiting for a new school year and losing months of sales!
If you would like to know more, please contact us to help you plan and execute this research at info@segmeasurement.com or 800-254-7670.
|
|
|
Three Questions to Answer When Designing an Assessment
There is a natural tendency to dive right in to writing test questions when developing an assessment. But, it is critical to be clear on the purpose and the intended use of the assessment before item writing or selecting items from a pool. Without clarity on purpose, there is risk that the assessment will not accurately meet your needs.
Be sure to consider these three questions when designing an assessment:
* How will the scores be used? This includes determining the decisions that will be made based on the results and what action will be taken. Understanding the decisions that need to be made and actions to be taken will help to define the assessment skill coverage, difficulty level, length, and type of items so that the test scores can be used as envisioned.
* Who will be taking the test? This includes defining the level of experience and skill the test takers have, any prerequisites for taking the assessment, range of reading abilities of the test takers, and why someone would be taking the test. Understanding who will be taking the test helps to define the content coverage, targeted difficulty level, and format of the assessment to ensure that the test is appropriate for the intended test takers.
* What are the logistical constraints for administration? This includes understanding the amount of time available to take the test, whether the test can be on paper or online, what the security needs for securing the content are, what precautions can be taken with regards to cheating, when the test can be administered, and how quickly the scores are needed. Understanding these parameters helps to define the test length, test format, test platform, security rules, administration window(s), and reporting plans so that the assessment created fits within the administration constraints.
These questions are relevant for low stakes classroom assessments through to high stakes certification and licensure exams, though the level of detail in planning for each type of test will vary greatly.
Once these questions are answered, the test blueprint and specifications can be developed. After the test specifications are developed, subsequent test development steps can be planned in accordance with the goals and purpose of the assessment.
|
Please contact us to help you plan and execute your assessment development atinfo@segmeasurement.com or 800-254-7670.
|
|
|
|
Recent Questions Asked by Educators and Publishers Considering Conducting an Efficacy Study
|
|
Here is the latest installment of this popular section of the newsletter with common questions and answers. If you have specific questions that are not covered here or if you are interested in exploring any of these questions further, please contact us. We are here to help.
How similar do the control and treatment groups need to be? The ideal approach is to randomly assign students/classes to treatment and control groups, yet this is often not possible in educational research. It is important to have the control and treatment group be as similar as possible in composition and size. For example, if you are investigating the use of your reading product on students who are struggling readers in the third grade, you want to be sure that your control/comparison group also consists of struggling readers in the third grade. If your user group is from a specific district, it would be best to have the control group also be from the same district. If you are looking specifically at a certain subgroup (i.e., ethnicity, gender, economic status, geographic region, or non-native speakers), the subgroup should be equally represented in both the treatment and control groups.
How do we use study feedback to make changes to our program? While it is always reassuring to get positive feedback about your product and hear folks say it is fine as is, constructive criticism can be even more helpful to identify ways in which your product can be improved or expanded. When teachers or students provide information regarding something that did not work as expected or something that was difficult to find or use, this is valuable information regarding the usability and utility of your product. We help to look for trends and themes in the feedback that can be used to help make a short list of areas to focus for future revisions of the program or additional training and documentation.
|
What questions do you have? Let us know what questions you want answered. Contactinfo@segmeasurement.com to share your feedback.
|
|
SEG At Upcoming Conferences
Let's Meet!
We are looking forward to seeing our colleagues and meeting new friends at the upcoming conferences. We are participating in several sessions & we invite you to join us.
|
|
Look for us at these upcoming conferences:
-
EdNET, September 25 - 27, Dallas, TX
-
NCTM Regional Conference, October 31 - November 2, Philadelphia, PA
-
NJEA Convention, November 5, Atlantic City, NJ
-
E-Learn World Conference on E-Learning, November 14 - 16, Washington DC.
We would love to meet with you and discuss how we can help you build strong assessments and get the proof of effectiveness you need for success.
If you would like to meet with a representative from SEG Measurement to discuss how we might help you with your assessment and research needs, please contact us at info@segmeasurement.com. |
About SEG Measurement
Building Better Assessments and Evaluating Product Efficacy |
SEG Measurement conducts technically sound product efficacy research for educational publishers, technology providers, government agencies and other educational organizations, and helps organizations build better assessments. We have been meeting the research and assessment needs of organizations since 1979. SEG Measurement is located in New Hope, Pennsylvania and can be accessed on the web at
|
|
|
|
