From jordan.heide at state.or.us Fri Sep 25 07:48:18 2015 From: jordan.heide at state.or.us (HEIDE Jordan) Date: Fri, 25 Sep 2015 14:48:18 +0000 Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment Message-ID: We'd like to request your review of the attached proposed Oregon Extended Assessment Observational Rating Assessment (ORA)(see attached Administration Instructions and ORA) designed to provide instructional and functional information for teachers and parents in in four domains: attention, math concepts, receptive and expressive communication. The ORA is designed to be administered to students who are not able to access the academic tasks presented on the Oregon Extended Assessment (ORExt), even when extensive, universal supports are provided. The ORA will be incorporated into the Extended Assessment beginning 2015-16. Given the description and purpose of the ORA, we are seeking feedback on the questions below by 9/30/15: 1. Is the purpose of the ORA clearly stated? 2. Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)? 3. Does the ORA appear to be a useful tool for the intended population (i.e., face validity)? 4. Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales? 5. Are the ORA scoring instructions clear? 6. Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt? 7. Do you feel that clear directions regarding the use of the results from the ORA have been provided? 8. Do you have any recommended edits/changed to the ORA? Please submit your feedback to Brad Lenhardt (brad.lenhardt at state.or.us). Finally, we would appreciate it if you would keep this information confidential at this time. Once we finalize the ORA we will distribute it through the typical ODE communication channels. Sincere thanks for your expert consideration and assist -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ORExt_ORA_AdminInstruct2016_V10.docx Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document Size: 106334 bytes Desc: ORExt_ORA_AdminInstruct2016_V10.docx URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ORExt_ORA_2016_V10.docx Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document Size: 111282 bytes Desc: ORExt_ORA_2016_V10.docx URL: From jordan.heide at state.or.us Fri Sep 25 12:33:03 2015 From: jordan.heide at state.or.us (HEIDE Jordan) Date: Fri, 25 Sep 2015 19:33:03 +0000 Subject: [Assessment_Advisory_Committee] Opportunity: Community College Placement Processes Work Group In-Reply-To: References: Message-ID: Dear Assessment Advisory Committee members, Work Group Description As the fall term begins, work is underway to convene a workgroup to examine issues surrounding community college placement. This examination will include work to rethink community college testing and placement practices, to think about issues such as whether campuses might use multiple measures, and/or adopt similar or even shared placement practices across campuses - with the ultimate goal of placing students in the highest possible class in which they can be successful. The work will conclude with recommendations to the legislature, in accordance with HB 2681 (see attached), for processes and strategies for course placement in community colleges. For additional information about the work to redesign developmental education in Oregon and specifically the initial work on assessment practices, please see page 6 of the attached report. Work Group Recruitment In order for the work group to be successful in developing recommendations about community college placement practices that truly reflect the best interests of Oregon students, we need representation from many different perspectives. Specifically, we are looking for: * high school teachers * teacher mentors who work with high school teachers * high school administrators * high school counselors * high school students Meeting Schedule The meeting schedule follows. For the in-person meetings, the Community College Placement Processes Work Group will participate in the scheduled meetings of the Developmental Education Redesign Workgroup from 9:00-3:00 then will hold its own two-hour meeting from 3:00-5:00. Please note that substitute, travel, and meal reimbursements are not available for this workgroup due to budget limitations. * October, webinar (date and time to be determined; estimated length: 90 minutes) * October 23, in-person meeting at Chemeketa, 9:00-5:00 * November, webinar (date and time to be determined; estimated length: 90 minutes) * December 4 in-person meeting at Salem Center for Business and Industry, 9:00-5:00 Response Requested Please email Cristen McLean (Cristen.McLean at state.or.us) by Thursday, October 1, if you are available to participate. Thank you for your consideration of this request. Best, Cristen Cristen McLean Policy Analyst Office of Learning | Instruction, Standards, Assessment, & Accountability Unit | Oregon Department of Education Office: 503.947.5842 | Fax: 503.378.5156 | *cristen.mclean at state.or.us -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: OR Report Year 2 Final.pdf Type: application/pdf Size: 658169 bytes Desc: OR Report Year 2 Final.pdf URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 2681 Enrolled.pdf Type: application/pdf Size: 20567 bytes Desc: 2681 Enrolled.pdf URL: From jordan.heide at state.or.us Fri Sep 25 15:13:42 2015 From: jordan.heide at state.or.us (HEIDE Jordan) Date: Fri, 25 Sep 2015 22:13:42 +0000 Subject: [Assessment_Advisory_Committee] Text-to-Speech on Tablets and Chromebooks In-Reply-To: References: Message-ID: Dear Assessment Advisory Committee members, ODE is requesting your feedback on whether to change its policy regarding allowing districts to use text-to-speech on tablets and Chromebooks. Issue: For technical reasons, the pause feature is not available for text-to-speech (TTS) on tablets and Chromebooks. Students can stop TTS reading in the middle of an item or passage, but there is no way to have TTS resume reading where it was stopped. If the student activates TTS again for the same item or passages it will start again at the beginning. The pause feature, which allows TTS to resume reading where it was paused, is available on desktops and laptops. Current Policy: ODE has disallowed the use of TTS on tablets and Chromebooks on the basis that the inability to pause TTS and then resume again at the place it was paused would disadvantage students testing on tablets and Chromebooks when they encountered a long passage or item. It's important to note that while this restriction is noted in all documentation and user guides, AIR is not able to actually block TTS on tablets or Chromebooks. AIR and ODE received a lot of calls from districts that were testing students on tablets and Chromebooks and wanted to be able to use TTS. They questioned why it was not available and stated that it was a great inconvenience to not be able to use it. Proposed Workaround: AIR has proposed some workarounds as an alternative to the pause feature if ODE decides to support TTS on tablets and Chromebooks in 2015-2016: * Students can select the text they want read aloud and TTS will read only the selected text. Therefore, if a student pauses a passage in the middle and wants the TTS to resume where the passage was paused, the student can highlight the remainder of the passage. * We can include in the documentation the recommendation that if a student will use TTS a desktop or laptop computer should be used for the ELA performance task test, which is the test that has the longest passages. To inform ODE's decision-making on whether to allow TTS use on tablets and Chromebooks with the proposed workarounds, please respond with your feedback by Wednesday, September 30. Thank you as always for your input! -------------- next part -------------- An HTML attachment was scrubbed... URL: From stewartw at gladstone.k12.or.us Mon Sep 28 08:23:41 2015 From: stewartw at gladstone.k12.or.us (Bill Stewart) Date: Mon, 28 Sep 2015 08:23:41 -0700 Subject: [Assessment_Advisory_Committee] Text-to-Speech on Tablets and Chromebooks In-Reply-To: References: Message-ID: Jordan, Interesting situation! Since the 'problem' can't be fixed and the function can't be blocked, it leaves ODE and DTCs caught in the middle. The proposed workaround seems reasonable, with a few caveats. 1) I'd suggest changing 'support' to 'allow' under the bold header *"Proposed Workaround"* 2) Include a *STRONGLY *worded disclaimer that* this is a **workaround* of an unfixable *technical issue *and that districts should* use it with caution. *Sort of a 'buyer beware' statement... 3) Include, if possible, the *likelihood of future fixes* that would *align the function* of tablets with the existing (or future) function of laptops and desktops. This could inform districts that are considering further investment in tablets for assessment. I hope this helps! Bill On Fri, Sep 25, 2015 at 3:13 PM, HEIDE Jordan wrote: > Dear Assessment Advisory Committee members, > > > > ODE is requesting your feedback on whether to change its policy regarding > allowing districts to use text-to-speech on tablets and Chromebooks. > > > > *Issue*: For technical reasons, the pause feature is not available for > text-to-speech (TTS) on tablets and Chromebooks. Students can stop TTS > reading in the middle of an item or passage, but there is no way to have > TTS resume reading where it was stopped. If the student activates TTS again > for the same item or passages it will start again at the beginning. The > pause feature, which allows TTS to resume reading where it was paused, is > available on desktops and laptops. > > > > *Current Policy*: ODE has disallowed the use of TTS on tablets and > Chromebooks on the basis that the inability to pause TTS and then resume > again at the place it was paused would disadvantage students testing on > tablets and Chromebooks when they encountered a long passage or item*. > It?s important to note that while this restriction is noted in all > documentation and user guides, AIR is not able to actually block TTS on > tablets or Chromebooks.* AIR and ODE received a lot of calls from > districts that were testing students on tablets and Chromebooks and wanted > to be able to use TTS. They questioned why it was not available and stated > that it was a great inconvenience to not be able to use it. > > > > *Proposed Workaround: *AIR has proposed some workarounds as an > alternative to the pause feature if ODE decides to support TTS on tablets > and Chromebooks in 2015-2016: > > ? Students can select the text they want read aloud and TTS will > read only the selected text. Therefore, if a student pauses a passage in > the middle and wants the TTS to resume where the passage was paused, the > student can highlight the remainder of the passage. > > ? We can include in the documentation the recommendation that if > a student will use TTS a desktop or laptop computer should be used for the > ELA performance task test, which is the test that has the longest passages. > > > > To inform ODE?s decision-making on whether to allow TTS use on tablets and > Chromebooks with the proposed workarounds, please respond with your > feedback by *Wednesday, September 30*. > > > > Thank you as always for your input! > > > > _____________________________________________________ > Assessment_Advisory_Committee mailing list > Assessment_Advisory_Committee at listsmart.osl.state.or.us > > http://listsmart.osl.state.or.us/mailman/listinfo/assessment_advisory_committee > Hosted by the Oregon State Library. The Library is not responsible for > content. > Questions related to message content should be directed to list owner(s) > or the sender of the message, by phone or email. > Technical questions? Call 503-378-8800. > > -- Bill Stewart Special Projects Gladstone SD stewartw at gladstone.k12.or.us 503-805-8680 - cell -------------- next part -------------- An HTML attachment was scrubbed... URL: From bbain at ttsd.k12.or.us Mon Sep 28 10:57:20 2015 From: bbain at ttsd.k12.or.us (Bain, Brian) Date: Mon, 28 Sep 2015 17:57:20 +0000 Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment In-Reply-To: References: Message-ID: <016B0BFF1DF04E44B5C8ED9AACF3243A7BFA367C@EX3.ttsd.ttsd.k12.or.us> I don't know that I'm particularly qualified to answer these questions, but here's my thoughts. I hope these are being asked for Special Education directors as well. 1. Is the purpose of the ORA clearly stated? Yes. 1. Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)? Yes. I appreciate that there is a clear threshold for when to discontinue to the ORext and transition to ORA. 1. Does the ORA appear to be a useful tool for the intended population (i.e., face validity)? Yes. Thought I'm not familiar with working with these students. 1. Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales? Yes. I appreciate the detail, and the example lesson for Level of Independence. I wonder if more examples may be necessary for some of the other domains/prompts, such as examples for anticipating/predicting coming events. 1. Are the ORA scoring instructions clear? Relatively, there's always going to be some level of judgment in a rating scale. 1. Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt? Yes. 1. Do you feel that clear directions regarding the use of the results from the ORA have been provided? Yes. 1. Do you have any recommended edits/changed to the ORA? Can we call it the Oregon Observational Rating Assessment (orORA)? "This student will not be taking the OAKS test; he will be taking ORext or orORA." ;) Brian Bain District Assessment Coordinator Tigard-Tualatin School District 6960 SW Sandburg St, Tigard, OR 97223 503-431-4120 - bbain at ttsd.k12.or.us From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us] On Behalf Of HEIDE Jordan Sent: Friday, September 25, 2015 7:48 AM To: assessment_advisory_committee at listsmart.osl.state.or.us Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment We'd like to request your review of the attached proposed Oregon Extended Assessment Observational Rating Assessment (ORA)(see attached Administration Instructions and ORA) designed to provide instructional and functional information for teachers and parents in in four domains: attention, math concepts, receptive and expressive communication. The ORA is designed to be administered to students who are not able to access the academic tasks presented on the Oregon Extended Assessment (ORExt), even when extensive, universal supports are provided. The ORA will be incorporated into the Extended Assessment beginning 2015-16. Given the description and purpose of the ORA, we are seeking feedback on the questions below by 9/30/15: 1. Is the purpose of the ORA clearly stated? 2. Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)? 3. Does the ORA appear to be a useful tool for the intended population (i.e., face validity)? 4. Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales? 5. Are the ORA scoring instructions clear? 6. Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt? 7. Do you feel that clear directions regarding the use of the results from the ORA have been provided? 8. Do you have any recommended edits/changed to the ORA? Please submit your feedback to Brad Lenhardt (brad.lenhardt at state.or.us). Finally, we would appreciate it if you would keep this information confidential at this time. Once we finalize the ORA we will distribute it through the typical ODE communication channels. Sincere thanks for your expert consideration and assist -------------- next part -------------- An HTML attachment was scrubbed... URL: From stewartw at gladstone.k12.or.us Mon Sep 28 11:24:49 2015 From: stewartw at gladstone.k12.or.us (Bill Stewart) Date: Mon, 28 Sep 2015 11:24:49 -0700 Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment In-Reply-To: References: Message-ID: Jordan & Brad, Please see my comments below, in CAPS....I hope they are useful. Bill On Fri, Sep 25, 2015 at 7:48 AM, HEIDE Jordan wrote: > We?d like to request your review of the attached proposed Oregon Extended > Assessment Observational Rating Assessment (ORA)(see attached > Administration Instructions and ORA) designed to provide instructional and > functional information for teachers and parents in in four domains: > attention, math concepts, receptive and expressive communication. The ORA > is designed to be administered to students who are not able to access the > academic tasks presented on the Oregon Extended Assessment (ORExt), even > when extensive, universal supports are provided. The ORA will be > incorporated into the Extended Assessment beginning 2015-16. > > Given the description and purpose of the ORA, we are seeking feedback on > the questions below *by 9/30/15*: > > 1. Is the purpose of the ORA clearly stated? YES, BUT IF IT WERE ME, I > WOULD REVERSE THE ORDER OF THE FIRST TWO SENTENCES...GIVE THE 'WHY &WHAT' > FIRST. > 2. Is the intended population who will participate in the ORA clear > YES, THE INTENDED POPULATION SEEMS CLEAR. HOWEVER THE DECISION RULE IS > *NOT* CLEAR! (i.e., is the decision rule used to determine whether to > administer the ORA clear)? THE QA INTRO STATEMENT AND THE ITALICIZED > STATEMENT SEEM OUT OF ORDER WITH THE PARAGRAPH THAT FOLLOWS. > 3. Does the ORA appear to be a useful tool for the intended population > (i.e., face validity)? ON THE SURFACE, IT DOES SEEM LIKE IT *COULD* BE > USEFUL, BUT I'D LIKE SOME SPECIFICS ABOUT HOW IT WILL ACTUALLY BE USED. > WHAT IS THE REQUIRED USE VS AN ALLOWABLE OR OPTIONAL 'BEST PRACTICE' USE > OF THE TOOL, IF ANY? > 4. Are the ORA administration instructions clear, particularly in the > areas of describing the LOI and COM scales? SORT OF... WHY ARE THERE TWO, > FAIRLY REDUNDANT DOCUMENTS? ALSO WHY ARE THERE NO CUES FOR SCORING ITEMS > 1-4 IN THE EXPRESSIVE DOMAIN? SEEMS ODD (CONFUSING TO USERS?) TO HAVE CUES > FOR THE ALL REACTIVE DOMAIN ITEMS AND JUST ONE ITEM WITH CUES IN THE > EXPRESSIVE. WHERE IS THE "STEP 1, STEP 2..." SEQUENCE IN THE ADMIN > DIRECTIONS? OBVIOUSLY, I THINK IT IS NEEDED. > 5. Are the ORA scoring instructions clear? GENERALLY, BUT IT WOULD BE > CHALLENGING FOR SOME USERS IF IT WEREN'T FOR THE SCORING AND SUMMARY PAGES. > 6. Do you feel that IEP teams will be able to use this information for > students who are so low functioning that they cannot access the academic > demands of the ORExt? PROBABLY, BUT SOME EDITING FOR CLARITY AND SEQUENCING > WOULD BE HELPFUL. > 7. Do you feel that clear directions regarding the use of the results > from the ORA have been provided? NOT AS CLEAR AS NEEDED. NOT LINEAR ENOUGH > FOR SOME USERS. & BRAD > 8. Do you have any recommended edits/changed to the ORA? SEE ABOVE. > > > > Please submit your feedback to Brad Lenhardt (brad.lenhardt at state.or.us). Finally, > we would appreciate it if you would keep this information confidential at > this time. Once we finalize the ORA we will distribute it through the > typical ODE communication channels. > > > > Sincere thanks* for your expert consideration and assist* > > _____________________________________________________ > Assessment_Advisory_Committee mailing list > Assessment_Advisory_Committee at listsmart.osl.state.or.us > > http://listsmart.osl.state.or.us/mailman/listinfo/assessment_advisory_committee > Hosted by the Oregon State Library. The Library is not responsible for > content. > Questions related to message content should be directed to list owner(s) > or the sender of the message, by phone or email. > Technical questions? Call 503-378-8800. > > -- Bill Stewart Special Projects Gladstone SD stewartw at gladstone.k12.or.us 503-805-8680 - cell -------------- next part -------------- An HTML attachment was scrubbed... URL: From CARLSON_CATHERINE at salkeiz.k12.or.us Mon Sep 28 12:17:36 2015 From: CARLSON_CATHERINE at salkeiz.k12.or.us (Catherine Carlson) Date: Mon, 28 Sep 2015 19:17:36 +0000 Subject: [Assessment_Advisory_Committee] Text-to-Speech on Tablets and Chromebooks In-Reply-To: References: Message-ID: I agree with Bill?s suggestions. Catherine Carlson Coordinator Testing & Evaluation/Student Records Department Salem-Keizer School District 503-399-5590 FAX 503-391-4088 Supporting student achievement through information, professional development, and assessment services From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us] On Behalf Of Bill Stewart Sent: Monday, September 28, 2015 8:24 AM To: HEIDE Jordan Cc: assessment_advisory_committee at listsmart.osl.state.or.us Subject: Re: [Assessment_Advisory_Committee] Text-to-Speech on Tablets and Chromebooks Jordan, Interesting situation! Since the 'problem' can't be fixed and the function can't be blocked, it leaves ODE and DTCs caught in the middle. The proposed workaround seems reasonable, with a few caveats. 1) I'd suggest changing 'support' to 'allow' under the bold header "Proposed Workaround" 2) Include a STRONGLY worded disclaimer that this is a workaround of an unfixable technical issue and that districts should use it with caution. Sort of a 'buyer beware' statement... 3) Include, if possible, the likelihood of future fixes that would align the function of tablets with the existing (or future) function of laptops and desktops. This could inform districts that are considering further investment in tablets for assessment. I hope this helps! Bill On Fri, Sep 25, 2015 at 3:13 PM, HEIDE Jordan > wrote: Dear Assessment Advisory Committee members, ODE is requesting your feedback on whether to change its policy regarding allowing districts to use text-to-speech on tablets and Chromebooks. Issue: For technical reasons, the pause feature is not available for text-to-speech (TTS) on tablets and Chromebooks. Students can stop TTS reading in the middle of an item or passage, but there is no way to have TTS resume reading where it was stopped. If the student activates TTS again for the same item or passages it will start again at the beginning. The pause feature, which allows TTS to resume reading where it was paused, is available on desktops and laptops. Current Policy: ODE has disallowed the use of TTS on tablets and Chromebooks on the basis that the inability to pause TTS and then resume again at the place it was paused would disadvantage students testing on tablets and Chromebooks when they encountered a long passage or item. It?s important to note that while this restriction is noted in all documentation and user guides, AIR is not able to actually block TTS on tablets or Chromebooks. AIR and ODE received a lot of calls from districts that were testing students on tablets and Chromebooks and wanted to be able to use TTS. They questioned why it was not available and stated that it was a great inconvenience to not be able to use it. Proposed Workaround: AIR has proposed some workarounds as an alternative to the pause feature if ODE decides to support TTS on tablets and Chromebooks in 2015-2016: ? Students can select the text they want read aloud and TTS will read only the selected text. Therefore, if a student pauses a passage in the middle and wants the TTS to resume where the passage was paused, the student can highlight the remainder of the passage. ? We can include in the documentation the recommendation that if a student will use TTS a desktop or laptop computer should be used for the ELA performance task test, which is the test that has the longest passages. To inform ODE?s decision-making on whether to allow TTS use on tablets and Chromebooks with the proposed workarounds, please respond with your feedback by Wednesday, September 30. Thank you as always for your input! _____________________________________________________ Assessment_Advisory_Committee mailing list Assessment_Advisory_Committee at listsmart.osl.state.or.us http://listsmart.osl.state.or.us/mailman/listinfo/assessment_advisory_committee Hosted by the Oregon State Library. The Library is not responsible for content. Questions related to message content should be directed to list owner(s) or the sender of the message, by phone or email. Technical questions? Call 503-378-8800. -- Bill Stewart Special Projects Gladstone SD stewartw at gladstone.k12.or.us 503-805-8680 - cell -------------- next part -------------- An HTML attachment was scrubbed... URL: From CARLSON_CATHERINE at salkeiz.k12.or.us Mon Sep 28 16:24:34 2015 From: CARLSON_CATHERINE at salkeiz.k12.or.us (Catherine Carlson) Date: Mon, 28 Sep 2015 23:24:34 +0000 Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment In-Reply-To: <016B0BFF1DF04E44B5C8ED9AACF3243A7BFA367C@EX3.ttsd.ttsd.k12.or.us> References: <016B0BFF1DF04E44B5C8ED9AACF3243A7BFA367C@EX3.ttsd.ttsd.k12.or.us> Message-ID: I agree with Brian's points, although I am wondering how to mark a student if they do not even perform at a level 1. I think the directions and criteria for administering the test are clear, and this option would could provide much more useful information for students who simply can't take the extended assessments. I assume that special ed. folks will have a chance to weigh in as well, since they know student needs the best. Catherine Carlson Coordinator Testing & Evaluation/Student Records Department Salem-Keizer School District 503-399-5590 FAX 503-391-4088 Supporting student achievement through information, professional development, and assessment services From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us] On Behalf Of Bain, Brian Sent: Monday, September 28, 2015 10:57 AM To: HEIDE Jordan; assessment_advisory_committee at listsmart.osl.state.or.us Subject: Re: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment I don't know that I'm particularly qualified to answer these questions, but here's my thoughts. I hope these are being asked for Special Education directors as well. 1. Is the purpose of the ORA clearly stated? Yes. 1. Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)? Yes. I appreciate that there is a clear threshold for when to discontinue to the ORext and transition to ORA. 1. Does the ORA appear to be a useful tool for the intended population (i.e., face validity)? Yes. Thought I'm not familiar with working with these students. 1. Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales? Yes. I appreciate the detail, and the example lesson for Level of Independence. I wonder if more examples may be necessary for some of the other domains/prompts, such as examples for anticipating/predicting coming events. 1. Are the ORA scoring instructions clear? Relatively, there's always going to be some level of judgment in a rating scale. 1. Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt? Yes. 1. Do you feel that clear directions regarding the use of the results from the ORA have been provided? Yes. 1. Do you have any recommended edits/changed to the ORA? Can we call it the Oregon Observational Rating Assessment (orORA)? "This student will not be taking the OAKS test; he will be taking ORext or orORA." ;) Brian Bain District Assessment Coordinator Tigard-Tualatin School District 6960 SW Sandburg St, Tigard, OR 97223 503-431-4120 - bbain at ttsd.k12.or.us From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us] On Behalf Of HEIDE Jordan Sent: Friday, September 25, 2015 7:48 AM To: assessment_advisory_committee at listsmart.osl.state.or.us Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment We'd like to request your review of the attached proposed Oregon Extended Assessment Observational Rating Assessment (ORA)(see attached Administration Instructions and ORA) designed to provide instructional and functional information for teachers and parents in in four domains: attention, math concepts, receptive and expressive communication. The ORA is designed to be administered to students who are not able to access the academic tasks presented on the Oregon Extended Assessment (ORExt), even when extensive, universal supports are provided. The ORA will be incorporated into the Extended Assessment beginning 2015-16. Given the description and purpose of the ORA, we are seeking feedback on the questions below by 9/30/15: 1. Is the purpose of the ORA clearly stated? 2. Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)? 3. Does the ORA appear to be a useful tool for the intended population (i.e., face validity)? 4. Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales? 5. Are the ORA scoring instructions clear? 6. Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt? 7. Do you feel that clear directions regarding the use of the results from the ORA have been provided? 8. Do you have any recommended edits/changed to the ORA? Please submit your feedback to Brad Lenhardt (brad.lenhardt at state.or.us). Finally, we would appreciate it if you would keep this information confidential at this time. Once we finalize the ORA we will distribute it through the typical ODE communication channels. Sincere thanks for your expert consideration and assist -------------- next part -------------- An HTML attachment was scrubbed... URL: From stewartw at bctonline.com Mon Sep 28 18:04:01 2015 From: stewartw at bctonline.com (Bill Stewart) Date: Mon, 28 Sep 2015 18:04:01 -0700 Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment In-Reply-To: References: <016B0BFF1DF04E44B5C8ED9AACF3243A7BFA367C@EX3.ttsd.ttsd.k12.or.us> Message-ID: Catherine, Without regard to the rest of the questions, I think your comment - ?I assume that special ed. folks will have a chance to weigh in as well, since they know student needs the best.? - is spot-on, especially the last part. Bill Bill Stewart 503-805-8680 - c > On Sep 28, 2015, at 4:24 PM, Catherine Carlson wrote: > > I agree with Brian?s points, although I am wondering how to mark a student if they do not even perform at a level 1. I think the directions and criteria for administering the test are clear, and this option would could provide much more useful information for students who simply can?t take the extended assessments. I assume that special ed. folks will have a chance to weigh in as well, since they know student needs the best. > > Catherine Carlson > Coordinator > Testing & Evaluation/Student Records Department > Salem-Keizer School District > 503-399-5590 > FAX 503-391-4088 > Supporting student achievement through information, professional development, and assessment services > > > > From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us] On Behalf Of Bain, Brian > Sent: Monday, September 28, 2015 10:57 AM > To: HEIDE Jordan; assessment_advisory_committee at listsmart.osl.state.or.us > Subject: Re: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment > > I don't know that I'm particularly qualified to answer these questions, but here's my thoughts. I hope these are being asked for Special Education directors as well. > > > Is the purpose of the ORA clearly stated? > Yes. > Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)? > Yes. I appreciate that there is a clear threshold for when to discontinue to the ORext and transition to ORA. > Does the ORA appear to be a useful tool for the intended population (i.e., face validity)? > Yes. Thought I'm not familiar with working with these students. > Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales? > Yes. I appreciate the detail, and the example lesson for Level of Independence. I wonder if more examples may be necessary for some of the other domains/prompts, such as examples for anticipating/predicting coming events. > Are the ORA scoring instructions clear? > Relatively, there's always going to be some level of judgment in a rating scale. > Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt? > Yes. > Do you feel that clear directions regarding the use of the results from the ORA have been provided? > Yes. > Do you have any recommended edits/changed to the ORA? > Can we call it the Oregon Observational Rating Assessment (orORA)? "This student will not be taking the OAKS test; he will be taking ORext or orORA." ;) > > > > Brian Bain > District Assessment Coordinator > Tigard-Tualatin School District > 6960 SW Sandburg St, Tigard, OR 97223 > 503-431-4120 - bbain at ttsd.k12.or.us > > From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us ] On Behalf Of HEIDE Jordan > Sent: Friday, September 25, 2015 7:48 AM > To: assessment_advisory_committee at listsmart.osl.state.or.us > Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment > > We?d like to request your review of the attached proposed Oregon Extended Assessment Observational Rating Assessment (ORA)(see attached Administration Instructions and ORA) designed to provide instructional and functional information for teachers and parents in in four domains: attention, math concepts, receptive and expressive communication. The ORA is designed to be administered to students who are not able to access the academic tasks presented on the Oregon Extended Assessment (ORExt), even when extensive, universal supports are provided. The ORA will be incorporated into the Extended Assessment beginning 2015-16. > Given the description and purpose of the ORA, we are seeking feedback on the questions below by 9/30/15: > Is the purpose of the ORA clearly stated? > Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)? > Does the ORA appear to be a useful tool for the intended population (i.e., face validity)? > Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales? > Are the ORA scoring instructions clear? > Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt? > Do you feel that clear directions regarding the use of the results from the ORA have been provided? > Do you have any recommended edits/changed to the ORA? > > Please submit your feedback to Brad Lenhardt (brad.lenhardt at state.or.us ). Finally, we would appreciate it if you would keep this information confidential at this time. Once we finalize the ORA we will distribute it through the typical ODE communication channels. > > Sincere thanks for your expert consideration and assist > _____________________________________________________ > Assessment_Advisory_Committee mailing list > Assessment_Advisory_Committee at listsmart.osl.state.or.us > http://listsmart.osl.state.or.us/mailman/listinfo/assessment_advisory_committee > Hosted by the Oregon State Library. The Library is not responsible for content. > Questions related to message content should be directed to list owner(s) or the sender of the message, by phone or email. > Technical questions? Call 503-378-8800. -------------- next part -------------- An HTML attachment was scrubbed... URL: From David.Marshall at miltfree.k12.or.us Tue Sep 29 07:08:35 2015 From: David.Marshall at miltfree.k12.or.us (Marshall, David) Date: Tue, 29 Sep 2015 14:08:35 +0000 Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment In-Reply-To: References: Message-ID: <96A4E26934AE77478523E4BB348DBC8F73C5670D@MFUSD-MAIL.mfusd.local> My apologies for the delay in responding. I agree with Brian and Catherine's summaries. This tool does add needed clarification to the test selection process for special needs students. In the past, this process was definitely one of the more "fuzzy" areas of state assessments. So, specifically addressing the question of the ORA, it seems to provide needed decision process guidance. It appears that the ORA and the extended assessment manual are intending to get us as a system to truly having only the most impacted 1-2% of disabled students in the state take the extended assessments. That part seems appropriate and valid. I am concerned about the combination of SBAC being potentially harder to access for special needs students AND the lack of a "3-5%" assessment. The gap between extended and OAKS was large. The gap between SBAC and extended appears to be ginormous (ok, really, really big). So while the ORA appears to provide some good guidance, the much larger issue is the lack of a more complete assessment system for our disabled students. Sorry if I got a bit off topic, but I'll take every opportunity I can to get on my "3-5%" soapbox. David From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us] On Behalf Of HEIDE Jordan Sent: Friday, September 25, 2015 7:48 AM To: assessment_advisory_committee at listsmart.osl.state.or.us Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment We'd like to request your review of the attached proposed Oregon Extended Assessment Observational Rating Assessment (ORA)(see attached Administration Instructions and ORA) designed to provide instructional and functional information for teachers and parents in in four domains: attention, math concepts, receptive and expressive communication. The ORA is designed to be administered to students who are not able to access the academic tasks presented on the Oregon Extended Assessment (ORExt), even when extensive, universal supports are provided. The ORA will be incorporated into the Extended Assessment beginning 2015-16. Given the description and purpose of the ORA, we are seeking feedback on the questions below by 9/30/15: 1. Is the purpose of the ORA clearly stated? 2. Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)? 3. Does the ORA appear to be a useful tool for the intended population (i.e., face validity)? 4. Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales? 5. Are the ORA scoring instructions clear? 6. Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt? 7. Do you feel that clear directions regarding the use of the results from the ORA have been provided? 8. Do you have any recommended edits/changed to the ORA? Please submit your feedback to Brad Lenhardt (brad.lenhardt at state.or.us). Finally, we would appreciate it if you would keep this information confidential at this time. Once we finalize the ORA we will distribute it through the typical ODE communication channels. Sincere thanks for your expert consideration and assist -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsuggs at pps.net Tue Sep 29 11:21:40 2015 From: jsuggs at pps.net (Joseph Suggs) Date: Tue, 29 Sep 2015 18:21:40 +0000 Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment In-Reply-To: References: Message-ID: My comments/thoughts (apologies if any of these are duplicative of others'). And I'll also note that a large-district perspective may be quite different on the utility of this tool than smaller districts. 1. Is the purpose of the ORA clearly stated? Yes 2. Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)? I think so, although perhaps some things should be stated even a little more explicitly. Specifically, if I understand correctly, students must complete at least 20 items on the ORExt to count for participation regardless of whether the ORA is used. What needs more clarity is whether this will be a required component or will be an optional tool. For Portland, unless this is mandated, I think it is extremely unlikely that any teacher will use it as we already have similar tools in place. In addition our teacher's union will likely see this as a workload issue if required and will want to take away one of our other assessments. One more question: is the decision to use the ORA tied to student abilities on specific ORExt domains? I'm guessing most teachers would be able to figure this out, but may be worth making more explicit in the instructions. 3. Does the ORA appear to be a useful tool for the intended population (i.e., face validity)? Yes, except that in Portland we already use similar (and slightly more comprehensive) tools that we would likely not replace with the ORA. Per comment above, if ORA is mandated for these students this will put us in a difficult position whether we have to drop something else (per union negotiation) that we would prefer to use over the ORA. 4. Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales? The only thing not clear to me here is whether there is another (secure) document that includes prompts or interaction scenarios that would be used. The instructions give a fairly concrete example (using a beverage) but the scoring instructions are more generic. Is there a set of prompts that will be used for the ORA? If not, is this assessment time bound/on demand or can teachers rely on (recent) past experience to complete ratings? Most of the content on the rating is already being attended to in the classroom. I'm rambling a bit; I guess my suggestion is more clarity is needed as to whether this is a constrained assessment or an looser, non-time-bound observational tool. 5. Are the ORA scoring instructions clear? Yes 6. Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt? Maybe not useful for these severely impacted students. As previously mentioned, in Portland we already do additional assessments for this group of students and couple that with how closely teachers know and work with students already, the utility for the teacher seems limited. 7. Do you feel that clear directions regarding the use of the results from the ORA have been provided? Yes 8. Do you have any recommended edits/changed to the ORA? What is the anticipated PD addition for this assessment? The instructions seem relatively clear so that additional training would be limited; but in some cases, may be needed. Joe From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us] On Behalf Of HEIDE Jordan Sent: Friday, September 25, 2015 7:48 AM To: assessment_advisory_committee at listsmart.osl.state.or.us Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment We'd like to request your review of the attached proposed Oregon Extended Assessment Observational Rating Assessment (ORA)(see attached Administration Instructions and ORA) designed to provide instructional and functional information for teachers and parents in in four domains: attention, math concepts, receptive and expressive communication. The ORA is designed to be administered to students who are not able to access the academic tasks presented on the Oregon Extended Assessment (ORExt), even when extensive, universal supports are provided. The ORA will be incorporated into the Extended Assessment beginning 2015-16. Given the description and purpose of the ORA, we are seeking feedback on the questions below by 9/30/15: 1. Is the purpose of the ORA clearly stated? 2. Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)? 3. Does the ORA appear to be a useful tool for the intended population (i.e., face validity)? 4. Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales? 5. Are the ORA scoring instructions clear? 6. Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt? 7. Do you feel that clear directions regarding the use of the results from the ORA have been provided? 8. Do you have any recommended edits/changed to the ORA? Please submit your feedback to Brad Lenhardt (brad.lenhardt at state.or.us). Finally, we would appreciate it if you would keep this information confidential at this time. Once we finalize the ORA we will distribute it through the typical ODE communication channels. Sincere thanks for your expert consideration and assist -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsuggs at pps.net Tue Sep 29 13:28:43 2015 From: jsuggs at pps.net (Joseph Suggs) Date: Tue, 29 Sep 2015 20:28:43 +0000 Subject: [Assessment_Advisory_Committee] Text-to-Speech on Tablets and Chromebooks In-Reply-To: References: Message-ID: I agree as well. I definitely support making this a recommendation rather than an impropriety when the interface allows it to be accessed when less appropriate. As long as we?re on the technology topic, I?d recommend a communication push out some time in the near future that re-caps current expectations for upcoming technology needs including whether a stylus will still be required in 2016-17 and perhaps even a technology recommendation or at least pros and cons list by type of device related to assessment use. That may already exist somewhere and I just need to catch up on my reading. Joe From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us] On Behalf Of Catherine Carlson Sent: Monday, September 28, 2015 12:18 PM To: Bill Stewart; HEIDE Jordan Cc: assessment_advisory_committee at listsmart.osl.state.or.us Subject: Re: [Assessment_Advisory_Committee] Text-to-Speech on Tablets and Chromebooks I agree with Bill?s suggestions. Catherine Carlson Coordinator Testing & Evaluation/Student Records Department Salem-Keizer School District 503-399-5590 FAX 503-391-4088 Supporting student achievement through information, professional development, and assessment services From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us] On Behalf Of Bill Stewart Sent: Monday, September 28, 2015 8:24 AM To: HEIDE Jordan Cc: assessment_advisory_committee at listsmart.osl.state.or.us Subject: Re: [Assessment_Advisory_Committee] Text-to-Speech on Tablets and Chromebooks Jordan, Interesting situation! Since the 'problem' can't be fixed and the function can't be blocked, it leaves ODE and DTCs caught in the middle. The proposed workaround seems reasonable, with a few caveats. 1) I'd suggest changing 'support' to 'allow' under the bold header "Proposed Workaround" 2) Include a STRONGLY worded disclaimer that this is a workaround of an unfixable technical issue and that districts should use it with caution. Sort of a 'buyer beware' statement... 3) Include, if possible, the likelihood of future fixes that would align the function of tablets with the existing (or future) function of laptops and desktops. This could inform districts that are considering further investment in tablets for assessment. I hope this helps! Bill On Fri, Sep 25, 2015 at 3:13 PM, HEIDE Jordan > wrote: Dear Assessment Advisory Committee members, ODE is requesting your feedback on whether to change its policy regarding allowing districts to use text-to-speech on tablets and Chromebooks. Issue: For technical reasons, the pause feature is not available for text-to-speech (TTS) on tablets and Chromebooks. Students can stop TTS reading in the middle of an item or passage, but there is no way to have TTS resume reading where it was stopped. If the student activates TTS again for the same item or passages it will start again at the beginning. The pause feature, which allows TTS to resume reading where it was paused, is available on desktops and laptops. Current Policy: ODE has disallowed the use of TTS on tablets and Chromebooks on the basis that the inability to pause TTS and then resume again at the place it was paused would disadvantage students testing on tablets and Chromebooks when they encountered a long passage or item. It?s important to note that while this restriction is noted in all documentation and user guides, AIR is not able to actually block TTS on tablets or Chromebooks. AIR and ODE received a lot of calls from districts that were testing students on tablets and Chromebooks and wanted to be able to use TTS. They questioned why it was not available and stated that it was a great inconvenience to not be able to use it. Proposed Workaround: AIR has proposed some workarounds as an alternative to the pause feature if ODE decides to support TTS on tablets and Chromebooks in 2015-2016: ? Students can select the text they want read aloud and TTS will read only the selected text. Therefore, if a student pauses a passage in the middle and wants the TTS to resume where the passage was paused, the student can highlight the remainder of the passage. ? We can include in the documentation the recommendation that if a student will use TTS a desktop or laptop computer should be used for the ELA performance task test, which is the test that has the longest passages. To inform ODE?s decision-making on whether to allow TTS use on tablets and Chromebooks with the proposed workarounds, please respond with your feedback by Wednesday, September 30. Thank you as always for your input! _____________________________________________________ Assessment_Advisory_Committee mailing list Assessment_Advisory_Committee at listsmart.osl.state.or.us http://listsmart.osl.state.or.us/mailman/listinfo/assessment_advisory_committee Hosted by the Oregon State Library. The Library is not responsible for content. Questions related to message content should be directed to list owner(s) or the sender of the message, by phone or email. Technical questions? Call 503-378-8800. -- Bill Stewart Special Projects Gladstone SD stewartw at gladstone.k12.or.us 503-805-8680 - cell -------------- next part -------------- An HTML attachment was scrubbed... URL: