FAQ: Quill Reading for Evidence Frequently Asked Questions

Table of Contents:

1. What is Quill Reading for Evidence?
2. What distinguishes Reading for Evidence from other online reading tools?
3. What is the content of Reading for Evidence activities?
4. What will students experience in a Reading for Evidence activity?
5. How is Quill's artificial intelligence (AI) Feedback Bot able to provide students with the right feedback on their original writing?
6. Where do Reading for Evidence activities fit with students’ other work on Quill? What kind of pacing do you recommend?
7. How is Reading for Evidence's feedback scaffolded and differentiated?
8. What responses will not be accepted in a Quill Reading for Evidence activity?
9. Will Reading for Evidence's feedback be accurate 100% of the time? What should students do if it's not accurate?
10. Why can't I create Quill Evidence activities with a text and/or stem of my choosing?
11. What are some best practices for using Reading for Evidence in my classroom?
12. For whom are these activities most appropriate? How does the tool support students who may need more scaffolding, like multilingual learners and students with IEPs?
13. With which standards do Reading for Evidence activities align?
14. How can I access & assign Reading for Evidence activities?
15. How can I assess student performance & progress in Reading for Evidence activities?
16. How has Quill ensured that Reading for Evidence activities are accessible to people with disabilities?
17. What research informed the development of Reading for Evidence?
18. Where can I share my feedback on Reading for Evidence?
19. Where can I get support and/or find resources related to Reading for Evidence?

1. What is Quill Reading for Evidence?

Quill Reading for Evidence is Quill's newest learning tool—and our first tool that supports students in building both language and reading skills. Reading for Evidence provides short, nonfiction texts for students and then engages them through open-ended writing prompts. Quill's artificial intelligence (AI) Feedback Bot automatically evaluates student responses and serves custom feedback—per prompt—to coach students to strengthen their writing and better represent key ideas from the text. Designed for students in 8th-12th grade, each activity takes 15-20 minutes to complete.

2. What distinguishes Reading for Evidence from other online reading tools?

In most online reading tools, students read a passage and then answer multiple-choice questions. Many students read the questions and answer choices first, then just scan the text for the “right answer,” moving on as soon as they’ve found it. Alternatively, in a Reading for Evidence activity, students must have read and understood what they read in order to complete the sentence stems effectively. They have to think and write about the content, not just select A, B, C, or D. What’s more, when students struggle, the tool coaches them—showing them where to reread, suggesting information to include in their response, etc. Then, immediately, they get the chance to implement that feedback and to revise, multiple times.

With Reading for Evidence, not only do you not have to provide the feedback, but students get up to 15 rounds of feedback in each activity, and the opportunity to revise in response, in real time. The feedback isn’t generic like “Be more specific” or “That’s incorrect.” Each activity’s feedback is custom-built for the particular text and writing prompts—allowing for much more targeted feedback. We’ve also put a lot of thought into the tone, style, and content of our feedback—consulting with experts in the field of writing instruction and feedback, having teachers review the feedback, and analyzing student responses to various kinds of feedback. We’ve designed the feedback to guide students in asking themselves the kinds of questions that effective writers ask themselves as they write. With practice, students internalize these questions, ultimately transferring that thinking to their independent writing outside of the tool.

3. What is the content of Reading for Evidence activities?

Reading for Evidence activities feature non-fiction passages written at an 8th-9th grade reading level. These texts cover a wide range of topics, from 21st-century real world issues to major historical moments and key literary themes.

Our growing Evidence library includes activities aligned to several free curricula, including:

• WordGen Weekly, an interdisciplinary curriculum developed by the Harvard Graduate School of Education and the SERP Institute.
• World History: 1200 CE - Present, a high school history course developed by OER Project.
• The free AI literacy curriculum created by aiEDU, a non-profit that creates equitable learning experiences that build foundational AI literacy.

These curriculum pairings are optional—our Reading for Evidence activities are designed to fit seamlessly into a variety of English, social studies, and science classrooms!

4. What will students experience in a Reading for Evidence activity?

After students read the text in a Reading for Evidence activity, they are prompted to highlight sentences about a key idea, and to write 3 sentences about a key idea using the connectives because, but, and so. As students write and revise, Quill’s Feedback Bot coaches them to improve the accuracy and specificity of their evidence, refine their logic and syntax, and correct any grammatical errors.

Like in Quill Connect, students can make up to 4 revisions for each prompt, and teachers can see all of students’ revisions in the Activity Analysis report.

You can view a video demo of the tool as well read a detailed walk-through of the tool here. We also encourage you to play through an activity yourself!

5. Where do Reading for Evidence activities fit with students’ other work on Quill? What kind of pacing do you recommend?

We recommend using Reading for Evidence activities once students have at least started working on their diagnostic recommendations, and they certainly are great to use any time after students have worked their way through those. We generally consider a healthy Quill “diet” to be 2-4 Quill activities per week. Given that Evidence activities are a bit more intensive and time-consuming than our other independent practice activities, we recommend expecting students to complete 1-2 Evidence activities a week, or about 10 per semester.

We recommend introducing and framing both the "because/but/so" strategy and the Evidence tool itself before students begin activities. We have a great pack of activities that does just that! The pack provides students with increasingly-challenging practice with the connectives in the Quill Connect and Quill Proofreader tools, and culminates with two Reading for Evidence activities. We recommend doing the first Evidence activity as a whole class and then having students do the second with a partner. You can check out that pack here.

There are also a number of Quill Lessons you can use to introduce the connectives themselves (Quill Lessons is our whole-class, synchronous instructional tool. Each Quill Lesson provides a lesson plan and interactive slides to teach a particular grammatical concept). You can find them in the activity library by filtering to Lessons and then to the concepts of “Complex Sentences” and “Compound Sentences.”

6. How is Quill's artificial intelligence (AI) Feedback Bot able to provide students with the right feedback on their original writing?

First our Curriculum team writes a text for a Reading for Evidence activity, as well as the sentence stems students will complete to show their understanding of that text. In order to ensure the AI can be highly accurate, 33-66% of each text includes evidence that can be used to answer the sentence stems. The curriculum team then comes up with categories of strong and weak responses (such as “responses that incorrectly focus on X” or “responses that provide a reason rather than an effect”), write at least one hundred responses for each of those categories, and then expand the responses that fall into these categories using machine learning. They also write custom feedback for each of those categories--usually several pieces (layers) of feedback for each. This feedback is highly specific to that activity's text--this is what makes it so special! (of course this is also why it takes many hours to create just one Evidence activity!).

Next, the team trains Quill's AI Feedback Bot to compare a student's response to each of those categories, determine which category the response falls into, and therefore, which feedback it needs. At this point, the activity is published to a small cohort of teachers who have opted-in to beta testing these early-stage activities with their students (Interested? Email hello@quill.org and let us know you want to beta test new Evidence activities!). Once the team has collected hundreds of authentic student responses, they can expand and refine those categories and their corresponding feedback, and retrain Quill's AI Bot. The activity goes through another, wider round of testing, its categories and feedback get refined again, and then, it is published to everyone on Quill!

At this point, the activity's feedback has a high degree of accuracy (although it's never 100%). The team also regularly checks up on published Evidence activities to make sure their feedback algorithms are performing well and to make any needed tweaks. Through this methodical, iterative process, Quill's curriculum team is able to "teach" Quill's AI Bot to serve specific, targeted feedback that coaches students to refine their logical reasoning, clarify their ideas, provide specific, precise evidence, and correct any grammatical or mechanical errors.

For reference, below is a list of just a few of Reading for Evidence’s feedback algorithms. Evidence will provide actionable feedback if students:

• Write more than one sentence or more than 100 words
• Write a response that is less than 2 words
• Use profanity
• Use because of to complete the because prompt
• Use so that to complete the “so” prompt
• Use should, must, or ought (expressing opinion)
• Give a command and/or begin a response with a verb
• Use first- or second-person pronouns
• Copy word-for-word (or nearly word-for-word) from the text
• Misuse commonly confused words such as there, their, and they’re
• Make a subject-verb agreement error
• Omit an apostrophe when using a contraction
• Omit ending punctuation
• Misuse an article
• Make a spelling error

7. How is Reading for Evidence's feedback scaffolded and differentiated?

The most unique aspect of Reading for Evidence is that it provides students with targeted and actionable feedback based on the specific misconceptions and/or areas of growth they have demonstrated in their submissions. This makes for a customized experience for each student. As a teacher expressed to us upon seeing Reading Evidence for the first time: "Individual feedback is the ultimate form of differentiation."

There are also multiple layers of feedback algorithms for the same “category” of response—why? Just as you would not give a student who made the same mistake a second time the exact same feedback, Evidence won't either! When students make the same kind of error again, they will receive a second layer of feedback--feedback that addresses the same issue, but with a different question and usually with more scaffolding. For example, a second layer of feedback might highlight the specific paragraph of the text that they misunderstood or missed and need to reread.

Feedback in Evidence is also prioritized—so if a student has written a response with faulty logic or lacking evidence, as well as mechanical and spelling errors—feedback about that faulty logic or lack of evidence is going to be triggered first. We don’t want students wasting time and energy adding punctuation or correcting misspelled words when they actually need to start over, go back to the text, and write something quite different.

8. What responses will not be accepted in a Quill Reading for Evidence activity?

Quill Reading for Evidence activities are designed for students to use evidence from a passage to complete three sentences. Our feedback bot will guide students through revising their sentences to use strong evidence, accurate mechanics, and correct spelling. There are some responses that the feedback bot will not accept that we have outlined below.

1. Using the same evidence for multiple stems. While a piece of evidence may work for multiple conjunctions in one activity, our passages are designed so that the strongest evidence for each conjunction is always different. If students use the same evidence for the "but" and the "so" stems but that evidence is strongest for the "but" stem, Quill will push them to select a different piece of evidence for the "so" stem. We want to encourage students to find the strongest possible evidence for each conjunction and to give students the opportunity to use evidence from multiple parts of the passage.

2. Using plagiarized evidence. Quill Reading for Evidence activities discourages plagiarism by giving students feedback when they've used ten or more consecutive words from the passage in their response. We selected ten or more consecutive words as the cutoff because some of the passages include specific or technical language that students may not be able to paraphrase, so in some instances using under ten consecutive words may be unavoidable. However, responses that use ten or more consecutive words can typically be partially or entirely paraphrased in the student's own words. Furthermore, the tool may also give feedback on shorter phrases that are taken directly from the passage when those phrases can be easily paraphrased.

3. Using opinions and first-person point-of-view. Quill Evidence requires that students provide evidence-based responses. This requirement allows Quill to narrow down the variety of potential responses, making the feedback more effective. For this reason, you can expect that your students will see similar feedback if using opinionated language or first-person pronouns.

4. Using "because of" and "so that." We want to encourage students to use the conjunction "because" to show a relationship of cause using evidence from the text, so students who start a "because" response with "of" will be directed to instead start their response using a noun and focus on a consequence. Furthermore, when a student responds with a "so" response that has the sentiment of a "so that" response, they will receive feedback that encourages them to shift their focus to a consequence and start their response using a noun.

5. Profanity. Students cannot include any profanity in their responses. If they do, they will receive the feedback to revise their work and make sure to use appropriate language.

6. Colloquial spellings. We discourage colloquial spellings of words like "gotta," "shoulda," "wanna," etc. by guiding students to update the spelling of these words.

9. Will Reading for Evidence's feedback be accurate 100% of the time? What should students do if it's not accurate?

The feedback won't be accurate 100% of the time (although we're getting pretty close!), and it’s good for students to know this! If they do receive feedback they find confusing, please encourage them to let us know during the activity--either by clicking the "Report a Problem" button under the feedback, or on the optional emoji survey at the end of every Reading for Evidence activity. Please also encourage them to do their best to implement the feedback that they receive, even if it feels off.

Please also let them know that each activity’s feedback will get more and more accurate as more students complete the activity (since Quill is regularly retraining the algorithms in each activity based on those authentic student responses).

10. Why can't I create Quill Evidence activities with a text and/or stem of my choosing?

While we would love to allow you to customize activities or create your own activities, the complex artificial intelligence systems we use are unique to each activity and text. Each activity uses its own set of artificial intelligence models that have been trained and calibrated by a team of experts to provide feedback on a specific passage and stem. This means that even making a small change to the text or stem requires dozens of hours of technical work. If you're looking for a Quill tool that allows you to customize content, check out Quill Lessons!)

11. What are some best practices for using Reading for Evidence in my classroom?

The following are some of our recommendations for integrating Reading for Evidence into your instruction and curriculum--read the full article about them here!

1. Introduce students to Reading for Evidence and the connectives they will be using. Frame the practice they are about to do. Explaining the function of the connectives because, but, and so and providing examples can be particularly helpful. Click here for a handout you can use with students. It can also be very helpful to walk students through the onboarding slides at the beginning of an Evidence activity (they'll see these on their first 3 Evidence activities) and emphasize that the tool will coach them to write sentences that are: evidence-based, specific, and original. Check out this article for more ideas for introducing students to the tool.

2. Use a gradual release of responsibility, beginning with modeling, for your students’ first Reading for Evidence activity or activities. After you’ve introduced the tool and walked students through its onboarding, a gradual release approach can be very effective. Read more about using this approach with Evidence here.

3. Align Reading for Evidence activities to your curriculum when/where possible and invite students to make connections. Help students make connections between the content of activities and your course content. For example, you might assign "Are Wind Farms a Good Alternative Energy Source?" after students have read the chapter in The Boy Who Harnessed the Wind in which the protagonist builds a windmill.

4. Extend the practice students are getting with the connectives because, but, and so. Have students use because, but, and so to extend their responses about course content. You can also consult the free WordGen Weekly resources for additional lessons and activities on some Reading for Evidence topics. This article has for more ideas for using the connectives in course content.

12. For whom are these activities most appropriate? How does the tool support students who may need more scaffolding, like multilingual learners and students with IEPs?

This first iteration of the Reading for Evidence tool is most appropriate for general education students in grades 8-12. However, the tool can provide valuable practice for students who don't fall into this category. Here are some ways you can support students who may need additional scaffolds:

• Introduce the main idea of the text and front-load vocabulary. For example, images/illustrations, videos, magazine/newspaper articles, realia, etc. can be used to help students access an activity text and activate relevant schema.
• Use browser extensions to provide read aloud and language support. Over the next several months and years, we will embed more scaffolds within the tool. For example, students will be able to hover over challenging vocabulary to see a student-friendly definition, etc. However, in the meantime, Evidence integrates with a variety of extensions that can provide important support.
• Model how to find evidence and how to incorporate feedback. There is so much metacognition at work when completing these activities. Pulling back the curtain on that for students can be powerful—and empowering!

This article expands on the tips above and links to extensions that integrate with Evidence.

13. With which standards do Reading for Evidence activities align?

The skills practiced in Reading for Evidence activities most closely align with the following English Language Arts Common Core Standards:

• RI.9-10.1: Cite strong text evidence to support analysis of what the text says explicitly.
• RH.9-10.2: Determine the central ideas or information of a primary or secondary source; provide an accurate summary of how key events or ideas develop throughout the text.
• W.9-10.1b: Develop claim(s) and counterclaims fairly, supplying data and evidence for each while pointing out the strengths and limitations of both claim(s) and counterclaims in a discipline-appropriate form and in a manner that anticipates the audience’s knowledge level and concerns.
• L.9-10.1: Demonstrate command of the conventions of standard English grammar and usage when writing or speaking.
• L.9-10.2: Demonstrate command of the conventions of standard English capitalization, punctuation, and spelling when writing.

14. How can I access & assign Reading for Evidence activities?

You can view and assign Reading for Evidence activities individually from the Activity Library, or in thematic packs from the Featured Activity Packs page.

15. How can I assess student performance & progress in Reading for Evidence activities?

Quill has several reports that include data about student performance in Reading for Evidence activities:

• The Activity Summary Report
• The Activity Analysis Report
• The Concepts Report
• The Standards Report

Learn more about how Evidence activities are scored, and how to access Quill data reports.

16. How has Quill ensured that Reading for Evidence activities are accessible to people with disabilities?

Reading for Evidence was built with accessibility in mind from the start to ensure there are no barriers for those with visual, auditory, motor, or cognitive disabilities. We want the tool to work for everyone. Evidence complies with the Web Content Accessibility Guidelines (WCAG) 2.1, Level A, and AA. If you have accessibility concerns, please reach out to us at hello@quill.org.

17. What research informed the development of Reading for Evidence?

The writing practice students do in Reading for Evidence is based on a strategy in which students use the connectives because, but, and so to expand a provided sentence stem about content they are learning. The tool and the practice it provides is also based on a wealth of research on writing instruction and the writing-to-learn approach. You can learn more about the tool's evidence-base here.

18. Where can I share my feedback about Reading for Evidence?

This is a new tool and we are eager to hear from you! We strive to ensure that our activities are accurate, appropriate, culturally responsive, and accessible, and we take concerns about an activity seriously. Please email support@quill.org with any concerns, and your message will be directed to a member of our curriculum team who can help. You can also share your feedback or requests about the tool here.

19. Where can I get support and/or find resources related to Reading for Evidence?

If you need specific support or want to report an issue, you can live chat with our support team through the green message bubble on the bottom right corner of any page on Quill, or email us at support@quill.org.

We also encourage you to explore the library of resources designed to support your implementation of Reading for Evidence.