Program Evaluation Appendix #2: Individual Student Responses

APPENDIX #2

Individual Student Responses

Laverne’s Response

Evaluator #1 – Kathryn M.

Do you agree with the evaluations or not? Why or why not?

a.  A rubrics is certainly a requirement for not only the students to follow their progress, but also to understand what teachers’ expect from the student. Including a rubric was an oversight.

b.  Basic vocabulary instructions including illustrations is a consideration. Establishing instructions in a requirements document that include the examples Kathryn listed is imperative for the student to recognize during the course readings.

c.  Many years ago I studied American Sign Language for two semesters. The first day of the first semester the teacher did not speak for the first 15 minutes of the class. She wrote sentences on the board and then signed the sentences to the class. By presenting the class with sentence frames to communicate furnished us, the students, with the ability to understand her expectations of us when she would point to a student for a response. Therefore, I completely agree with providing sentence frames throughout the program.  

Evaluator #2 – Bertha L.P.

Do you agree with the evaluations or not? Why or why not?

a.  I agree with Bertha’s suggestion to determine the proficiency level of the program. Blending students with different skill-sets; beginners may cause frustration or a proficient English speaking student may cause boredom. Assessing the students’ abilities prior to the start of the lesson is helpful in grouping and planning for the curriculum. Without understanding the students’ ability to read or speak English, the teacher will be challenged with delivering a productive and meaningful lesson.

b.  I disagree with Bertha’s recommendation of familiarizing yourself (the teacher) with stories of the students’ ethnic backgrounds. There are too many stories from ethnic backgrounds to comb through and understand. I found that child stories or folktales can be obscure, therefore ask the students’ to submit a list of stories they are interested in practicing to complete the assignment for the teacher's approval. By providing a list, the teacher has the ability to omit unwanted tales.

c.  Offering the students to present to the community may be intimidating. Especially for those students who are not comfortable with their speech. However, celebrating success is important for all students.

How would you improve your program design? In your evaluation paper, you need to specifically tell us HOW you want to improve your design based on the comments from your reviewers, not just what you will do. For example, it is not enough to just say: I would gather the program information. You need to tell us how. You need to transform your ideas into operational strategies.

After reading the professionals evaluations, I realized they delivered constructive criticism to develop a successful and meaningful program for ESL students. Distributing an aptitude survey either prior to the start of the program or the first day of the session to address the needs of every student will help in grouping by skill-set.

The importance of rubrics is to enhance students’ ability to communicate their ideas and become accountable for their work. Therefore, for the teacher to evaluate the student's’ performance, our program must include a rubric with step-by-step instructions of expectations for each session, #1 – #4. Examples of frame sentences, along with online links to learning vocabulary, suggested materials, plus a list of books and folktales used in previous classes ought to be included in the rubric as well.

Celebrating with fellow classmates, family, and friends at the end of the program to recognize the achievements of every student with a ceremony presenting certifications is a possibility. Also, offer the students an opportunity to speak of their journey towards learning English as a second language.

Marti’s Response

Do you agree with the evaluations or not?  Why or why not?

I appreciate both evaluators taking time from their busy schedules to evaluate our program.  Their experience and insights are invaluable.

Both evaluators complimented the modeling of storytelling and multiple practice opportunities before participants tell their own stories for the class and the camera. Since I felt that both were very important, I appreciate the confirmation.  

My own experience as an elementary teacher shows! For example, during group sessions, the teacher is to tell “The Three Little Pigs,” and then lead a brief discussion of the story. One evaluator asked, “Brief discussion on what?” An elementary teacher would take for granted what to ask, point out, etc. However, this program is for adults. I can see that some of the comments we received are actually pointing out the difference between writing instructions for an adult program and writing lesson plans for an elementary classroom.

Both evaluators suggested adding a rubric or other evaluation instrument to give participants more feedback. Since the program was to be informal, I didn’t include a formal evaluation instrument, hoping that feedback from fellow participants and from people who would visit the website later would suffice. Either we need to emphasize the informality of the program from the beginning, or we need to develop an appropriate rubric with instructions for its use. I suspect the latter will make the program more practical and more likely to be implemented.   

One of the evaluators suggested that the teacher study stories from the students’ ethnic backgrounds before the beginning the program.While I can see how such a familiarity would help the teacher help students select stories and do translations, unless the teacher was working with a group of students from one specific background, I’m concerned that this could be overwhelming. Without knowing well in advance who will be part of the class, the teacher can’t know which cultures’ stories to learn.  Also, I would be leery of limiting participants to stories the teacher is familiar with.   Part of the experience is for each participant to select a story, whether it be an old fable or a childhood experience, that has meaning for the participant.

How would you improve your program design?

In designing the program, we imagined adult English Language Learners who were already studying American English in other settings and were already somewhat fluent in the language. However, we didn’t describe a level of proficiency required to participate. Bertha L.P., one of our evaluators, wrote, “No mention of the students’ level of proficiency indicated in the program. Assessing the students’ ability to understand basic common everyday expressions is critical to their being able to produce/use the new language,” (P., 2016, point 3). Kathryn M., another evaluator, wrote, “Pre-teaching basic vocabulary will help students to understand the expectations and projects. Consider using terms, definitions, and images to help students understand key words,” (M., 2016, point 3). In order to address these concerns, I would suggest that we:

• Work with an experienced teacher of adult English Language Learners to develop appropriate entrance requirements. This could be a combination of a vocabulary test, conversational skills test, certificate of completion from an adult ELL program, teacher recommendation, and the like. An experienced teacher could give us guidance as to the skill level needed to participate in this project.
• Test-run the program with people for whom English is a second language but who have lived in the United States for multiple decades. Their feedback would be invaluable, as they could tell us what they had struggled with and what came easily, as well as what was enjoyable and what was stressful, when learning American English.

Both evaluators suggested adding a rubric or other evaluation instrument to give participants more feedback. In order to accomplish this, I suggest that we:

• Work with a teacher who is experienced in teaching languages to adults and develop an appropriate rubric.  
• Video an adult model telling a story and then a teacher evaluating the presentation using the rubric so that participants would understand better how the evaluation will be accomplish.
• Create a certificate of completion to give participants at the end of the program.
• “Beef up” computer training so that participants can view the feedback they receive online.

One of the evaluators mentioned making the presentation available to a wider community. Although the stories will be posted online, where they can be seen by anyone with Internet access, community availability should be better facilitated:

• Partner with a preschool teacher who will agree to show her students the videos and to type their feedback.  This helps program participants tailor their presentations to a specific audience and ensures that there will be an audience outside of the class itself.
• Direct future classes to view the previous classes’ websites. This bolsters the audience and also gives future classes examples to follow.

Mandy’s Response

Yes, I do agree with both evaluations. I thought both individuals were able to delve into the program, and see some issues that could be problematic if not addressed. I hadn't considered the different levels of English proficiency between the learners. Including a rubric would be important as well. I did not necessarily agree that the instructor needed to be familiar with the folktales prior to the course, as I feel that the topic should be open for the student to pick. I also thought a celebration at the end of the program was a good idea. Depending on the comfort level of the students, perhaps family and friends could be invited and then the student's work could be used in material advertising the course for later terms. I feel more confident in the program, after having these professionals review and validate our program. I would recommend the following improvements to our program design:

• Establish and include a rubric for the expectations of the presentation.
• Illustrations of the stories used as examples should be incorporated, storybooks for example.
• Determining the proficiency of the students prior to the course, so that students at similar levels can be grouped.
• When the students complete their group work, perhaps rotating leadership and responsibilities so that all can contribute (similar to our own group project).
• The instructor can show his/her own video as an example.

Kylies’s Response

Do you agree with the evaluations or not? Why or why not?

I do agree with both of the evaluations. Both of the evaluations gave great feedback about being more specific and giving more details. Adding more details and structure to the program design will allow our design to be clearer and avoid misunderstandings. It will also allow for the person looking at our program design to get a more concrete visual of how the program will work.  Adding some additional details will avoid unnecessary questions and make for a more concrete program design.

How would you improve the design?

I would improve the design of our program by simply adding some more specific details.  I would give the students a variety of examples of what is expected from them such as the teacher writing on the board, visually showing them examples of examples of the project, YouTube video, or giving them a paper explaining what is expected from them. This would provide a better set of directions that lay out step by step what they need to do.

In order to have a class that may have a variety of students at different skill levels there will be a to need implement some form of basic testing to get a better idea of where the student starts and the progression they make. This will give a better understanding of how much actual progress was made through the course of the program. This will also give a better understanding of where to start each student at. The program will do best if the teacher uses translated texts, peers, and even volunteers that are fluent in the language as well as English. Every day before breaking off into small group activities the teachers, assistant teachers, and any volunteer will use that time to introduce themselves that follow a specific framework.  That will be written on the board. My name is Kylie and I am from Indiana. Every day the teacher can add another aspect to the introduction.  My name is Kylie and I am from Indiana. I work as an ASL interpreter.  Using the fill in the blank method (with their own information) the students will have examples from the teacher, assistant, and volunteers but also be able to see it visually on the board. The students will also be allowed the option to tell a story from memory or a story that they originally learned in their native language in English. To get to the level where they are comfortable doing this we can use picture books that will help aid the students in interpreting different stories.

Allow structured time to go over words that may have arose in the books or conversations that may not be known.  Then writing the words on the board. Also, putting the words into context. A good way to help students understand is by reading example stories and then showing a video of the same story being read will hopefully allow the students to better retain the information. The social aspect is a very needed piece to this program design. Having group activities daily that allow them to work in small groups will allow a social interaction but also allow them the chance to improve their language skills by simply interacting. Each day after introductions have the students pair off and take turns telling the story of their day. This can be used as a social aspect as well as a warm up activity. This will allow to them to tell a narrative about themselves that will include things they use or do daily. This could be a huge help to some students that are lacking communication skills needed throughout their daily routines.  There will always be options for different forms of support. Support could be iPad that translate, pictures, word cards, and visual aids if needed. This should be something that will help build their confidence and their language skills.  

        The progression of this program will take the form of finding out what level each student is at and discussing their options of appropriate stories for their level. Then when that level is mastered moving to harder stories. When each level is mastered the student will continue to move onto harder levels that will contain more details and substance. The further the student goes in the program, the more detailed the stories they tell and the more informative they will be needed to move to the next level. The teacher will meet with the students individually to help them select a story that will meet the set of guidelines that are appropriate for whichever level the student falls into. Guidelines could include difficulty level, word choices, pronunciation, and general understanding of the text.

When the students are to the point that they have to tell their story in front of the class the class will be given a peer evaluation. This will allow the student to better understand how they are doing from other English as a second language students. This will also allow for the student to know their strengths and areas that they need to work on improving.

        During the week between session #2 and #3 the teacher will meet with the students allowing a time to go over any questions and concerns they may have. Also, the student can read or tell their story to the teacher and this could be used as a comparison for how much improvement has taken place between this point and the final project. Session #3 we will schedule time for each conference. When it is time to post their story the students will be given directions and walked through the entire process by example. The teacher will post a story they have completed and use that to show the class using a computer and projector. During session #4 the student will be given feedback on a template that they would have been given previously to completing the assignment. That way the student will know exactly what is expected of them.  This template will allow for a set number of comments that will focus on positive and how they can improve.  At the end of the program the students would be given an evaluation so they can make suggestion of how to improve the program. This will allow for the program to constantly be able to grow and improve.  All of these are ways that could be used to improve our program design as a whole.  

Kim’s Response

I agree with most of the feedback. Many of the comments seemed to be about clarifying what level our learners were at. Once we establish that, it would clear up a lot of the questions the reviewers had. More specific instructions need to be provide and a clearer definition of at what level the program could be appropriately used. While this program could be adapted to many different levels of English language learners, a base level should be established for our particular program so we can provide clear expectations of the learners as well as better define the expected outcomes. ***As a group we need to define this*** It should also be made clear whether all the students will have the same “L1”. I do not think this is necessary, overall but for the sake of our program design, it might be easiest.

We will clearly define that a basic literacy level in L1 is required of all learners, something equivalent to a U.S. Middle School literacy level would be enough.

The instructor will provide level appropriate examples of each stage of the program, from introductions through storytelling and commenting.

In order to connect the stories to their lives, students will introduce their story. They will explain how they know the story. For example, when I was little, my mom read to me. This story was my favorite. She read it every night. Post storytelling conversations/comments will include peers saying whether or not they know the story, what they liked about the story, what the person could do to improve, and ask one question about the story or how the introduction. For example, “What is porridge? (Goldilocks)” or “What other stories did your Mom read to you?” These questions are not necessarily for cognitive understanding of the story but instead for continued language practice. For storytelling aids, students may draw pictures, cut them out of magazines, or bring in a picture book of the story if it is available.

As the class progressed with other language learning curriculum and skill development, students would be expected to periodically re-tell their story to the class. Students’ storytelling would need to include any new words or skills that were appropriate, for example, using the correct tenses after a lesson on past present and future.