EdTech 505 Week 6: American Evaluation Association

Leave a comment

American Evaluation Association

I wish there were a tab in their webpage called “Do you need an evaluator?”.  When the toilet stops working, I understand that I need to call a plumber, but it’s not clear to me at what point do I need to call an evaluator.  From this web page, it is obvious that you cannot easily put your finger on “evaluation”, and say “here it is.”  This is evidenced by the massive amount of text dedicated to their policies page, and the fact that it was changed 6 times, just last year.  However, I know as an organization gets bigger, and each department narrows its sights on their objectives, I suppose it would be helpful to have some sort of consultant that tries to look at the big picture and promote efficiency.

AEA 365

I found this link within the webpage, which made me feel human again while reading about “evaluation”.  The most recent post talked about randomized control trial (RTC), which mostly went over my head, but what was most interesting were the big names that the author used with it:  Institute of Education Sciences and The U.S. Department of Education.  It suggests that the influence of these two institutes is causing the evaluation industry to conform to RTC.  It gave some practical advice and links for someone who wants to learn more about applying RTC.

Reflection

As I was reading through these webpages, the same feeling emerged as when I have read the course textbook; evaluation is necessary and serves a purpose, but I’m probably not the guy to make a career out of it.  I am willing to learn and perhaps I will see things differently, but I imagine that most of the time, when someone is evaluating a program, I will be on the microscope tray, rather than looking through the eyepiece.

As I read about evaluation, here are some other questions that pop in my mind, and I’m curious what might be the answer:

  • Does every program have a life cycle?  If so, when do you decide to let it die or evaluate it?
  • Is it possible to get bogged down in evaluation to such an extent that you sacrifice efficiency?
  • How has evaluation influenced the history of the world?  For example, how did evaluation or lack of, affect the fall of the Roman Empire?
Advertisements

EdTech 523: Best Practices For Online Teaching

Leave a comment

“Where do I begin?” is the most difficult challenge when I am given the instruction, “Develop a list of principles for effective online instruction”. This medium for instruction is definitely making waves in the field of education.  However, the scope of education is so broad (extending across stages of mental and physical development, social class, subject areas, and to a certain extent psychological and psychosocial environments) that it is quite challenging to pinpoint principles of effective online instruction that is all encompassing.  Nonetheless, I am going to attempt to discuss these principles in general terms, without targeting a specific educational demographic mentioned in parenthesis.

  • Form an online community

A sense of community gives education context by which we form meaning and purpose.  Many educational theories include a social element as part of their basis; in theory terms it is referred to as community of practice and community of learners (Jonassen & Land, 2012).   As education moves more into online environments these practices of community may look different from the traditional form of education, yet they are necessary to establish. The community edifies the learner through his or her participation.  Even as I write this, I am motivated by the fact that someone in my learning community will read it.

One popular idea related to community in online learning is “social presence” (Palloff & Pratt, 2006). Earlier attempts at distance education and online learning were not as successful due to the isolation of the learner from the source of instruction.  Even now, many still believe that online learning is inferior to traditional classrooms because of this same perception.  However, advancement in web tools are creating more opportunities for social presence online, and in some ways create more enriching learning communities than the traditional counterpart.

  • Establish a regulated learning environment

One key concept related to a regulated learning environment is netiquette (Rice, 2011).  In order for the learners of an online community to feel safe and valued, guidelines need to be set for acceptable and unacceptable behavior online.

  • Be aware of digital inequality and learning disabilities

I have included the gap in digital inequality as a reminder to myself because sometimes its easy to overlook that some students have not obtained certain skills with computers and online tools.  Many online educators are developing software, hardware, and online instructional techniques as part of a plan called universal design, which considers the needs of all learners without sacrificing the content (Hall, Strangman, & Meyer).

  • Engage the virtual senses

Information is not only transmitted in text.  Tap into the visual and auditory techniques for sharing information. This is a reminder to use the vast amount of multimedia resources available online.  In many ways technology tools provide for a more enriching learning experiences than the traditional classroom because it gives the learner more direct and individual contact to multimedia sources.  Besides textual information, video recordings, audio recordings, and screencasts can provide the teacher with great tools for communicating information to a student.

  • Chunk educational scaffolding in time segments

Deadline related tasks encourage students to participate regularly with the course and the instructor (Graham, Cagiltay, Lim, Craner, & Duffy, 2001).  Tasks should be presented with scaffolding techniques so that the learners can build a foundation in which knowledge and understanding of a specific concept can grow (Patnoudes, 2012)

  • Provide collaboration with measurable objectives

Students can collaborate by providing opportunities for open discussion with a topic or by having students work together to accomplish a task (Carwile, 2007). The online educational environments provide many opportunities for the constructivist approach to learning.

Questions and Response:

What does good online instruction look like?

It returns the “awe” to learning.  Online instruction is a breath of fresh air for many learners and teachers who have watched traditional forms of education stagnate in old practices, or turn their back on modern social practices. It is hard to speak of good online instruction in a specific sense because the opportunities are so numerous.  So generally speaking, good online instruction is open to the possibilities.  Since I have chosen this road, I have developed my own personal motto for any problem that arises in the epistemological practices of education; if technology does not have the solution, it will soon. 

Does it look the same for all grade levels and content areas?

As I mentioned in my introduction, the scope of education is very wide.  The list of best practices is intended to be general enough that it can apply to all levels of education.  However, the teacher will certainly have to accommodate specific strategies to meet the cognitive and maturity levels of his or her students.

Will effective face-to-face teachers be effective online teachers?

Yes and no.  Good teaching is a versatile skill that can transfer easily from traditional learning environments to non-traditional online environments.  However, each teacher is an individual, harnessing skills, experiences, and special talents into his or her instruction.  Therefore, not everyone is going to be equally as effective in one environment as the other.

References:

Jonassen, D., & Land, S. (Eds.). (2012). Theoretical Foundations of Learning Environments (2nd ed.). Routledge. pages 38-50

Palloff, R. M., & Pratt, K. (2007). Building Online Learning Communities: Effective Strategies for the Virtual Classroom (2nd ed.). Jossey-Bass. page 30

Rice, K. (2011). Making the Move to K-12 Online Teaching: Research-Based Strategies and Practices (1st ed.). Allyn & Bacon. page 79

Graham, C., Cagiltay, K., Lim, B.-R., Craner, J., & Duffy, T. (2001, April). The Technology Source Archives – Seven Principles of Effective Teaching: A Practical Lens for Evaluating Online Courses. Retrieved February 25, 2013, from http://technologysource.org/article/seven_principles_of_effective_teaching/

Patnoudes, E. (2012, September). How To Integrate Education Technology With Scaffolding | Edudemic. Retrieved February 25, 2013, from http://edudemic.com/2012/09/scaffolding-education-technology/

Hall, T., Strangman, N., & Meyer, A. (2011, January). Differentiated Instruction with UDL | National Center on Accessible Instructional Materials. Retrieved February 25, 2013, from http://aim.cast.org/learn/historyarchive/backgroundpapers/differentiated_instruction_udl

Carwile, J. (2007). A Constructivist Approach to Online Teaching and Learning. Education. Retrieved February 25, 2013, from http://www.vccaedu.org/inquiry/inquiry-spring-2007/i-12-Carwile.html

3D GameLab Guildsite – Let the journey begin!

Leave a comment

3D GameLab Guildsite – Let the journey begin!.

This is a potential teacher resource for alternative reality.  It is connected to my university.

EdTech 505 Week 5: Gap Analysis

Leave a comment

Peer Structure and Support

Brief Overview:  This is a classroom instructional management design that requires students to create peer assessments of literary content and analyze peer responses.  The purpose of this evaluation is to measure the effective use of Google Web 2.0 tools while writing the peer assessment project.

Needs Analysis

As I have indicated previously, this is a program that I have successfully implemented in another school, but I had not yet applied the use of web tools, therefore the peer assessments were created by students and printed out for classroom distribution.  Additionally, when one student failed to meet the deadline, it affected the whole class.  In my current school, creating these assessments in web-based format, we can save the consumption of paper and the web-based collaboration feature makes the whole team accountable to the deadline and they don’t have to rely on one student.

The Goal

The objective is to make the peer assessment process more efficient by using web-based tools.  An additional objective with this current group of students is to measure the effectiveness of peer assessment for developing analytical literacy skills.  At the end of the evaluation, a recommendation can be made to continue with this program for future literacy activities.

The Program (Bridge)

All the students will read the same selected text.  The facilitator will distribute the assessment tasks to student directors, who will meet together to discuss those tasks as they relates to the deadline.  The student directors will meet with their team of students to delegate responsibility among the members.  Each team will work together to form assessment artifacts that target the objective and they will determine what are the appropriate responses to meet those objectives.

Students will be provided time with a computer to create a collaborative document, questionnaire, spreadsheet, and presentation, in addition, time for taking the peer assessments of other students and to analyze the results of their own assessment.


Peer Structure and Support

Philosophy and Goal

Through the process of assessing peer skills and knowledge, the students become more aware of their own ability to interpret literature and analyze peer responses.

Needs Assessment

In order to develop critical thinking skills for the students, the educational experience needs to be relevant for the learner.

The program facilitator needs to provide

  • rich literature for the assessment tasks
  • examples of assessment tasks
  • feedback on assessment artifacts

Students have a need to make a learning experience more relevant by

  • analyzing text for peer assessment tasks
  • analyzing peer responses of assessment tasks

Program Planning

  1. The students will take a pre-survey about assessment tasks.
  2. The whole group of students will read the selected text.
  3. Student groups are formed with a director, who discusses peer assessment tasks and coordinates the collaboration of the team.
  4. Each team will create an online assessment that targets the group’s assessment tasks.

Implementation and Formative Evaluation

During this phase the facilitator will review the assessments created by each group to see if they properly understood the assessment tasks and to clarify any mis-guided assessment artifacts.  Once the peer assessments are ready for distribution, the whole class will respond to the quizzes created by their peers.

Summative Evaluation

After the students have responded to the peer assessments, each team will collect and analyze the data.  They will put together an expository presentation that shows the anonymous responses from the class.  They will identify positive and negative response characteristics to their intended assessment tasks.

EdTech 505 Week 5: Evaluation in Program and Planning Cycles

Leave a comment

ADDIE Model

ADDIE

I remember learning about ADDIE in the EdTech 503, Instructional Design, and the evaluation component was easily understood within the whole ADDIE cycle.  As an educator, my mind is already trained to see evaluation as a component of instruction.  Now that I’m taking EdTech 505, Evaluation, that component has become harder to grasp.  I feel like I have been trying to cut out the piece of the pie called “Evaluate” to see if it tastes different from the rest of the pie.  In other words, even though the pie does have separate pieces, it is all made from the same ingredients; one piece cannot be completely independent from the others.

The ABCs of Evaluation, p.51

Evaluation: Program Cycle

This model does not stray much from the ADDIE model, but you can make the distinction with the purpose of the model.  ADDIE relates more specifically to instructional design, whereas the Program cycle on the right can relate to instruction or any active part of a system or organization, whether it relates to instruction or not.  This model does account for both formative evaluation and summative evaluation, which the ADDIE models does not distinguish.  Also, this model suggest that implementation strategies can change according to the formative evaluation during one rotation of the cycle.

The Planning-Evaluation Cycle

The Planning-Evaluation Cycle

This model does not fit as easily into an educational or instructional situation.  Even though the components of ADDIE and the Program Cycle appear in this model, it is distributed quite differently from the other two models.  For example, this cycle includes analysis and design as part of the evaluation phase.  Nor, does this model clearly distinguish between formative and summative evaluation, it almost suggest that the whole evaluation process is formative.  It appears that this model would be good to analyze some function or feature of an established system, and based on the results in the evaluation phase, the ADDIE model could be applied as an instructional model within the planning phase, which would address the needs that were discovered during the evaluation phases.

References:

EdTech 523: Collaboration Web-Based Style

Leave a comment

This is a unique position; I’m looking at education from the simultaneous perspective of a teacher and a student.  There are many advantages to this dual perspective; one in particular is that, as a teacher, I can imagine the challenge of a student trying to balance the many tasks that he or she has.  When it comes to collaboration in education, I believe that the teacher’s perspective is quite different from that of the student.  Teachers are often quite considerate of the variables that can affect collaboration negatively, so they carefully plan the tasks to avoid the pitfalls as much as possible.  During this process the teachers become invested in the expected outcomes and, if like me, begin to form fairytale images of how it all plays out.  However, many students groan at the though of collaboration, but after the student becomes invested (usually motivated by a grade), they get to encounter all of the unforeseen problems that even the teacher did not anticipate.  So, what to do now?  As a teacher, I try to offer support, but as a student, I try to solve the problem.  The maturity level of the student also plays a big part.

Fortunately, as an EdTech student, I have worked collaboratively with some very mature and capable students. These experiences have all been online and it served my professional skills to participate in these collaborative tasks.  As a K-12 teacher, many of my collaboration fairy tales have not come true, yet, it has been the challenges that help me to plan and facilitate more effectively.

  • Do you see value in Web-based collaborative tools?

I have experienced first hand the value of collaborative tools both as a teacher and a student.  My “student” benefit is more obvious.  If it weren’t for the web-based collaboration tools, I would not be able to study this program and collaborate with my peers, while living in Saudi Arabia.  As a teacher, I have also been able to use web based tools, even though I teach in a traditional classroom setting.  Web based tools have helped me communicate better with my students by creating an online network, sending detailed instructions, and recording information multimedia.  Additionally, some of these tools have helped me track accountability, especially in the collaboration tasks.

  • What are potential pitfalls in implementing collaborative activities using Web-based tools?

As a teacher, the pitfall has been the digital inequality of my students.  Even though all of my students have access to most of the latest technology, not all of them are accustomed to using it in the way that I require, or they just don’t know how to navigate through web-based tools.  Many of the issues have been related to maturity level.  Students don’t remember their password, or they prefer to be spoon-fed the instructions rather than trying to be problem solvers.  Also, when students have the opportunity to be sneaky, they always seem to find the capacity.  Some web-based tools are not set with accountability measures so it is hard to track fraudulent identities or other undesirable activities.

EdTech 505: Week 4, Project Evaluation Design Format

Leave a comment

For this week’s assignment, I decided to fill out the chart suggested in our textbook, but the data that I input in the chart is part of my evaluation project.   The chart is called “Evaluation Design Format” and the data represents my evaluation of the Peer Structure and Support program, which divides students in groups that create informal assessments of the content for their peers.

I have inserted the table below, but perhaps it can best be seen in the attachment.

505 Project Evaluation Design Format

Peer Structure and Support

Evaluation Project, Spring 2013

 

Evaluation Questions

Activities to Observe

Data

Source

Population Sample Design

Data Collection

Responsibility

Data Analysis

Audience

  1. Are students able to collaborate effectively to assess their peers’ ability to understand and interpret text information?
  • Assessment departments
  • Group Task Analysis
  • Collaborating on effective assessment objectives.
  • Prepare sample responses
Elements of Literature Textbook (Selected Short Stories) One group of 20 students from the advanced English Class.
  • Student directors checklists
  • Classroom observation and notes from the teacher.

 

  • Teacher determines student leaders.
  • Leaders share expectations with their group
  • Teacher observes class activities

 

  • Student Directors feedback
  • Teacher’s feedback
  • Assessment content

 

  • Teacher
  • LRC Director
  • MYP Coordinator
  • English Department Chair
  • Parents
  1. Are Google Forms used effectively for students to evaluate their peers’ literacy and reading skills?

 

  • Creating and sharing a Google Forms
  • Appropriate question styles for objective.
  • Peer Share the questionnaire.

 

Google Accounts

  • Forms
  • Spreadsheet

Selected Short Stories

One group of 20 students from the advanced English Class.
  • Students digitally share data with the evaluation process

 

  • LRC director observes online activity of students
  • Teacher reviews peer assessment submitted
  • Assessement content
  • Assessment format
  • Digital sharing
 
  • Teacher
  • LRC Director
  • MYP Coordinator
  • English Department Chair
  • Parents
  1. Are students able to collect, interpret, and, present anonymous assessment data from their peers?

 

  • Review results in a Google Spreadsheet.
  • Share selected results in a Google Presentation
  • Present results anonymously.

 

Google Accounts

  • Forms
  • Spreadsheet
  • Presentation
  • Google+
  • YouTube

 

One group of 20 students from the advanced English Class.
  • Presentation checklists
  • Shared presentation material.

 

  • Students collaborate on peer responses that meet standards
  • Teacher observes student presentations

 

  • Student Peer response and content
  • Teacher feedback about presentation

 

  • Teacher
  • LRC Director
  • MYP Coordinator
  • English Department Chair
  • Parents

 

Older Entries