The professor gave us the following for reflection:

Whether you are looking to choose students from a class (or from employees in a workplace) for a special project or you are looking for a pool of people to take a survey, the first thing you will need to do is decide on how you will choose your sample, or group of students or employees.

So, based on this week’s readings (The ABC’s of Evaluation, Chapter 8),

  • What are some ways you could choose that sample?

The book identifies two categories for gathering sample groups: probability sampling and non-probability sampling.  Once I determined what I wanted to evaluate, the process that I used most resembled a non-probability sampling and more specifically a purposive sample.  For me it was a practical and efficient decision.  Since I already knew the framework of the program that I want to evaluate, I decided to chose my own group of students and evaluate the program with them.  I could have sought another teacher to run the program with his group of students, but this would have been a major time imposition for both of us, considering the training and the information exchange that would be involved.  I have a former colleague who is running a version of my same program with his group of students, but we are so far away geographically that the observations and information exchange would be limited.  Nonetheless, the sample student population would not be random in either of those cases, unless you determine that the administrative scheduling was done in a random sort of way.

  • What experiences have you had in choosing samples?

I have never been in a situation where I was able to pool a sample group from a greater population.  In my limited sampling of populations, I have always had smaller manageable groups so, I could survey the whole group.  In the program that I am evaluating, I had to make decisions about groups and student leaders, but my decisions were based more on track record than anything else.  The program that is being evaluated is based on student hierarchy, which creates a social dynamic and the students are usually not accustomed to it.  However,  I have set up the survey in such a way that I can measure attitudes, opinion, and impact based on student roles. 

  • What are some things to watch out for and/or avoid in selecting samples?

When I think of my own evaluation project, I have to laugh a little when I read this question.  Because I decided to evaluate a program in which I am the facilitator, I had to be very selective of my group.  I work with students at a very volatile age of 14-15 years old.  I was with the same group of students last year.  All of them have matured, but some more than others.  Also due to culture and family issues in some students and the administration, there are variables that I cannot control, making it difficult to initiate some educational endeavors among several students.  There was really only one group that I could depend on in terms of maturity and responsibility, so it was a no-brainer when it came to selecting that group. 

In the case that a sample needs to be random, there are so many uncertainties among the people that are selected.  If it is crucial that everyone in the sample needs to give feedback then I don’t know if the sample can truly be random.  There are always people who are not able to participate or do not respond in a timely manner, which makes it difficult for evaluators to depend on them.  In general education, a sample of a particular class has a good probability of having a mix of skills and academic level, but as education levels become more specific, you will likely have a group of students with similar goals and interests.  Regardless, most educational sample groups are preselected groups that neither the teacher or the evaluator has selected.

Advertisements