A Guide to Talking about Homeschool Evaluation

Apples to Apples? Maybe Not

I

n my previous life as an engineer, I focused on product and process evaluation—everything from design, to execution, to interpretation and reporting. Thinking and talking about evaluation was so much fun that it led me to train others in it. Everything about it was easy for me. Well…at least, it was until I became a homeschooler. When I became a homeschooler, I’d stumble—repeatedly—when talking about evaluation.

What was going on? It wasn’t like me. I searched the inter-webs and asked other homeschoolers to find out. The upshot of it was that I’d come face to face with a sort of homeschooling truth:

Even though evaluation might be easy to do, talking about it is a tricky thing.

Of course, I wanted to know why.

In studying the problem, I realized we tend have the familiar, Western institutional model in mind when we talk about education and assessments. But, homeschooling is a departure from the Western educational model. And, while the point may seem obvious, less obvious is how the range of departure—from slight to significant—complicates things.

Homeschoolers have different worldviews, values, practices, and behaviors—even amongst other homeschoolers. Therefore, views about learning and knowing (let alone, evaluation practices) may contrast notably from the Western model, suggesting greater commonality with Indigenous models than the Western one. Likewise, practices may be a combination—or hybridization—of practices common to Western and Indigenous models.

In other words, when we compare evaluation views and practices of the mainstream to a specific homeschool model, are we comparing apples to apples, Granny Smiths to Fujis, apples to oranges, or apples to apple-oranges?

Further, we tend to think of evaluation as a partner to learning. We sort of forget that it’s separate from it (even in the Western model) and follows a purpose. Because worldviews about learning and knowing differ, the purpose of evaluation differs.  Depending on worldview, there may be no reason to assess learning, making evaluation irrelevant.

As I thought about these things, it occurred to me that my conversations about homeschool evaluation needed to start here:

Evaluation is the child of a purpose arising from worldview.

Wise Whys—or Reasons We Evaluate Learners

Depending on the model, the purpose of evaluation differs. For example, the Western model’s purpose is to identify learner and learning system deficiencies for multiple learning systems (e.g., schools and school districts) and for large populations of learners. Comparative in nature, Western evaluation compares the abilities and knowledge of individual learners of a specific population, to other learners of that population, and against state/federal/instructor learning objectives.

For homeschoolers, evaluation purposes vary for several reasons. Depending on the state, state/federal objectives can differ for homeschoolers or comparative assessment against the objectives may not be required. Then, even if homeschoolers look to the objectives for guidance, they may find correlations between age and performance against those objectives imprecise. For example, Duncan is reading at a high school level, writing at a high school level, and studying college astronomy. However, his age and social maturity indicate that he’s in seventh grade. Finally, a homeschooler’s worldview informs their evaluation purpose—even if it indicates that there is no purpose for evaluation.

Evaluation may be irrelevant because learning and knowing is:

  • Revealed through doing.
  • Arrived at through means invisible to those who would evaluate (e.g., by way of dreams, unobserved experience, and so forth).

Or, evaluation may be used to:

  • Identify achievement for celebration, including specific learning achievements or a learner’s becoming (which may be linked to the past, present, and future).
  • Assess an individual’s contributions and participation in a community as it pertains to overall strengths of the community.
  • Ensure compliance to a set of rules or standards.
  • Learn what a learner knows, extending learning, understanding, and insight to kinship groups or the larger community.
  • Compare learner performance to the state/federal objectives—after all, the departure from the Western model may be slight.
  • Etc.

Who and When?

When it comes to evaluation, focus and timing are important considerations, too. Who are we evaluating and when is the best time to conduct the evaluation?

Western assessments focus on multiple learners of a specific age and level of ability. When it comes to who is evaluated, demographics matter. Equally important are frequent and scheduled evaluations—to optimize learning systems within the classroom, school, and district; before learner age/ability indicates transition to the next lesson or level; and to ensure learner ability is appropriate for the next lesson or level.

For homeschoolers, the focus differs—even if departure from the Western model is slight. In general, there no opportunity in the day-to-day to compare peers. (Though, there is an opportunity to compare an individual with peers, annually, via annual testing or another normative assessment.) Therefore, the focus is limited to individual learners or learning collaborators. (Of course, this limitation may not hold true for homeschoolers attending online public schools, co-op schools, or alternative schools.)

As for when to conduct an evaluation, it depends upon the homeschooler’s purpose for it.

The Nitty-Gritty of Assessment

At its core, an evaluation is a means for collecting information. What that information signifies—even when there’s an obvious score or letter grade realized through the assessment—is arrived at through interpretation. And, there are various ways to collect that information as the following illustrates.

Assessments Based on Timing and Comparison

The Western model furnishes a variety of comparative assessments linked by schedule:

Diagnostic, placement, and learning preference/style assessments are common and may be realized through:

  • Personality Tests
  • Interactions/Discussions
  • Observations
  • Pretests/Quizzes
  • Initial Learning System Maps

The following assessment are common:

  • Performance-Based Assessments to evaluate learner ability to apply knowledge and skills in a sustained, long-term way.
  • Formative Assessment conducted throughout the learning and resulting in feedback for the learner and learning system.
  • Ipsative Assessment which compares the prior and current performances of an individual as a measure of progress—it’s a great way to identify strengths and weaknesses of the overall system, too.

The following lists various approach options for these assessments:

  • Observation
  • Interaction
    • Discussion
    • Scrum meeting
    • Mini Retrospective
  • Evaluation of an expressive artifact:
    • Map
    • Proposal
    • Journal Entry
    • Performance
  • Quiz/Test
  • Comparative analysis of current learning system map to initial map

Summative assessments evaluate outcomes and achievements through any of the following options:

  • Evaluation of completed work
    • Project
    • Paper
    • Performance
  • Test
  • Observation
  • Survey/Questionnaire

Other, Non-Comparative Assessment Options

While there are many non-comparative evaluation types, the following demonstrate purposes and approach options for evaluation types that, generally, fall outside of the Western model’s scope. When the purpose is to:

  • Identify achievement for celebration, Reggio Emilia’s approach to making learning visible provides a great example. As co-learners, Reggio Emilia teachers observe, record, and collect learning artifacts/works. Next, they organize these in context of what is valued by relevant kinship and social groups (.e.g., learner’s family, the teachers, the school, and the community at large). Finally, they’ll celebrate achievements by sharing them with these relevant groups.
  • Learn from learners, the best approach is through interaction—whether that is through observation or discussion. Unlike a lecture from a content expert, however, the exchange is mutual—both parties engage with each other to ask, discuss, and try out concepts.
  • Assess an individual’s contributions and participation to a community, formal and informal surveys are common.

The Problem with Interpretation

Often, conversations about evaluation have as much to do with an interpretation of the results as how you arrived at those results. For example, how do you know that a particular set of results means that learning has happened or that a learning system is optimized?

Honestly, interpretation is problematic—even for the Western model. Whether results are qualitative or quantitative, objectively obtained or subjectively gleaned, accepting an interpretation about them requires a leap of faith and agreement. At some point, we must agree that the results mean x rather than y.

The Western model has time and wide acceptance on its side for how results are interpreted—often, with rubrics to support interpretation. For example, meeting most of the listed criteria indicates lesson learned; so does an A on a test. Yet, homeschoolers don’t have this history and instead of rubrics or a scoring system, deliberation may be the mode of interpretation. Consequently, it becomes necessary, often, to connect the dots between observed patterns/themes and interpretations when talking about evaluation.

Now, Back to that Tricky Conversation

These days, I’m more sure-footed when talking about evaluation. And, when I start out by saying that homeschooling is a departure from what we know of school, it clears the way for the point that it all comes down to purpose. From there, it seems to be a lot easier to talk about the inner-workings of it all, including how results translate into meaningful interpretation.

So, even though it’s still a tricky subject, I can talk about homeschool evaluation with confidence. Better still, I can help others see how homeschooling is shaped by the worldviews of homeschoolers—worldviews that are as culturally diverse as homeschoolers themselves.