Saturday, May 29, 2010

Study/Future Margith A. Strand/ May 29, 2010

The purpose of this study is to construct the design of elucidating the mechanisms of the "cycle" of the interactions which are inherent within the nature of the distance education field. The Adaptive Structuration Theory is one of the main constructive ideas upon which the platform will be developed and shown to be based on structure and systems; the imaginal phenomenological basis of the approach will be established to be the foundation for which one of the better pathways to the online course format is delivered.

Researchers in the future can find that they are on their own advent of their philosophy and construction of design as online facilitators. They can find that perhaps all or most of the learning approaches are inherent in the construction of the educational training and teaching present on the online venue. The distance approach is a viable option to study and present research to be based on most of the other learning processes which have been studied by past educators.

F(x) = y / Domain and Range and Context Function

Based on the Adaptive Structuratioon Theory model, we have come up with a function driven context platform method for ascertaining the meta-derivative effect size condition for a semiotic work found in either a literary setting or an online driven setting. The F(x) is the context function, where the x is the domain as defined as the word or the sentence section of a paragraph. The y is the range and is defined as the meaning of the semiotic expression. The variation in the delivery of the expressions can be altered by the function F( ) and this is known as the Contextual function and obviously, in cases, can alter the meaning of the sentence or can change the outcome of the paragraph structure. Given that these are "played" within the context of the Adaptive Structuration Theory, where the emphases is on the time, place and space conditions of a situation, we can see the "play" in variations and conditions can result in the ability to use the variations as a series of testing conditions for meaning studies in semiotics.

My Comment on Function Derivatives : Semiotics- Margith A. Strand

"The semiotic function and the genesis of pictorial meaning"

Göran Sonesson

in Tarasti, Eero, ed.,Center/Periphery in representations and institutions. Proceedings from the 3rd Annual Meeting and Congress of The International Semiotics Insitute, Imatra, Finland, July 16-21, 1990.: International Semiotics Institute, Imatra 1992; ss 211-256.

One way of approaching the subject matter of semiotics may be to consider what is implied by the notion of semiotic function. There are two classical contexts for the use of this phrase. One is found in the work of the linguist and proto-semiotician Louis Hjelmslev writing in the forties. To him, the semiotic function simply means that any sign must involve an expression serving as vehicle for a content. But as soon as we look a little closer into his notion of sign, things start out to be very complicated, so it will be more convenient to begin at the other end. The second classical locus appears spread out all other the numerous writings of the cognitive psychologist Jean Piaget. Therefore, we will first attend to Piaget, his admirers and critics, to both of which I happen to belong. But before we can begin to deepen our insights into semiotics by way of a discussion of the semiotic function, something must be said, in rough outline, and at a more abstract level, about semiotics in general and pictorial semiotics in particular.


Deriving from the context of the work above, I can formulate a "contextual semiotic methodology" of ascertaining the sentence and the paragraph contextual material in a new way of formulating the links of an linguistic expression which utilizes the "context" rather than the content of the work under consideration. Here, we are in the "thematics" mode, and as supported by the earlier diagram which is shown which expresses the connectivity between the "emergence" of the intent of the work, and the final outcome. The "stem word associations" have been replaced by the "contextual derivations" and the more-sentence structure friendly modes of language and idea expression. The connectivity between "discourse" and "Discourse" are still of importance, and the "domain" and the "range" functionalities of the method are considered to be the associated conrrespondence of the "code" variables in the proper "function" [contextual] locales (semiotic placements).

Semiotic History/ Function/ May 29, 2010/ Pictorial Function Semiotics

http://filserver.arthist.lu.se/kultsem/sonesson/ImatraCourseTx1.html

Contextual Derivation Analysis Chart for Sentence/Paragraph Structure/Systems use

http://www.mywebspiration.com/view/454527a1993c

Contextual Thematic Diagram for Semiotic Derivation Analysis

http://www.mywebspiration.com/view/454527a1993c

Friday, May 28, 2010

Recontextualisation_ by Gunilla Jansson

Studies in Higher Education
Vol. 31, No. 6, December 2006, pp. 667–688
ISSN 0307-5079 (print)/ISSN 1470-174X (online)/06/060667–22
© 2006 Society for Research into Higher Education
DOI: 10.1080/03075070601004275

Recontextualisation processes as sensemaking
practice in student-writers’
collaborative dialogue

Gunilla Jansson*

University of Stockholm, Sweden
01236gSTCGO013utoaSruu.0niycnH1gdi067li0leioiEl-0lnet8lr5ay0sa_a 0.0a JDjlAif/a 7nan0oAen_n9d 3rcHs2rs 0(FetsRs0ipio7morcg0eran5nlhis3benn@0ee2ect7arr3)in 0r /sE.2c1o6s Lh0d4gr0d 0ut7m1idin6c00sat-k0o1tai4 7oH.2s4n7uiXg5.sh e(eorn Elidnuec)ation
The research introduced in this article is placed in the field of socially mediated learning processes
and draws from fieldwork in a highly diverse sector in higher education. It explores the potential of
peer scaffolding as a means of making sense of tutor comments. The data consist of recordings of
conversations in two collaborative writing groups on two laboratory reports. The participants were
students with a second-language background enrolled in a course which was part of a one-year
Master’s programme in computer science at a Swedish university. The analysis is based on transcriptions
of episodes in the recordings, where the students are engaged in conversations about
different aspects that can be related to the unfolding text. All instances in these episodes where the
students are making use of teachers’ comments on explicitness with respect to logical reasoning are
coded. A qualitative analysis of the interaction reveals how aspects from teacher voices are extracted
from the institutional frame, paraphrased and put into the students’ colloquial talk. The findings
indicate that peer collaboration plays an important role in enabling students to use the metaknowledge
available in the educational setting as a tool for their learning of academic literacies.
Pedagogical implications are discussed in terms of the potential of peer scaffolding as a means to
support and develop teachers’ discourse around writing.

Introduction
In the last two decades, higher education in Sweden, as in many other countries, has
undergone significant change. Since the 1980s, a main objective of Swedish national
education policy has been to reduce participation disparities among social and
cultural groups (Strand, 2000). Today’s student population has grown in both size
and diversity. Swedish concerns about the non-participation of specific social and
cultural groups are echoed in the current national debate about widening access to
higher education.
*Department of Scandinavian Languages, University of Stockholm, S-106 91 Stockholm, Sweden.
Email: gunilla.jansson@nordiska.su.se
Downloaded By: [Fielding Graduate University] At: 17:27 21 May 2010
668 G.

Cultural Context: Eli Gottlieb

Learning How to Believe: Epistemic
Development in Cultural Context
Eli Gottlieb

Mandel Leadership Institute
Jerusalem, Israel

Over the last decade, researchers have become increasingly interested in students’
beliefs about the nature of knowledge and how these beliefs develop. Although initial
psychological accounts portrayed epistemic development as a domain-independent
process of cognitive maturation, recent studies have found trajectories of epistemic
development to vary considerably across contexts. However, few studies have focused
on cultural context. This article examines the role community values and practices
play in fostering particular epistemological orientations by comparing the
epistemological beliefs of 5th, 8th, and 12th graders (N = 200) from General and Religious
schools in Israel regarding 2 controversies: belief in God and punishment of
children. In both controversies, older participants were less likely than younger participants
to consider the controversy rationally decidable. However, this shift
emerged earlier in the God controversy than in the punishment controversy. In the
God controversy, General pupils were less likely than Religious pupils to consider
the question rationally decidable or their own beliefs infallible. But no such school
differences were observed in the punishment controversy. Qualitative and quantitative
analyses linked these differences to divergent discourse practices at General and
Religious schools, suggesting that the relations between learning and epistemic development
are more intricate than has been assumed hitherto.
Epistemology is an area of philosophy concerned with questions of what knowledge
is and how it is justified. Although few people give these questions such detailed
and sustained attention as professional philosophers, anyone attempting to
acquire, produce, or evaluate knowledge relies, at least implicitly, on some set of
epistemological beliefs. Such beliefs are of obvious interest to educators. To understand
how students acquire, evaluate, and justify knowledge, we need to

Excerpts from Social Semiotics / Lilie Chouliaraki

The Contingency of Universality’: Some
Thoughts on Discourse and Realism

LILIE CHOULIARAKI

Constructionism and realism are dominantly regarded as incompatible meta-theories. In
this article, I argue rather that a realist epistemology offers some premises that can usefully
ground discourse analysis in social scienti. c research. This has implications for the latter’s
modes of theorising and its potential for social criticism. The argument is in two moves.
First, I discuss how the concept of discourse . gures in three major critical traditions of social
theoretical, sociological and political theoretical thinking. I argue that, for different reasons
each, these traditions offer a less than satisfactory answer to the ontological question of how
discourse . gures in the social and/or to the epistemological question of how discourse is
operationalised in theory and research. Second, I turn to a discussion of critical realist
meta-theory. I critically discuss Bhaskar’s ontology of the real and suggest that realist
epistemology should be combined with a constructionist ontology, along the lines of feminist
theorising. This is useful in providing discourse theory and analysis with more effective
accounts on the nature of the social and on the modes of social inquiry. In so doing, I sketch
a view of discourse informed by critical realist elements and point to its conceptual and
analytical advantages.
Constructionism and realism are dominantly regarded as incompatible meta-theories.
Although neither can be reduced to a homogeneous . eld in itself, the two
positions are generally used as binary poles, which organise contemporary debates
on what the social world is and how we can study it. Such debates, more often than
not, exclude the possibility of a fruitful dialogue between the two positions. This
polarisation has implications concerning the theorising and research of social life in
critical social sciences. Roughly put, the world is either seen as an effect of discourse,
constructed through social practices of signi. cation, or as pre-existing social practice
in the form of some intransitive referent. Constructionists accuse realists of essentialism,
of insisting on the illusion of some pure existence, whereas realists accuse
constructionists of idealism, of the illusion that all existence is contingent on
language and signi. cation. For some characteristic arguments against realism see,
for example, Edwards et al. (1995), Laclau & Mouffe (1985) and Potter (1997), and
for some against constructionism see, for example, Bhaskar (1997/1998a) and
Archer (1998).
My own starting point is a constructionist perspective centred on the concept of
discourse. Discourse . gures here as constitutive of social practice. Speci. cally,
ISSN 1035-0330 print; 1470-1219 online/02/010083-32 Ó 2002 Taylor & Francis Ltd
DOI: 10.1080/10350330220130386
Downloaded By: [Fielding Graduate University] At: 17:21 21 May 2010
84 L.



p.92 Social Semiotics Commentary

First, the meaning–power relationship. Laclau & Mouffe combine the Foucauldian
view of discourse as . elds of signi. cation enunciated from positions of
power with Derrida’s view of semiosis as an in. nite play of signi. cation, which
precludes the . xity of meaning. Derrida (1978) speci. es the logic of the semiotic as
a logic of iterability. The semiotic is seen as a ‘differential structure’, a structure that,
precisely because it lacks a centre, cannot arrest meaning in its fullness. So, in any
practice, there is always a meaning surplus (‘supplement’) that exceeds its context
and that expands the possibilities for re-signi. cation without limit. The performative
force of the text (its capacity to constitute the social) resides not in the social logic
that produces it (a´ la Bourdieu), but in the semiotic logic itself, in the sign’s
necessity/capacity to break with prior contexts and assume new contexts, hence new,
unpredictable meanings: this ‘breaking force (‘force de rupture’) is not an accidental
predicate but the very structure of the (…) text’ (Derrida 1978, in Butler 1997a: 148;
emphasis added).

p.96
Discourse therefore is the materiality of the social, insofar as it both has
a speci. c material logic, the performative logic of the semiotic (cf. Derrida), and
insofar as it has material effects: it ‘materialises’ everything there is (Laclau &
Mouffe 1990; Butler 1993). This view comes closest to the Foucauldian view of
discourse already set out than either of the views already discussed. The materialities,
which phenomenologically appear to us as referents outside the semiotic, are
Downloaded By: [Fielding Graduate University] At: 17:21 21 May 2010
The Contingency of Universality 93
not real objects, but rather truth effects. They establish their positive identities
precisely at the moment when the meaning–power relationship that brings them into
being manages to conceal itself—and genealogy is that analytical practice that
exposes the status of such material positivities as contingent.
However, does the claim that all is discursively constituted necessarily imply that
the epistemological-analytical ‘language’ of discourse should be the ‘language’ of the
social overall? Laclau and Mouffe’s ‘language’ does exactly that: . eld of discursivity,
 oating signi. ers, elements, discourse, articulation, moments, nodal points—all
provide a rich and sophisticated theorisation of the social as a terrain of discursive
articulations, which emphasise its openness and indeterminacy, its constitution and
change through constant ‘resigni. cations’.4 But if the task of critical, including
genealogical, analysis is to show how certain materialities are postulated as more
physical, hence as more ‘immediately real’, than others, then we need to study both
how this ‘immediate’ materiality is ‘given’ in a practice and how it is ‘open’ to new
classi. catory struggles and meanings. As well as employing Laclau and Mouffe’s
existing theoretical language, which accounts for the latter (the openness of discourse),
we also need a language that accounts for the possibility of relative closure.
This means accounting for physical materiality in its speci. city: as a modality of
materiality that is indeed discursively constituted but whose logic of constitution
(and thus its ‘degree of resistivity’ to closure) makes it function differently from
the materiality of the linguistic semiotic as temporally unfolding interaction in the
here-and-now of a practice.5 So, although Laclau and Mouffe acknowledge the
extra-semiotic, their epistemological ‘language’ binds the social and its ‘physical’
materialities too close to discourse. This is a consequence, I have argued, of their
turn to Derrida’s view of signi. cation and his emphasis on the unlimited, rather than
conditionally limited, reiterative function of the semiotic (see Butler 1997a: 150–51
for a critique on Derrida’s view of iterability as inherent in the differential structure
of language, ‘abstracted from its social operation’). As a result, Laclau and Mouffe’s
‘language’ of description of the social ignores what might be called the ‘multi-materiality’of
social practice and underestimates the extent to which other, non-semiotic
(but always semiotised) modalities of matter may bear a performative force on social
practice.

Title Page _ Context and Content

Wednesday, May 26, 2010

Basics:)

Adaptive Structuration Theory is process and structure: social factor
ERG is physicodynamic motivation theory
Discourse is reality in writing
Semiotic is another word for meaning and expression
discourse....Discourse...Contextual meaning...Social Effect
Imaginal "soul" phenomenology....color? expression?



To prove that Adaptive Structuration Theory is the process and systems-dynamics approach to the positivistic, imaginal distance learning and teaching field of education: process orientation of which is Discourse Analysis, with all of its social factorization.

The outcome is to facilitate and improve the process of instruction on the online platform delivery with the enablement from the semiotic of information patterning.
"Calling of the course" and the result of the social factor?
Communication.....Expression...Abstract to Concrete.

At this current time, there are discrepancies in the level of performance on the part of Instructors as well as students, at all accreditation levels, and we can see that these discrepancies can be alleviated by improving the expectations in the technical and academic apsects of the Discourse field.

Students and facilitators are in need of a academic set of background material which will add to the backdrop of support systems structure and processes in that we can facilitate the outcome to be at a higher level of achievement with a set of ascertained goals and objectives.

With the academic set of material, which is intended to support the mission objective of the student, he or she can be aware of his and her higher expectations as is required by the online format of delivery, given the course outcome objectives and course orientations.

The Discourse processes will be determined and attained through the investigation of the discourse and the meta-derived processes which will support the steps of the Adaptive Structuration Theory model through the ERG [Existence/Relatedness/Growth] and the Organizational Behavior theories.

How does Adaptive Structuration Theory enable the student to [collaborate] in his or her class with the following domain and dimensional concept variables: situated meanings, and cultural context.

Within the domain-reach fashion of the student approaching the context from his or her Instructors performativity and practice level, is it possible to discursively determine how effective an Instructor is in his or her course platform?

My Thematics idea for analyzing sentence construction (semiotic)/ Margith Strand

http://www.mywebspiration.com/view/452240af1a1

Tuesday, May 25, 2010

Imaginal Phenomenology/ May 25, 2010

http://www.phenomenologyonline.com/inquiry/48.html

Monday, May 24, 2010

Dissertation Progression / Fielding Graduate University [2010/Summer] / May 24, 2010 A New Universality Approach to Online Education:Margith A. Strand

Constructivistic online education is not new to the field of Education, it is the basic construct of the online delivery mechanism which is based on the premise of meaning and context to the platform. It is laid on the foundation of motivation, insight and knowing the parameters for which the student has challenged himself or herself to embark on a course during which he or she will succeed or attempt to rejuvenate their efforts again. In the records of academia, we have seen constructivism as a basis of knowledge which holds information for the reader and the practitioner in the foundations of structure and systems in that a wide variety of literature has been produced with these referents. In my mind, the constructivistic approach is one where the facilitator is one who must insight enthusiasm and encourage with a sense of Group Dynamics, Group Processes, Comparative Value Judgements, Personality, Performativity, Work design, Comparative Attitudes, and a good set of work ethics to go by for himself or herself.

Among the basic psychological foundations which are the essential designs laid to deliver the understanding in the comprehension of the human aspect is Maslow's Hierarchy of of "Needs." Here we have the basic "bottom-rung," which is labelled "physiological," the next, "security," then, the next level "self-esteem," and the highest with the "self-actualization."

Of course, we can apply these ideas to the contexts of the online student, and yet, I feel that a more appropriate theory lies in the ERG (Existence/Relatedness/Growth) theory where we can consider the paramaters of the individual to be within the domain of social atomism and reductionistic approach of the student as a thinker and a feeling person who drives himself or herself within the construction of the classroom and "needs" a hierarchy of control and gentility. One of the ways in which we can assure the delivery of such a course layout must be by examining the design and the approach of the interactions of the facilitator expression, or in more academic terms, platform wording, and in the author of this paper's mind, semiotic expressions. We find that one of the best ways to articulate the meaning of the delivery is to use the "imaginal (soul) phenomenology" which is a new area of pychology in which the "soul-factor" of the language-use is expressed as a part of the academic realm of study.

From a systemic perspective, constructivism is a part of the whole, and the smaller component is Adaptive Structuration Theory. The whole is the global approach and the AST is one of the components of that global approach in that it is needed to define the entire picture which is the joining the links of the disparage between "Discourse" and "discourse" in the realm of the online educational world, within a small sample set.

The Adaptive Structuration Theory results in the construct of structure as understood by Marshall Scott Poole. AST is concerned with the time, place, and space aspects which are comprised within the world of the technologically-driven course platform associated with the online educational arena. We find that within this construction of the course format, the "techno" is hidden and the social aspects are more important in the delivery of the class in that the Group Dynamics, as well as the Work Function, comprised of the Work Team Dynamics and the Organizational Behavioral aspects. ERG (Existence/Relatedness/Growth)is the backbone in terms of psychodynamic theory, and the organizational management of the organizational behavioral theories which lie behind the students and the facilitator of the course modular matrix may be considered as the pathway to the sectional perspective of AST.

AST is a change process dynamic which deals with the realm of structure and change and within the Structuration part, it deals specifically with the cognition of the time and space and place aspects of the locale, whether that be in the context of the mind, or the technological reality of the convention. We find that AST deviates from the realm of "techno" in that the concept of "Social action" is of paramount importance and the fact that the machinery is of minimal significance is of paramount virtue and construct of the theoretical development.

Imaginal Phenomenology is the other "backbone" of the swemiotic delivery concept in that the "soul" factor of the psychology and the intent of the individula/group, must be considered within the nature of the course, class and institutional endeavor, including the online approach to education.

Within the construction of a "good" course, we can consider the group processes, group dynamics, and the cultural models which are a part of the concept varialble for the meta-analytical process we are going to embark on; the other concept varialbe can be considered as the "situated meaning," which is the diference in the definition of the term: discourse and Discourse. Discourse means Social Action. The definition for the term "discourse" is "situated meanings." By my method, we are going to look at the innate difference in the terminology of the same word, with two different meanings, which will be called: internal meta-analysis of the semiotic delivery.

One change which I will bring about is the "cultural model" into cultural "context."
The content analysis of the threaded discussions will result in the meta-analysis if one is aware and seeks to find the change in the "discourse to Discourse" with an affirmative y/n approach to a question involving the two concept variables. The other concept variables are "power" and "performativity."

Meta-analyses are carried out to confirm the trend by examining the effect-sizes. Meta-analyses will be used as a model-basis in a qualitative sense to see the "effect-sizes" and the results of quantitative analysis on a qualitatively defined set of literature bases. As defined, the "r" factor is the strength of the relationship between the variables, including concept variables, and the "d" factor is the strength of the difference between the concept variables. In the best case scenario, we hope to find the "discourse to Discourse" enablement by quantitative manners to be shown by a large r-factor, displaying the strength in the relationship between the discourse into an effective Discourse and approved social strength in the semiotic wordings of the courses examined.

The concepts variables "situated meanings" and "cultural contexts" will be used, along with "power" and "meaning," as a second set; the third set is "power" and "performativity."

Tuesday, May 18, 2010

Saturday, May 15, 2010

Discourse Analysis and Complex Systems

This article is similar to my Discourse Analysis and Adaptive Structuration Theory
Margith A. Strand May 15 2010

Discourse analysis and complex adaptive systems:
Managing variables with attitude/s
Charl Walters & Roy Williams, w.w associates, Reading, U.K.
Charl.w.w@ntlworld.com, Roy.w.w@ntlworld.com
Abstract: There have been long-standing debates about the relative values of quantitative vs. qualitative research, and of
positivism vs. critical theory in management studies. In this paper we discuss the value of discourse theory and the tools of
discourse analysis in the context of complex adaptive systems theory, which can usefully be seen as a synthesis of the thesis of
modernism and the antithesis of post-modernism. Discourse’ has been developed and used in several disciplines, to interesting
effect. It is now time to systematise the notion of discourse, and the tools of discourse analysis, both theoretically and
practically, so that they can better be applied to management research, and to management practice.
Keywords: Discourse, modernism, post-modernism, complex adaptive systems, communities of practice
1. Introduction
Traditionally there have been debates on the
merits of positivism vs. critical theory,
quantitative vs. qualitative methodology, and
modernism vs. post-modernism. However, we
believe that these discussions can be
approached quite differently. Complex
adaptive systems theory can help us to
achieve a “synthesis of modernism and postmodernism”
(Byrne in Rihani, 2002:72).
Similarly, discourse theory and discourse
analysis can provide us with practical tools to
apply this ‘synthesis’ to current management
issues.
It is important to see the different modes of
social organisation from modernist, to postmodernist,
to Complex Adaptive Systems as,
by and large, cumulative modes of social
organisation, not as substitutive or oppositional
theories. The point about complex adaptive
systems theory, and discourse theory, is that
different modes of social organisation and
different epistemologies are applied in different
contexts, and for different purposes. The
cumulative development of these different
modes is outlined in Figure 1.
2. Variables
Variables are the basis of research. There
have been two fundamental developments in
epistemology and research methodology,
which share a strong common thread – their
radical scepticism. The first development, in
science, overturned the uncritical conservatism
of metaphysical and traditional practices, by its
insistence on the criteria of falsifiability and
replicability as the basis for rational truth and
knowledge. This resulted in a body objective
knowledge, which was stripped of context and
subjectivity, and which was highly
commodified, which is why much of it is
defined quantitatively. Like money, quantified
scientific results are highly exchangeable,
which is both a result of scientific method, and
a constituent part of it.
But science left culture largely undisturbed, by
confining scientific scrutiny to the ‘social
sciences’, which were not entirely successful in
producing the same kind of ‘objective’
knowledge that could be produced in the
natural sciences. The ‘individuals’ who were
the object of study in the social sciences were
not as amenable to the reductionist
quantifiability of the natural sciences, nor did
they perform very well on the other metrics of
the natural sciences – predictability and
determinism. ‘Social control’ and even ‘social
engineering’ were tried, but remained elusive.
This led to a long-standing, and still largely
unresolved debate in the social sciences, of
which management research is a part, on the
divide between the two “cultures”: ‘science’
and ‘culture’, or natural and social sciences.
Natural and social sciences are embedded in
language and social practices, much of which
has become ‘naturalised’. The second
development, the radical scepticism of postmodernism,
challenged the uncritical
conservatism of the epistemology of language
itself; thereby challenging culture and society,
and the notions of rationality, truth and the
individual, which had become ‘royal game’
within the research establishment. It brought
the radical scepticism of science under its own
radical scrutiny
2.1 Dispersed Subjects
The semiotics of post-modernism contested
the notion of the individual, particularly the
apparently ‘objective’ notions of gender, race
and class, and deconstructed the extent to
which these are socially constructed, and the
extent to which the notion of the individual is
also socially constructed. Post-modernists
replaced the ‘individual’ with the notion of the
72 Charl Walters et al
http://www.ejbrm.com ©MCIL 2003 All rights reserved
dispersed subject; no longer one ‘identity’ - the
individual - but ‘identities’: occupying various
subject positions, within different discourses
and discursive communities. Identity is no
longer seen as ‘individual’ but as a process
and a contestation within overlapping and even
contradictory discourses – i.e. an intersection
of the personal and the social.
Just as an ‘individual language’ is a non
sequitur, so too the notion of the ‘individual’
itself came to be seen as a non-sequitur (a
useful riposte to Margaret Thatcher’s claim that
there was “no such thing as ‘society’ ”). The
idea of people shifting their identities was
nothing new, except that it was now
underpinned by a radical and systematic
epistemological critique of language and
culture. It is of course now also enhanced by
the hardware and the architecture of postmodernism,
i.e. the networked society.
2.2 Variables with Attitude
The point that discourse theory and complex
adaptive systems theory make is that human
‘variables’, which are ubiquitous as the objects
of study of management research, are quite
simply not ‘objects’ but rather, subjects with
identities (in fact, changing subjects).
Moreover, this has specific methodological and
epistemological implications.
Rihani writes that:
The new discoveries [in physics] did not
prove Newton to have been in error.
Essentially, they revealed circumstances
where linear methods yielded excellent
results, and others where they did not.
More fundamentally, they established
beyond dispute that some phenomena,
now referred to as non-linear systems, are
essentially probabilistic. They do not
conform to the four golden rules associated
with linearity: order, reductionism,
predictability and determinism. Causes and
effects are not linked; the whole is not
simply the sum of the parts; emergent
properties often appear seemingly out of
the blue; taking the system apart does not
reveal much about its global behaviour;
and the related processes do not steer the
systems to inevitable and distinct ends
(2002: 68).
Snowden, writing on knowledge management,
comes to much the same conclusion. He
illustrates the difference between systems
which are not adaptive, and human systems
which are adaptive - precisely because they
are made up of humans who have identities, or
(what we refer to as) subject positions; people
who are “subjects”, rather than “objects”:
Human systems are complex; a complex
[adaptive] system comprises many
interacting agents, an agent being anything
that has identity. We all exist in many
identities; the author can be son, father or
brother in different contexts; similarly with
work group identities, both formal and
informal along with various social
groupings. As we fluidly move among
identities, we observe different rules,
rituals, and procedures unconsciously. In
such a complex system, the components
and their interactions are changing and can
never be quite pinned down. The system is
irreducible. Cause and effect cannot be
separated because they are intimately
intertwined Two examples make this
clearer: Consider what happens in an
organization when a rumour of reorganisation
surfaces: the complex human
system starts to mutate and change in
unknowable ways; new patterns form in
anticipation of the event. On the other
hand, if you walk up to an aircraft with a
box of tools in your hand, nothing changes.
A feature of a complex system is the
phenomenon of retrospective coherence in
which the current state of affairs always
makes logical sense, but only when we
look backwards. Organisations tend to
study past events to create predictive and
prescriptive models for future decisions
based on the assumption that they are
dealing with a complicated system in which
the components and associated
relationships are capable of discovery and
management. (Snowden 2002:17).
There are particular methodological
consequences of this. Human variables must
be regarded as variables with identities, which
are (particularly in a networked society)
dispersed subjects. They are capable of acting
powerfully and ‘changing the subject’ from time
to time, and context to context. In short, they
are ‘variables with attitude’, and are unlikely to
be amenable to behaving as predictable and
deterministic ‘objects’, no matter how complex
the researcher’s psychological description and
analysis of the subject is.
3. Complex Adaptive Systems
In complex adaptive systems it is not useful to
look for directly, and predictably linked, causes
and effects; instead, what one has to look for
are emergent properties, attractors, and fitness
landscapes. The solutions within complex
adaptive systems are those that allow for
Electronic Journal of Business Research Methods, Volume 2 Issue 1 (2003) 71-78 73
http://www.ejbrm.com ©MCIL 2003 All rights reserved
interaction between the ‘subjects’ with a
substantial amount of freedom, but within
supportive rules.
A large shift in mind-set is required
…from one suited to linear, highly
predictable, systems to an approach based
on non-linear, less predictable systems, in
which internal chaotic interactions between
local actors produce self-organised …
order (Rihani op cit: xv).
He goes on to say
for a system to exist in a state of selforganised
Complexity, its internal elements
should be capable of interacting at an
appropriate level of connectivity and in
accordance to suitable local rules (p9).
In other words, a network of communication
between the people within the system, and a
certain degree of freedom within a set of
mutually acceptable rules are pre-conditions
for ‘self-organised complexity’ to emerge. Selforganised
complexity is distinct from either
chaos or order, and the
management of complex adaptive systems
is therefore a reiterative process that relies
on slow, and uncertain evolution (ibid).
It is vital to note that this does not mean a
linear approach is never valid; this is not an
either/or approach. In the management of
objects rather than subjects, a linear systems
approach could be valid and appropriate.
It is also important to realise that complex
adaptive systems are not merely systems
which are complex, and which adapt –
because they are made up of people with
identities. They are particular systems, which
have to be managed to elicit self-organised
complexity (a fine balance somewhere
between order and chaos), and which share
four common traits:
• They have active internal elements
that furnish sufficient local variety to
enable the system to survive as it
adapts to unforeseen circumstances
• They systems’ element are lightly but
not sparsely connected
• The elements interact locally according
to simple rules to provide the energy to
maintain stable global patterns, as
opposed to rigid order or chaos
• Variations in prevailing conditions
result in many minor changes and a
few large mutations, but it is not
possible to predict the outcomes in
advance (Rihani, op cit: 81).
4. Discourse
We are concerned in this paper to outline, both
theoretically and practically, ways in which a
combination of discourse theory and complex
adaptive systems theory can indeed offer us a
dialectical synthesis of the thesis of
modernism/positivism and the antithesis of
post-modernism. In order to do so, we need to
build on the radical scepticism common to both
modernism and post-modernism, and to
extrapolate and build further on the notion of
the dispersed subject. To this effect, we need
to insert the dispersed subject back into the
social – to put post-modernism back into
sociology (without the linear and positivistic
reductionism), back into communities of
practice and discourse communities, but not
‘communities’ as they have been traditionally
defined.
Rihani and Snowden both view linear and nonlinear
systems as what we would call different
discourses - different ways of making sense of
particular contexts (or all contexts, if you wish
to be reductionist) and acting within them.
There is an overlap between this notion of
discourse, (and discourse communities that
support and maintain particular discourses)
and the concept of Communities of Practice
(CoP). CoPs, as used in the ICT world, refer to
groups of people who may be organised very
informally, and who do not necessarily
maintain, or seek to maintain, any integrated
and sustained discourse over any considerable
period of time. Although there may be
similarities between this (ICT) understanding of
a Community of Practice and the concept to
we refer, there are distinct differences in that
discourses are generally more structured and
more stable
Discourses can be characterised in the
following ways
1. Discourses in broad terms serve two
related purposes, to make sense of the
environment, and to order it accordingly. Or as
Ferguson says, discourse is an “interpretative
grid”, but it is also “a conceptual ‘apparatus’ …
that does something” (1994: xiv).
2. It is quite possible to approach the same
issue from the point of view of quite different
discourses.
3. The best intentions do not always work
out in practice. Discourse is first and foremost
about what actually happens. This might relate
only ironically or paradoxically to what was
intended.
74 Charl Walters et al
http://www.ejbrm.com ©MCIL 2003 All rights reserved
4. And in overall terms, a discourse can be
distinguished from other discourses by:
• Its primary concerned and /or focus
• How it identifies its key issues
• What kinds of solutions it advocates
and implements
• What assumptions it makes about the
desirability of and necessity for change
• How it changes and develops in terms
of participation and consultation.
• The people who constitute the
community of practice.
• How it relates to other discourses –
does it take precedence or not and, if
so, under what circumstances?
We need to return to point 3, i.e. that what
actually happens might relate only ironically or
paradoxically to what was intended.
Discourses are sustained and systematic ways
of articulating, making sense of the
environment, and ordering it accordingly:
discourse is an “interpretative grid”, but it is
also “a conceptual ‘apparatus’ …that does
something” (Ferguson, ibid). The group of
people who support and maintain particular
discourses are its discourse community. Other
groups may of course contest these
discourses.
4.1 Discourses of Economic
Development
A classic analysis of what actually happens
within a particular discourse is that of
economic development in Lesotho, in
Ferguson’s book The Anti-Politics Machine
(1994). He locates the
intelligibility of a series of events and
transformations not in the intentions of one
or more animating subjects, but in the
systematic nature of the social reality
which results from those actions (op. cit:18,
emphasis added)…
and, continues
the outcomes of planned social
interventions can end up coming together
into powerful constellations of control that
were never intended and in some cases
never even recognised, but are all the
more effective for being ‘subject-less’... It is
this emphasis on the ‘systematic nature of
the resultant social reality’ that is the core
of the notion of discourse here. It includes
a framework for making sense of the world
and for planning interventions, but it also
includes what the anthropologist knows full
well, namely “how easily structures can
take on lives of their own (op. cit: 17).
Ferguson is not just stating that these are
unintended outcomes. He says that it is often
the case in economic development that
…outcomes, that at first appear as mere
‘side effects’ of an unsuccessful attempt to
engineer an economic transformation,
become legible in another perspective as
unintended … elements in a resultant
constellation that has the effect of
expanding the exercise of a particular sort
of state power while simultaneously
exercising a powerful depoliticising effect”
(op. cit: 21)
- hence, the “anti-politics machine”. More
specifically, Ferguson says later on:
the ‘development’ apparatus in Lesotho is
not a machine for eliminating poverty, that
is incidentally involved with the state
bureaucracy; it is a machine for reinforcing
and expanding the exercise of bureaucratic
state power, which incidentally takes
‘poverty’ as its point of entry …
depoliticising both poverty and the state …
Such a result may be no part of the
planners’ intentions – indeed it almost
never is – but resultant systems have an
intelligibility of their own (op. cit: 255-6).
4.2 Discourses of Distance Education
Ferguson’s analysis of economic development
in Lesotho is similar to Yates and Orivel’s
analyses of the management of distance
education. They found that distance education
paradoxically often exacerbates inequity while
increasing access. (Yates 2000, Orivel 2000).
Distance and Open Learning generally aims to
provide access to quality education. The most
important factors are:
accessibility, cost, distance, equity of
opportunity, and interaction in a supportive
environment.
These are the stated aims of distance learning.
But what actually happens? Or to put it another
way, what is the effect of discourses of
distance education on educational practices
and provision, and what are the realities?
Perraton writes that research on distance
learning can be interpreted in two ways. On
the one hand, it has provided new forms and
levels of access, and therefore increased
equity in education, while on the other hand, it
is a second-rate system used to offer a
shadow of education while withholding its
substance
Electronic Journal of Business Research Methods, Volume 2 Issue 1 (2003) 71-78 75
http://www.ejbrm.com ©MCIL 2003 All rights reserved
…[and] an inefficient way of containing
educational demand without meeting it …
[that] helps insulate the elite system from
pressures that might otherwise threaten its
status or ways of working (in Yates
2000:230).
Yates continues
paradoxically, basic education Open and
Distance Education (ODE) systems which
are set up to provide extended educational
opportunity to underprivileged groups also
often exploit those who work for them,” and
he says that “there are situations where
distance learners are required to pay a
disproportionate part of the cost of their
education, compared with those who
attend more conventional institutions. This
represents a kind of double inequity for
those who cannot access conventional
provision (p236).
Yates also cites Oliveira and Orivel (1993):
The Brazilian teacher education project,
Logos II … can be seen as reinforcing
inequality… by a strange inequitable quirk
of policy, ODE learners may be said in
some instances to be subsidising the
inefficiencies of conventional education. In
such a case, ODE is masking, rather than
addressing, issues of social equity and
democracy” (op. cit: 237).
He quotes examples of the Malawi College of
Distance Education which provided education
for more than 50% of secondary education
students, on only 20% of the secondary
education budget, and the Papua New Guinea
College of Education which similarly provided
education for 50% of secondary school pupils,
on only 5% (five percent!) of the secondary
education budget (ibid). Not only does this
highlight the practice of providing cheap, and
often inferior quality education (as in the cases
discussed), but it also raises the question
seriously under-paid and /or inadequately
supported staff.
So we have to be very aware of not only how
distance education is, or is not, satisfying its
own internal, or intended outcomes, but also
how it functions within the broader provision of
education and social equity – what the
discourse does, as well as what it says it does,
and more importantly, what it begins to
represent.
Distance education can be analysed within
different management discourses: that of input
compliance, or administration, or that of
outcomes management. Interestingly in
Perraton’s comments above (ibid) he not only
says that distance learning can be interpreted
in two different ways, but that “the evidence will
fit either interpretation”. This is confusing and
unhelpful. It would be better to analyse the
different discourses that are at stake here,
examining who maintains, defends and uses
them, to what purpose, in which contexts and
who successfully or unsuccessfully challenges
them. Discourses at this level are not ‘equally
valid’ options. They are political and social
choices that have implications for what
happens, what works, and for whom it works.
Within a discourse of input compliance for
instance, one might use Perraton’s data to
conclude that ‘access’ had been successfully
provided. Then again, within an outcomes
discourse, one might conclude that access had
only been provided at the expense of equity
and quality, and that it also functioned as a
cross-subsidy from the poor to those who were
already privileged and already had inequitable
access to educational resources. Furthermore,
one might conclude that the discourse was
politically successful mainly in that it deceived
the socially excluded into believing that what
was being provided was part of welfare
provision and promoted equity, rather than
actually being a ‘dis-welfare’; in this case a
double dis-welfare in that the recipients of diswelfare
usually subsidise the beneficiaries of
welfare elsewhere in the system.
Orivel (2000) points out that:
The least developed countries have a
simple choice to make: either they
introduce new technologies in their schools
at the expense of expanding school
opportunities to currently excluded
children, or they concentrate their limited
resources on educational expansion, and
thus renounce the chance to develop new
technologies in their school systems. As
long as GDP per capita remains highly
unequal from one country to another, the
capacity of new technologies to reduce the
education gap will not constitute a viable
option (op. cit. 138).
He takes a traditional, linear, economic
approach to the discourse of Open and
Distance Learning. An Economic Discourse
looks for, and at, discrete variables that can be
reduced to numbers. In this case, ‘technology’
is one discrete variable, and ‘education’ the
other. ‘Technologies’ are further reduced to
NICT (“new information and communication
technologies”), which are further reduced to
CAI (computer aided instruction), which is
further simplified by saying
76 Charl Walters et al
http://www.ejbrm.com ©MCIL 2003 All rights reserved
the best assumption one can make is that
one hour of learning in both cases [face-toface
teaching vs. CAI] generates on
average the same educational outcome.
At this stage the content has been so oversimplified
that one wonders whether Orivel, is
talking about anything at all, apart from
numbers)
The only alternative for Orivel is the low-tech
approach of teachers’ talk-and-chalk. He
believes that there is, in effect, a ‘simple’
choice between low-tech and high-tech. And
goes on to argue that for as long as we have
large inequities in GDP per capita, virtually no
developing country should use ICT in
education. That might be a very long time, and
seems a rash statement, even though he later
qualifies this by specifying that the threshold
for the use of ICT in education is $7,300 per
capita GDP. Nevertheless, that too will be a
long time in coming in many countries. There
are many problems here, apart from the
extensive economic reductionism.
Technological change and the adoption and
implementation of new technologies doesn’t
happen in discrete variables – certainly not
that discrete.
What is needed is not the sudden
transformation of education through the use of
technology, but rather the step-wise addition of
features for communication and learning; not a
complete change of systems. Besides, CAI has
not been at the top of anyone’s priority a list
since the late 1980’s. What learners and
educationalists are exited about now are the
possibilities that ICT offers for interaction with
humans, not machines. CAI will continue to
have its place, if it can be justified in terms of
costs, but it’s only a drill-and-practice box, and
an interesting box to help you ask further
questions. The enthusiasm for ‘expert systems’
has also faded, and what is left are useful
simulation and modelling systems which are
best used to find better questions, rather than
better answers. What’s more, these simulation
and modelling systems are even more
specialised and more expensive than CAI, and
are only appropriate and feasible in a business
strategy environment, not a basic learning
environment.
Not every teacher needs to have a desktop
computer or a high-powered laptop. The issue
is that learners and teachers need to be
connected to humans and machines in a
network that as a whole will provide them with
incrementally better learning and teaching, and
personal knowledge management
opportunities, as well as support through a
carefully structured environment, which
includes materials that form a well-designed
and integrated package. Too often e-Learning
‘opportunities’ are planned as low-budget
ventures with little regard to the participants.
It’s not a numbers game to see if everyone can
master every skill that is available, nor should it
be seen as a purely economic venture,
although one clearly has to take account of
budgets and costs.
Within a network configuration approach to
learning-and-communication we are no longer
dealing with independent variables in the strict
sense of the term. Orivel’s approach assumes
independent (and discrete) variables: he
assumes that we are dealing with inputs such
as CAI/NICT from the outside, which are
applied to/inserted into a fenced-off domain
called ‘education’, much as an economist
would approach issues of production and
consumption. Orivel’s application of traditional
economic discourse to (N)ICT-enhanced
education is not an example of a ‘possible’ and
‘equally valid’ discourse, which one might
choose to analyse networked learning. It is
simply wrong. It’s not applicable at all. The
whole point about networked distance learning
is that it is a configuration within which learning
takes place, and where all of the learners
(consumers) are potentially contributors
(producers) as well. The learners and teachers
are part of a network (which extends to many
others, quite outside the education sector),
which as a whole makes up the networked
learning environment. They are no longer
consumers of externally produced goods and
services, in the way that the earlier
beneficiaries of mass education were, and if
they continue to be seen as such, networked
learning will never take off.
Orivel makes revealing comments about the
key factors that allow developing countries to
achieve some measure of mass education in
the current context. He cites the example of
China, to which Rihani also refers (op. cit:
chapter 3). China, says Orivel, is a “special
case, where the dependency ratio [the ratio of
people in the workforce to people not in
employment] has fallen dramatically, and
where unit costs are also very low. This
unusual combination of factors allows China to
allocate a lower percentage of GNP to
education without sacrificing the objective of
expanding education opportunities. In addition,
China is able to allocate more resources to
physical investment” (op. cit: 146). In other
words, because China has lowered the birth
Electronic Journal of Business Research Methods, Volume 2 Issue 1 (2003) 71-78 77
http://www.ejbrm.com ©MCIL 2003 All rights reserved
rate, the ‘dependency’ of young people - for
funds to pay for their education, on older
people - who are taxed to provide those funds,
is lower. In comparison to other developing
countries, more people are working and paying
taxes, and fewer people are at school. That,
combined with low ‘unit costs’ (i.e. low
teachers salaries) enables China to provide
mass education without eating too much into
GNP.
This is a sound argument, and a good
economic analysis. The answer would seem to
be plain. Education for All can be achieved;
you just need to reduce the birth rate, and
everything will fall into place. But then Orivel
ducks the issue, and defaults to the comfort
zone of a linear economic discourse. He says,
very tellingly, that although this argument
may have some policy relevance in terms
of priorities”, one must remember, “it is
easier to manipulate unit costs than
fertility rates” (ibid, emphasis added).
In other words we are back to the overriding
linear (modernist) discourse in which
‘education costs’ and ‘reproductive health’ are
regarded as discrete variables and domains,
an economic discourse in which such messy
(and unmanageable) variables as ‘fertility
rates’ are seen as ‘unsuitable’ for intervention,
especially given the reductionist / correct,
political environments of today. It’s a discourse
in which objectifiable, quantifiable, discrete,
independent variables are fore grounded, and
in which continuous, complex adaptive human
subjects and their behaviours are excluded
because they are ‘more difficult to manipulate’.
4.3 Primary Health Care Management
Discourse analysis can also assist in analysing
seemingly coherent management domains
such as Primary Health Care, where there are
in fact a number or disparate discourses in
operation. These can undermine the very
notion of a Primary Health Care system. One
only needs to think of the ‘subjects who make
up the communities of practice that intersect
across the sets of ‘sub-disciplines’ of primary
health care, all of whom may be intent on
doing their job well, but few if any of them
effectively contributing to the management of a
Primary Health Care system. The examples
from Distance Education and Development
Discourse call attention to areas of
contestation in the Health Care sector, and
foreground how theories of discourse and
Complex Adaptive Systems can inform the way
we manage the demands presented by
opposing subject positions, and ‘variables with
attitude’.
5. Conclusion
We have examined the development of
different modes of social organisation and
knowledge, and outlined the ways in which
modernism and post-modernism can be said to
share a fundamental characteristic – radical
scepticism. We have also argued that complex
adaptive systems can be seen to be a
synthesis of the thesis of modernism and the
antithesis of post-modernism. Finally, we’ve
touched on the how complex adaptive systems
theory can be used to systematise the use of
‘discourse’ in the management of development
economics and distance education. This
requires a shift in our thinking from linear to
non-linear systems, and from objective
variables to human ‘variables with attitude’ –
i.e. with identities as ‘dispersed subjects’. In
terms of both discourse and complex adaptive
systems, it is important to note that the
intelligibility of a series of events and
transformations [is to be found] not in the
intentions of one or more animating
subjects, but in the systematic nature of
the social reality which results from those
actions” (Ferguson 1994:18).
References
Ferguson, J. (1994) The Anti-Politics Machine.
University of Minnesota Press, USA.
Henriques, J et al, (1984) Changing the
Subject. Routledge, London
Orivel, F. (2000) Finance, costs and
economics, in: Yates and Bradley (ed):
Basic Education at a Distance.
Routledge, London.
Perraton, H. (2000). Open and Distance
Learning in the Developing World.
London: Routledge
Rihani, S. (2002) Complex Systems Theory
and Development Practice, Zed Books,
London.
Snowden D.J. (2002) Complex Acts of
Knowing: Paradox and Descriptive Selfawareness.
Journal of Knowledge
Management Vol 6, No. 2, (May).
Williams, R.T. (1993) Texts and Discourses: a
framework for the production of meaning,
in: Mass Media for the 90s. De Beer, A.S.
(ed) van Schaik, Pretoria, South Africa.
Yates, C. (2000) Outcomes, what have we
learned? in: Yates and Bradley (ed): Basic
Education at a Distance. Routledge,
London.
Pre-scientific Modernist Post-Modernist Complex Adaptive Systems
Threshold Criteria Language, tool-making
Culture & tradition
Radical Scepticism of Nature
“2 cultures”: culture and science
Radical Scepticism of Language,
Society, Culture (& Science)
Dispersed subjects
Virtual CoP.
Networked Society
Configurations of Discourses
Nature Nature Nature
Science
Ecology Ecology
Society Culture & Tradition Culture & Individual identity & choice Dispersed Subjects, contesting
agency and structure
Virtual Communities of Practice/
Configurations of Discourse
Communities.
Social ecologies
Person Pre-determined Individual Identity – “made” Subject positions constructed and
contested.
In & out of Subject Positions in
various CoP.
Basis of Information
System
Traditional Use Commodified Information (exchange) Pastiche Virtual Communities of Practice.
Capital Tradition & Craft Technology
Algorithms
Commodities – goods and objective,
procedural information – context and
subject stripped.
Algorithms, Objects, Contexts,
Commodities as Objects,
Deconstruction,
Decontextualisation.
Algorithms, Deconstructed and
Decontextualised Objects, BUT
subject to social ecolologies of virtual
CoP.
Cumulative MODES
(not “eras”)
Pre-scientific Modernist Post-Modernist Complex Adaptive Systems
Variables Metaphysics, within which
is Nature and Culture.
Nature and Society as Objects of Study,
with some cognisance of Individual
Identity, the unconscious, etc.
Highly constructivist notion of
variables as ‘socially constructed’.
‘Individuals’ replaced by ‘dispersed
subjects’.
Different types of variables, for which
different modes of discourse are
appropriate. ‘Subject positions’
within various CoP.
Epistemology Experience
Tradition
Metaphysics
Metaphysics & Truth
Reason
Falsifiablility
Commoditised, Objectified
knowledge/procedural information.
Rational Truth
Ironic Experience/ configurations
Just-in-context Strategic Knowledge
Useful algorithms & discourses
Virtual/immediate experience/
configurations.
Just-in-context Strategic Knowledge
Ecologies and Configuration of CoP
Management Tradition, Obedience Compliance, Administration
….. Executive Management
Executive Management of
Configurations
Ecologies and Configurations of CoP.
Chaos, Emergent Properties, Sticky
Events, Historical Accidents
Markets Local, barter + Commodities Globally transparent Virtual / CoP.
Entrepreneurs ? Preserving, expanding New algorithms New configurations New ecologies & configurations.
Methodology Preserving & defending
traditions
Positivism, anthropology Deconstruction, Discourse Theory,
ethnography.

Meta-Analysis of Distance Education

The Effects of Distance Education on
K–12 Student Outcomes:
A Meta-Analysis
October 2004
Meta-Analysis of Distance Education
Learning Point Associates 2
The Effects of Distance Education on
K–12 Student Outcomes:
A Meta-Analysis
October 2004
Cathy Cavanaugh
University of North Florida
Kathy Jo Gillan
Duval County Public Schools
Jeff Kromrey
University of South Florida
Melinda Hess
University of South Florida
Robert Blomeyer
North Central Regional Educational Laboratory
1120 East Diehl Road, Suite 200
Naperville, Illinois 60563-1486
(800) 356-2735  (630) 649-6500
www.learningpt.org
Copyright © 2004 Learning Point Associates, sponsored under government contract number ED-01-CO-0011.
All rights reserved.
This work was originally produced in whole or in part by the North Central Regional Educational Laboratory
with funds from the Institute of Education Sciences (IES), U.S. Department of Education, under contract number
ED-01-CO-0011. The content does not necessarily reflect the position or policy of IES or the Department of
Education, nor does mention or visual representation of trade names, commercial products, or organizations
imply endorsement by the federal government.
Meta-Analysis of Distance Education
Learning Point Associates 3
Contents
Abstract ........................................................................................................................4
Introduction ..................................................................................................................5
Distance Education in the K–12 Context .............................................................5
Characteristics for Success ..................................................................................6
Teaching and Learning Theory............................................................................7
Purpose of the Study.....................................................................................................8
Method .......................................................................................................................10
Location and Selection of Studies......................................................................11
Limitations of the Review.................................................................................13
Coding of Study Features ..................................................................................13
Calculation of Effect Sizes ................................................................................14
Statistical Analysis of Effect Sizes ....................................................................15
Results........................................................................................................................15
Characteristics of the Study...............................................................................15
Overall Effects on K–12 Distance Education.....................................................16
Publication and Methodological Variables.........................................................18
Distance Education Variables............................................................................18
Instructional and Program Variables..................................................................19
Discussion ..................................................................................................................19
Implications for Research and Practice..............................................................19
Conclusions ................................................................................................................21
The Need for Prospective Study in Virtual Schooling........................................22
Recommendations for K–12 Online Learning Policy and Practice ....................23
References ..................................................................................................................26
Appendix: Coded Variables and Study Features in the Codebook ...............................32
Meta-Analysis of Distance Education
Learning Point Associates 4
Abstract
The community of K–12 education has seen explosive growth over the last decade in distance
learning programs, defined as learning experiences in which students and instructors are
separated by space and/or time. While elementary and secondary students have learned through
the use of electronic distance learning systems since the 1930s, the development of online
distance learning schools is a relatively new phenomenon. Online virtual schools may be ideally
suited to meet the needs of stakeholders calling for school choice, high school reform, and
workforce preparation in 21st century skills. The growth in the numbers of students learning
online and the importance of online learning as a solution to educational challenges has increased
the need to study more closely the factors that affect student learning in virtual schooling
environments. This meta-analysis is a statistical review of 116 effect sizes from 14 webdelivered
K–12 distance education programs studied between 1999 and 2004. The analysis
shows that distance education can have the same effect on measures of student academic
achievement when compared to traditional instruction. The study-weighted mean effect size
across all outcomes was -0.028 with a 95 percent confidence interval from 0.060 to -0.116,
indicating no significant difference in performance between students who participated in online
programs and those who were taught in face-to-face classrooms. No factors were found to be
related to significant positive or negative effects. The factors that were tested included academic
content area, grade level of the students, role of the distance learning program, role of the
instructor, length of the program, type of school, frequency of the distance learning experience,
pacing of instruction, timing of instruction, instructor preparation and experience in distance
education, and the setting of the students.
Meta-Analysis of Distance Education
Learning Point Associates 5
Introduction
The community of K–12 education has seen explosive growth over the last decade in distance
learning programs, defined as learning experiences in which students and instructors are
separated by space and/or time. While elementary and secondary students have learned through
the use of electronic distance learning systems since the 1930s, the development of online
distance learning schools is a relatively new phenomenon. Online virtual schools may be ideally
suited to meet the needs of stakeholders calling for school choice, high school reform, and
workforce preparation in 21st century skills. The growth in the numbers of students learning
online and the importance of online learning as a solution to educational challenges has increased
the need to study more closely the factors that effect student learning in virtual schooling
environments.
Beginning in the 1930s, radio was used simultaneously to bring courses to school students and to
help teachers learn progressive Deweyan methods of teaching (Bianchi, 2002), in what might
have been among the earliest professional development school models. From that point on,
television, audio and videoconferencing, the Internet, and other technologies have been adapted
for the needs of young learners. This meta-analysis is a statistical review of web-delivered K–12
distance education programs between 1999 and 2004 conducted in order to determine how
student learning in online programs compares to learning in classroom-based programs, and to
identify the specific factors that influence student learning.
Distance Education in the K–12 Context
The many thousands of K–12 students who participate in online education programs are attracted
to virtual schooling because it offers advantages over classroom-based programs. Among the
benefits of distance education for school-age children are increases in enrollment or time in
school as education programs reach underserved regions, broader educational opportunity for
students who are unable to attend traditional schools, access to resources and instructors not
locally available, and increases in student-teacher communication. Students in virtual schools
showed greater improvement that their conventional school counterparts in critical thinking,
researching, using computers, learning independently, problem-solving, creative thinking,
decision-making, and time management (Barker & Wendel, 2001). Academic advantages over
traditional classroom instruction were demonstrated by students in Mexico’s Telesecundaria
program, who were “substantially more likely than other groups to pass a final 9th grade
examination” administered by the state (Calderoni, 1998, p. 6); by students taking a chemistry by
satellite course (Dees, 1994); and by students learning reading and math via interactive radio
instruction (Yasin & Luberisse, 1998). Virtual school developers and instructors continue to
refine their practice, and in so doing, they learn from reports of both successful and unsuccessful
programs.
Virtual schooling, like classroom schooling, has had limited success in some situations. In an
online environment, students may feel isolated, parents may have concerns about children’s
social development, students with language difficulties may experience a disadvantage in a textheavy
online environment, and subjects requiring physical demonstrations of skill such as music,
physical education, or foreign language may not be practical in a technology-mediated setting.
Meta-Analysis of Distance Education
Learning Point Associates 6
For example, Bond (2002) found that distance between tutor and learner in an online
instrumental music program has negative effects on performance quality, student engagement,
and development and refinement of skills and knowledge. While distance learning was viewed as
beneficial for providing the opportunity for elementary school students to learn a foreign
language, Conzemius and Sandrock (2003) report that “the optimal learning situation still
involves the physical presence of a teacher” (p. 47). Virtual school students show less
improvement than those in conventional schools in listening and speaking skills (Barker &
Wendel, 2001). Highly technical subjects such as mathematics and science have also proven to
be difficult to teach well online. The Alberta Online Consortium evaluated student performance
on end-of-year exams among virtual school students across the province, and found that virtual
school student scores in mathematics at grades 3, 6, 9, and 12, and the sciences at grades 6 and 9
lagged significantly behind scores of nonvirtual school students (Schollie, 2001).
Given instruction of equal quality, groups of students learning online generally achieve at levels
equal to their peers in classrooms (Kearsley, 2000). Equality between the delivery systems has
been well documented over decades for adult learners, and while much less research exists
focusing on K–12 learners, the results tend to agree. “Evidence to date convincingly
demonstrates that , when used appropriately, electronically delivered education—‘e-learning’—
can improve how students learn, can improve what students learn, and can deliver high-quality
learning opportunities to all children” (National Association of State Boards of Education, 2001,
p. 4). Many studies report no significant differences between K–12 distance education and
traditional education in academic achievement (Falck et al, 1997; Goc Karp & Woods, 2003;
Hinnant; 1994; Jordan, 2002; Kozma et al, 2000; Mills, 2002; Ryan, 1996), frequency of
communication between students and teachers (Kozma et al), and attitude toward courses
(McGreal, 1994).
Although various forms of technology-enabled distance education for pre-college students have
been in use for nearly a century, rapid change in technology and the educational context have
resulted in a small body of research relevant to today’s conditions that can serve to guide
instructors, planners, or developers. The temptation may be to attempt to apply or adapt findings
from studies of K–12 classroom learning or adult distance learning, but K–12 distance education
is fundamentally unique.
Characteristics for Success
A primary characteristic that sets successful distance learners apart from their classroom-based
counterparts is their autonomy (Keegan, 1996) and greater student responsibility (Wedemeyer,
1981). By the time they reach higher education, most adults have acquired a degree of autonomy
in learning, but younger students need to be scaffolded as part of the distance education
experience. Virtual school teachers must be adept at helping children acquire the skills of
autonomous learning, including self-regulation. Adult learners more closely approach expertise
in the subjects they study and in knowing how to learn, due to their long experience with the
concepts and with meta-cognition, whereas children are relative novices. This distinction is
important because experts organize and interpret information very differently from novices, and
these differences affect learners’ abilities to remember and solve problems (Bransford, Brown, &
Meta-Analysis of Distance Education
Learning Point Associates 7
Cocking, 1999), and their ability to learn independently. Expert learners have better developed
metacognition, a characteristic that children develop gradually.
A second characteristic that differentiates successful distance learners from unsuccessful ones is
an internal locus of control, leading them to persist in the educational endeavor (Rotter, 1989).
Research has found that older children have more internal locus of control than younger children
(Gershaw, 1989), reinforcing the need for careful design and teaching of distance education at
K–12 levels. Younger students will need more supervision, fewer and simpler instructions, and a
more extensive reinforcement system than older students. Effective online programs for young
learners include frequent teacher contact with students and parents, lessons divided into short
segments, mastery sequences so student progress can grow in stages, and rewards for learning
such as multimedia praise and printable stickers or certificates.
Young students are different from adult learners in other ways. Piaget’s stages of cognitive
development, in particular preoperational (2 to 7 years), concrete operational (7 to 11 years), and
formal operational (11 years to adulthood) outline the phases in development toward adulthood.
The stages offer pedagogical guidance for delivering effective web based education, which
should focus on the major accomplishments of learners in these stages. Each stage is
characterized by the emergence of new abilities and ways of processing information (Slavin,
2003, p. 30), which necessitates specialized instructional approaches and attention to each child’s
development. Since adults have progressed through these stages of cognitive development,
delivery of web based education at the adult level need not concentrate on methods that help the
learner develop these cognitive skills. In contrast, web-based instruction for students in their
formative years must include age appropriate developmental activities, building on the students’
accomplishments in and through the cognitive stages. For example, an online mathematics or
science lesson designed for students at the preoperational stage needs to use very concrete
methods, such as instructing the student to develop concepts by manipulating and practicing with
real-world objects. The concept can built upon for students in the concrete operational stage
using multimedia drag-and-drop manipulations and representations, or realistic simulations. At
the formal operational stage, students are capable of using symbols, language, and graphic
organizers to continue to learn the concepts in more abstract ways.
Teaching and Learning Theory
Piaget helps us to understand that learning should be holistic, authentic, and realistic. Less
emphasis should be placed on isolated skills aimed at teaching individual concepts. Students are
more likely to learn skills while engaged in authentic, meaningful activities. Authentic activities
are inherently interesting and meaningful to the student. Web-based technology offers a vast
array of opportunities to help expand the conceptual and experiential background of the student
(Bolton, 2002, p. 5).
Neo-Piagetian theorists have expanded on Piaget’s model of cognitive development. Among
others, Vygotsky proposed that historical and cultural context play significant roles in helping
people think, communicate, and solve problems, proposing that cognitive development is
strongly linked to input from others. Vygotsky’s theory implies that cognitive development and
the ability to use thought to control our own actions require first mastering cultural
Meta-Analysis of Distance Education
Learning Point Associates 8
communication systems and then learning to use these systems to regulate our own thought
process. He believed that learning takes place when children are working within their zone of
proximal development. Tasks within the zone of proximal development are ones that children
cannot yet do alone but could do with the assistance of more competent peers or adults (Slavin,
2003, p. 43–44). When working with children using web-based technology, teachers must offer
students activities that make use of the web’s powerful tools for collaborative learning, and are
within their zone of proximal development. Online communities can provide a supportive
context that makes new kinds of learning experiences possible (Bruckman, 1998, p. 84–85).
Constructivism, a widely used theory in distance education, is founded on the premises that by
reflecting on our experiences and participating in social-dialogical process (Duffy & Cunnigham,
1996), we construct our understanding of the world we live in. Each of us generates our own
"rules" and "mental models," which we use to make sense of our experiences. Learning,
therefore, is simply the process of adjusting our mental models to accommodate new experiences
(Brooks & Brooks, 1993). Children have not had the experiences that adults have had to help
them construct understanding. Therefore, children construct an understanding of the world
around them that lacks the rich experiences that adults have had. Scaffolding or mediated
learning is important in helping children achieve these cognitive understandings (Slavin, 2003, p.
259), and are essential components of web-based learning experiences for children. Online
learning environments, when designed to fully use the many tools of communication that are
available, is often a more active, constructive, and cooperative experience than classroom
learning. In addition, technologies that are easily employed in online environments, such as mind
mapping tools and simulations, are effective means for helping students make meaning of
abstract phenomena and strengthen their meta-cognitive abilities (Duffy & Jonassen, 1992).
Purpose of the Study
With the emphasis on scientifically-based research and the call for evidence-based program
decisions in the federal No Child Left Behind Act of 2001, scientific evidence is needed to guide
the growing numbers of online school developers and educators. Many studies of K–12 distance
education have been published, but a small proportion of them are controlled, systematic,
empirical comparisons that fit the definition of “scientific,” as it is defined by the U.S
Department of Education and described at the What Works Clearinghouse website,
http://www.w-w-c.org/. This study is an effort to search for and collect the studies that fit the
definition of scientific research on K–12 distance education programs, and to draw conclusions
about the effectiveness of distance education for K–12 students based on the synthesized
findings of the studies.
Meta-analysis is an established technique for synthesizing research findings to enable both a
broader basis for understanding a phenomenon and a parsing of influences on the phenomenon.
Several recent meta-analyses related to distance education have been published in recent years
(see Table 1).
Meta-Analysis of Distance Education
Learning Point Associates 9
Table 1
Summary of recent meta-analyses in distance education
Author(s), Date Focus N of studies Effect Size
Machtmes & Asher, 2000. Adult telecourses 30 -0.0093
Cavanaugh, 2001. Academic
achievement of K–
12 students
19 +0.015
Allen, Bourhis, Burrell, &
Mabry, 2002.
Student satisfaction
among adult
learners
25 +0.031
Bernard, Abrami, Lou,
Borokhovski, Wade,
Wozney, Wallet, Fiset, &
Huang, 2003.
Student
achievement,
attitude, retention
232 +0.0128
Shachar & Neumann, 2003. Student
achievement
86 +0.37
Ungerleider & Burns, 2003. Networked and
online learning
12 for achievement
4 for satisfaction
0 for achievement
-0.509 for
satisfaction
The Sachar and Neumann study was the only one to have found a moderate effect for distance
education. Only one of the recent meta-analyses in distance education focused on K–12 learners,
and it included web-based programs along with the analog conference and broadcast programs
no longer in common use in today’s virtual schools. The explosion in virtual schools, especially
virtual charter schools in the United States, has necessitated a fresh look at the knowledge base.
The need is for research that guides practitioners in refining practice so the most effective
methods are used. Given sufficient quantity and detail in the data, meta-analysis is capable of not
only comparing the effectiveness of distance education programs to classroom-based programs,
but it can compare features of various distance education programs to learn what works. For
example, synchronous programs can be compared to nonsynchronous programs. Meta-analysis is
a tool that allows looking in detail at virtual schooling practice and results, and it can lead to
better informed practice and improved results.
Several advantages can result from a synthesis of studies of the effectiveness of distance
education programs for K–12 learners. Because all of the studies included in this review drew
data from school-based classes, the review can provide valuable insight into the practical
effectiveness of K–12 distance education. Controlled experimental research may offer findings
of theoretical interest but may not be generalizable to complex learning settings such as virtual
schools or classes. The uncontrollable cultural and social variables naturally present in a school
or class, whether online or on-ground, make a statistical synthesis a more exact test of the
strength of K–12 distance education. The effects of virtual learning would have to be strong and
consistent to be measurable across a range of natural milieus.
The purpose of this meta-analysis is to provide a quantitative synthesis of the research literature
of web-based K–12 distance education from 1999 to the present, across content areas, grade
Meta-Analysis of Distance Education
Learning Point Associates 10
levels, and outcome measures. The first goal was to determine the effects of distance education
on K–12 student outcomes, specifically academic achievement. The second goal was to identify
the effects on student outcomes of the features of distance education, including content area,
duration of use, frequency of use, grade level of students, role of the instructor, type of school,
timing of interactions, and pacing of the learning.
From the literature, the meta-analysis seeks to answer the following questions:
1. Is distance education as effective, in terms of student achievement, as classroom-based
instruction?
2. To what extent are student outcomes related to the features of a distance education system
(duration of use, frequency of use, role of the instructor, timing of interactions, and pacing of the
learning)?
3. To what extent are student outcomes related to features of the educational context (content
area, school type, and grade level)?
4. To what extent are results related to study features (year, type of publication, various potential
threats to validity)?
Meta-analysis, the use of statistical analysis to synthesize a body of literature, is appropriate for
answering questions such as these because it allows comparison of different studies by
computing an effect size for each study. Meta-analysis is used to estimate the size of a
treatment’s effect, and allows investigation into relationships among study features and outcomes
(Bangert-Drowns, 2004). The inclusion of a study in a meta-analysis is limited by several
factors, the most significant of which is the reporting of the information needed to compute
effect size. Very often, reports released by virtual schools and other distance education programs
do not include mean scores, comparison group scores, sample sizes, or standard deviations.
Nonetheless, the meta-analytic technique is a way to identify effects or relationships in literature
that may not be evident otherwise (Lipsey & Wilson, 2001).
Method
This quantitative synthesis is a meta-analysis of empirical studies published since 1999 that
compared the effects of web-delivered distance education with classroom-based learning on K–
12 student academic performance. Since 1999 the sophistication in the use of distance learning
tools has improved, but the types of tools available to schools have remained approximately the
same. The stages of the meta-analysis were identification and retrieval of applicable studies,
coding of study features and findings, and data analysis. These stages are described below.
For the purposes of this meta-analysis, studies were included in the analysis if they met the
following criteria for inclusion. The studies must:
• Be available as a journal article, dissertation or report in English between 1999 and 2004.
• Compare K–12 students in a distance education group to a nondistance education group,
or compare the distance education group before and after distance education.
• Use web-based telecommunications, such that at least 50 percent of the students’
participation in the course or program occurred at a physical distance from the instructor.
Meta-Analysis of Distance Education
Learning Point Associates 11
• Be quantitative, experimental, or quasi-experimental studies for which effect size could
be computed, the outcome measures were the same or comparable, and the N was 2 or
greater.
• Use student academic achievement, motivation, attitude, retention, or conduct as outcome
variables.
Location and Selection of Studies
Numerous databases, journals, websites, and bibliographic resources were searched for studies
that met the established inclusion criteria. In each case, search terms included:
 cybercharter
 cyberschool
 distance education
 distance learning
 elearning
 mlearning
 online school
 open learning
 open school
 schoolnet
 telelearning
 virtual charter
 virtual school.
Electronic searches were systematically conducted in the following databases:
 Dissertation Abstracts
 ERIC
 JSTOR
 Kluwer
 ProQuest Education
 PsychInfo
 Wilson Education.
Web searches were performed using the Google, Teoma, Grokker, MetaCrawler, and AltaVista
search sites.
Abstracts in the following distance education journals were examined:
 American Journal of Distance Education
 Computers & Education
 Distance Education
 Journal of Distance Education
 Journal of Distance Learning
 Open Learning.
Meta-Analysis of Distance Education
Learning Point Associates 12
Abstracts in the following educational technology journals were examined:
 Association for the Advancement of Computing in Education journals
 Australasian Journal of Educational Technology
 British Journal of Educational Technology
 Canadian Journal of Educational Communication
 Canadian Journal of Learning and Technology
 Computers in the School
 Educational Technology & Society
 Educational Technology Research and Development
 Journal of Computer Mediated Communication
 Journal of Computing in Childhood Education
 Journal of Educational Computing Research
 Journal of Information Technology Education
 Journal of Interactive Media in Education
 Journal of Research on Technology in Education.
Abstracts in American Educational Research Journal were examined, as were abstracts in the
following electronic journals:
 Australian Educational Computing
 Australian Journal of Educational Technology
 Electronic Journal for the Integration of Technology in Education
 International Journal of Educational Technology
 International Review of Research in Open and Distance Learning
 Journal of Asynchronous Learning Networks
 Journal of Interactive Online Learning
 Online Journal of Distance Education Administration
 TechKnowLogia: International Journal of Technologies for the Advancement of
Knowledge and Learning
 Turkish Online Journal of Distance Education.
In addition, abstracts were examined in the following conference proceedings:
 American Education Research Association
 Canadian Association for Distance Education
 EdMedia
 E-Learn/WebNet
 Society for Technology in Teacher Education.
The web sites of several distance education organizations and over 200 virtual schools were
browsed for studies, and the director of each virtual school was contacted at the email address
listed on the school’s website to request studies. The department of education website for each
state was browsed for report cards for state virtual charter schools. The reference lists of the six
recent meta-analyses of distance education shown in Table 1 were reviewed for potential studies.
Meta-Analysis of Distance Education
Learning Point Associates 13
Of the thousands of abstracts that were reviewed, 80 full-text articles, dissertations, or reports
concerning DE and traditional instruction at K–12 level were obtained and evaluated for
inclusion in the analysis. Independently, two researchers read all collected studies to determine
eligibility for inclusion based on the stated criteria. Fourteen of the studies were found to meet
all criteria for inclusion. Of the 66 studies that were examined and excluded, 28 percent were
descriptive reports, 14 percent reported on uses of telecommunications or other educational
technology that did not meet the definition of distance education, 25 percent reported results
without control or comparison group data, and 33 percent included summary data only or did not
provide data sufficient to compute effect size.
Limitations of the Review
For literature on K–12 distance education to be meaningfully synthesized, the inclusion criteria
had to be narrowly specified. This synthesis included studies with data on the performance of
grades 3–12 students in web-based distance learning programs compared to students in
classrooms. Measures of performance present in the literature do not draw a complete picture of
the full range of effects that students experience as a result of participation in distance education.
Qualitative studies, strict experimental studies, narrative reports, and other designs offer
information not acquired in this analysis. Although the inclusion criteria were designed to allow
a wide range of studies to be analyzed so that a comprehensive knowledge of K–12 distance
education would result, a small number of studies was analyzed. The results should be
interpreted with caution.
Coding of Study Features
Coding of study features allows the meta-analyst to unravel different study factors related to
variations in the phenomenon from factors related to method (Lipsey & Wilson, 2001). The
coding used in this analysis was identified from research on K–12 distance education and from
variables typically coded in contemporary meta-analyses in education. A trial conducted on a
small sample of studies led to the addition of variables in the codebook that were not present in
the initial set of variables. Each study was coded independently by two researchers according to
the established coding procedure. The full codebook is included in Appendix A. The initial interrater
agreement across all coded variables was 85 percent. Discrepancies between researchers
were discussed and resolved. The entire dataset was reviewed for the presence of discrepancies
and unexpected values.
Fourteen studies, with a total of 116 outcomes, had data sufficient to include in the analysis (see
Table 2). The dependent variable in this synthesis was student outcome measured by instruments
appropriate to the individual study given at the end of the distance education period which varied
from a few weeks to an entire academic year. The measures included district, state, or national
examinations, as well as teacher or researcher designed tests of academic performance.
The studies were coded on 45 factors, categorized into five groups: identification of studies,
distance education features, instructor/program features, study quality features, and sources of
invalidity (see Appendix A). Of particular interest were the variables associated with distance
education features (e.g. duration of the experience, role of the distance learning, role of the
Meta-Analysis of Distance Education
Learning Point Associates 14
instructor, timing of the interactions) and instructor/program features (e.g. amount of teacher
preparation for distance teaching, setting of the students). In many cases, however, the literature
failed to report the detail needed to make meaningful comparisons on these factors. The levels of
each variable were compared by computing average effect sizes for each level, but examination
of interactions among the different variables was not practical due to the small number of effect
sizes available.
Calculation of Effect Sizes
The effect sizes estimated for each study outcome were computed using Cohen’s d, defined in
this meta-analysis as the difference between the nondistance learning group and the distance
learning posttest mean scores divided by the average standard deviation. A correction factor for
small sample bias in effect size estimation (Hedges, Shymansky, & Woodworth, 1989) was used
in cases in which sample sizes were small. The unit of analysis was the study outcome. For
studies in which more than one independent group of students was evaluated, independent effect
sizes were estimated for each group, were weighted to avoid study bias, and were included in the
aggregated effect size estimate. A positive effect size, with a 95 percent confidence interval not
encompassing zero, is an indication that the distance learning group outperformed the
nondistance learning group.
Table 2
Selected study features and effect sizes for 14 studies of web-based K–12 distance education
Author, year Grade
level
Subject area School
type
Outcome
measure
Instructional
role of the
distance
learning
Timing of
interactions
N Weighted
mean
effect size
(d)
95% CI
for d
(upper/lo
wer)
Alberta
Consortium
2001*
3, 6, 9,
12
English,
mathematics,
science, social
studies
Mix of
public
and
private
National
test
Course Asynchronous 13–
397
-0.028 0.141/-
0.197
Alaska
Department of
Education and
Early
Development
2003*
4–7,
9–12
Reading,
writing,
mathematics
State
charter
State and
national
tests
Full program Synchronous 7–67 -0.005 0.303/-
0.313
Colorado
Department of
Education
2003a*
3–6 Reading,
writing,
mathematics
State
charter
State test Full program Asynchronous 33–45 -0.028 0.261/-
0.276
Colorado
Department of
Education
2003b*
7–8 Reading,
writing,
mathematics
State
charter
State test Full program Asynchronous 9–55 -0.029 0.199/-
0.258
Colorado
Department of
Education
2003c*
3–6 Reading,
writing,
mathematics
State
charter
State test Full program Combination
synchronous
asynchronous
14–23 -0.013 0.440/-
0.466
Colorado
Department of
Education
2003d*
7–8 Reading,
writing,
mathematics
State
charter
State test Full program Combination
synchronous
asynchronous
10–21 -0.013 0.449/-
0.475
Goc Karp & 9–12 Physical Public Class Portion of Asynchronous 19 -0.253 0.357/-
Meta-Analysis of Distance Education
Learning Point Associates 15
Woods 2003* education assignments course 0.863
Indiana
Department of
Education,
2004*
3, 6 Reading,
mathematics
State
charter
State test Full program Unspecified 17–18 0.001 0.470/-
0.468
Minnesota
Department of
Education
2003*
5 Reading,
mathematics
State
charter
State test Full program Unspecified 26 0.014 0.398/-
0.371
Mock 2000* 12 Science Public Teacher
made test
Portion of
course
Asynchronous 7 -0.472 0.472/-
1.416
Stevens 1999* 12 Science Public Teacher
made test
Portion of
course
Unspecified 21–33 -0.029 0.497/-
0.556
Washington
Office of the
Superintendent
of Public
Instruction
2003*
7 Reading,
Writing,
mathematics,
listening
State
charter
State test Full program Asynchronous 12–15 0.002 0.540/-
0.537
Wisconsin
Department of
Public
Instruction
2003
3 Reading State
charter
State test Full program Asynchronous 57 -0.016 0.243/-
0.276
Texas
Education
Agency 2003*
9–11 English,
mathematics,
science, social
studies
State
charter
State test Full program Combination 15–21 -0.014 0.445/-
0.474
* indicates studies yielding multiple effect sizes
Statistical Analysis of Effect Sizes
The test for heterogeneity (Q), based on Hedges and Olkin (1985), was used to determine
whether the effect sizes of the studies were homogenously distributed, in other words, to learn
whether the distribution of effect sizes around their mean was what would be expected from
sampling error alone (Lipsey & Wilson, 2001). The Q value for the weighted effect sizes was
1.485, and was considered to be homogeneous, indicating that the variance observed was likely
to be due to sampling error. Therefore, the fixed-effects model was used to estimate variance
(Kromrey & Hogarty, 2002). Study feature analyses were performed to determine the extent to
which student outcomes were moderated by the study variables. Statistical Analysis System
(SAS) software was used for the analyses. Effect size comparisons were done for the variables:
grade level, content area, duration and frequency of the distance learning experience,
instructional role of the distance education, pacing of the instruction, role of the instructor,
timing of the interactions, and types of interactions, as well as for various study quality and
invalidity factors.
Results
Characteristics of the Studies
The 14 studies included in the analysis yielded 116 independent effect sizes drawn from a
combined sample of 7561 students whose performance as a result of participation in a distance
education program was compared to control groups in which students did not participate in
Meta-Analysis of Distance Education
Learning Point Associates 16
distance education. Sixty one percent of the study results had sample sizes of less than 50, and 16
percent had sample sizes above 100. All but one of the studies included more than one
comparison, and the average number of comparisons per study was 8, ranging from one to 38.
Eighty six percent of the studies were organization reports, 7 percent were published articles, and
7 percent were dissertations. All of the studies were published between 1999 and 2004, with
eleven published during 2003 and 2004, and three published from 1999 through 2001. Two
studies were published in Canada, and the other twelve were published in the U.S.
A range of distance learning structures was examined in the literature. Half of the studies
reported on programs that used asynchronous timing in instruction. Three studies documented a
program that used a combination of synchronous and asynchronous instruction, one program was
delivered synchronously, and the remaining programs did not report on instructional timing. Ten
of the studies reported results of student participation in full year-long distance learning
programs, one included data for distance learning courses, and three studies focused on portions
of courses delivered at a distance for less than a semester. Thirteen studies included data from
programs in which students participated approximately five days per week, and the other study
did not indicate the frequency of student participation. The diversity of distance learning
structures is an indication of the wide range of educational uses to which it is being applied:
enhancement or extension to classroom instruction, school courses, and full-time educational
programs.
The studies encompassed a variety of instructional features. The bulk of the results, 75 percent,
occurred in the secondary grades, 6–12. The other results concern elementary age children, in
grades 3–5. Results from seven academic content areas were reported. Thirty percent of the
results came from tests of reading ability, followed by mathematics, which accounted for 26
percent of the results. Writing was the subject for 16 percent of the results, science was the topic
of 14 percent, and social studies made up 9 percent of the results. Three percent of results came
from physical education comparisons, and one percent from a test of listening. National tests
were used to compare outcomes in one study, state tests were used in nine studies, teacher made
tests were used in two studies, and one study reported data from both state and national tests.
Overall Effects of K–12 Distance Education
The analysis resulted in an overall weighted effect size not significantly different from zero, a
result that is consistent with the results of recent meta-analyses of distance education (see Table
1), which tend to show that distance education is as effective as classroom instruction. The
weighted mean effect size across all results was -0.028, with a standard error of 0.045 and a 95
percent confidence interval from -0.116 to 0.060. The average unweighted Cohen’s d was -0.034,
and the median effect size was -0.015. The effect sizes varied considerably among the studies.
Figure 1 displays the full range of effect sizes calculated for the 116 results across the horizontal
axis, and the number of results having each effect size on the vertical axis. The spike in the
number of results around the zero effect size is an indicator of the tendency of the overall effect
size. Unweighted effect sizes ranged from -1.158 to 0.597, with a standard deviation of 0.157,
indicating that some applications of distance education appeared to be much better than
classroom instruction and others were much worse.
Meta-Analysis of Distance Education
Learning Point Associates 17
Distribution of unweighted effect sizes
0
20
40
60
80
100
-1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
Effect size
Number of results
Figure 1. Distribution of unweighted effect sizes of 116 outcomes
The 95 percent confidence intervals also show wide variability in their size, as displayed in
Figure 2. Only one confidence interval did not encompass zero, and all but three effect sizes fell
between 0.5 and -0.5. Each of the fourteen studies and all except one of the 116 outcomes within
the studies had individual effect sizes that did not differ significantly from zero, indicating that in
almost every comparison, students in distance education programs performed as well as students
in classroom-based programs.
Meta-Analysis of Distance Education
Learning Point Associates 18
Figure 2. 95 percent confidence intervals for individual effect sizes of 116 outcomes.
Of the 45 factors coded in the study, the following 30 were examined to determine sources of
significant variation in effect sizes. Ten of the remaining variables were used for identifying the
studies or computing effect size, and the other five could not be compared because the studies
did not include the data for coding the variables, or the variable was not a relevant factor in the
studies. The variables that went uncoded due to the absence of data were the frequency of
student participation in distance learning, the level of preparation of the teachers in distance
education, and the amount of experience of the teachers in distance education. The variables that
were not relevant factors for the studies were control for the effects of a second testing, and
control for the effects of a pretest. Analysis of variance was not meaningful for some of the
variables because of missing data in the studies, resulting in a high number of cases in which a
value of “unspecified” was coded for the variable.
Publication and Methodological Variables
Twenty variables were coded to discover whether publication or methodological variables
accounted for variation in effect sizes. The publication features included the year of publication,
the type of publication, and the region of publication. The methodological variables related to the
testing sequence in the study, the type of achievement measure used in achievement studies,
pretest equivalency measures, study design, statistical power, and control for 12 potential sources
95 PercentConfidence Intervals for Individual Effect Sizes
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
Meta-Analysis of Distance Education
Learning Point Associates 19
of invalidity. None of the variable comparisons resulted in effect sizes significantly different
from zero (see Table 2 and Figure 2).
Distance Education Variables
Eleven variables were used to identify the features of the distance education experience that may
play a role in student performance. They were duration of the program, frequency of use of
distance learning, instructional role of the program, number of distance learning sessions,
duration of distance learning sessions, pacing of the instruction, role of the instructor, timing of
the interactions, type of interactions, amount of teacher preparation for distance instruction, and
amount of teacher experience in distance instruction. Because of the individualized nature of
distance education, only two of the studies indicated specific numbers and durations of distance
learning sessions, and they were studies of limited partial-course experiences. Half of the studies
did not indicate whether students or instructors set the pace within the distance learning
timeframe, while three of the programs were completely self-paced, and four were designed for
students to set their pace within parameters set by the instructor. In terms of the role of the
instructor in teaching, one program was fully moderated, five were nonmoderated, four used a
combination of moderated and nonmoderated activities, and four did not indicate the instructors’
role. Ten programs used a combination of interactions among students, content, instructors, and
others; one limited interactions to student-content; and three did not specify interaction types. No
studies described the levels of instructor preparation or experience required of or possessed by
the instructors. All levels of each distance education variable had effect sizes not significantly
different from zero.
Instructional and Program Variables
The five variables that indicated the extent to which instructional and program factors played a
role in student outcomes were grade level, school type, content area, the qualifications of the
teacher in the teaching field, and the setting of the students. Twelve of the studies indicated that
the instructors were certified teachers, and the other two studies did not describe the credentials
of the instructors. In five of the programs, students participated from home or a nonschool
location, four programs are designed such that students completed some work from home and
some in a school setting, in three programs, students did their distance learning work while at a
school, and two programs did not specify the setting of the students. All instructional and
program factors had effect sizes that were effectively zero.
Discussion
The literature reviewed in this meta-analysis includes results from 116 comparisons of grades 3–
12 web-based distance education programs with classroom-based teaching, including data for
7561 students. The questions of the effectiveness of distance education for K–12 student
performance, and of the factors influencing its effectiveness were addressed using fixed-effects
effect size estimation. The findings confirm those of other recent meta-analyses of distance
education programs, and provide a needed update to the meta-analysis focused on K–12 students
which was completed in 1998 just as the web-based systems were beginning to be studied in
virtual schooling. The analysis showed that for the factors examined, distance learning did not
Meta-Analysis of Distance Education
Learning Point Associates 20
outperform or underperform classroom instruction. The number of studies was small, and many
studies did not report detailed information, so the results should be viewed as indications of
tendencies rather than prescriptions for practice. What has been learned from these results is that,
based on the best research available on online K–12 distance education programs, such programs
are effective for student learning. Prior to this point, the field has relied on small individual
studies, syntheses that included outdated analog technology, and syntheses that included adult
learners.
Implications for Research and Practice
Distance education as it has been implemented at the K–12 level over the past decade has
improved over time according to several measures: providing access to education and choice in
course offerings to increased numbers of students, offering education to a larger range of grade
levels and ability levels, using more interactive and widely accessible technologies, and leading
students to academic success on a wider range of achievement instruments. The effect of
distance education on learning may be moderated by several factors, existing as it does in a very
complex web of educational, technological, and social dynamics. Factors such as the design of
the distance learning system, the demands of the content, the abilities and disabilities of the
student, and the quality of the teacher are likely to be influential factors, as they are in
conventional educational enterprises. The consistency of the effects shown in the studies
analyzed in this review suggest that as distance education is currently practiced, educators and
other stakeholders can reasonably expect learning in a well-designed distance education
environment to be equivalent to learning in a well-designed classroom environment.
How will K–12 distance education realize greater potential and maximize it effectiveness? How
will designers and managers of K–12 distance education programs make better decisions in order
to design and deliver a more effective program? The answers lie in changes in the ways
policymakers and researchers do their work in this complex context. In order for distance
education to be evaluated, data must be collected and reported in detail. Such data collection
begins with identification of goals. Policymakers and evaluators must enter into a partnership in
which common goals are identified, an evaluation plan is acted on, and detailed reporting
follows. Evaluation must be seen as a tool to support policy setting and decision making (Means
& Haertel, 2004). It is no longer enough to ask whether distance education is effective, we need
to understand why (Sabelli, 2004). We need to know how to make it more effective, what factors
contribute most to effectiveness, and in what contexts the factors operate. Acquiring this
knowledge requires consensus on a definition of effectiveness that goes beyond standardized
tests, and a system for identifying and measuring factors that influence effectiveness. As Means
and Haertel stress, “many studies of the effects of technology-supported innovations are hindered
by a lack of measures of student learning commensurate with the initiative’s goals” (p. 99).
One factor warranting special consideration in assessing the effectiveness of virtual schooling is
teacher quality. In classrooms, teacher effectiveness is a strong determiner of differences in
student learning, far outweighing differences in class size and heterogeneity (Darling-Hammond,
2000). Based on the similarities in student outcomes between distance and classroom learning,
there is every reason to expect that teacher preparation is critical in distance education. However,
there has been very little formal preparation available addressing the unique nature of online
Meta-Analysis of Distance Education
Learning Point Associates 21
instruction and very little time for teachers to develop their expertise as online instructors. As
professional development becomes more common and expertise grows, student success is likely
to grow as well.
As second factor that is growing in importance in K–12 distance education is the emergence of
virtual charter schools. By 2002, there were about 2000 charter schools nationwide, and the No
Child Left Behind Act allows public schools that “chronically fail” to make adequate yearly
progress to be restructured as charter schools (Nelson, Rosenburg, & Van Meter, 2004, p. 1).
According to state department of education websites, there are now almost 100 virtual charter
schools operating. This synthesis includes data from ten virtual charter schools, all of which
performed at levels equivalent to nonvirtual public schools in their states. In contrast, the 20004
report on charter school achievement on the National Assessment of Education Progress (Nelson
et al) provides evidence that charter schools overall are underperforming when compared to
noncharter public schools. Charter school students had significantly lower achievement in grades
4 and 8 math and reading, even when eligibility for free or reduced price lunch and urban
location were factored into the comparison. When minority status was used as a factor, it was
found that black and Hispanic charter school students scored lower in 4th grade math and
reading, but the difference was not significant. The fact that virtual charter school students were
not shown to score lower than nonvirtual school students in this meta-analysis is an indicator of
the success of distance education for K–12 learning.
Teacher quality and classification as a charter school have been recognized as factors that can
influence student learning in classrooms, but little data is available about the influences of these
factors in virtual schooling. Practitioners and policymakers in K–12 distance education are urged
to use data-driven decision making, and to do so they must be informed by experience and data
must be available. In 2004, there have been fewer than ten years of accumulated experience and
too little detailed research published on web-based distance education methods. The lack of
detail in the research to date hinders thorough investigation of the factors influencing practice,
and limits what can be learned for the improvement of practice. A coordinated research and
reporting effort is needed in order to improve the cycle of conducting research on practice and
applying research to improve practice.
Conclusions
Students can experience similar levels of academic success while learning using
telecommunications and learning in classroom settings. While distance learning as it is practiced
in today’s virtual schools uses technology that is less than ten years old and advances rapidly, the
literature has shown that a student’s education online can be as effective as it is in a classroom,
provided that a classroom with the appropriate course is accessible to the student. As the power
of communication technology and educational technology grow, the skill of distance educators
and designers will be challenged to provide experiences that use that power to provide an
experience for students that improves on classroom instruction with its limits of time and place.
Research in K–12 distance education is maturing alongside the technology and those who use it,
but current web-based distance education systems have only been studied for about the last five
years at the K–12 level, a very short time in which to build a body of literature.
Meta-Analysis of Distance Education
Learning Point Associates 22
This meta-analysis represents an investigation into the literature on K–12 web-based distance
education with attention on the factors likely to influence student performance. The result shows
variation in the degree of success students have experienced, and a need for more information if
firm conclusions are to be drawn. Blomeyer (2002) stated the recommendation well: “Support
for additional professionally designed and executed program evaluations and scientific
educational research should be given a high priority in all public and private agencies supporting
effective implementation and use of online learning in K–12 learning communities” (page 10).
The importance of knowledge about effective virtual schooling cannot be overstated, because of
the current boom in the numbers of virtual schools and students, and because of the essential role
virtual schools can play in school reform movements and workforce development efforts. As of
spring, 2004, there were roughly 2,400 publicly-funded cyber-based charter schools and state
and district virtual schools in 37 U.S. states, with an estimated 40,000 to 50,000 students
participating in online courses, according to Susan Patrick, Director of the U.S. Department of
Education's Office of Educational Technology (Fording, 2004). With recent and continued
growth in virtual schools, virtual school leaders and policy makers will need a strong research
foundation on which to base decisions.
Several groups in the U.S. have identified school reform, particularly high school reform, as
priorities in coming years. The U.S. Department of Education has identified high school reform
models that support student achievement, and has recognized small school size, scheduling
choice, charter schools, career academies, early college initiatives, and student engagement as
research-based models that contribute to improved student achievement (U.S. Department of
Education, 2004). The National Governor’s Association has formed a task force to study
redesigning high schools in order to make them “more rigorous and relevant to the lives of
America’s youth” (National Governor’s Association, 2004). The task force initiative responds to
employers’ needs for more skilled and better educated workers by suggesting that reforms
include choices in high school programs and opportunities to earn college credit or professional
credentials. The National Association of Secondary School Principals in 2004 published
Breaking Ranks II, which calls for reforming high schools to become more rigorous and
personalized (National Association of Secondary School Principals, 2004), and the National
High School Alliance has developed the Catalog of Research on Secondary School Reform. The
catalog compiles studies of effective school reform programs, including those based on early
college, smaller schools, student interests and learning styles, at-risk student needs, talent
development, and career academies (National High School Alliance, 2004). Each of the reform
models described and recommended by these groups is an example of a strength that has been
shown by virtual schools. By offering scheduling flexibility, personalization, freedom from a
large physical school, engaging tools of distance learning, opportunities to accelerate learning,
and access to rigorous academic programs, virtual schools are not just important examples of
school reform models, but virtual schools may represent the best hope for bringing high school
reform quickly to large numbers of students.
Another strength of virtual schools is their unique capability for immersing students in
information and communication technologies (ICT). An international effort is underway to
improve ICT literacy as a “contribution to the development of human capital” (Educational
Testing Service, 2001, p. iii). An international panel convened by the Educational Testing
Meta-Analysis of Distance Education
Learning Point Associates 23
Service determined that ICT skill is needed by citizens to function in the current technological
climate, and that ICT skills are needed to help people worldwide meet fundamental needs,
making ICT literacy a global objective. The development of ICT literacy begins with access to
technology, and many publicly-funded virtual schools have found ways to bridge the access
divide by providing computers to students. Virtual school students must develop ICT skills to be
successful in online learning, and they may become the sought-after employees of the near
future. Because of the global need for ICT skills and their role in virtual schools, demand could
rise for data on effective virtual schools as more are developed worldwide.
The Need for Prospective Study in Virtual Schooling
An important step toward improving the state on virtual schools research was taken in 2004
when the U.S. Department of Education hosted an E-learning Summit to explore the status of K–
12 e-learning in the U.S. The DOE Office of Educational Technology is showing leadership by
identifying e-learning as a priority in the new National Educational Technology Plan.
Technology, including e-learning, is seen as a force that can transform education because of the
power of e-learning to individualize, personalize and differentiate instruction. Plans for the
Federal role in e-learning leadership will include development of an e-learning clearinghouse
listing programs for students, a process for addressing quality and accreditation issues, and
support for developing online content. Such initiatives begin to bring knowledge and expertise to
more stakeholders, assist policymakers and practitioners in accessing information, and serve as a
focal point for guiding future work that will improve outcomes across the spectrum.
As a relatively recent innovation in the sometimes slow-moving world of education, distance
education has been shown over decades with every variety of technology to work effectively
although it works in very different ways than classroom instruction does, it meets different
needs, and serves different audiences, having had far less time in which to mature, as evidenced
by the studies included in this meta-analysis. The literature contains reports on distance
education programs in which student outcomes exceed those in conventional classrooms (see
citations in “Distance Education in the K–12 Context” section), but in order to make use of such
data in syntheses such as this one, complete data need to be reported.
Recommendations for K–12 Online Learning Policy and Practice
Policy-makers and practitioners should continue to move forward in developing and
implementing K–12 distance education programs when those programs meet identified needs
and when they are designed and managed as carefully as traditional education programs. The “no
significant difference” result reported here and elsewhere lends confidence to distance educators
that their ongoing efforts are likely to be as effective as classroom-based education. This
synthesis, considered together with current policy and recent research findings, demonstrates that
students of many types and ages can learn in many content areas using the flexibility and choices
afforded by distance education. In their recent article, New Millenium Research for Educational
Technology: A Call for a National Research Agenda, Roblyer and Knezek (2003) recommended
a focus and priorities for a future technology research agenda. The focus, they stated, should be
providing a rationale for technology use. The priority is to explain why students and educators
should use technology.
Meta-Analysis of Distance Education
Learning Point Associates 24
Optimally, the research on K–12 distance education would recommend specific practices that
would lead to results that exceed those in conventional education settings. The barriers that
prevent such recommendations include:
 a limit on the educational expertise focused on distance education as an area of study. A
small subset of educational researchers have elected to focus on virtual schooling, either
as doctoral candidates, faculty, program directors or independent evaluators.
 a rather short-sighted view of the purposes of distance education, a lack of consensus
about the goals of distance education, and an accompanying lack of evaluation directed at
assessing progress toward those goals. Distance education has been seen primarily as a
substitute for classroom instruction, rather than a potentially more effective way of
learning. Until the goal is established of reaching a higher potential, research will
continue to determine whether distance education is as effective as classroom instruction,
rather than looking for ways that distance education can excel.
 a failure to take into account the complexity of systems in which distance education
operates. Complexity is difficult to quantify, but virtual schooling evaluation and
research can begin to track a greater range of influences, leading to a more thorough
understanding of its effects.
 a paucity of research and reporting that includes details sufficient for quantitative
synthesis. Most reports on virtual schooling released in the past omit sample sizes, mean
scores, standard deviations, and other details needed for big-picture synthesis.
For distance education to add a prospective agenda to the archive of valuable retrospective study
that currently guides the field, five major action recommendation must be addressed by online
learning practitioners, online learning district-level leadership, and Federal and State educational
policy makers:
1. First, the broader educational community needs to become better informed about K–12
online learning and distance education, to foster better communication among the widest
range of experts and practitioners who have the potential to contribute to advances in the
field.
This crucial informational campaign requires professionals working in distance education
in any capacity to network by participating in conferences, publishing articles and papers,
and contributing to discussions locally and globally where people who are not involved in
distance education can learn.
2. Second, the community of distance education policy makers, researchers, and
practitioners should develop and articulate a long-range view of the intended and
expected benefits of distance education and become advocates for suitably long-term
studies of its effects.
The list of potential benefits should be broad, and should be a close match to the benefits
or “effects” anticipated for any educational experience. Curriculum content should
include a liberal education in which knowledge, skills, and dispositions are developed
that successful students need in order to enjoy a full life in a democracy. But effects and
Meta-Analysis of Distance Education
Learning Point Associates 25
benefits should also include academic literacies, technology skills, and academic
standards.
This list of crucial, performance-based knowledge, skills, and dispositions must serve as
a guide in the stages of design, implementation, and evaluation of programs. Consensus is
needed on the goals of distance education, and plans should follow to evaluate progress
toward those goals. Distance education program directors should see researchers as
partners in informing practice.
3. Third, because education occurs in a dynamic context, and the rapid change in the
technology used in distance education adds to the complexity, evaluation of distance
education programs needs to account for more of this complexity than has so far been the
practice.
A common “codebook” or heuristic descriptive system should be created and refined to
ensure that outcomes from distance and online learning programs can be accurately
compared to other online and distance programs and to face-to-face instruction. A
descriptive system supporting comparative analysis of all varieties of traditional and
online and distance learning delivery systems will dramatically increase both the
generalizability of results and the synthesizability of research findings available to inform
development, implementation and institutionalization of online and distance learning
programs.
4. Finally, standards are needed for reporting the academic and programmatic outcomes of
distance education programs. Many K–12 distance education program directors collect
admirable amounts of data, and conduct in-house analyses, but until there are standards
set to guide the reporting of data, educational research will remain limited to examining
results from only a small, enlightened subset of these programs.
5. The actions recommended require coordination and leadership. Leadership should begin
at the national level and include professional organizations like the North American
Council on Online Learning and International Society for Technology in Education. The
United States Department of Education and the leading professional organizations and
groups should assume a leadership role organizing a national distance learning and online
learning community of practice to work toward enacting these essential action
recommendations.
Distance educators belong to a wide variety of overlapping professional groups and
associations that have the potential to contribute to a powerful and effective coalition.
The larger coalition needed to weld a broader professional consensus should serve as a
central clearinghouse for information about K–12 online and distance education, a
matchmaking service for programs and evaluators, and as an organizational focus for
organizing national efforts to support online and distance learning policy, program
development, and professional development.
Meta-Analysis of Distance Education
Learning Point Associates 26
Learning, progress, and data-driven decisions require the availability of relevant data. The K–12
distance education and online learning communities certainly have the infrastructure for sharing
that information. What is needed now is an adequate and uniform system for describing
academic and programmatic outcomes within and across a variety of programs and instructional
delivery systems, and uniform metrics and standards that can support comparisons within and
across the various delivery systems and instructional modalities.
With ubiquitous availability of good information on the performance of all K–12 educational
programs and instructional systems, parents and practitioners, policymakers and national
political leadership will be able to make the very best informed decisions about how to best
educate and equip all our children for life and success during the ensuing twenty-first century.
Meta-Analysis of Distance Education
Learning Point Associates 27
References
References marked with an asterisk indicate studies used in the meta-analysis.
Allen, M., Bourhis, J., Burrell, N., & Mabry, E. (2002). Comparing student satisfaction with
distance education to traditional classrooms in higher education: A meta-analysis. The
American Journal of Distance Education, 16, 83–97.
*Alaska Department of Education and Early Development. (2003). Spring 2003 benchmarks,
Delta/Greely school district - Delta Charter Cyber School. Retrieved July 19, 2004, from
http://www.eed.state.ak.us/tls/assessment/AsmtVer/SchoolAsmtVerSuptSearch.cfm
Arkansas Virtual School. (2003). 2002–2003 testing overview. Dardanelle, AR: author.
Bangert-Drowns, R., Hurley, M., & Wilkinson, B. (2004). The effects of school-based writingto-
learn interventions on academic achievement: A meta-analysis. Review of Educational
Research, 74 (1), 29–58.
Barker, K., & Wendel, T. (2001). E-learning: studying Canada's virtual secondary schools.
Kelowna, BC: Society for the Advancement of Excellence in Education. Retrieved April
11, 2004, from http://www.excellenceineducation.ca/pdfs/006.pdf
Bernard, R. M., Lou, Y., Abrami, P.C., Wozney, L., Borokhovski, E., Wallet, P.A., Wade, A., &
Fiset, M. (2003, April). How does distance education compare to classroom instruction? A
meta-analysis of the empirical literature. Symposium conducted at the annual meeting of
the American Educational Research Association, Chicago, IL.
Bianchi, W. (2002). The Wisconsin school of the air: Success story with implications.
Educational Technology & Society 5 (1).
Blomeyer, R. (2002). Virtual schools and e-learning in K–12 environments: Emerging policy and
practice. NCREL Policy Issues, 11. Naperville, IL: North Central Regional Educational
Laboratory. Retrieved May 2, 2004, from
http://www.ncrel.org/policy/pubs/pdfs/pivol11.pdf
Bolton, J. (2002). Web-based distance education: Pedagogy, epistemology, and instructional
design. University of Saskatchewan. Retrieved April 21, 2004, from
http://www.usask.ca/education/coursework/802papers/boulton/boulton.pdf
Bond, A. (2002). Learning music online: An accessible program for isolated students.
Kensington Park, SA: Australian National Training Authority. Retrieved April 12, 2004,
from http://www.ncver.edu.au/research/proj/nr1013.pdf
Bransford, J., Brown, A., & Cocking, R. (Eds.). (1999). How people learn. Washington, DC:
National Academies Press.
Meta-Analysis of Distance Education
Learning Point Associates 28
Brooks, J. & Brooks, M. (1993). The case for constructivist classrooms. Alexandria, VA:
Association for Supervision and Curriculum Development.
Bruckman, A. (1998). Computer supported cooperative work. The Journal of Collaborative
Computing, 7, 47–86.
Calderoni, J. (1998). Telesecundaria: Using TV to bring education to rural Mexico. Education
and Technology Technical Notes Series: World Bank Human Development Network.
Retrieved April 8, 2004, from
http://wbln0018.worldbank.org/HDNet/HDdocs.nsf/C11FBFF6C1B77F9985256686006D
C949/1635F1703FE053B385256754006D8C3F/$FILE/telesecundaria.pdf
Cavanaugh, C. S. (2001). The effectiveness of interactive distance education technologies in K–
12 learning: A meta-analysis. International Journal of Educational Telecommunications,
7(1), 73–88.
* Colorado Department of Education. (2003a). Branson alternative school, grades 1–6 school
accountability report. Denver, CO: Author. Retrieved July 19, 2004, from
http://reportcard.cde.state.co.us/reportcard/pdf/2003_1750_0948_E.pdf
* Colorado Department of Education. (2003b). Branson alternative school, grades 7–8 school
accountability report. Denver, CO: Author. Retrieved July 19, 2004, from
http://reportcard.cde.state.co.us/reportcard/pdf/2003_1750_0948_M.pdf
* Colorado Department of Education. (2003c). Connections academy, grades 1–6 school
accountability report. Denver, CO: Author. Retrieved July 19, 2004, from
http://reportcard.cde.state.co.us/reportcard/pdf/2003_0880_1887_E.pdf
* Colorado Department of Education. (2003c). Connections academy, grades 7–8 school
accountability report. Denver, CO: Author. Retrieved July 19, 2004, from
http://reportcard.cde.state.co.us/reportcard/pdf/2003_0880_1887_M.pdf
Conzemius, A; & Sandrock, P. (2003). Developing world language programs in elementary
grades. Madison, WI: Wisconsin State Department of Public Instruction. (ERIC Document
Reproduction Service No. ED 480156)
Darling-Hammond, L. (2000). Teacher quality and student achievement: a review of state policy
evidence. Education Policy Analysis Archives, (8)1.
Dees, S. (1994). An investigation of distance education versus traditional course delivery using
comparisons of student achievement scores in advanced placement chemistry and
perceptions of teachers and students about their delivery system (satellite course).
Unpublished dissertation, Northern Illinois University, DeKalb.
Meta-Analysis of Distance Education
Learning Point Associates 29
Duffy, T., & Cunningham, D. (1996). Constructivism: Implications for the design and delivery
of instruction. In Jonassen, D. (Ed.), Handbook of research on educational
telecommunications and technology. New York: Macmillan Library Reference.
Duffy, T., & Jonassen, D. (Eds.). (1992). Constructivism and the technology of instruction: A
conversation. Hillsdale, NJ: Lawrence Erlbaum Associates, Publisher.
Educational Testing Service. (2001). Digital transformation: A framework for ICT literacy.
Retrieved October 9, 2004, from http://www.ets.org/research/ictliteracy/ictreport.pdf
Falck, A-K; Husu, J; Kronlund, T., Kynaslahti, H., Salminen, J., & Salonen, M. (1997). Testing
virtual classroom in the school context. Distance Education. 18, (2), 213.
Fording, L. (2004). Education, 21st century-style. Newsweek. Retrieved September 9, 2004, from
http://www.msnbc.msn.com/id/4633126/
Gershaw, D. (1989). Line on life: locus of control. Retrieved May 29, 2004, from
http://www3.azwestern.edu/psy/dgershaw/lol/ControlLocus.html
* Goc Karp, G., & Woods, M. (2003). Wellness NutriFit online learning in physical education
for high school students. Journal of Interactive Online Learning, 2 (2). Retrieved April 9,
2004, from http://www.ncolr.org/jiol/archives/2003/fall/03/index.html
Hedges, L., Shymansky, J., & Woodworth, G. (1989). Modern methods of meta-analysis.
Washington, DC: National Science Teachers Association.
Hinnant, E. (1994). Distance learning using digital fiber optics: a study of student achievement
and student perception of delivery system quality. Unpublished dissertation, Mississippi
State University, Starkeville.
* Indiana Department of Education. (2004). School snapshot: Irvington community school.
Indianapolis, IN: Author. Retrieved July 20, 2004, from
http://mustang.doe.state.in.us/SEARCH/snapshot.cfm?schl=1537
Jordan, A. (2002). An investigation into the effects of online teaching and learning on
achievement outcomes at the secondary level. Unpublished dissertation, Fayetteville State
University, Fayetteville, NC.
Kearsley, G. (2000). Online education. Belmont, CA: Wadsworth.
Keegan, D. (1996). Foundations of distance education. London: Routledge.
Kozma, R., Zucker, A., Espinoza, C., McGhee, R., Yarnall, L., Zalles, D., & Lewis, A. (2000).
The online course experience: evaluation of the virtual high school’s third year of
implementation, 1999-2000. Arlington, VA: SRI. Retrieved December 8, 2003, from
http://www.govhs.org/Images/SRIEvals/$file/SRIAnnualReport2000.pdf
Meta-Analysis of Distance Education
Learning Point Associates 30
Kromrey, J., & Hogarty, K. (2002). Estimates of variance components in random effects metaanalysis:
sensitivity to violations of normality and variance homogeneity. American
Statistical Association Joint Statistical Meeting, 1963–1968.
Lipsey, B., & Wilson, D. (2001). Practical meta-analysis. Thousand Oaks, CA: SAGE
Publications, Inc.
Machtmes, K., & Asher, J. W. (2000). A meta-analysis of the effectiveness of telecourses in
distance education. The American Journal of Distance Education, 14 (1), 27–46.
McGreal, R. (1994). Comparison of the attitudes of learners taking audiographic
teleconferencing courses in secondary schools in northern Ontario. Interpersonal
Computing and Technology, 2 (4), 11–23. Retrieved April 12, 2004, from
http://www.helsinki.fi/science/optek/1994/n4/mcgreal.txt
Means, B., & Haertel, G. (2004). A blueprint for a national research agenda to evaluate
educational technology. In Means, B. & Haertel, G. (Eds.). Using Technology Evaluation
to Enhance Student Learning. New York: Teachers College Press.
Mills, S. (2002). School isn’t a place anymore: an evaluation of virtual Greenbush online
courses for high school students. University of Kansas: Schiefelbusch Institute for Life
Span Studies. Retrieved May 2, 2004, from
http://media.lsi.ku.edu/research/vgeval/vgevaluationreport.pdf
* Minnesota Department of Education. (2003). 2003 report card: cyber village academy.
Minneapolis, MN: Author. Retrieved July 21 2004, from
http://education.state.mn.us/ReportCard/2003/RCF402507010.pdf
* Mock, R. (2000). Comparison of online coursework to traditional instruction. Unpublished
thesis, Michigan State University, East Lansing. Retrieved April 29, 2004, from
http://hobbes.lite.msu.edu/~robmock/masters/abstract.htm
National Association of Secondary School Principals. (2004). Breaking ranks II: Strategies for
leading high school reform. Retrieved October 9, 2004, from
http://www.nassp.org/breakingranks/breakingransk2.cfm
National Association of State Boards of Education. (2001). Any time, any place, any path, any
pace: Taking the lead on e-learning policy. Alexandria, VA: author.
National Governor’s Association. (2004). Governors leading high school reform efforts.
Retrieved October 9, 2004, from
http://www.nga.org/nga/legislativeUpdate/1,1169,C_ISSUE_BRIEF^D_7342,00.html
National High School Alliance. (2004). Catalog of research on secondary school reform.
Retrieved October 9, 2004, from http://www.hsalliance.org/research/index.asp
Meta-Analysis of Distance Education
Learning Point Associates 31
Nelson, F., Rosenburg, B. & Van Meter, N. (2004). Charter school achievement on the 2003
national assessment of educational progress. Washington, DC: American Federation of
Teachers. Retrieved September 9, 2004, from
http://www.aft.org/pubs-reports/downloads/teachers/NAEPCharterSchoolReport.pdf
Roblyer, M. & Knezek, G. (2003). New millennium research for educational technology: A call
for a national research agenda. Journal of Research on Technology in Education, 36 (1),
60–71.
Rotter, J. B. (1989). Internal versus external control of reinforcement: A case history of a
variable. American Psychologist, 45, 489–493.
Ryan, W. (1996). The effectiveness of traditional vs. audiographics delivery in senior high
advanced mathematics course. Journal of Distance Education/Revue de l'enseignement à
distance (11)2. Retrieved April 13, 2004, from http://www.cade-aced.ca/en_pub.php
Sabelli, N. (2004). Policy, planning, and the evaluation of learning technology. In Means, B. &
Haertel, G. (Eds.). Using Technology Evaluation to Enhance Student Learning. New York:
Teachers College Press.
* Schollie, B. (2001). Student achievement and performance levels in online education research
study. Edmonton, Alberta: Alberta Online Consortium. Retrieved April 25, 2004, from
http://www.albertaonline.ab.ca/pdfs/AOCresearch_full_report.pdf
Shachar, M. & Neumann, Y. (2003). Differences between traditional and distance education
academic performances: a meta-analytic approach. International Review of Research in
Open and Distance Learning. October.
Slavin, R. (2003). Educational psychology theory and practice, 7th Ed. Boston, MA: Pearson
Education, Inc.
Southern Regional Education Board. (2003). Essential principles of high-quality online teaching:
guidelines for evaluating K–12 online teachers. Atlanta, GA: author. Retrieved May 2,
2004, from http://www.sreb.org/programs/EdTech/pubs/PDF/Essential_Principles.pdf
* Stevens, K. (1999). Two Canadian approaches to teaching biology, chemistry, mathematics
and physics to senior high school students in virtual classes. Australasian Science
Education Research Association. Retrieved April 9, 2004, from
http://www.tellearn.mun.ca/pubs/virtual.html
* Texas Education Agency. (2003). 2002–2003 campus performance: Southwest Virtual
Preparatory School. Austin, TX: Author. Retrieved July 22, 2004, from
http://www.tea.state.tx.us/perfreport/aeis/2003/campus.srch.html
Meta-Analysis of Distance Education
Learning Point Associates 32
Ungerleider, C.S., & Burns, T.C. (2003). A systematic review of the effectiveness and efficiency
of networked ICT in education: A state of the field report to the Council of Ministers of
Education, Canada and Industry Canada. Unpublished report.
U.S. Department of Education. (2004). High school reform models that support student
achievement. Retrieved October 9, 2004, from
http://www.ed.gov/about/offices/list/ovae/pi/hs/reform.html
* Washington Office of the Superintendent of Public Instruction. (2003). Internet academy 2002–
2003 WASL results. Olympia, WA: Author. Retrieved July 22, 2004, from
http://reportcard.ospi.k12.wa.us/Reports/summary.aspx?schoolId=1164&reportLevel=School
Wedemeyer, C.A. (1981). Learning at the back door: Reflections on non traditional learning in
the lifespan. Madison: University of Wisconsin Press.
* Wisconsin Department of Public Instruction. (2003). Successful school guide: Wisconsin
Virtual Academy. Madison, WI: Author. Retrieved July 22, 2004, from
http://data.dpi.state.wi.us/data/questions.asp?fullkey=011945040100&DN=Northern+Ozau
kee&SN=Wisconsin+Virtual+Academy&TYPECODE=6&CTY=45&ORGLEVEL=SC
Yasin, K. & Luberisse, Y. (1997). Meeting the needs of a new democracy: Multichannel learning
and interactive radio instruction in Haiti: a case study. Washington, DC: USAID.
Retrieved April 29, 2004, from http://ies.edc.org/pubs/book11.htm
Meta-Analysis of Distance Education
Learning Point Associates 33
Appendix
Coded Variables and Study Features in the Codebook
A. Identification of studies
1. Study number (“study”).
2. Finding/hypothesis number (“finding”).
3. Author name (“author”). Last name of first author.
4. Year of publication (“year”).
5. Number of findings/hypotheses within study (“number”).
6. Country (“country”).
Unspecified=0,
USA=1,
Canada=2,
Mexico/Central America/South America=3,
Europe=4,
Asia=5,
Africa=6,
Australia/Pacific=7,
Multinational=8,
Other=9.
7. Grade level of students (“grade”).
Unspecified=00,
grades 1–12 use 01 to 12,
Mixed primary (K–2) =13,
Mixed intermediate (3–5) =14,
Mixed middle (6–8) =15,
Mixed high (9–12) =16,
K–12=17,
other=18.
8. School type (“school”).
Unspecified=0,
Public district sponsored=1,
Public state sponsored=2,
Private=3,
Other=4,
Charter=5,
Combination=6.
Meta-Analysis of Distance Education
Learning Point Associates 34
9. Content area (“content”).
Unspecified=0,
Reading/language arts=1,
Mathematics=2,
Social studies=3,
Science=4,
Computers/technology=5,
Foreign language=6,
Arts=7,
Physical education=8,
Other=9,
Writing=10.
10. Type of publication (“publication”).
Published journal article=1,
Journal article in press=2,
Book chapter=3,
Report=4,
Dissertation=5,
Conference paper=6.
B. Distance learning features
1. Duration of distance learning experience (“duration”).
Less than one semester=1,
One semester=2,
More than one semester=3.
2. Frequency of distance learning experience (“frequency”).
Unspecified=0,
From 5 to 7 days per week=1,
From 1 to 4 days per week=2,
From 1 to 3 days per month=3,
Less than monthly=4.
3. Instructional role of distance learning (“role”).
Unspecified=0,
Full-time educational program=1,
Courses to supplement an educational program or partial educational program=2,
Supplement to a specific course=3.
4. Number of distance learning sessions (“dlnumber”).
Unspecified=0,
List number of sessions.
Meta-Analysis of Distance Education
Learning Point Associates 35
5. Duration of distance learning sessions (“dlduration”).
Unspecified=0,
List average minutes per session.
6. Pacing of distance learning instruction (“pacing”).
Unspecified=0,
Completely self-paced=1,
Student sets pace within instructor-determined parameters=2,
Pacing completely specified by program or instructor=3.
7. Instructor role (“instructrole”).
Unspecified=0,
Fully moderated=1,
Nonmoderated=2,
Combination=3,
Other=4.
8. Timing of interactions (“timing”).
Unspecified=0,
Synchronous=1,
Asynchronous=2,
Combination=3,
Other =4.
8. Type of interactions (“interaction”).
Unspecified=0,
Student—content=1,
Student—instructor=2,
Student—student=3,
Student—others=4,
Combination=5,
Other=6.
C. Instructor/program features
1. Amount of teacher preparation in distance learning (“instructprep”).
Unspecified=0,
List hours of preparation.
2. Amount of teacher experience in distance learning (“instructexp”).
Unspecified=0,
List years of experience.
Meta-Analysis of Distance Education
Learning Point Associates 36
3. Qualifications of teacher in the teaching field (“instructqual”).
Unspecified=0,
Certified in content area=1,
Certified but teaching out of field=2,
Alternative or provisional certification=3,
Uncertified=4,
Other=5.
4. Setting of students during distance learning (“setting”).
Unspecified=0,
Home=1,
School=2,
Other=3,
Combination=4.
D. Study quality features
1. Student sample size (“sample”). Actual sample size.
2. Measure of academic outcome (“achmeasure”).
Standardized test=1,
Researcher-made test=2,
Teacher-made test=3,
Other=4.
3. Testing sequence (“testseq”).
Unspecified=0,
Pre-post=1,
Post only=2,
Other=3.
4. Pretest equivalency (“preequiv”). Have the initial differences between groups been
accounted for?
Unspecified=0,
Statistical control (ANCOVA, regression)=1,
Random assignment=2,
Statistical control and random assignment=3,
Gain scores=4,
Other=5.
5. Reported reliability of measures (“reliability”).
Unspecified=00,
Actual reliability statistic.
6. Effect size coefficient (“effsize”).
Actual coefficient.
Meta-Analysis of Distance Education
Learning Point Associates 37
7. Statistics used in determining effect size. (“esstats”).
Means=1,
t-value=2,
F-value=3,
Chi-square=4,
Other=5.
8. Weight (“weight”).
One divided by the actual number of findings/hypotheses in the study.
E. Sources of Invalidity
1. Type of Design (“design”).
Quasi-experimental/nonrandomized one group pretest-posttest=1,
Nonrandomized static-group comparison=2,
Nonrandomized pre-post control group=3,
Time series=4,
Randomized posttest-only control group=5,
Randomized pre-post control group=6,
Other=7.
2. History (“history”). Control for specific events occurring between the first and second
measurement in addition to the experimental variable.
Adequately controlled by design=1,
Definite weakness of design=2,
Possible source of concern=3,
Not a relevant factor=4.
3. Maturation (“maturation”). Control for processes within the participants operating as a
function of the passage of time.
Are there processes within participants operating as a function of the passage of time,
such as growing older or more tired, that might account for changes in the dependent
measure?
Adequately controlled by design=1,
Definite weakness of design=2,
Possible source of concern=3,
Not a relevant factor=4.
4. Testing (“testing”). Control for the effect of taking a test upon the scores of a second
testing.
Adequately controlled by design=1,
Definite weakness of design=2,
Possible source of concern=3,
Not a relevant factor=4.
Meta-Analysis of Distance Education
Learning Point Associates 38
5. Instrumentation (“instrument”). Control for changes in calibration or observers' scores
that produce changes in the obtained measurement.
Adequately controlled by design=1,
Definite weakness of design=2,
Possible source of concern=3,
Not a relevant factor=4.
6. Statistical Regression (“regression”). Control for group selection based on their extreme
scores.
Adequately controlled by design=1,
Definite weakness of design=2,
Possible source of concern=3,
Not a relevant factor=4.
7. Selection Bias (“selection”). Control for biases resulting in the differential selection of
comparison groups.
Adequately controlled by design=1,
Definite weakness of design=2,
Possible source of concern=3,
Not a relevant factor=4.
8. Mortality (“mortality”). Control for differential loss of participants from the
experimental and control groups.
Adequately controlled by design=1,
Definite weakness of design=2,
Possible source of concern=3,
Not a relevant factor=4.
9. Selection-Maturation Interaction (“selectmatur”). Control for interaction between
extraneous factors such as history, maturation, or testing and the specific selection
differences that distinguish the experimental and control groups.
Adequately controlled by design=1,
Definite weakness of design=2,
Possible source of concern=3,
Not a relevant factor=4.
10. Reactive or Interaction Effect of Testing (“testeff”). Control for the influence of pretesting
on the participants' responsiveness to the experimental variable, making the
results for a pre-tested population unrepresentative of the effects of the experimental
variable for the unpre-tested universe from which the participants were selected.
Adequately controlled by design=1,
Definite weakness of design=2,
Possible source of concern=3,
Not a relevant factor=4.
Meta-Analysis of Distance Education
Learning Point Associates 39