VOL. 17, No. 1, 63-74
The Journal of Distance Education (JDE) (1986-2000) was subjected to a content analysis (235 items) that focused on item type, topic, research method, and biographical information about first authors. A coding scheme was created for these categories. The distribution was balanced between male and female authors, national and international viewpoints, and a broad range of topics. In the subset of items coded as empirical studies, the highest frequencies of (a) item topic were foundations of distance education and technology/medium; (b) language of presentation: English; (c) institutional affiliation of author: higher education; (d) type of data collected and analyzed: qualitative.
La Révue de l'éducation à distance (RED) (1986-2000) a été assujetti à une analyse de contenu (235 items) qui s’est concentrée sur le type de l’item, le sujet, la méthode de recherche et l’information bibliographique des premiers auteurs. Un système de codification a été créé pour ces catégories. La répartition a été équilibrée entre les auteurs hommes et femmes, les points de vue nationaux et internationaux et une vaste étendue de sujets. Dans le sous-ensemble des items codifiés comme étant des études empiriques, la plus haute fréquence était pour (a) sujet de l’item : les bases de l’éducation à distance et technologie/média; (b) langue de présentation : anglais; (c)affiliation institutionnelle de l’auteur : enseignement supérieur; (d) type de données recueillies et analysées : qualitatif.
In 2000 the following bit of hyperbole found its way into Rourke’s master’s thesis: “The University of Alberta alone has over 14 000 registrants in WebCT courses.” At the time the claim seemed fantastic and served as a useful eyecatcher. This year 100,000 are registered (Smith, 2001). As our Vice-President (Academic) exclaimed, “The growth has been absolutely phenomenal” (p. 3). The phenomenal growth of online education has sparked a great deal of interest in topics such as teaching at a distance, learning autonomously, and using technology to deliver content and support interaction. Although the medium may be novel, the discussion is not entirely original. Since 1986 the Journal of Distance Education (JDE) has addressed many of these issues in a scholarly and practical manner. The current popularity of the JDE’s subject matter, its substantial history, and its presence during a volatile period in distance education warrants a systematic examination of its content.
The Canadian Association of Distance Education (CADE) began publishing the JDE in 1986 to “promote and encourage scholarly work of an empirical and theoretical nature that relates to distance education in Canada and throughout the world” (CADE, 2001). Since then 23 issues of the JDE have been published, with over 200 items presented in 15 volumes. Currently the journal is published twice a year and has an annual subscription rate of approximately 800 (M. Haughey, personal communication, 2001). The journal is distributed as a benefit of membership in the Canadian Association of Distance Education (see www.cadeaced. ca) and the bulk of the subscriptions come from these members. In addition, all the journal’s back issues are available online on the CADE Web site.
Although the journal’s mandate has an academic tenor, university faculty represent only one segment of subscribers. Sturrock (1990) characterized the audience as mostly “busy practitioners in the field, preoccupied with enrolments, budgets, space problems, staffing, budgets and so on” (p. 1). In addition to serving these groups, Layton (1992) indicated that the editors attempt to “offer a balance among a number of concerns including: official language and gender representation, institutional level, the authors’ numerous interests and areas of expertise, and national and international contexts” (p. 1). Thus scholarly discussion of distance education finds a unique home in the JDE, which is diverse in content, authorship, and audience.
Occasionally, periodicals such as the JDE are the object of content analyses (Dick & Dick, 1989; Klein, 1997; Koble & Bunker, 1997; Scriven, 1991; Suen & Stevens, 1993). In other cases, a field is reviewed and numerous relevant sources including journals and dissertation abstracts are consulted (Berge & Mrozowski, 2001). The intention is not so much to scrutinize a particular journal, but to examine a field based on evidence found in its leading publications. These analyses can be sorted into two types based on their purpose. The purpose of evaluative analyses is to critique a specific element of the field. Reeves (1995), for example, offered a critique of research methodology in the field of instructional technology by analyzing five years of research studies appearing in the Educational Technology Research and Development Journal and the Journal of Computer Based Instruction. Suen and Stevens (1993) conducted a similar critique of research methods in the field of distance education by sampling items from the American Journal of Distance Education (AJDE).
Evaluative analyses are less common than descriptive analyses. The purpose of the latter is to generate descriptive information on various qualities of a journal’s content. These analyses have several approaches including describing a journal’s content, comparing it with other journals, identifying trends in the field, and proposing future directions (for the field or the journal). Koble and Bunker (1997) collected information on the authors, audience, and items published in the AJDE from 1987-1995. This provided the foundation for a discussion of trends in research and practice in the field of distance education. When the amalgamation of the Journal of Instructional Development with the Educational Communications and Technology journal was proposed, Dick and Dick (1989) compared the content of both journals to determine the appropriateness of the merger. On taking over as the editor of the “development” section of ETR&D, Klein (1997) conducted a content analysis of his section of the journal and compared his results with those of a survey of readers’ wishes. The comparison was used to guide the editor in soliciting and selecting items for publication in the journal. Foley, Keener, and Branch (1994) used the content analysis technique to assess the contribution of women in an educational technology journal, and Hawkridge (1999) offered a nostalgic look back at the British Journal of Educational Technology (BJET) from its inception in 1970. Scriven (1991) reviewed items from the first 10 years of the Australian journal Distance Education. He identified item contributions of authors by country, classified items into categories, and attempted to identify trends.
The JDE has also been the subject of a content analysis. During its third year of operation, Sturrock and Howard (1989) surveyed the abstracts of 33 manuscripts submitted to the JDE to determine the priorities and concerns of distance educators. Ultimately, the purpose of these reviews has been to make an observation or statement about the field. The particular journal that is sampled is regarded simply as a reflection of what practioners or researchers in that field are doing.
The current descriptive review of the JDE was undertaken to achieve four objectives. First, we sought to identify several characteristics of the authors, the items, and the research methods employed in the studies. Second, we wished to compare this information with the aims and purposes of the JDE, and third, with other analyses of similar publications. Finally, we wished to identify trends in distance education. This information will be of interest to many groups in distance education; however, we wished to provide this information primarily for the benefit of researchers who should be aware of the state of the distance education literature, its gaps, and areas of saturation.
Content Analysis
Content analysis is a research technique that has quantitative and qualitative characteristics. In the quantitative tradition, it is defined as “the objective, systematic, and quantitative description of the manifest content of communication” (Berelson, 1952, p. 18). Many of the activities of the content analyst fit neatly into this positivistic perspective. Other activities, however, are better described in qualitative nomenclature. Frick (1998) describes content analysis as the process of analyzing material through the use of emergent categories, culminating ultimately in data reduction. This study contained elements of both approaches.
The principal apparatus for a content analysis is the coding scheme, which is a system of categories and subcategories into which segments of content are coded. Three objects were selected for coding in this analysis: the items, their authors, and the research processes used in the items that were coded as empirical research items. Several categories and codes were defined for each object (see Tables 1 and 2). A comprehensive review was undertaken that included all items published in the JDE from 1986 to 2001. All content including peer-reviewed items, book reviews, dialogues, editorials, and letters to the editor were included in the initial analysis.
Item Categories
The JDE invites three broad categories of submissions: (a) scholarly items and research papers; (b) for-your-information reports; and (c) dialogue pieces (JDE, 2001).
This typology is useful for orienting potential contributors; however, it is less useful for the content analyst. Therefore, several other sources were consulted in an attempt to establish categories and descriptions for item type, including the Publication Manual of the American Psychological Association (4th ed., APA, 1994), the Educational Resource Information Centre’s (ERIC) typology of publication types (ERIC, 2001), and the Canadian Business and Current Affairs (CBCA) education database (CBCA, 2001). Ultimately, a typology developed by Dick and Dick (1989) for their review of ETR&D and JID, and later adapted by Klein (1997) and Reeves (1995) was judged to be the most useful for this project. Dick and Dick’s schema was embellished with one category from the ERIC typology: viewpoint. Also missing from Dick and Dick’s analysis were categories for editorial, publication review, and announcements, which are regular features of the JDE. All categories and their descriptions are listed in Table 1.
In addition to type, items were also categorized by the volume and issue in which they appeared and the language in which they were presented.
Item Topics
Following the example of Koble and Bunker (1997), topic in this study refers to the primary subject of the item, not the scope of the item; only one topic was assigned per item. An attempt was made to classify item topics using an inductive process where categories were extracted from the items during the review. The categories that emerged were not sufficiently different from those that are generally used to organize introductory textbooks in the field of distance education. Therefore, tables of contents from several distance education textbooks were consolidated to form the basis of the item topic typology (see Table 2) (Cyrs, 1997; Keegan, 1996; Lockwood, 1995; Moore & Kearsley, 1996; Stewart, Keegan, & Holmberg, 1983; Tate & Mills 1999).
First Author Categories
Categories for first author include name, sex, institutional affiliation, country in which institution is located, and province in which the institution is located if the code for country was Canada.
Types of Data
For the subset of items that were identified as empirical studies, we categorized the types of data that were collected and analyzed. Categories included qualitative, quantitative, or both.
To establish the dependability of the categorization decisions, one author coded all the items and a second author independently coded a 10% random sample of those items. The proportion of simple agreement between the coders was 0.93.
Item Type
Figure 1 illustrates the percentage of each of the items types. Four item types account for approximately 70% of the total number of items: Empirical, Description, Publication Review, and Viewpoint. The five item types that constitute empirical and theoretical works (evaluation, literature review, methodological, empirical, and theoretical) account for almost 30% of the total.
It is encouraging to see empirical research items as the highest in frequency of item type. This coincides with the JDE’s aim of providing a forum for empirical work in distance education. However, the most com-
mon design for empirical studies was descriptive. Gall, Borg, and Gall (1996) characterize descriptive research as “the most basic of methods, which involves simply describing the characteristics of individuals or other phenomena” (p. 373). This content analysis, for example, would be categorized as descriptive. Certainly there is a place for descriptive designs in distance education, but whether this should occupy three quarters of our research is open for discussion. Perraton (1988) argues that this is a function of the lack of theoretical foundations in distance education. For “without a theoretical basis, research is unlikely to go beyond data gathering” (p. 1).
Exploring Perraton’s (1988) assertion, we noted a dearth of theoretical items published in the JDE. This is a common criticism of distance education publishing, which is unfortunate, for as Garrison (2000) explains:
Theoretical inquiry is central to the vitality and development of a field of practice. The theoretical foundations of a field describe and inform the practice and provide the primary means to guide future developments. The power of ideas, as represented in our theories, influences practice directly by focusing perspective, revealing knowledge, and suggesting alternatives. Since ideas and ideals shape distance education practice, attention and effort must be devoted to the development of coherent, rigorous, and valid theory. (p. 3)
Authors such as Bates, Garrison, Holmberg, and Moore published extensively during the period reviewed, and published in the JDE, but they chose to direct the bulk of their theoretical publications elsewhere (Holmberg, 1999; Moore, 1993; Garrison, 1997, 2000). Two factors may be contributing to this. First, other journals in the field, particularly the AJDE, have significantly higher subscription rates than the JDE, which may be an attractive feature for authors seeking to influence theoretical developments. Second, most of the parenthetical citations to Holmberg, Moore, and Garrison refer to books or chapters in books. The length of most journal articles may be too restrictive for a thorough presentation of a theoretical work. Substituting for theoretical items in the JDE were viewpoints, the ERIC term for position papers, opinion papers, and essays, or what Koble and Bunker (1997) called conceptual discussions.
Item Topic
In terms of topical representation, there seems to be a nearly linear decrease across the 10 categories, ranging from a high of 13.6% for Foundations and 13.2% for Technology/Media to less than 1% for Trends. Percentages for the remaining topics are: Administration (12.3%), Instructional Design (10.6%), Learner Characteristics (9.8%), Editorial (8.1%), International Perspectives (6%), Faculty Characteristics (4.7%), Evaluation (4.3%), and Student Support Services (1.3%). The high proportion of items addressing foundations was influenced by a preoccupation in issues published in the 1980s with the status of distance education as a discipline and whether it warranted its own graduate programs.
The predominant language of JDE items was English (75%); items presented in French accounted for only 12% of the total. The remaining 13% appeared in both French and English, which is generally the case for editorials, or in Spanish in a special issue on distance education in Latin America. The fact that items are submitted by authors from several English-speaking countries aside from Canada may be contributing to the predominance of English-language items.
Author Categories
Representation was nearly equal between male (45%) and female (43%) first authors. Information concerning the sex of the remaining 12% of authors was unavailable. In terms of institutional affiliation, the overwhelming percentage of first authors affiliated themselves with higher education (72%), and an additional 16% indicated their affiliation was CADE. These authors are also probably affiliated with higher education institutions. The remaining 22% gave their affiliation as government, professional association, private, or K-12. A probable reason for this imbalance is the higher education job description, which places great weight on dissemination of scholarly work.
Approximately 48% of the first authors came from outside Canada, predominantly the United Kingdom (14%), the United States (7%), and Australia (6%). The remaining 22% of the journal’s content was submitted by authors from 20 countries around the globe.
Representation by Canadian province generally follows the population trends with Alberta, Ontario, British Columbia, and Quebec each contributing respectively 13.2%, 12.8%, 10.2%, and 8.1% of the items. The disproportionately high representation of Alberta authors may be due to the presence of Athabasca University. However, the same logic does not apply to Quebec, the home of TéléUniversité, because many of its authors seek publication in French-language journals. Percentages of items contributed by authors from the remaining provinces were: New Brunswick (2.1), Saskatchewan (1.7), Newfoundland (1.3), Manitoba (1.3), and Nova Scotia (0.9).
Research Processes
Descriptive information on the research processes used in the subset of articles coded as empirical research was also gathered. The amount of information concerning these issues was inconsistent across articles and insufficient to allow an accurate classification of the studies’ research perspectives. Thus classification was limited to whether the authors had collected and analyzed qualitative or quantitative data. Thirty-one percent of the empirical studies collected and analyzed qualitative data; 25% relied solely on quantitative data. An additional 31% of the empirical articles collected and analyzed both qualitative and quantitative data. These percentages appear to reflect a 20-year trend to employ qualitative methods increasingly in educational research, and to combine the two in an attempt to provide more definitive answers to research questions (Reeves, 1996).
Mandate
The second purpose of this study was to determine whether the results are consistent with the JDE’s aims. One aim, to present a balance of French and English items, on a breadth of topics, submitted by male and female authors, from national and international perspectives, has been met as noted by the data presented above.
Evidence for the second aim—the presentation of scholarly work of an empirical and theoretical nature—was ascertained by calculating the percentage of items that were coded as Empirical, Literature Review, Evaluation, Methodological, and Theoretical. Together the items formed a cumulative 29% of the total journal content. If items coded as Viewpoint are added to this list, the cumulative percent increases to 44.7%. The most prolific contributor was Burge, who contributed six items in the journal’s 15-year history. There does not appear to be a single author or group whose perspective dominates the journal.
Comparison With Other Analyses
The third goal of this study was to compare the results from the analysis of the JDE (1986-2001) with analyses of other scholarly and professional journals devoted to distance education: Koble and Bunker’s (1997) analysis of the American Journal of Distance Education (AJDE) (1987-1995), Scriven’s (1991) analysis of the Australian Journal Distance Education (DE) (1979-1989), and Sturrock and Howard’s (1989) early analysis of the JDE.
The analysis revealed four interesting comparisons; the first pertains to breadth. The JDE presented items submitted by authors from 24 countries, whereas the AJDE presented items from only 10 countries. The DE fell in between, with contributions from 17 countries. Further, Canadian items did not dominate the JDE to the same degree that US items dominated the AJDE (54.7% vs. 69.8%). Australian authors were even less dominant in the DE (34.86%).
The second comparison pertains to topics. The emphasis on particular topics across groups was similar as evidenced through frequency. Foundations of distance education and technology/media were the most frequent item categories for the JDE and the AJDE. Least frequent categories across all journals included faculty characteristics and student support services.
The third comparison pertains to research processes. A higher percentage of empirical research items appeared in the AJDE, and of these a higher proportion was quantitative (73%) than the JDE (29%). The JDE’s presentation of quantitative, qualitative, and mixed methodology items appeared to be more balanced. Neither Scriven (1991) nor Sturrock and Howard (1989) conducted a comparable analysis on the DE or the early volumes of the JDE.
Finally, the AJDE has a smaller proportion of descriptive items (15%) than the JDE, in which descriptive items were the second most frequent item type. Scriven’s (1991) and Sturrock and Howard’s (1989) categories for item type are not interchangeable with categories of the other two analyses.
Trends
To determine whether any trends were apparent in the journal, graphs were drawn depicting the percentage of each category by volume. The only visually discernible trend was found for the category item type in which a modest upward trend is apparent in the proportion of empirical items. No other trends were discernible in the remaining variables. Trends were anticipated on topics such as online learning; however, the first issue of the JDE begins with an article by Bates (1986) on the topic of computers as communication tools rather than tools for presenting preprogrammed instruction. Other categories, such as instructional design or learner characteristics may be too general to be sensitive to trends.
As of this writing, CADE has been publishing the JDE for over 15 years, producing a total of 23 issues and over 200 items. This sample and time frame was sufficiently large to prompt an analysis of the journal in an attempt to reveal some characteristics about its authors, the items they contribute, and the research processes they use. In turn these items, appearing in a leading journal, revealed something about the field of distance education.
American Psychological Association. (1994). Publication manual of the American Psychological Association (4th ed.). Washington, DC: American Psychological Association.
Bates, A.W. (1990). Third generation distance education: The challenge of new technology. (ERIC Document Retrieval Service No. ED 332682)
Berelson, B. (1952). Content analysis in communication research. New York: Free Press
Berge, Z.L., & Mrozoski, S. (2001). Review of research in distance education, 1990-1999. American Journal of Distance Education, 15, 1-15.
Canadian Association of Distance Education. (2001). Journal of Distance Education/Revue de L’Éducation à Distance. [Online] Available: http://www.lib.unb.ca/Texts/JDE/
Canadian Business and Current Affairs. (2001). Publication types.
Cyrs, T. (1997). Visual thinking: Let them see what you are saying. In T. Cyrs (Ed.), Teaching and learning at a distance: What it takes to effectively design, deliver, and evaluate programs (pp. 27-32). San Francisco, CA: Jossey-Bass.
Dick, W., & Dick, D. (1989). Analytical and empirical comparisons of the Journal of Instructional Development and Educational Communication and Technology Journal. Educational Technology Research and Development, 37, 81-88.
Educational Resource Information Centre. (2001). Publication types. [Online] Available: http://ericir.syr.edu/Eric/Help/pubtypes.shtml
Foley, A., Keener, P., & Branch, R. (1994). Women’s contributions to instructional technology journals. Educational Technology Research and Development, 42, 55-62.
Frick, U. (1998). An introduction to qualitative research. London: Sage.
Gall, M., Borg, W., & Gall, J. (1996). Educational research: An introduction (6th ed.). New York: Longman.
Garrison, D.R. (2000). Computer conferencing: The post-industrial age of distance education. Open Learning, 12, 3-11.
Hawkridge, D. (1999). Thirty years of BJET. British Journal of Educational Technology, 30, 293-304.
Holmberg, B. (1999). The conversational approach to distance education. Open Learning, 14, 58-60.
Keegan, D. (1996). Foundations of distance education. London: Routledge
Klein, J. (1997). ETR&D—Development: An analysis of content and a survey of future direction. Educational Technology Research and Development, 45, 57-62.
Koble, M., & Bunker, E. (1997). Trends in research and practice: An examination of The American Journal of Distance Education. American Journal of Distance Education, 11, 19-38.
Layton, M. (1992). Editorial in English. Journal of Distance Education, 7, 1. [Online] Available: http://cade.athabascau.ca/vol7.1/05_editorial_english.html
Lockwood, F. (1995). Open and distance learning today. London: Routledge.
Moore, M. (1993). Theoretical principles of distance education. In D. Keegan (Ed.), Theoretical principles of distance education (pp. 22-39). London: Routledge.
Moore, M.G., & Kearsley, G. (1996). Distance education: A systems view. New York: Wadsworth.
Perraton, H. (1988). A theory for distance education. In D. Sewart, D. Keegan, & B. Holmberg (Eds.), Distance education: International perspectives (pp. 34-45). New York: Routledge.
Reeves, T. (1995). Questioning the questions of instructional technology research. [Online] Available: http://www.hbg.psu.edu/bsed/intro/docs/dean/
Reeves, T. (1996). Established and emerging evaluation paradigms for instructional design. In D.L. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 163-178). New York: Macmillan.
Rourke, L. (2000). Exploring social interaction in computer conferencing. Unpublished master’s thesis, University of Alberta.
Scriven, B. (1991). Ten years of “Distance Education.” Distance Education, 12, 137-145.
Smith, R. (2001, March 22). WebCT use spreads among U of A courses. UofA Express News.
Stewart, D., Keegan, D., & Holmberg, B. (1983). Distance education: International perspectives. New York: Routledge.
Sturrock, J., & Howard, D. (1989). Looking in the mirror: The developing concept ofdistance education and the Journal of Distance Education. International Council for Distance Education Bulletin, 21, 54-59.
Sturrock, J. (1990). Editorial in English. Journal of Distance Education, 5, 1. [Online] Available: http://cade.athabascau.ca/vol5.1/4_editorial-engl.html
Suen, H., & Stevens, R. (1993). Analytic considerations in distance education research. American Journal of Distance Education, 7(3), 61-69.
Tate, A., & Mills, R. (Eds.). (1999). The convergence of distance and conventional education: Patterns of flexibility for the individual learner. London: Routledge
Liam Rourke is a doctoral candidate in the Learning, Development and Assessment (Instructional Technology) program, Department of Educational Psychology, University of Alberta.
Michael Szabo is a professor of educational psychology in the University of Alberta and has been involved in instructional technologies and distance education for the better part of three decades.
ISSN: 0830-0445