The Growing Impact of Open Access Distance Education Journals: A Bibliometric Analysis

Olaf Zawacki-Richter, Terry Anderson and Nazime Tuncay

VOL. 24, No. 3

Abstract

Open access dissemination resonates with many distance education researchers and practitioners because it aligns with their fundmantal mission of extending access to learning opportunity. However, there remains lingering doubt whether this increase in access comes at a cost of reducing prestige, value (often determined in promotion and tenure hearings) or reference of the work by other authors. In this article, we examine 12 distance education journals (6 open and 6 published in closed format by commercial publishers). Using an online survey completed by members of the editorial boards of these 12 journals and a systematic review of the number of citations per article (N = 1,123) and per journal issue between 2003 and 2008, we examine the impact, and perceived value of the 12 journals. We then compute differences between open and closed journals. The results reveal that the open access journals are not perceived by distance eductation editors as significantly more or less prestigious than their closed counterparts. The number of citations per journal and per article also indicates little difference. However we note a trend towards more citations per article in open access journals. Articles in open access journals are cited earlier than in non-open access journals.

Résumé

La diffusion par accès direct reçoit l’aval de nombreux chercheurs et praticiens du domaine de l’éducation à distance parce que celle-ci s’inscrit dans la lignée de leur mission fondamentale qui est d’augmenter l’accès aux opportunités d’apprentissage. Toutefois, un doute subsiste quant à savoir si cet accès accru se réalise ou non au coût d’une diminution du prestige, de la valeur (souvent déterminée au cours des entrevues de promotion et de titularisation) ou de la référence aux travaux par les autres auteurs. Dans cet article, nous examinons 12 revues d’éducation à distance (dont 6 sont à accès direct et 6 sont publiées en format fermé par des éditeurs commerciaux). À l’aide d’un sondage en ligne rempli par des membres des comités de rédaction de ces 12 revues et d’une revue systématique du nombre de citations par article (N = 1,123) et par édition de revue parue entre 2003 et 2008, nous examinons l’impact et la valeur perçue des 12 revues. Nous calculons ensuite les différences entre les revues à accès direct et les revues fermées. Les résultats révèlent que les éditeurs de l’éducation à distance ne perçoivent pas les revues à accès direct comme étant significativement plus ou moins prestigieuses que les revues fermées. Il y a aussi peu de différences au niveau du nombre de citations par journal et par article. Toutefois, nous notons une tendance vers un nombre plus élevé de citations par article dans les revues à accès direct. Les articles dans les revues à accès direct sont cités plus tôt que ceux des revues à accès non direct.

Introduction

Publishing in distance education has two critically important functions. At the professional level the results of innovations, experiments, theoretical insights and pedagogical interventions increase the effectiveness and efficiency of education as we learn from and build upon the works of others. From an academic perspective, publication and review by peers, has become the means by which research is shared and as importantly the means by which contribution is measured and recognized and this has become a primary determinant of promotion and reputation. From both perspectives publication is of critical relevance for all those seriously involved in distance education.

But how does one know which is the best outlet for publication? Obviously one wants to publish in sources that are widely exposed within both professional and academic communities and published in journals that are known by reputation to have high standards for process, peer review and revision. The explosion of interest and increase in number of publications that release material in open access formats also complicates the decision making for authors. Open Access publication may result in higher exposure, but open access journals may be perceived as having a lower value than older more well established closed outlets.

The purpose of this research is to survey 12 major distance education journals from 2003 to 2008 (N = 1,416 articles), to investigate the relative prestige and perceived quality of these journals and to determine if open access journals are becoming more influential in distance education publications.

This study is therefore guided by the following questions:

• Which of the 12 most well-read journals produce the highest number of citations per article?
• How do distance education experts evaluate the prestige of open access journals in contrast to the proprietary ones?
• Which journals are perceived by distance education experts as being the most widely read?
• Do open access journals produce more citations over time than closed ones?

Our hypothesis is that the impact of open access journals in terms of average citations per article and year (b1) between 2003 and 2008 is significantly higher than the impact of closed journals (b2), thus:

H1: b1 > b2

Literature Review: Open Access and Closed Access

In 2006 there were over 23,000 scholarly journals, publishing 1.4 million articles a year, and creating a $5 billion industry that employs approximately 90,000 persons (Ware, 2006). Distance education as practice, scholarship and focus of research is one of thousands of disciplines that create both consumers and producers of scholarly journals. The global trend towards increasing numbers of distance education researchers, programs, courses, teachers and students has been matched by increase in the number and the variety of distance education journals.

Scholarly journals are published under one of two models. In the older 'closed' publication model, a commercial publisher, university press or a professional association edits and publishes the journal and generates revenue from the sale of individual issues or articles, subscriptions or, more often today, by sale of aggregated databases of articles to libraries. Open access journals are most often published by a sponsor (often a university, professional organization, or government agency) and the content is distributed freely to subscribers on the open web and via aggregated databases. Importantly, no fees or subscription costs, institutional or organizational membership is required in order to read the full text of open access journal articles. The Directory of Open Access Journals (DOAJ) indexes more than 4,820 full text and quality controlled scholarly journals, covering most academic disciplines (DOAJ, 2010).

Harnad et al. (2008) have argued that a hybrid model exists for wider distribution of scholarly work that has advantage for both commercial publishers and researchers. They refer to open access publishing as the "gold option" but have concerns with its sustainability. They argue that a "green model" in which authors archive their full text work on institutional repositories allows for both commercial viability of closed access journals and access by the public. Although Harnad et al. (2008) claim that over 90% of closed published journals allow self archiving, research has shown that the percentage of authors who do self archive is very low, unless this is a requirement to do so as a condition of their employment (Thomas & McDonald, 2007). In addition, some publishers demand a quarantine period of time (typically one year) before authors are allowed to self-archive copyright materials. Finally, the distributed nature of self archives makes serendipitous browsing of related articles more challenging (i.e., special issue articles are not gathered together) and forces users to be very efficient search engine users. Finally, our research notes that search engines often miss critical articles in self-archived repositories. The use of specific peer-to-peer-networks for open content and social search engines could be solutions for sharing and retrieving information generally — not just for the scientific community.

Given the high stakes associated with publication in research awards and individual promotion and tenure hearings, there has been considerable interest in determining the impact of journal publication in all fields of study. The study of the effect of specific journal publication is referred to as bibliometrics – a generic term for a whole range of specific measurement indicators. Its purpose is to measure the output of scientific and technological research through data derived not only from scientific literature but patents as well (Okubo, 1997).

Open Access journals have a broadly similar citation pattern and impact to other journals, but may have a slight tendency to earlier citations (TSI, 2004). Proponents of open access argue that any form of distribution restriction results in the loss of potential research impact (see Harnad et al, 2004). A number of studies in different subject areas demonstrate that open access increases impact (e.g., Lawrence, 2001; Brody et al., 2004; Kurtz, 2004; Hajjem, Harnad & Gingras, 2005; Hitchcock, 2005; Eysenbach, 2006; Norris, Oppenheim & Rowland, 2008; Kousha & Abdoli, 2009).

Researchers often discover differences among disciplines in relationship to the impact of open versus closed publication. Lawrence (2001) showed that free online availability substantially increases a paper's impact and that more highly cited articles and more recent articles in computer science are significantly more likely to be online. He reported that citations in computer science conference papers were three times higher for open access proceedings than for those published in non-open access formats. Hajjem, Harnad and Gingras (2005) conducted a multidisciplinary research study of a 12-year sample (1992-2003) of nearly 14 million articles to present a more general view of citation impact of open access journals in 10 different disciplines including biology, psychology, sociology, health, political science, economics, education, law, business, and management. They extracted citation data from the ISI database and used a robot to crawl the Web for locating open access (self-archived) versions of the articles published in non-open access journals. The overall results showed that open access articles had more citations than non-open access articles. The citation advantage of open access articles varied from 36% to 172% by discipline and year.

Other studies present contradicting findings. For instance Davis et al. (2008) found no evidence of a citation advantage for open access articles in the first year after publication. They concluded that the citation advantage from open access reported widely in the literature may be an artefact of other causes. Davis (2009) conducted a research study on biomedical sciences journals and found that the open access advantage is declining. This may be due to the increase in access afforded to researchers associated with universities that subscribe to databases containing thousands of closed peer reviewed journals, coupled with the increase in self-archiving, providing open access to copyright (closed) articles.

From this literature we can conclude that disciplinary differences exist among use, perceptions of prestige, access and citation results between open and closed publication models and that these results may be changing over time. Thus, it is important to research the relative impact of publications in the field of distance education and attempt to quantify the impact over time.

Method and Sample

Selected Journals

Table 1 provides an overview of the 12 journals (6 open access and 6 closed access) that were reviewed for this study. They were selected based upon of their reputations as the most prominent and recognized journals in the field of distance education.

All full-papers published in these journals between 2003 and 2008 are in the sample (N = 1,416). Editorials, field notes, book reviews and other non-research papers are excluded. The year 2003 was selected as the starting date because the Asian Journal of Distance Education and the International Journal of Distance Education Technologies were released in 2003 for the first time. Since articles need at least one year to generate citations (McVeigh, 2004), the year 2008 was chosen as a cut-off date.

Table 1: Selected Distance Education Journals

*latest issue published in January, 2008; re-launched in 2009 in an open access format (http://journal.ignouonline.ac.in/iojp/index.php/IJOL)

How to Measure "Impact" of a Journal?

According to Elbeck & Mandernach (2009), there are two general approaches to scholarly journal bibliometric assessmentFN1:

The first is based on citation analysis (e.g. Baumgartner & Pieters, 2003; Jobber & Simpson, 1988). The advantages and disadvantages of citation analysis to measure impact are discussed in the following section. The second approach to journal ranking examines perceptions of style or content, acceptance rates, readership, distribution and prestige and other qualitative assessments of a journal (e.g., Mylonopoulos & Theoharakis, 2001; Nisonger, 1999; Hult, Neese, & Bashaw, 1997; Luke & Doke, 1987).

In this study we follow Elbeck & Mandernach's suggestion to combine citation and perception data to measure the impact of a journal.

Citation Analysis

Many of the studies mentioned above determine citation counts based upon the proprietary ISI (or Web of Science, WoS) ranking system developed and maintained by Thompson Publishing — a major commercial publisher. Meho & Yang (2007) argue that ISI databases are "no longer sufficient because new databases and tools that allow citation searching are now available" (p. 2105). Critics of ISI citation databases note that they a) cover mainly English-language journals from North America and Europe; b) are limited to only 8,700 journalsFN2; c) do not count citations from books and conference proceedings; and d) provide different coverage between research fields.

Regrettably, ISI does not rate any of the 12 major distance education journals. Our own experience suggests that Thompson has little interest in indexing and calculating impact factors for journals in relatively small disciplines and especially those that compete with it as open access publications. Similar to ISI, Elsevier, producer of the Scopus citation database, provides access to over 15,000 titles, among them only IRRODL and IJDET.

Fortunately, we were able to gather data on citations on most of the 12 journals using Google Scholar (GS) data. GS automatically extracts bibliographic information and cited references from online full-text documentsFN3. We used "Publish or Perish" which is a free software program that retrieves and analyzes academic citations from GS and presents statistics such as the total number of papers/citations, the average number of citations per year/author, Hirsh's h index and other related parameters.FN4

Although GS finds articles from all the 12 journals, GS does not find all full papers that were published between 2003 and 2008. Table 2 provides an overview of GS coverage.

Table 2: Google Scholar Coverage

* Total number of articles published in the journals between 2003 and 2008
** Total number of articles found by GS in the same time period

Due to the low coverage of EURODL in GS we decided to exclude this journal from the citation analysis since it would not lead to valid results.

The number of citations grows over time. Therefore, the data was collected for all journals within two weeks in October 2009. In this time period GS generated a stable number of citations so that the results are comparable between journals.

As the main indicators for the impact of a journal we use the average number of cites per paper and Hirsh's h index: "A scientist [or a journal] has index h if h of his/her Np papers have at least h citations each, and the other (Np-h) papers have no more than h citations each" (Hirsch, 2005, p. 16569).

Journal Prestige

To collect perception data, an online survey was sent to all members of editorial boards of the 12 journals. The questionnaire was completed by 83 of 277 editorial board members (30% response rate).

Recipients were instructed as follows (Nelson, Buss & Katzko, 1983): "Please assign numbers to the journals listed below according to how you would rate them, where 5 = outstanding, 4 = excellent, 3 = good, 2 = adequate, 1 = poor. You may leave blanks if you do not recognize a journal" (p. 475).

Furthermore, the editors were asked to rate the journals according to their exposure to journal's scholarly articles, where 5 = outstanding (I read or scan most articles in nearly every issue), 4 = excellent (I read many articles and scan the table of contents in nearly every issue), 3 = good (I often read articles that I retrieve from this journal), 2 = adequate (I occasionally read articles from this journal), 1 = poor (I almost never or never look at articles from this journal).

Limitation: We acknowledge the bias of editorial board members who are more aware of their 'own' journal and are likely prejudiced towards perceiving quality in that journal, however we also believe that these are likely most informed consumers of distance education research and thus their opinion and perceptions of value are useful. The following two tables show that editors rate their own journal significantly higher than non-editors and that they are – as to be expected – more exposed to their journal than to others.

Table 3: Differences in Ratings of a Journal Comparing Non-editors and Editors

*significant at alpha = 0.05 level; a = closed, b = open

By Fisher’s combined probability test (cf. Winer, 1971, p. 49), we have χ²(24) = 84.86, p < .001, which indicates that there is a statistical significant difference between editors and non-editors in terms of their ratings by combining all the journals.

Table 4: Differences in Exposure to a Journal Comparing Non-editors and Editors

Here we have χ²(24) = 102.92, p < .001, which indicates that there is also a statistical significant difference between editors and non-editors in terms of their exposure to the journals by combining all the journals.

Results

Citation analysis

The following table shows the results of the citation analysis for the 12 journals and for the time period between 2003 and 2008 (N = 1,123 papers). The most cited papers are presented in Table 6. The top two papers were published in open access journals.

Table 5: Citation Analysis of 12 Distance Education Journals (N = 1,123 papers since 2003)

a = closed, b = open

At this stage, three journals have to be excluded from the further analysis: EURODL, because the data is not complete (only 33% of articles covered with GS, cf. Table 2) and furthermore the Asian Journal of Distance Education (AsianJDE) and the Indian Journal of Open Learning (IJOL) because the total number of citations is so low that they would produce a bias, i.e. the results would be inflated by a small number of 'big hits' or 'one hit wonders' (highly cited articles), which are not representative for the journal.

The development of the impact of a journal can be described in terms of cites per paper which decline from 2003 to 2008 (Table 5). The years are recoded (2003 = 1, 2004 = 2, etc.) and a simple logarithmic regression is performed using the year as the single predictor which produces the best fit, i.e., y = b₀ + b₁* ln(year). The dependent variable that we use for each journal is 'cites per paper'.

We interpret the regression coefficients (b1 values) as a measure for the slope of the curve to describe the development of impact based on cites per paper. The steeper the curve declines the more impact a journal looses from 2003 to 2008. Figure 1 exemplary illustrates the citation slopes for OJDLA (R2 = .975) and AJDE (R2 = .940).

Table 6: Most Cited Papers in the 12 Journals Published Between 2003 and 2008 (as of October 8, 2009)

a = closed, b = open

The development of the impact of a journal can be described in terms of cites per paper which decline from 2003 to 2008 (Table 5). The years are recoded (2003 = 1, 2004 = 2, etc.) and a simple logarithmic regression is performed using the year as the single predictor which produces the best fit, i.e., y = b₀ + b₁* ln(year). The dependent variable that we use for each journal is 'cites per paper'.

We interpret the regression coefficients (b₁ values) as a measure for the slope of the curve to describe the development of impact based on cites per paper. The steeper the curve declines the more impact a journal loses from 2003 to 2008. Figure 1 exemplary illustrates the citation slopes for OJDLA (R² = .975) and AJDE (R² = .940).

Figure 1: Citations per paper from year 1 = 2003 to 6 = 2008 (OJDLA and AJDE)

Table 7 depicts that the slopes are all negative and ln(year) significantly predicts (p < 0.05) the dependent variable 'cites per paper', which indicates that 'cites per paper' decreases with the natural logarithm of the year, which in turn decreases with the year. If we look at the R-squares, it ranges from .765 to .975 which provides a moderate to excellent fit of the regression model.

Table 7: Logarithmic Regression (nine journals)

* significant at alpha = 0.05 level; a = closed, b = open

QRDE behaves quite differently compared to the other journals (Figure 2). Applying a cubic model defined by the equation y = b₀ + (b₁<*t) + (b₂*t²) + (b₃*t³) provides a much better fit with R² = .996.

Figure 2: Citations per paper from year 1 = 2003 to 6 = 2008 for QRDE (logarithmic and cubic model)

Figure 3 shows the percentages of the six years total citations (2003 = 100%) between open access (JDE, IRRODL, OJDLA, TOJDE) and closed access (AJDE, DE, IJDET, OL) journals and reveals that the articles in open access journals are cited more quickly. QRDE is excluded from this analysis due to its poor fit to the logarithmic model.

Figure 3: Percentages of 6 year total citations (observed values)

To test for the difference between slopes of open access and closed journals a simple logarithmic regression is performed with the two groups (open/closed).

Table 8: Logarithmic Regression (two journal groups)

* significant at alpha = 0.05 level

To test for the difference between the two regression coefficients a z-test is carried out based on the formula suggested by Paternoster et al. (1998, p. 862):

In comparing the slopes of these two regression coefficients, closed journals decrease faster (i.e., lose impact) than open journals. However, the two slopes are not statistically significant different from each other at the 5% confidence level (z = 1.73, p = .08).

Thus, we have to accept H₀: b₁(open) = b₂(closed).

Journal Prestige Analysis

Rating of Journals

The average ratings for each journal are presented in the following table.

Table 9: Rating of Distance Education Journals by Editorial Board Members

* total number of ratings; a = closed, b = open

Figure 4 depicts error bar graphs with a 95% confidence interval showing a top group of five journals (IRRODL, DE, AJDE, JDE, and OL), a second group of three journals (EURODL, IJDET, and QRDE) followed by a third group that was rated significantly lower than the top group (OJDLA, TOJDE, AsianJDE, and IJOL). The top rated journal is IRRODL which might be due to the relative high number of editors who participated (cf. Table 3).

Figure 4: Error Bar Graph for Journal Ratings (95% confidence interval, with 1 = poor and 5 = outstanding)

When comparing the open to the closed journals we find the mean rating for the open journals to be 3.43 with SD of .62; closed journals had slightly higher, but not significantly different prestige ratings averaging 3.62 with SD of .515. However, it is useful to note that the standard deviation is considerably higher for the open access journals, which may be related to their relative youthfulness.

Exposure to the Journals

Exposure to a journal was rated on a scale from 1 = poor (I almost never look at articles from this journal) to 5 = outstanding (I read or scan most articles in nearly every issue). Exposure and ratings of each journal (cf. Table 9) are correlated on a highly significant level in a positive direction (Table 10).

Table 10: Exposure to Journals by Editorial Board Members and Correlation between Rating and Exposure

* Pearson correlation between rating and exposure, significant at the 0.01 level (2-tailed); a = closed, b = open

When comparing the open to the closed journals we find the mean rating for access to the open journals to be 3.08 (SD = .81); closed journals had almost identical access ratings averaging 3.09 (SD = .73).

Correlations of exposure data reveal that there are regional clusters of some journals in terms of their readership, e.g. readers of the Asian Journal of Distance Education also read the Indian Journal of Open Learning (r = .815, p < 0.01). And there is another group of North American journals: Readers of the Quarterly Review of Distance Education are also exposed to articles in the American Journal of Distance Education (r = .470, p < 0.01), the Online Journal of Distance Learning Administration (r = .639, p < 0.01), and the Journal of Distance Education (r = .410, p < 0.01). It is likely that open access journals are read more often, especially in developing countries, where access to closed issues is not available. However, this cannot be confirmed based on our data.

Conclusions

This study provides multiple ratings for each of the 12 most popular distance education journals. We hope the results will guide both authors and readers in selecting outlets for their research work, and input for their reading of distance education issues, theories and research results. We note however that the longitudinal data demonstrates that the number of citations produced by articles in a journal has changed over time and likely the perceptions of prestige and access change as well. Thus, these results are a snap shot of ratings in October 2009. Readership and publication outlet also has regional implications, as distance educators and researchers may wish to read and publish in journals that are related to and read by audiences from specific countries or regions – others will value a journal for its international scope. Thus, the effect of regional importance, though not discussed nor assessed in this article, may have an important bearing on journal readership.

These results show that in the distance education discipline open and closed journals are not significantly different in terms of prestige or access as assessed by experts nor differentiated in the number of citations. However, analysis of the slopes of graphs of increasing citations over time after release indicates that open access journals are more quickly cited and may be increasing in number of citations per article in recent years. This study should be repeated in future years to gauge the impact over time of open versus closed journals.

We conclude this article with a brief discussion of the most important attribute of open access publications- that being accessibility to all citizens of the world, not just for those employed at well-funded universities. Publishing in open access journals is but one of the attributes of open scholarship. Gideon Burton (2009) argues that "Open Access is more than a new model for scholarly publishing; it is the only ethical move available to scholars who take their own work seriously enough to believe its value lies in how well it engages many publics and not just a few peers." Distance educators and scholars in many areas of the world do not have access to closed research publications. Unfortunately, the data collected does not allow us to speak to the accessibility of open versus closed journals in developing countries nor by those distance educators who work in businesses, schools and many colleges whose libraries do not subscribe to scholarly journals – an interesting perspective for future research.

We hope the results of this study demonstrate that publishing in open access journals does not negatively impact the number of citations, nor the perceptions of value by experts in the field.

1. The bibliometric approach is an appropriate method to measure the impact of a journal but it should be very carefully used for rating the "quality" of an individual researcher in hiring, promotion and tenure decisions. Weingart (2005) advocates for an 'informed peer review': "The future of the higher education and research system rests on two pillars: traditional peer review and ranking. The goal must be to have a system of informed peer review, which combines the two. However, the politicized use of numbers (citations, impact factors, funding, etc.) appears unavoidable" (p. 117).
2. Ulrich's Periodicals Directory (Bowker, NJ) lists approximately 22,500 active scholarly, refereed journals (Meho & Yang, 2007).
3. Compared to ISI/WoS and Scopus, data collecting and cleaning using GS is very time consuming. Although we did not measure the exact time needed to carry out the citation analysis (at least four weeks of a well-trained, full-time Research Assistant), our experience supports Meho & Yang's (2007) findings: "The WoS data took about 100 hours of collecting and processing time, Scopus consumed 200 hours, and GS a grueling 3,000 hours" (p. 2105).
4. http://www.harzing.com/pop.htm (accessed 5 Nov, 2009); a new tool (the Scholarometer) which is based on GS data as well is available at http://scholarometer.indiana.edu/ (accessed 15 March, 2010).

References

Baumgartner, H., & Pieters, R. (2003). The structural influence of marketing journals: A citation analysis of the discipline and its sub-areas over time. Journal of Marketing, 67(2), 123-39.

Brody, T., Stamerjohanns, H., Harnad, S. Gingras, Y. Vallieres, F. & Oppenheim, C. (2004). The effect of Open Access on Citation Impact. Presented at: National Policies on Open Access (OA) Provision for University Research Output: an International meeting. Southampton University, Southampton UK.

Burton, G. (2009, April Academic Evolution Blog Scholar or Public Intellectual? [Web log message]. Retrieved from: http://www.academicevolution.com/2009/04/index.html

Davis, P., Lewenstein, B., Simon, D., Booth, J., & Connolly, M. (2008). Open access publishing, article downloads, and citations: randomised controlled trial. British Medical Journal, 337-568. DOI: 10.1136/bmj.a568

Davis, M. (2009). Author-Choice Open-Access Publishing İn The Biological And Medical Literature: A Citation Analysis. Journal of the American Society for Information Science and Technology, 60(1), 3 - 8.

DOAJ (Directory of Open Access Journals). Retrieved March 15, 2010, from: http://www.doaj.org

Elbeck, M., & Mandernach, B. J. (2009). Journals for computer-mediated learning: publications of value for the online educator. International Review of Research in Open and Distance Learning, 10(3), 1-20.

Eysenbach G. (2006). Citation advantage of Open Access articles. PLoS Biol 4(5): e157. DOI:10.1371/journal.pbio.0040157

Hajjem, C., Harnad, S., & Gringras, Y. (2005). Ten-year cross disciplinary comparison of the growth of OA and how it increases citation impact. Retrieved August 26, 2009, from http://eprints.ecs.soton.ac.uk/11688/01/hajjem.pdf

Harnad, S. (1990). Scholarly skywriting and the prepublication continuum of scientific inquiry. Psychological Science 1, 342-343, Retrieved November 9, 2009, from: http://www.cogsci.soton.ac.uk/~harnad/Papers/Harnad/harnad90.skywriting.html

Harnad, S. (1991). Post-Gutenberg Galaxy: The fourth revolution in the means of production of knowledge. Public-Access Computer Systems Review, 2(1): 39 - 53. Retrieved October 1, 2009 from http://www.cogsci.soton.ac.uk/~harnad/Papers/Harnad/harnad91.postgutenberg.html

Harnad, S., Brody, T., Vallieres, F., Carr, L., Hitchcock, S., Gingras, Y, Oppenheim, C., Stamerjohanns, H., & Hilf, E. (2008). The access/impact problem and the green and gold roads to open access: an update. Serials Review 34(1), 36-40.

Harnad, S.,Brody, T.Vallières, F.Carr, L.,Hitchcock,S., Gingras,Y.,Oppenheim,C., Stamerjohanns, H.,Hilf, E.(2004). The access/impact problem and the green and gold roads to Open Access. Serials Review, 30(4), 310-314.

Harnad, S., & Brody, T. (2004). Comparing the impact of Open Access (OA) vs. non-OA articles in the same journals. D-Lib Magazine, 10(6). Retrieved November 10, 2009 from: http://www.dlib.org/dlib/june04/harnad/06harnad.html

Harter, S. P. (1996). The impact of electronic journals on scholarly communication: A citation analysis. The Public-Access Computer Systems Review, 7. Retrieved November 10, 2009, from http://info.lib.uh.edu/pr/v7/n5/hart7n5.html/

Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46).

Hitchcock, S., Brody,T, Gutteridge,C., Carr, L. & Harnad, S. (2003).The impact of OAI-based search on access to research journal papers. Serials, 16(3), 255-260.

Hult, G.T.M., Neese, W.T., & Bashaw, R.E. (1997). Faculty perceptions of marketing journals. Journal of Marketing Education, 19(1), 37-52.

ISI. (2004). The impact of Open Access Journals: A Citation Studyfilm. Thomson ISI. [Online]. Available: http://www.isinet.com/media~presen~g,/acropact-oa-jos. [Accessed: October 10, 2009].

Jobber, D., & Simpson, P. (1988). A citation analysis of selected marketing journals. International Journal of Research in Marketing, 5(2), 137-42.

Kousha,K., Abdoli, M. (2009). The citation impact of open access agricultural research: a comparison between OA and Non-OA publications. World Library and Information Congress 75th IFLA General Conference and Assembly. http://www.ifla.org/files/hq/papers/ifla75/101-kousha-en.pdf [Accessed: October 04, 2009]

Kurtz, M.J. (2004). Restrictive access policies cut readership of electronic research journal articles by a factor of two. Harvard-Smithsonian Centre for Astrophysics, Cambridge, MA. http://opcit.eprints.org/feb19oa/kurtz.pdf

Lawrence, S. (2001). Free online availability substantially increases a paper's impact. Nature, 411, 521. Retrieved Ocotber 2, 2009, from http://www.nature.com/nature/debates/eaccess/Articles/lawrence.html

Luke, R.H., & Doke, E.R. (1987). Marketing journal hierarchies: Faculty perceptions, 1986-1987. Journal of the Academy of Marketing Science, 15(2), 74-8.

McVeigh, M. E. (2004). Open Access Journals in the ISI citation databases: Analysis of impact factors and citation patterns. Retrieved November 10, 2009 from http://science.thomsonreuters.com/m/pdfs/openaccesscitations2.pdf

Meho, L., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105-2125.

Mylonopoulos, N.A., & Theoharakis, V. (2001). Global perceptions of IS journals. Communications of the ACM, 44(9), 29-37.

Nelson, T. M., Buss, A. R., & Katzko, M. (1983). Rating of scholarly journals by chairpersons in the social sciences. Research in Higher Education, 19(4), 469-497.

Nisonger, T.E. (1999). JASIS and library and information science journal rankings: A review and analysis of the last half century. Journal of the American Society for Information Science, 50(11), 1004-1020.

Norris, M.,Oppenheim, C., Rowland, F. (2008). The citation advantage of open-access articles. Journal of the American Society for Information Science and Technology. DOI: 10.1002/asi.20898 Retrieved from: http://dx.doi.org/10.1002/asi.20898

Okubo, Y. (1997). Bibliometric indicators and analysis of research systems: Methods and examples. Translated from the French by the OECD (Paris: OECD, 1997).

Paternoster, R., Brame, R., Mazerolle, P., & Piquero, A. (1998). Using the correct statistical test for the equality of regression coefficients. Criminology, 36(4), 859-866.

Thomas, C., & McDonald, R. H. (2007). Measuring and comparing participation patterns in digital repositories. D-Lib Magazine, 13(9/10), 1082-9873.

Ware, M. (2006). Scientific publishing in transition. Bristol: Mark Ware Consulting.

Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 62(1), 117-131.

Winer, B. J. (1971). Statistical principles in experimental design (2nd ed.): McGraw Hill.

Olaf Zawacki-Richter is Professor at the FernUniversität in Hagen (Germany) and the University of Maryland University College (USA). He teaches courses on educational technologies, instructional design, educational management and research methods. E-mail: olaf.zawacki-richter@fernuni-hagen.de

Terry Anderson is Professor and Canada Research Chair in Distance Education at Athabasca University — Canada’s Open University. He teaches educational technology courses in Athabasca University Masters and Doctorate of Distance Education programs. E-mail: terrya@athabascau.ca

Nazime Tuncay is a doctoral candidate in the Computer Education and Instructional Technology Department at Near East University (North Cyprus). She holds a MSC in Applied Mathematics and Computer Science. Her research interests include distance education, e-education, vocational education and web tools. E-mail: nazime07@yahoo.com