The Effectiveness of Traditional vs. Audiographics Delivery in Senior High Advanced Mathematics Course |
Walter F. Ryan
VOL. 11, No. 2, 45-55
A study was conducted to compare the differences in 1) the achievement of students who studied senior high advanced mathematics by distance education and students who studied senior high advanced mathematics by traditional means and 2) the success in postsecondary calculus courses of distance education senior high advanced mathematics graduates and traditional senior high advanced mathematics graduates. Results showed that distance education students were as successful in both achievement in senior high advanced mathematics and in success in postsecondary calculus courses as traditionally taught students. Probable factors contributing to the similar results for both groups are discussed.
L'objectif de cette étude est de comparer les différences entre 1) la performance d'élèves de niveau fin secondaire qui ont suivi des cours de mathématiques avancées via un mode d'éducation à distance versus celle d'élèves du même niveau dont les cours de mathématiques avancées étaient dispensés par des moyens traditionnels, et 2) le succès dans des cours de calcul de niveau postsecondaire de ces deux mêmes cohortes d'élèves. Les résultats indiquent que les élèves du profil éducation à distance réussissaient aussi bien, tant dans leurs cours de mathématiques avancées du cycle terminal du secondaire que dans leurs cours de calcul au niveau du postsecondaire, que les élèves qui avaient suivi leur cours de mathématiques avancées selon un mode d'enseignement traditionnel. On y aborde l'apport de certains facteurs contribuant à la similitude des résultats obtenus pour chacun des deux groupes.
Research on the effectiveness of distance delivery of courses using telecommunication technologies in K-12 is scarce (Moore, 1989). Some studies suggest that there is no significant difference in student achievement between telecommunication-based distance education courses and traditional delivered courses in K-12 (Batey & Cowell, 1986; Martin & Rainey, 1993; Schmidt, Sullivan, & Hardy, 1994). Others have suggested that tele-communication-based distance education courses improved quality and depth of learning for the students (Conboy, 1993; Kelleher, 1983) and student retention rates (Alberta Education, 1988, 1989a, 1989b). Because significant growth is occurring in the number of distance-delivered senior high school courses, student achievement in these courses and the success of these students in later university courses merits further study.
This paper examines the relative effectiveness of distance education in a natural experiment that allowed direct comparisons between traditional and distance student achievement in a senior high advanced mathematics program. This natural experiment presented itself in a context where a provincial department of education decided to deliver an advanced mathematics sequence of courses to its small schools. This paper also evaluates the relative success in university studies between traditional and distance student graduates from senior high advanced mathematics.
The natural experiment for this study was the senior high advanced mathematics program in the province of Newfoundland and Labrador, Canada. The secondary system in Newfoundland and Labrador has three distinct streams of mathematics courses: the advanced mathematics stream, the academic mathematics stream, and the practical mathematics stream. The advanced mathematics course stream is intended for the higher ability mathematics student who plans to study mathematics-related disciplines in postsecondary studies (Newfoundland and Labrador Department of Education, 1994). The advanced mathematics sequence has three courses: Advanced Mathematics 1201, Advanced Mathematics 2201, and Advanced Mathematics 3201. Senior high students who choose the advanced mathematics sequence normally study one of these course in each year of senior high school.
Because of its dispersed population, Newfoundland and Labrador has many small senior high schools. Most of these schools do not have the resources or the expertise to offer the advanced mathematics sequence of courses. Consequently, in 1985, the Government of Newfoundland and Labrador set up a committee to study the state and needs of the small schools in the province and this committee recommended that “greater use of technology be made in program delivery in small schools, especially in small high schools” (Riggs, 1987, p. 28). In response to this report, the Provincial Department of Education launched the delivery of Advanced Mathematics 1201, the first course in the advanced mathematics sequence, to 13 small high schools in September 1988. By the 1990-1991 school year, all three senior high advanced mathematics courses were offered by distance education.
The Newfoundland and Labrador Department of Education employs an audiographics distance education system to deliver the advanced mathematics courses to the small high schools in the province. The audiographics system consists of an audioconferencing system for voice exchange and a telewriter and monitor to draw and convey graphical images. Microphones are used at both the distance education instructor transmission site and the distance education student sites for voice communication. The distance education students are on-line with their distance education instructor every second school day. The distance education students are not normally supervised when they are on-line with the distance education instructor. On the days that they are not on-line, the students complete assigned work as outlined by their instructor. The distance education students are usually assigned to the library or to another mathematics class in their school on the days that they are not on-line.
The distance education advanced mathematics students in Newfoundland and Labrador use the same text materials as are used in the traditional classroom. The distance education students also have a study guide that highlights the main points in the concepts and procedures under study, details the pace of the lessons, and assigns the work to be completed. In each unit of study, students are expected to complete homework assignments on a regular basis and submit them for correction and comment by fax to the distance education resource teacher. Additionally, in each unit of study, distance education students complete a major assignment and submit the completed assignment to the instructor. The grade on these assignments is used for summative evaluation. Students also have an exam at the end of each unit of study. The courses each have two major tests, one at midyear and one at the end of the year. The final grade is a composite of the results of the major assignments, unit tests, and the two major tests.
The distance education advanced mathematics courses are treated the same as traditional courses in the participating high schools. An on-site teacher, usually the principal or assistant principal, is responsible for keeping school records on the progress of student, for contact with parents concerning the progress of the student, and for forwarding the summative evaluation reports to the parents.
Subjects were 7867 students who studied Advanced Mathematics 3201, the third course in the advanced mathematics sequence, in the province of Newfoundland and Labrador, Canada, for the school years 1991-92, 1992-93, 1993-94, and 1994-95. Of these 7867 students, 315 completed Advanced Mathematics 3201 by distance education. Specifically, there were 38 distance education Advanced Mathematics 3201 students in 1991-92, 85 in 1992-93, 88 in 1993-94, and 104 in 1994-95.
Academic Achievement
To evaluate academic achievement in advanced mathematics, the final course grade in Advanced Mathematics 3201, as recorded by the Public Exam Division of the Department of Education of the Government of Newfoundland and Labrador, was selected. This final grade was the average between an instructor-assigned grade and a grade in a provincial administered common exam. All students, both distance education and traditional students, completed this exam. Each school year, the provincial Department of Education arranged and supervised the preparation of the Advanced Mathematics 3201 common exam. After students had completed it, the Public Exam Division of the Department of Education supervised its marking. The study used the final course grade rather than the grade in the provincial administered exam because the Public Exam Division of the Department of Education of the Government of Newfoundland and Labrador had the complete data set of the final course grades in Advanced Mathematics 3201 available for all four years investigated in the study but did not have a complete data set available for the grade in the provincial administered common exam.
The study used a covariate, the average of the student’s highest grade 12 grade in an academic science course, academic English course, and an academic social studies course, to adjust the group means in academic achievement for Advanced Mathematics 3201. This covariate was selected because both distance education and traditional Advanced Mathematics 3201 students would have studied most of these courses by traditional means. Additionally, this covariate should have a high correlation with the final Advanced Mathematics 3201 grade of the student.
This study equalized the sample sizes between the traditional and the distance education students for each year investigated in this study. The Public Exam Division of the Department of Education of the Government of Newfoundland and Labrador used computers to select randomly the same number of traditional Advanced Mathematics 3201 students as distance education Advanced Mathematics 3201 for each year investigated. Therefore, the study used the final grades from 38 traditional taught students in 1991-92, 85 in 1992-93, 88 in 1993-94, and 104 in 1994-95 to compute the traditional Advanced Mathematics 3201 group means.
Academic Success
To evaluate the relative academic success in postsecondary studies of the distance education and traditional Advanced Mathematics 3201 graduates, the two groups of graduates were compared in their participation rates in postsecondary studies, by their pass rate in their first postsecondary calculus course, and by the number of calculus courses taken in postsecondary studies. To collect the data for analysis to measure these three attributes of academic success, a survey instrument was forwarded to all distance education Advanced Mathematics 3201 graduates for the years 1991-92, 1992-93, 1993-94, and 1994-95 and an equal number of traditional Advanced Mathematics 3201 graduates for each of these years. The sample for the traditional Advanced Mathematics 3201 graduates was obtained by stratifying the total provincial traditional Advanced Mathematics 3201 student population based on student enrolments in each school board in the province. Each school board then randomly selected the specific number of traditional students for each of these years. Thus, the final samples for academic success consisted of 315 distance education and 315 traditional Advanced Mathematics 3201 graduates.
The survey instruments forwarded to the distance education and traditional advanced mathematics graduates requested information from the graduates on their attendance at postsecondary institutions, whether they completed calculus courses in their postsecondary studies, the number of calculus courses completed in postsecondary studies, and how much success they had in their first postsecondary calculus course. A space for additional comments on the survey instruments permitted graduates the opportunity to add clarifying explanations.
To compare the academic achievement in advanced mathematics of distance education and traditional students, a 2X1 analysis of covariance (two groups, one factor) was completed using the final grades for each of these two groups in Advanced Mathematics 3201 for the school years 1992-93, 1993- 94, 1994-95, and for the combined results for the years 1991-92, 1992-93, 1993-94, and 1994-95. Thus, the smallest sample for the ANCOVA had 85 distance education Advanced Mathematics 3201 students and 85 traditional Advanced Mathematics 3201 students. This sample size ensured robustness of the ANCOVA. Type one error for each of the ANCOVA tests was set at 0.01. Consequently, the true rate for type one error for the four ANCOVA tests was equal to or less than 0.04.
Analysis of the results demonstrated that the Advanced Mathematics 3201 final grades for each year evaluated approximated normal distributions and had equal variances in the different cells. The covariate had both a strong correlation, ranging from a low of 0.67 to a high of 0.76, and a linear relationship with the Advanced Mathematics 3201 final grades in each cell for each year analyzed. Finally, the slope of the regression lines for each group in each year analyzed were approximately equal (greatest difference been 0.05 for the combined set of results, 0.44 for distance education group and 0.49 for the traditional group). Thus, all the necessary conditions for an ANCOVA test were satisfied.
Analysis of covariance demonstrated no significant difference between students who studied advanced mathematics by distance education and by traditional means in their achievement level in Advanced Mathematics 3201. As can be seen in Table 1, the largest F-value was 1.27. This corresponded to a significance level of 0.26 and was not significant.
In each ANCOVA, the observed mean for the distance education advanced mathematics group was adjusted upwards and the observed mean for the traditional advanced mathematics group was adjusted downward. This resulted in a reduction of the differences between the two group means for 1992-93 and 1993-94 and for the combined results for 1991-92, 1992-93, 1993-94, and 1994-95 and an increase of the differences between the two means for 1994-95. Therefore, the use of the covariate in this analysis reduced the difference of the achievement level means in Advanced Mathematics 3201 between students who studied advanced mathematics by distance education and by traditional means.
To evaluate the relative academic success in postsecondary studies of the distance education and traditional Advanced Mathematics 3201 graduates, three chi-squared tests of independence were completed. An alpha level of 0.01 was selected as the criterion to determine significance for each test because three separate tests were completed.
To collect the data on academic success, survey instruments were forwarded to 302 distance education advanced mathematics graduates and 315 traditional advanced mathematics graduates. Survey instruments were forwarded to only 302 of the 315 distance graduates because the school boards in Newfoundland and Labrador did not have the addresses of 13 of the graduates.
As noted in Table 2, completed calculus survey instruments were returned by 177 of the distance education Advanced Mathematics 3201 graduates and by 202 of the traditional sample. In the returned surveys from the traditionally taught group, nine calculus surveys were discarded because of incomplete information. Thus, the statistical analysis for academic success used calculus survey results from 177 distance education advanced mathematics student graduates and from 193 traditional graduates. These numbers represented 56.19% of the total distance sample and 61.27% of the traditional sample.
To evaluate the two groups in their participation rates in postsecondary studies, a chi-squared test of independence was run comparing the two groups in the categories: students who attended a postsecondary institution after graduation from high school and students who did not attend a postsecondary institution after graduation. This analysis with one degree of freedom yielded a chi-squared value of 0.43, resulting in a significance level of 0.51. Therefore, no significant relationship existed between distance education advanced mathematics graduates and traditional graduates in their participation rates in postsecondary studies.
Note. The top number in each cell represents the actual number and the bottom number in each cell represents the expected value.
To evaluate the two groups in their pass rate in their first postsecondary calculus course, a chi-squared test of independence was run comparing the two groups in the categories: students who passed their first postsecondary calculus courses in their first attempt; students who attempted but failed their first postsecondary calculus course in their first try; and students who did not take courses in calculus at postsecondary institutions. This analysis with two degrees of freedom yielded a chi-squared value of 0.66, resulting in a significance level of 0.72. Thus, no significant relationship existed between distance and traditional graduates in their pass rate in their first postsecondary calculus course.
Note. The top number in each cell represents the actual number and the bottom number in each cell represents the expected value.
To evaluate the two groups in the number of calculus courses taken in postsecondary studies, a chi-squared test of independence was run comparing the two groups in the categories: students who took one calculus course in postsecondary study; students who took two calculus courses in postsecondary study; students who took three calculus courses in post-secondary study; and students who took four or more calculus courses in postsecondary study. This analysis with four degrees of freedom yielded a chi-squared value of 1.57, resulting in a significance level of 0.81. Thus, no significant relationship existed between distance education graduates and traditional classroom graduates in the number of calculus courses taken in postsecondary studies.
Note. The top number in each cell represents the actual number and the bottom number in each cell represents the expected value.
The results of this study suggest that the academic achievement and academic success of students who complete senior high mathematics courses by distance education does not differ significantly from students who complete senior high mathematics courses by traditional means. Specifically, the data show that students who studied senior high advanced mathematics by distance education achieved the same level as the traditionally taught student, participated in postsecondary studies at a comparable rate, and had the same relative success with postsecondary calculus courses.
It may be argued, of course, that the results for academic achievement are simply a function of the self-selecting process for students who studied the advanced mathematics by distance education. Indeed, the participation rate in advanced mathematics in the distance education schools was lower than the participation rates in advanced mathematics in the traditional schools. However, over the four school years analyzed in this study, the gap between the participation rates narrowed from eight percentage points to five percentage points. In the academic year 1994-95, the participation rate in advanced mathematics in the traditional schools was 23% and in the distance education schools it was 18%. In light of these differences in participation rates, it is important to note that the analysis of the data for academic achievement for each year and the combined set of data for all four years produced similar results. Hence, it would appear highly unlikely that the difference in participation rates in senior high advanced mathematics was responsible for the similarity in results for the two groups in academic achievement.
Students who studied advanced mathematics by distance education lived in the small communities in the province of Newfoundland and Labrador. Students from these small communities generally participate in postsecondary studies at a lower rate compared to students from the larger communities in the province. It is interesting to note that the results of this study indicated that the distance education advanced mathematics students from these communities participated in postsecondary studies at the same rate as the traditional advanced mathematics students. Additionally, the distance education advanced mathematics graduate had the same relative success with postsecondary calculus courses as traditionally taught students. Therefore, the results of this study supports the initiative by the Department of Education of the Government of Newfoundland and Labrador to deliver advanced mathematics sequence of courses by distance education to the small schools in the province. Without the distance education delivery, students in these schools could not take the advanced mathematics courses and consequently would not be as prepared to take calculus courses in postsecondary studies.
One factor that may contribute in the success of the distance education students in their postsecondary studies is the independent study habits that they develop in studying senior high advanced mathematics by distance education. The distance education students who were the subjects for this study were required to complete much of their work independently. Consequently, the distance education student had to learn self-reliance in work habits to be successful in advanced mathematics. This self-reliance proved very helpful in postsecondary studies as illustrated by this anecdotal comment by a distance education instructor.
A girl was in talking to me yesterday. She is now studying in pilot school and has 80 hours of flying. I said, “How are you finding it now?” She said, “Good. Distance education made me work and study at home on my own. Because I did that in grades 10 to 12 in high school so now in pilot school when I have to study, I don’t mind studying four or five hours a night. Whereas if you are a regular student and you get slack at times with the teacher around, you are not ready to go to university, you are not ready to handle the responsibilities. Distance education makes one really aware of their responsibilities.
The findings of this study support the potential of distance education for delivery of mathematics courses to small senior high schools. However, the replication of this study under experimental control conditions would help support the conclusions emanating from this study. Including experimental controls in a replication study would guard against the effects of extraneous variables on the academic outcomes.
Walter F. Ryan
Assistant Professor of Education
Indiana University Southeast
4201 Grant Line Road
New Albany, IN, USA 47150
Alberta Education. (1988). Evaluation of phase 1 of the distance learning in small schools action research project, October, 1988. Barrhead, Alberta, Canada: Alberta Distance Learning Center.
Alberta Education. (1989a). Evaluation of phase 2 of the distance learning in small schools action research project, October, 1989. Barrhead, Alberta, Canada: Alberta Distance Learning Center.
Alberta Education. (1989b). Evaluation of phase 1 of distance learning project North, December, 1989. Barrhead, Alberta, Canada: Alberta Distance Learning Center.
Batey, A., & Cowell, R. (1986). Distance education: An overview. Northwest Regional Educational Laboratory. (ERIC Document reproduction Service No. ED 278 519)
Conboy, I. (1993). MAX-An Australian approach to audiographics in schools. In C. Latchem, J. Williamson, & L. Henderson-Lancett (Eds.), Interactive multimedia: Practice and promise (pp. 121-154). London: Kogan Page.
Kelleher, K. (1983). Tele-education: Teaching over the telephone with slow-scan video. Boulder, CO: Colorado Video. (ERIC Document reproduction Service No. ED 256 286)
Martin, E. E., & Rainey, L. (1993). Student achievement in a satellite-delivered high school science course. The American Journal of Distance Education, 7(1), 54-61.
Moore, M. G. (1989). Effects of distance learning: A summary of the literature. Washington: Congress of the U.S., Office of Technical Assessment.
Newfoundland and Labrador Department of Education.(1994). Advanced mathematics curriculum guide. St. John’s, Newfoundland and Labrador, Canada: Newfoundland and Labrador Department of Education.
Riggs, F. (1987). Report of the small school study project. St. John’s, Newfoundland and Labrador, Canada: Jesperson Press.
Schmidt, K. J., Sullivan, M. J., & Hardy, D. W. (1994). Teaching migrant students Algebra by audioconference. The American Journal of Distance Education, 8(3), 51-63.
Dr. Walter Ryan was an educator in Newfoundland and Labrador for 22 years. He completed his doctorate at Ohio University in 1996. He presently teaches at Indiana University Southeast. His main research interest is the use of technology to enhance mathematics instruction.
ISSN: 0830-0445