A Team-Based Collaboration Used for the Development of Transnational Online Distance Education Courses

Chris Crowley, Hailan Chen and Mercé Gisbert Cerver

VOL. 33, No. 2 2018

Abstract

Using a team-based course development approach, the University of British Columbia collaborated with the Asia-Pacific Network for Sustainable Forest Management and Rehabilitation (APFNet) and universities from Australia, the Philippines, Malaysia, China, and Canada to design and develop six transnational, online, distance-education courses covering diverse topics such as sustainable forest policy, international dialogues in global forestry issues and forest restoration. The team-based approach is commonly used in online course development to form a cost-effective managerial and development team that manages timelines, coordinates solutions, and oversees budgets, all of which are particularly challenging for an international collaboration among universities across the Asia-Pacific region, due to cultural differences and variations in teaching and learning practices.

This paper gives an overview of the design and development of transnational, online, distance-education courses, and the results of a pilot study that put the completed courses through an in-house, evidence-based quality enhancement rubric and a written survey reporting the leading professors’ satisfaction with the project management and instructional design services.

The quality enhancement rubric showed that the course designs were of high quality and the written survey revealed that the subject matter experts were satisfied overall.  However, many improvements were suggested and could be implemented to improve the quality of course design, and the collaboration in orientating subject matter experts to the team-based development process, and in implementing best practices for online course design.

Keywords: team-based approach, online learning, course development, transnational online distance education, sustainable forestry management, international collaboration.

Résumé

L’université de la Colombie-Britannique collabore avec le Réseau Asie-Pacifique pour la Gestion Durable et la Réhabilitation des forêts (APFNet) et des universités d’Australie, des Philippes, de Malaisie, de Chine et du Canada en vue de concevoir et développer six cours transnationaux de formation à distance en ligne. Fondés sur une approche de développement en équipe, ces cours couvrent divers sujets tels que la politique pour une forêt durable, les dialogues internationaux sur les enjeux mondiaux en termes de foresterie et la restauration des forêts. Le travail en équipe est couramment mis en œuvre dans le développement de cours en ligne afin de former une équipe de gestion et de développement rentable qui gère les délais, coordonne les solutions et supervise les budgets, éléments qui constituent de réels défis dans le cadre d’une collaboration internationale, entre universités de la zone Asie-Pacifique, en raison des différences culturelles et des pratiques d’enseignement et d’apprentissage. Cet article offre une vue d’ensemble de la conception et du développement des cours transnationaux d’enseignement à distance en ligne. Il présente également les résultats d’une étude pilote portant sur les cours réalisés analysés au travers du prisme d’une démarche maison visant l’augmentation de la qualité en se fondant sur des données factuelles, et d’une enquête écrite faisant état de la satisfaction des principaux professeurs concernant les services de gestion de projet et de conception pédagogique. La grille d’amélioration de la qualité met en avant des conceptions de cours de grande qualité et l’enquête écrite fait ressortir une satisfaction générale des experts de contenu. Cependant, dans l’ensemble, de nombreuses améliorations ont été suggérées et pourraient être mises en œuvre pour améliorer la qualité de la conception des cours et la collaboration en orientant les experts de contenu vers le processus de développement en équipe, et en favorisant l’implantation de bonnes pratiques de conception de cours en ligne.

Mots-clés : éducation en ligne, formation à distance, conception pédagogique.

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

Introduction

Sustainable Forestry Management and Rehabilitation Education in the Asia-Pacific Region

Appropriate education in sustainable forestry management (SFM) is one of the keys for regional development in the Asia-Pacific Region, however, universities in the region face several challenging issues including the lack of financial support and facilities, inappropriate curriculum design, decreasing enrollments, and access to international education systems (Forestry College Deans Meeting Mechanism in the Asia-Pacific Region: Asia-Pacific Network for Sustainable Forest Management and Rehabilitation, 2013).  The lack of a regionally recognized academic educational programs in SFM is believed to be a major impediment to the advancement of SFM in the entire Asia-Pacific Region.  To address these concerns, the APFNet funded the development of a comprehensive academic educational program aimed at helping senior forestry students, young faculty and mid-career professionals to acquire, exchange, and transfer knowledge of SFM and related issues (Forestry College Deans Meeting Mechanism in the Asia-Pacific Region: Asia-Pacific Network for Sustainable Forest Management and Rehabilitation, 2013).

Transnational Online Distance Education (TODE)

A traditional classroom teaching methodology was unsuitable to support the acquisition and exchange of SFM knowledge across the Asia-Pacific Region, due to time-consuming restrictions, geographical constraints, lack of flexibility and high costs.  It was determined that developing online, distance-education courses was a viable and economical option that would provide a high-quality learning experience for all students to enable them to understand the issues and build knowledge and skills related to SFM in the region.  Also, distance education is not a surprising choice for the APFNet, considering that the demand for these educational opportunities is growing on a global scale, especially in countries such as China and India where distance education has grown as an alternative to face-to-face learning (St. Amant & Flammia, 2012; Merola, 2017). 

The design and development of online, distance-education courses is a complex, creative process that requires a reasonable level of resource support and teamwork and has been shown to be an effective management approach for online course development (Schrum,1989).  The concept of team-based course development, in the context of team dynamics and course quality, is well described and supported by the literature (Oblinger & Hawkins, 2006; Puzziferro & Shelton, 2008).  Team-based course development helps ensure that resources are used efficiently and individual team members contribute appropriate skills and knowledge to improve course quality.  It is commonly used in online, distance-education course creation to form a cost-effective approach that uses best practices for course development workflow, timeline management, instructional design and budget oversight, all of which are particularly challenging for an international collaboration between large institutions (Crowley, Chen & Murano, 2015).  In the team-based model, Xu and Morris (2007) noted that very often it's the instructional designer who ensures that everyone on the team works efficiently toward achieving their common goals and avoids overlap between roles and responsibilities.

This project, involving universities from five different countries in the Asia-Pacific Region — Australia, the Philippines, Malaysia, China, and Canada — reflected an online, distance-education project that was borderless (Toprak & Genc-Kumtepe, 2013).  Education projects that cross international borders are considered transnational meaning that the learners can be located in a country different from the one where the awarding institution(s) is(are) based (McBurnie & Ziguras, 2001; Skidmore & Longbottom, 2015).

Team-Based Design and Development of the Transnational Online Distance Education (TODE) Courses 

For this TODE project, the University of British Columbia’s (UBC) Faculty of Forestry requested that UBC’s Centre for Teaching Learning and Technology (CTLT) provide a central support team for project management and pedagogical guidance, including a project coordinator, an instructional designer, a web designer, a graphic artist, and a media expert. The project coordinator was responsible for the overall central budget, coordinating the project development needs and, when required, to draw upon other experts available across campus, e.g., curriculum consultants, media producers, and librarians.  The instructional designer was the contact person for each lead professor in order to manage the timeline, provide pedagogical guidance, and communicate the requirements of web production and graphic design for course development.  In this paper, lead professor, subject matter expert and course author are used interchangeably.  Where multiple subject matter experts were involved in a single course, a lead professor at the home university was designated the course coordinator for development support and was the key contact for the instructional designer.

For a central support team used to collaborating within the learning ecosystem of a single post-secondary institution or, occasionally with other institutions inside a province, the transnational nature of this project presented a number of complexities to the team-based framework, including different languages, cultural differences, established epistemological approaches, creation of academic content, and learning platform selection.   In addition, inexperienced course authors found that one of the biggest challenges was often simply the amount of time required for online, distance-education course creation (Kang, 2001) and in this project not only were they inexperienced but they were located in diverse countries, which resulted in some communication challenges.

Before course design and development started, a number of key requirements were agreed upon at the beginning of the project including: 1) CTLT would lead communications and guide the development of the courses, 2) a module-based course development template would be established so that subject matter experts could easily incorporate their own course content and to give all the courses a similar structure and look and feel, and 3) an orientation workshop would be offered for the course authors. In addition, the instructional design approach would accommodate each course author’s epistemological approach, which for all the courses turned out to be in the objectivist tradition of learning, where knowledge would be transmitted for recall and understanding, with limited application and analysis (Driscoll, 2000).

UBC’s team-based, course development framework used a modified ADDIE online course development model for the collaborative effort between UBC instructional design staff and the faculty from the five different countries involved (University of British Columbia, n.d.).  The model incorporated best practices for online pedagogy as well as the practical implementation of an in-house quality enhancement checklist. The main goal of the model was to provide ample instructional design, web programming, media development and other support.  Also, due to the transnational scope of this project, course design and development were overseen by an UBC Forestry Academic Project Leader and an UBC APFNet Project Coordinator. 

The workflow of the course development included analysis, planning, design, development, and quality review. In the analysis stage, program needs and target learners were discussed and program goals and delivery mode were determined. The overall design, course outcomes, teaching methods, learning technologies, and timelines were explored and/or established.  In the planning stage, ' generic course templates' were created with similar look-and-feel lay-outs and graphics.  The instructional designer guided the development of two course templates by two UBC lead Forestry professors.  A three-day, on-site workshop oriented the course authors to curriculum development, the team-based development approach for online courses, course development specifications, copyright considerations, and educational video production. In the design and development stage, leading professors returned to their home universities to lead the development of the course overviews and course prototype modules, with the guidance of the instructional designer. They completed their courses, module by module, following the format and procedures of their prototype module.  Quality reviews of the course production were arranged after the entire project’s target completion date.

As requested in the APFNet proposal, each completed course featured self-directed learning modules composed of background information, video lectures, supplemental readings, self-directed learning activities in the form of self-tests, self-directed reflection questions, and/or self-directed discussion forums.  The project was completed in approximately 20 months, with one course unfinished due to difficulties at one of the participating institutions.

Despite the international growth in online, distance-education courses, there is little literature that has studied or commented on the use of a team-based course design and development framework in multinational collaborations with its many complexities as described above.  This paper will examine our experience in how a team-based course development concept regularly used in online, distance-education course development within a single institutional or, occasionally, provincial context can be used or adapted to create an effective transnational learning project despite the differences in language, culture and epistemological approaches.

Pilot Study

The objective was to meet the specific goals (outlined above) of the APFNet for the design and development of six TODE courses adapting a team-based, course development framework normally used within a single post-secondary institutional context to a project involving several international institutions with different languages, communication styles, cultures, and epistemological approaches.  In our pilot study, two questions were considered:

Methodology and Data Collection

Satisfaction with Course Development

The success of the team-based, course development approach was gauged through a written survey administered to the lead professors/course authors of the five courses developed.  The range of questions included: 1) instructor experience with online facilitation and course development, 2) project management and instructional design services provided to the instructor, and 3) open-ended questions soliciting instructor feedback on challenges and suggestions for changes in the development process.

Quality of the Course Design

Regardless of the course author satisfaction with the team-based approach, it was important to ensure a high-quality, successful course design.   To answer this question, a modified in-house Quality Enhancement (QE) rubric (University of British Columbia, n.d.) was used to determine to what degree the TODE courses met rigorous, quality, instructional-design standards. The QE rubric was not intended to probe deeply into the academic quality and content of the courses, except at a very basic level, but rather to look into the alignment of course learning outcomes, assessments, content, activities, and technologies as well as core considerations in course orientation, clarity, organization, and accessibility for a quality online course.

The TODE course designs were analyzed by two experienced instructional designers using the QE rubric. One instructional designer, who also led the design and development of the courses, was a certified Quality Matters (QM) peer reviewer and served as an internal instructional designer. The other instructional designer was a senior instructional designer who was not involved in the course development and was invited to evaluate the courses as an outside instructional designer.

In addition, six Forest Science graduate and upper-level, undergraduate students used the same rubric, with some minor modifications to reflect the student perspective, to analyze the courses from a learner perspective. Each student was randomly assigned two APFNet courses to assess using the QE rubric.

Results and Discussion

Satisfaction with Course Development

Leading Professor’s Experience with Online Course Development

Table 1 shows the leading professors’ rating of their experience in developing, teaching, and learning in an online environment and their experience working with an instructional designer in a team-based course design approach.

Table 1: Leading Professors’ Experience with Online Course Development and Delivery

 

Survey Statement

Average

1

I have background in developing online courses prior to the APF Net project

2.3

2

I have background in teaching online courses prior to the APF Net project.

2.0

3

I have experience learning in online courses prior to the APF Net project.

2.3

4

I have experience working with an instructional designer before.

1.7

Rating scale: “Strongly disagree 1  2  3  4  5  Strongly agree”.

Anecdotally, and from the survey, it was seen by the rating scores of 2.3, 2.0, and 2.3, respectively, that most of the leading professors lacked experience in developing, facilitating, and learning in an online environment, which is critical for online course development.  Furthermore, the 1.7 rating score indicated almost no experience working with an instructional designer and, probably, most of them were climbing a steep learning curve in the team-based online course development environment.

Leading Professors’ Satisfaction Rating on Project Management Service

Table 2 provides the leading professors’ rating on our concerned aspects regarding project management for this project.

Table 2: Leading Professors’ Satisfaction Rating on Project Management

Survey Statement

Average

1.

The course development process and specifications were made clear to me.

4.3

2.

The workshop held at UBC in Summer 2014 was very helpful.

4.3

3.

The timeline set by the participants at the Summer 2014 workshop was manageable.

4.0

4.

The pilot module development required from each course development team makes sense.

4.0

5.

The timeline reminders were well communicated throughout the course development process.

4.7

6.

The timeline management was flexible enough and effective for my course.

4.7

7.

Overall, CTLT’s project management approach to APF Net course development was effective.

4.3


Rating scale: “Strongly disagree 1  2  3  4  5  Strongly agree”.

Table 2 shows that, with a rating of 4.3 on the ‘overall effectiveness of project management approach’ category, leading professors were satisfied with the team-based, project management approach to the project.  They especially appreciated CTLT’s effort to communicate the development timelines throughout the process and the effort to allow reasonable flexibility on the timelines for individual courses, giving both aspects the highest rating of 4.7.  With a rating of 4.3, leading professors were positive that the three-day, on-site, course development orientation workshop was very helpful and that the development process and specifications were generally made clear to them. One low, but still positive, rating of 4.0, was given to the pilot module development. This suggests that the pilot module provided a helpful picture of what a module should look like and the effort involved in the production of a module.  It was not a surprise to see the other low 4.0 rating given to the timeline outlining milestone tasks, because even though the original timeline was jointly agreed upon by the participants at the workshop, later delays in meeting the milestone schedule happened to some teams. 

Leading Professors’ Satisfaction Rating on Instructional Design Support

Table 3 provides leading professors’ rating on our concerned aspects regarding Instructional Design support for this project.

Table 3: Leading Professors’ Satisfaction Rating on Instructional Design Support

Survey Statement

Average

1.

The advice and feedback from CTLT helped me to incorporate appropriate teaching approaches online.

4.3

2.

The advice and feedback from CTLT helped transition my   course well for delivery in an online environment.

4.3

3.

The course design templates were customized to my course and useful to follow while developing the course.

4.0

4.

The instructions from CTLT were clear and easy to understand.

4.0

5.

The advice and feedback on my course design were always timely from CTLT.

4.7

6.

CTLT listened to my suggestions for my teaching and learning practices and supported finding solutions for my approach. 

4.5

7.

The instructional design guidance and feedback from CTLT were valuable. 

4.3



Rating scale: “Strongly disagree 1  2  3  4  5  Strongly agree”.

The fact that there were no ratings under 4.0 suggests that the leading professors were generally satisfied with the instructional design support for the project.  They especially appreciated the effort to provide timely advice and feedback on their course design, as seen by a highest rating of 4.7.  With a rating of 4.5, it appears that the instructional design team did a good job of finding solutions and providing support to accommodate the teaching approach of individual course authors. The 4.3 rating shows that the leading professors appear to agree that the advice and feedback were valuable in helping them to incorporate appropriate online teaching approaches and creating the course in the online environment. The positive but low rating of 4.0 was given to the ‘customized course design templates for individual courses’ and ‘clear and easy instructions from CTLT’ categories.  This seems to indicate that providing clear instructions can be a challenge for a transnational project using a team-based approach, even though a considerable effort was made to provide clear and easy-to-follow instructions to guide course authors to work though the course development process using customized course design templates for individual courses.

Leading Professors’ Responses to Open Ended Questions on Challenges and Suggestions Grouped into Themes

Table 4 is a collection of the learning professors’ responses to three open-ended questions regarding the challenges they encountered, their advice to new professors, and comments to improve central support. The responses are tallied and grouped in categories of Media Development/New Delivery Approach, Time Commitment, Administration, Communication, and Project Quality and Management.

Media development in the form of video lectures was one of the major content development components in the project. The high number of comments by course authors proved that they went through a steep learning curve to convert PowerPoint presentations into short video lectures that would accommodate online learners’ expectations.  Technical support turned out to be one of the biggest challenges in this process. Despite the hands-on video production training in the workshop and consistent guidance and feedback during video production, many leading professors needed and sought local technical support from staff or video professionals to produce videos at their home universities.  Ultimately, two out of five courses produced video lectures with assistance from the professional video production team on their campus, two courses produced videos with the help of instructional assistants, and one course produced videos solely using the effort of course authors themselves.  The leading professors' advice to new professors who would want to start a similar venture was to be mentally and technically prepared for and ready to adapt to online teaching approaches.

Table 4: Leading Professors’ Responses to 'Open Ended' Questions on Challenges and Suggestions Grouped into Themes

Q1. List challenges you encountered during the design and development of your course.

Q2. What would be your advice to new professors who may be involved in the development of new APF Net courses?

Q3. Any comments, suggestions, or recommendations to improve our project management and instructional design support approach?

Media Development/New Delivery Approach

  • Media development – we are handicapped by slow Internet connectivity. (Q1)
  • New experience online. (Q1)
  • Technical support. (Q1)
  • Work with young staff who are more updated with media development. (Q2)
  • Study the technology well in advance. (Q2)
  • Be prepared to new teaching experience. (Q2)

Time Commitment

  • Time commitment. (Q1)
  • Time commitment – we encountered difficulty in beating earlier schedule due to strong typhoon that devastated our campus that led to out of electricity, Internet, telephone services for about 3 months. (Q1)
  • Difficult to get commitment from the (subject matter) expert. (Q1)
  • Time consuming. (Q1)
  • Work hard. (Q2)

Administration

  • Administration. (Q1)
  • Administration – one of our support team member resigned from the university that hamper our completion on time. (Q1)
  • Very challenging to push the leading expert to complete the course. (Q1)
  • Get more people involve in completing the course. Cannot depend on one or two leading prof. (Q2)

Communication

  • Communication. (Q1)
  • lengthy interaction with CTLT in terms of repeated requests for changes. (Q3)
  • Need more regular communications among relevant institutions. (Q3)

Project Quality and Management

  • Seek peer reviews to enhance the quality of your course. (Q2)
  • More time control over the course developers (instructors), because timely delivery of all courses is critical for the success of the project. (Q3)

The time commitment appeared to be underestimated by the course authors.  Despite new learning technologies and the team-based support, professors still found content creation time consuming and needed to put in a great effort to keep up with the schedule. In addition, there was a time commitment challenge not only for the leading professors but also for the course co-authors or subject matter experts in this collaborative environment.  To our knowledge and from the analysis of survey responses, the difficulties in making the time commitment were mainly caused by 1) the leading professors’ own administrative obligations, 2) busy time schedules, 3) the changing of development team members, and 4) uncontrolled natural hazards. For example, four out of five course authors were Forestry Deans and/or Academic Leads with all the ensuing administrative responsibilities. Three out of five courses involved more than one course author, in fact, one course had 10 course co-authors. Two universities reported changes to the course development team. One university was hit by a typhoon twice that paralyzed the university campus for weeks with electrical, telephone and Internet outages. 

As to the specific challenges encountered by the administration of course development at local institutions, one course was delayed by the resignation of a support team member and a second course was hampered by a faculty leadership change.  In the latter case, the original leading professor’s service to the university ended for reasons unrelated to the project, however, it did result in: 1) a new course lead author being assigned, which took a number of months for onboarding and 2) the course was incomplete at the end of the project schedule. When this happened, the local administration found it very difficult to find a new leading expert to complete the project. They were advised to consider getting more people involved in completing the course up front instead of depending on one leading professor to finish it.

Communications were also a concern — one comment identifying communication as one of the challenges was from the course with 10 co-authors. This may indicate that in the case of multiple course authors the challenges of clear communication exist not only between the leading professors and the instructional designer but also between the leading professor and co-authors.  Another response called for more regular communications among relevant institutions, indicating the need for institutions to update each other on their course development progress and to review each other’s work to ensure, for example, course content is not overlapping.  There was a complaint about the lengthy interaction with CTLT in terms of repeated requests for changes.  In fact, when communicating with leading professors for content edits, the instructional designer limited the review or feedback to no more than two rounds asking for clarifications or confirmation about proposed changes.  This signals that balancing the quality of work and course authors’ available time was a great challenge for the instructional designer. 

Regarding project quality and management, the lead professors provided two valuable pieces of advice.  One related to the improvement of the course by seeking peer review to enhance the quality of course design. The second comment related to time management by asking for a more rigorous approach to meet the scheduled timeline for course development, given the fact that one course was not completed and that the timely delivery of all courses was critical for the success of the project.  However, due to the abovementioned time commitment difficulties these were ongoing challenges for the instructional designer.

Leading Professors’ Responses to Open-Ended Questions Regarding Instructional Design Support

In analyzing the lead professors' responses in Table 5, it is apparent that a structured approach to the instructional design project management was recognized as crucial to the success of the project. 

Table 5: Leading Professors’ Responses to 'Open-Ended' Questions Regarding Instructional Design Support.


In what way did CTLT help you the most to design and develop your course?

Positive Responses

  • To develop the works step by step.
  • Providing templates for the work
  • Regular/timely feedback and supervision of the work
  • Flexibility in the schedule
  • Do things as [the name of instructional designer] did (asked). She is a great asset to UBC CTLT

Negative Responses

  • Not much help in design, but good help in development.

The responses indicate that the lead professors appreciated CTLT's expertise in guiding the process step-by-step, providing templates and regular facilitation.  The key role that the Instructional Designer fulfilled in course development was communicated anecdotally as well as in the survey. For example, when asked for recommendations to improve the instructional design approach, one comment from the subject matter expert was, "Do things as the [name of Instructional Designer] did. She is a great asset to UBC." It is interesting to see that the instructional designer’s effort to provide content templates as guidance and as a means of communication to accommodate best practice standards in course design may merely be considered by professors as development support instead of instructional design support.  This illustrates the constant challenge that instructional designers face when trying to demonstrate the value of their work.

Quality of Course Design

Analysis of the quality of course design for the five completed courses using responses to the QE checklist lead to the following results.

Average Rating for Course Quality

Charts 1 shows the average rating results for each of the five courses from both the instructional design and student perspective. The average score is an average rating of quality standards by instructional designers and students, respectively, regarding categories of General Course Information, Course Goals and Learning Outcomes, Learner Assessments, Course Materials, Learning Activities, Course Technology, and Course Look and Feel. 

Chart 1: Instructional Designer vs. Student Average Rating for Courses using the Quality Enhancement (QE) Rubric.

Rating scale: “Strongly disagree 1  2  3  4  5  Strongly agree”.

The instructional designer and student perspectives shown in Chart 1 indicate that, on average, there was agreement that the project successfully created well-designed TODE courses satisfying the learners’ need to understand the issues and build knowledge and skills related to SFM in the region.  The instructional designers’ average ratings for each course were slightly lower than students’ ratings for each course; 4.3 from the instructional designers and 4.6 from the students.  The lower average rating given by the instructional designers is probably due to their professional expertise and greater understanding and awareness of course quality. 

Chart 2: Instructional Designer vs Student Average Rating on Course Quality Components.

Rating scale: “Strongly disagree 1  2  3  4  5  Strongly agree”.

Chart 2 shows the instructional designer versus student average ratings on course quality components defined by the QE rubric. Overall, the ratings indicated that the courses were reasonably well created with clear goals and learning objectives, rich content, attractive designs consistent across each of the courses, intuitive navigation schemas, clear and concise instructions and consistent pedagogical scaffolding between modules and topics.  The content within each course corresponded with the course overall goal and topic learning objectives.

The chart also shows that Instructional Designers were more critical of the design of assessments and learning activities.  At the time of course review, it was observed that student assessments aligned with the learning objectives for most of the courses.  The assessments in some courses mainly measured pedagogical lower-level thinking (identify, recall and explain) through a series of self-directed, multiple-choice tests and reflection questions. The assessments in some of the courses included strategies to measure higher-level thinking (discuss, analyze, evaluate, develop) through a series of self-directed online discussions and activities.  In one course, the goals clearly stated that the students should be able to compare and analyze complex issues, however, assessments of these competencies were not present in the course yet.

Identified Quality Enhancement Aspects

Tables 6 and 7 show selected categories defining the quality of course design where the average rating was less than 4.0 from instructional designers and students using the QE rubric. With a rating scale of “Strongly disagree 1 2  3  4  5  Strongly agree”, scores of less than 4.0 indicate areas where the reviewers did not agree that the categories were meeting quality standards and therefore required higher priority for quality enhancement. The lower the score, the more room for improvement and the higher priority for enhancement.

Table 6: Instructional Designer Identified Quality Enhancement Aspects Across Courses Using the QE Rubric

Category

Statement

Rating

Course Technology

The course technologies are current.

3.9

Course Goals and Learning Outcomes

Instructions to students on how to meet the learning outcomes are clearly stated.

3.8

Learner Assessment

The assessment strategies/tools selected are appropriate to the student work being assessed.

3.8

Learning Activities

Learning activities are clearly integrated with specific instructional materials and linked to learning outcomes.

3.7

Learning Activities

Learning activities actively engage learners in meaningful and relevant learning throughout the course.

3.6

Learning Activities

Learning activities foster levels and types of interaction (e.g. student-content, student-student) that are appropriate to the course format and learning outcomes.

3.6

Learner Assessment

Learner assessment is sequenced, varied, and conducted on an ongoing basis throughout the course (formative and summative).

3.6

Learner Assessment

The learning activities and assessments are consistent with the learning outcomes.

3.5

Learner Assessment

Self-assessment activities with feedback are incorporated in the course.

3.1

Course Materials

The course contains equivalent alternatives to auditory and visual content for the vision or hearing-impaired students, such as a text transcript of a video clip, image, or animation.

2.1

Rating scale: “Strongly disagree 1  2  3  4  5  Strongly agree”.

Table 7: Student Identified Quality Enhancement Aspects Across Courses Using the QE Rubric

Category

Statement

Rating

Course Goals and Learning Outcomes

Instructions to students on how to meet the learning outcomes are clearly stated

3.8

Learner Assessment

Students have multiple opportunities to measure their own learning progress.

3.5

Course Technology

The course design accommodated the use of assistive technologies like screen readers.

3.5

Course Materials

The course contains equivalent alternatives to auditory and visual content for the vision or hearing-impaired students, such as a text transcript of a video clip, image, or animation.

3.4

Course Materials

The distinction between required and optional materials is clearly explained.

3.3

Learning Activities

Learning activities foster levels and types of interaction (e.g., student-content, student-student) that are appropriate to the course format and learning outcomes.

3.1

Learner Assessment

Self-assessment activities with feedback are incorporated in the course.

2.6

Rating scale: “Strongly disagree 1  2  3  4  5  Strongly agree”.

The table results indicate that, overall, students were less critical of the course design than the instructional designers, with the result being that fewer aspects of the categories were listed below 4.0 by students.  Instructional designers were more critical on the alignment of learning outcomes, learning activities, assessment strategies, and learning tools.  Five out of 10 statements scored below 4.0 by the instructional designers were ones that addressed issues of alignment, consistency, connection, appropriateness, and/or relevancy of course design components. The statements regarding ‘Self-assessment activities’, ‘Levels and types of interaction (e.g., student-content, student- student)’, and ‘equivalent alternatives to auditory and visual content such as text transcript to video clips’ were rated among or close to the lowest half by both the Instructional Designers and, students indicating some agreement of where the courses needed improvement.  The main improvement priorities identified by students were ‘self-assessment activities with feedback’ and ‘levels and types of interaction’, which were rated much lower than the Instructional Designers, 2.6 vs. 3.1 and 3.1 vs. 3.6, respectively, however, it should be remembered that the original instructional design request was for self-directed open educational courses without any instructor facilitation.  Regardless, the survey results indicated that the students’ expectations for feedback and interaction were higher than the instructional designers anticipated and will challenge designers to look for ways to increase feedback and interaction in such self-directed learning environments.  Also, students gave a low 3.3 rating score for the ‘distinction between required and optional materials’ category, signifying that the students' need for time management support wasn’t adequate.  This was another area not identified by the instructional designers’ scores that may require improvement to meet students' time management needs.

In follow-up interviews with four of the six students, they were asked for more detailed reasons for the low scores in these categories.  The interview transcriptions showed that students' concerns about the courses’ shortcomings, i.e., where the scores were lowest on the QE assessment, could be generalized into two areas, (1) the importance of feedback to support learning and (2) the need for the instructional design to support learning practices that economize student's time commitment. 

The importance of feedback was succinctly summed up by one student, "If you don't have feedback you can't assess your learning".  The students suggested that feedback could be from instructors, fellow students or even an automated system.  For example, an automated feedback system could include sample questions linked to sample answers or sample questions that are answered by directing students to specific content within a course, blog or textbook.  It was suggested that student-to-student feedback could be done through forums or discussion pages while instructor feedback could be given by direct responses to questions, quizzes or final exams and papers.

In the second area of student feedback, the practical aspects of organizing the course to support student time management, many students especially those actively working in the field, identified a need for clarity and definition of workloads.  As one student noted, "It is important to know what is actually necessary". For example, "clarify and specify the readings, indicate what is necessary and what is optional, indicating the most important links."

Conclusion

Conclusion and Limitations

To answer the first question of the pilot study, the project successfully created five well-designed TODE courses that satisfied students, lead professors, and instructional designers.  The team-based, course development process usually applied within a single post-secondary institution for online distance education course development was successfully adapted to create an effective transnational learning project despite the differences in language, culture, epistemological approaches and the diverse international locations of the course authors.  The survey results however, disclosed challenges that may have practical implications for similar transnational, course development projects. The results suggest that time commitment is one of the greatest challenges and, in fact, may be the greatest challenge for leading professors and content co-authors. Planned timelines will most likely need to be modified to accommodate both controlled and un-controlled factors encountered by individuals or international institutions. To have the project completed on time, the Instructional Designer needs to be firm on the ultimate target timeline while providing manageable flexibility on milestone tasks.

The survey results indicate that project coordinators and instructional designers need to continue to improve communications, i.e., providing clearer communications on processes and timelines. It may be a matter of communicating very clearly at the beginning to set expectations in terms of difficulty and time commitment.  Communications are very challenging with multiple and varied authors and co-authors and it is important to get a strong commitment to the course development process from administrators, academic coordinator, leading professors, and course co-authors. Administration issues may arise due to changing lead professors or team members. To manage the workload and the risk, one suggested strategy is to require a more team-based approach at each institution where at the start of the project, instead of a course being assigned to a single course author, the institution establishes a course content development team with some co-authoring support led by a leading professor. 

The second question about course quality is only partially answered because the surveys conducted to assess the quality of course design were limited to two instructional designers and six Forest Science students and the surveys to evaluate satisfaction with project management and instructional design support were limited to the subject matter experts. To further enhance course quality, however, it would be helpful to hear more from subject matter experts and students in the targeted regions.  The subject matter experts’ review could focus on whether or not the depth and breathe of the content for each course was at an appropriate academic level and relevant to the needs of the target audience.  To further understand the challenges of a transnational effort for course development, follow-up interviews with leading professors may reveal more insights on how to better manage a future project.

The student input was extremely valuable to provide a learner perspective on the quality of the course design to help identify the enhancement priorities.  Although generally less critical of the course design than the Instructional Designers, the students identified two major areas of concern that need immediate attention, feedback and economizing the student's time commitment.

Future Planning

Xu and Morris noted that it is the Instructional Designer who ensures that the team-based model works efficiently and the selection of a qualified Instructional Designer was pivotal to the success of the project.  The transnational nature of this project made for some exceptional demands on the Instructional Designer that required a patient but deft facilitative approach.  It was also an asset that the Instructional Designer was bilingual and familiar with some of the countries’ educational cultures and pedagogical approaches.

The successful development of the TODE courses that met the objectives of the Asia-Pacific Region’s Steering Committee as self-directed open education courses encouraged the APFNet to plan more online courses to create an online certificate SFM program involving additional universities from the Asia-Pacific Region.  Although these TODE courses can be used as a stepping stone to a credential program, much work would need to be done to meet the academic and instructional design quality standards for a certificate program.  The instructional design accommodated the objectivist approach of the course authors that focused on remembering and understanding information with some application and analysis.  For future accredited course developments, the instructional design challenge for the team would be to incorporate a more constructivist teaching approach that would support problem-solving, critical thinking, reasoning, and the active and reflective use of knowledge (Driscoll, 2000). Similarly, the current set of completed courses would need to be upgraded by the addition of engaging activities that, for example, facilitate student-to-student and student-to-instructor interactions and graded assessments that necessitate a higher order of meaningful thinking (analysis, synthesis, etc.).  The creation of a certificate from self-directed courses and new additional courses would require that the development team of experts to continue to closely collaborate with the academic review team to ensure the curriculum level meets the requirements of intellectual discourse.  The effort to design TODE courses to support high level meaningful learning will encourage the creation of SFM professionals who can think like sustainable foresters, problem solve, find creative solutions, manage their own learning, and practice good communications in the very complex world of sustainable forestry management.

Acknowledgement

The authors gratefully acknowledge the collaboration and financial support of the Asia-Pacific Network for Sustainable Forest Management and Rehabilitation (APFNet) for this transnational, online, distance education course development project.

References

Crowley, C., Chen H., & Murano, C.  (2015). Proceedings from 8th International Conference of Education, Research and Innovation: Managing the development of sustainable forestry online courses in an international collaboration.  Seville, Spain. Retrieved from https://library.iated.org/view/CROWLEY2015MAN

Driscoll, M. (2000). Psychology of learning for instruction (2nd ed.). Needham Heights, MA: Pearson Education Company.

Forestry College Deans Meeting Mechanism in the Asian-Pacific Region: Asia-Pacific Network for Sustainable Forest Management and Rehabilitation. (2013). Project document: Innovative sustainable forest management education in the Asia-Pacific Region.  Project document available upon request to Asia-Pacific Network for Sustainable Forest Management and Rehabilitation.

Kang, S. (2001) Toward a Collaborative Model for the Design of Web-Based Courses.  Educational Technology, 41(2), 22-30.

McBurnie, G., & Ziguras, C. (2001). The Regulation of Transnational Higher Education in Southeast Asia: Case studies of Hong Kong, Malaysia and Australia.  Higher Education 42: 85. doi:10.1023/A:1017572119543

Merola, R. (2017, Spring). What Does Data Tell Us about Cross-border Online Learning? International Higher Education, 89.

Oblinger, D., & Hawkins, B. (2006, January/February). The Myth about Online Course Development. Educause Review 41(1): 14-15.

Puzziferro, M., & Shelton, K. (2008). A Model for Developing High-Quality Online Courses: Integrating a Systems Approach with Learning Theory. Journal of Asynchronous Learning Networks, 12(3-4). Retrieved from http://www.anitacrawley.net/Articles/v12n3_puzzifero_0.pdf

Schrum, L. (1989). Online Education: A Study of Emerging Pedagogy. In B. Cahoo (Ed.). Adult learning and the internet. pp. 53-61.

Skidmore, M., & Longbottom, J. (2015). The future of transnational education.Retrieved from: http://www.obhe.ac.uk/newsletters/borderless_report_november_2011/future_transnational_educatio

St. Amant, K., & Flammia, M. (2012). Distance Education in the Global Age: An International Perspective on Online Learning. 2012 IEEE International Professional Communication Conference.

Toprak, E., & Genc-Kumtepe, E. (2013).  Cross-cultural communication and collaboration: Case of an international e-learning project.  Retrieved from http://www.eurodl.org/materials/contrib/2014/Toprak_Genc-Kumtepe.htm

University of British Columbia. (n.d.). Online and blended course quality checklist. Retrieved from http://wiki.ubc.ca/images/2/25/OnlineQualityCheckList_SiteFinal_Sep2016.pd

Xu, H., & Morris, L. V.  (2007). Collaborative Course Development for Online Courses. Innovative Higher Education, 32, 35-47.

Authors

Chris Crowley, Manager, Learning Design/Educational Course Consultants,  
Centre for Teaching Learning and Technology, The University of British Columbia. Email: chris.crowley@ubc.ca

Hailan Chen, Learning Design/Educational Course Consultant, Centre for Teaching Learning and Technology, The University of British Columbia. Email: hailan.chen@ubc.ca

Mercè Gisbert Cerver, Profesora, Departamento de Pedagogía, Universitat Rovira i Virgili. Email: merce.gisbert@urv.cat