ARELLO Education... · Web viewAssociation of Real Estate License Law ... In either case the Chair of the ECC will provide ... word processing files or Adobe® s that are

Association of Real Estate License Law Officials

Distance Education Certification ProgramPolicies and ProceduresUpdated December 2010

.

Copyright © 2010, The Association of Real Estate License Law Officials 1 | P a g e

1

2

34

5

6

7

8

9

10

11

12

13

14

15

17

18

1

TABLE OF CONTENTS

TABLE OF CONTENTS............................................................................................................. 2Program Administrative Policies and Procedures.......................................................3

Objectives of Distance Education Course Certification.........................................................3Eligibility Requirements for Providers Seeking Course Certification............................3Fees for Certification............................................................................................................................ 4Applying for Certification...................................................................................................................5The Course Review and Certification............................................................................................5Renewal of Certification......................................................................................................................6Appeals Procedure................................................................................................................................ 7Compliance after Certification is Obtained.................................................................................9Grounds for Suspension, Withdrawal or Conditioning of Certification......................10Course Audits........................................................................................................................................ 10

Course Review Policies....................................................................................................... 11Introduction To Course Review Policy......................................................................................111. Delivery Method Definitions..................................................................................................112. Course Design Requirements................................................................................................143. Interactivity...................................................................................................................................154. Evaluation and Assessment....................................................................................................165. Student Support Services........................................................................................................186. Other Requirements for Primary Providers...................................................................187. Course Content.............................................................................................................................218. Secondary Providers.................................................................................................................219. Instructors..................................................................................................................................... 22Addendum for Synchronous Internet Courses.......................................................................24


19

212223242526272829303132333435363738394041424344

45

2

Program Policies and Procedures

Objectives of Distance Education Course Certification

To foster excellence in professional real estate education by developing standards and guidelines for evaluating distance education effectiveness, while recognizing and respecting the diversity of courses in the real estate industry.

To promote educational and ethical standards of professional education.

To provide public assurance that real estate education offered via distanceeducation is of acceptable quality.

To provide guidance to providers in the continued improvement of the educational offering via distance learning.

To ensure that the certification process evaluates not only the presence of essential resources and processes but also the achievement of programmatic outcomes when applicable.

To encourage programs to view self-study and evaluations as a continuous obligation.

To provide feedback that will stimulate improvement of educational courses and related activities.

To ensure that the evaluation, policy and decision-making process undertaken by the certification process reflects the interests of the real estate community.

Eligibility Requirements for Providers Seeking Course CertificationEach provider applying for distance education course certification in real estate must demonstrate that it satisfies each of the general requirements listed below which taken together, define its eligibility to participate in ARELLO’s distance education certification process.

General Course and Provider Requirements:


46

47

48

4950515253

5556

575859

606162

63646566

676869

707172

7374757677

78

79

8081828384

85

3

Courses are offered within a context of a provider that has stated mission and objectives appropriate to career education.

Each student, potential student or other interested persons is provided an accurate and complete description of courses offered.

The provider admits students to courses and does not discriminate with respect to age, sex, ethnic background, color, creed, or disability.

Fees for CertificationAll fees will be paid directly to ARELLO, c/o Distance Education Certification Program. Fees may be paid by check or credit card. These fees will be in effect for the 3-year certification period. Course reviews cannot be started until the appropriate fee has been paid.

Note: all fees are effective July 1, 2006.

Primary Provider or Course Developer

$825 for the first course offered via different delivery methods $625 for the second and subsequent courses offered via the same delivery

method $470 for recertification after the three year certification period has expired

Secondary Provider or Branch School

$275 for the first course offered via different delivery methods $225 for the second and subsequent courses offered via the same delivery

method $195 for recertification after the three year certification period has expired

Other Fees

Course name change fee - $25 Primary providers that desire to change the clock hours of an existing

certified course may do so at recertification application time given new time study affidavits are submitted along with a $155 fee per course. (Note: this is in addition to the $470 recertification fee. Providers that substantially change the content of the course should submit the course as a new submission. ARELLO reserves the right to define what changes are considered substantial.)

Supplemental IDECC Certification - $75 per course. ARELLO’s subsidiary, IDECC (the International Distance Education Certification Center) offers IDECC Distance Education Certification that is very similar to ARELLO’s certification. IDECC Certification is primarily recognized in industries


8687

899091

929394

95

96979899100

101102

103104

105106107108109

110111

112113114115116

117118

119120121122123124125126127128129

4

outside real estate like appraisal. No additional paperwork is necessary for this supplemental certification. Certain restrictions apply; please contact our staff for details.

Applying for CertificationThe provider requesting course certification by ARELLO must first make application for certification. The application forms along with instructions on how to submit them are located, free of charge, at WWW.ARELLO.NET. Each course is reviewed and evaluated by the standards set forth in the ARELLO Distance Education Standards Manual. It is highly recommended that providers obtain and examine the ARELLO Distance Education Standards Manual prior to making application for certification. All applications are processed in the order they are received.

Through the application forms, the provider mobilizes its various elements to reflect on the purposes and effectiveness of their distance learning initiatives. Through this process the provider is expected to examine its distance learning courses in terms of strengths and weaknesses, and where problems or opportunities are identified, begin to work toward their solution or fulfillment.

Providers are expected to use ARELLO Distance Education Standards and these Program Policies and Procedures as the primary resource for completing their application for certification.

The Course Review and CertificationOnce the provider has paid the appropriate fee, completed and submitted the application and accompanying documentation to begin the review, the application will be evaluated by an educational consultant retained by ARELLO who possesses the necessary educational credentials to evaluate distance education courses. If a course is offered in multiple jurisdictions without substantial differences, the applicant need submit only one application for certification for that course. The educational consultant may contact a provider via the course review management system for additional information, as deemed necessary to complete the objective evaluation.

Receiving CertificationCertification is granted when a submission is in compliance with ARELLO standards for a course as determined by ARELLO and/or designees. ARELLO will not conditionally certify courses. Certification may be denied or withdrawn, as appropriate, upon a finding by ARELLO that the application does not comply with the standards.

If the review determines the course complies with standards, the provider will be promptly notified via the course management system that the course has been certified. A summary certificate listing a description of the course will be published


130131132

133

134135136137138139140141142143144145146147148149150151

152

153154155156157158159160161162163164165166167168169170171172

5

http://WWW.ARELLO.NET/

as proof of certification. Certificates are available for download in the course management system.

Handling DeficienciesOnce a course review has begun, fees are non-refundable. During the review, the reviewer may cite deficiencies in the submission for certification. ARELLO will provide two iterations of the course review to give the provider the opportunity to remedy the deficiencies. ARELLO shall afford the applicant a reasonable opportunity to address deficiencies in the application for certification. The provider will have forty-five(45) days to respond to the requests of the reviewer after which the course(s) will be withdrawn from the review process. Notice will be provided in writing via the course management system once a course is withdrawn from the review process. If the deficiencies have not been remedied after the second iteration of the review, the course will be withdrawn from the review process and fees forfeit. If the course is submitted again in a new submission, ARELLO shall assign the course to a different reviewer.

Access to the Application and Other DocumentationAccess to an application shall be restricted to ARELLO personnel involved in the certification process. ARELLO personnel shall take appropriate steps to protect the copyrights and trademarks of applicants. Upon completion of the certification or approval process of an applicant's submission(s), the identity of the providers to which a certification or an approval decision has been communicated shall become a matter of public record. However, the application material submitted shall remain restricted to ARELLO personnel except that it may be made available upon written request to ARELLO's regulatory agencies who request specific information; to any parties, such as secondary providers, who are also affected by the conditions and/or requirements; and to other parties if specifically authorized in writing by the applicant.

Denial of Certification or Recertification for Unethical OperationTo help ensure public protection against course providers that may operate unethically, ARELLO may deny certification of any course offered by a provider whose owners and/or instructors have a documentable record of breaching public trust. Breaching public trust may include but is not limited to felony convictions, and/or disciplinary action on a jurisdictional license. ARELLO reserves the right to perform limited background checks on those owners or instructors associated with certified courses. Organizations denied certification on this basis may appeal using the procedures described in these polices.


173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213

6

Renewal of CertificationPrior to the expiration of certification, it is the provider’s responsibility to file an application for a renewal of certification. Recertification for each course is required every three years.

In order to continue to use the “ARELLO Certified” credential for these courses, providers must successfully complete the recertification process for each course. Please note the following policy:

Application for recertification shall be submitted 90 days prior to the expiration of the current certification period. An application is deemed timely submitted if it is electronically submitted on or before the deadline. If ARELLO needs additional time to consider and take final action on the application for recertification, ARELLO may extend the current certification period for up to 120 days.

If the application for recertification is submitted anytime less than 90 days prior to the expiration date through the twelve month period following the expiration date, the application shall require an additional late filing fee of $100. During this fifteen-month period, certification can only be achieved by properly re-certifying the course. After the fifteen-month period, the course will no longer be eligible for recertification and must be filed as a new course.

Providers may obtain recertification information and applications at http://WWW.ARELLO.NET in the Documents area.

Appeals ProcedureDecisions made by course reviewers, the Program Director, the ARELLO Chief Executive Officer (CEO) and/or actions taken by the Program Director as a result of a course audit are appealable. Appealable action is limited to four grounds as follows:

1. ARELLO or its agents failed to consider all the evidence and documentation presented in favor of a provider’s application or audit documentation.

2. ARELLO or its agents acted erroneously by disregarding its own Standards.

3. There was a bias demonstrated by ARELLO or its agents.

4. The evidence provided to the course reviewers at the time of the decision was materially in error.


214

215216217218219220221

223224225226227228229

230231232233234235236237

238239240241

242

243244245246247248249250251252253254255256257

7

http://WWW.ARELLO.NE/

The provider must notify the Program Director in writing of its intention to petition for review of the decision no later than thirty (30) calendar days from the date the provider was notified of the adverse decision by ARELLO. The petition must relate to one of the grounds cited above and justification must be provided for the request. All documentation supporting the petition must be submitted to the Program Director no later than forty-five (45) days from the date the provider was notified of the decision. If the Program Director served as a reviewer of the course, the appeal shall be forwarded to the CEO for consideration.

The Program Director will be provided with a copy of the application, accompanying documentation, including any appended program responses, the letter of notification of the ARELLO action, and the written request which states the grounds for the appeal. The Program Director will have full access to all other documents and records filed with ARELLO and its agents during the certification process.

After receiving the written appeal, the Program Director will either uphold the original decision made by the ARELLO reviewer(s) or instruct them to take other action as necessary. The Program Director will notify the provider in writing of the decision within ten (10) days of his or her action.

If the provider wishes to appeal the decision of the Program Director , they may do so by making their request in writing to the ARELLO Chief Executive Officer (CEO) within thirty (30) days of the Program Director’s notification of the decision. The CEO shall have access to the necessary documentation and then may uphold the actions of the Program Director or instruct him to take other action as necessary. The CEO will notify the provider in writing within ten (10) days of his or her action.

If the provider wishes to appeal the decision of the CEO , they may do so by making their final appeal in writing to the Education Certification Committee (ECC) for consideration in a formal hearing.

Appeals are administrative, not-judicial, hearings.

During any hearing, the appellant unit and ARELLO shall have the following rights:

1. To be represented by a representative(s) of their choice.

2. To be present during any hearing.

3. To present oral testimony keeping within any established time limits.

4. If the provider waives personal appearance, the matter will be handled solely on briefs submitted.

The ECC’s deliberations following the conclusion of any hearing and its subsequent decision process will be in closed session.


258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303

8

All such petitions may only be heard at a scheduled meeting of the ECC. The full ECC may take one of two actions:

affirm the decision of the CEO and its educational agents, or reconsider the original decision and vote for other action.

In either case the Chair of the ECC will provide reasons for its actions and notify the provider in writing within forty-five (45) days following the meeting. If the ECC affirms the prior decision of the Program Director, CEO and its educational agents, there is no further remedy available to the provider. The decision is final.

Compliance after Certification is ObtainedOffering the Course Only As CertifiedProviders representing courses as certified to regulatory agencies or the public have the responsibility of only offering the course as it was certified. Providers are not allowed to manipulate the instructional design of the course without advanced notice to ARELLO and claim the course is certified.

Advertising of CertificationOnly after ARELLO grants certification may a provider utilize the ARELLO certification logo and so denote its certification status in official publications or advertising. Providers may advertise certified courses as “Course Instructional Design and Delivery Certified by the Association of Real Estate License Law Officials” as long as certification is current and in good standing. A secondary provider may not advertise that a course is ARELLO certified unless: 1) the primary provider has submitted the course and been granted certification, and 2) the secondary provider has also submitted the course to ARELLO and been granted certification.

Providers advertising certified courses shall ensure the delivery method, course titles and clock hours advertised are consistent with what is listed on the ARELLO certification summary certificate.

Advance Notice of Substantive Changes A provider considering substantive change(s) to a course(s) should notify ARELLO, in writing, of any proposed changes sixty (60) days in advance of the change. Substantive changes are generally defined as any changes in the course which will affect the certified instructional design and/or delivery. Changes in the course clock hours are also typically considered to be a substantive change.

Additional factors considered substantive include major content revisions, new administration and/or owners, new facilities or new locations. Additions, removals or modification of course instructors are also considered a substantive change to the course and shall be approved by ARELLO.


304305306307308309310311312313314

315

316317318319320321322323324325326327328329330331332333334335336337338339340341342343

344345346347

9

Change in Provider Accreditation, Approval or State Licensure StatusA provider of any ARELLO certified course must notify ARELLO within ninety (90) days of any denial, suspension, revocation, probation or any other final or interim adverse action affecting any accreditation status of the provider. A provider who has been granted certification must notify ARELLO in writing of any state agency or other regulatory body attempting to withdraw approval or licensure of the institution or any of its courses, programs or instructors.

Course Audits While standards and certification are positive components of maintaining quality distance education, other measures associated with enforcement of the standards are necessary to ensure courses are taught as certified.

At any point during the time a course is certified by ARELLO, reviewers may perform the following enforcement procedures, for primary and/or secondary providers that include but are not limited to:

Anonymously auditing courses. Randomly checking courses to ensure course design is consistent with

what was certified. Requesting the provider document historical information pertaining to

the amount of time courses are taking students to complete. Monitoring of advertising regarding certified courses.

It is important to note that ARELLO investigates complaints submitted by regulatory officials, students, or other providers regarding certified courses.

Grounds for Suspension, Withdrawal or Conditioning of Certification

Failure to comply with the responsibilities listed in this document shall be grounds for suspension, withdrawal or conditioning of its grant of certification from ARELLO. The sanctions against certified courses by ARELLO are appealable under the procedures listed in this document.


348349350351352353354355356

357

358359360361

362363364365

366367368369370371

373374375

376

377

378379380381382383384

10

Course Review Policies

Introduction To Course Review PolicyARELLO standards are applicable for a variety of delivery methods and education models that utilize any number of different technologies. While ARELLO is flexible in the application of its general standards in regard to a variety of delivery methods, ARELLO has found it necessary to provide course developers with additional specificity in regard to course review policy pertaining to common delivery methods prevalent in today’s market.

This policy document has been created to provide additional specificity to the ARELLO Distance Education Standards available at www.arello.net and pertains to typical delivery methods and/or technologies used in the design and delivery of distance education courses. ARELLO uses these policies in light of the standards to measure the quality of courses submitted for certification. The paragraphs below containing policy have been numbered for easy reference. ARELLO reviewers may reference this document and specific paragraphs during the certification review process.

1. Delivery Method DefinitionsARELLO has strict guidelines that constitute delivery method definitions, and certifies courses on an individual basis and by delivery method. For example, a provider has an ethics course that is delivered via the Internet and CBT (CDROM Based Training). In regard to ARELLO Certification, offering the course via the Internet and CBT would constitute two separate delivery methods thus requiring two separate certifications. Many providers choose to offer identical courses via several different delivery methods to accommodate market demands. While the list of definitions below does not encompass all delivery methods that exist, the following represent the most common delivery methods submitted to ARELLO. Some of the delivery methods below are listed only for clarification and are not eligible to be considered for certification. Please note, once a course is submitted to ARELLO, the review team will define the delivery method based upon ARELLO’s criteria. It is possible the delivery method you choose may not be described below. For more information about delivery methods not defined here, contact ARELLO.

1.1. CBT (CDROM Based Training) – CBT are asynchronous courses taught via a computer and typically delivered from a CD-ROM. CBT courses may also be “distributed” via the Internet but are not typically “delivered” via the Internet. The distinguishing feature of CBT courses is that an Internet


385386

387

388389390391392393394

395396397398399400401402403404

405

406407408409410411412413414415416417418419420421422

423424425426

11

connection is not needed to complete the course or run a substantial part of it. So for example, a course that is sent to the student on a CD-ROM and is installed and run on the student’s computer independent of an Internet connection would be considered a CBT course. Even if the student completion reporting were done within the CBT program via an Internet connection at the end of a course, the primary delivery method is still CBT. Typically, in a CBT course student progress information is kept on the student’s hard drive, not on a remote server as with Internet based courses.

1.2. Correspondence (not eligible for new certification) – Courses that use static print or static print in electronic media as the primary means by which to deliver content and assessments. Correspondence courses may consist of “computer files” that students simply print off to read and interact with. For example, a course consisting of a booklet and printed exercises that the student completes would be considered a correspondence course. In addition, word processing files or Adobe® PDF files that are used to simply distribute course material would also be considered correspondence since the purpose of the electronic file is to simply distribute the course content. Static electronic text files that are available on a web site may also be considered correspondence courses not necessarily “online courses.”

1.1. Web Enhanced Correspondence (not eligible for new certification) – Courses that combine several technologies to deliver a single course. Most often, web enhanced correspondence refers to correspondence courses that utilize print or electronic media to deliver content to the student. In addition, for interactive exercises such as quizzes or exams, web or other technologies may be used. For example, a course developer provides students with documents in which they could download from a web site and read as the primary method of content distribution. So far, this would be considered a correspondence course. However, the provider adds some interactive quizzes students must take on the Internet, thus adding another media to the instructional design strategy. This then would constitute a mixed media course.


Regarding courses offered via “correspondence” and/or “web enhanced correspondence” delivery methods:

a. ARELLO will cease to certify new courses using these delivery methods after September 30, 2009.

b. Recertification of courses certified via these delivery methods will be continued after September 30, 2009, but shall not have an expiration date that surpasses September 30, 2012. ARELLO will prorate recertification fees for courses not being allowed the full three year certification term. Providers that need to have recertification fees prorated should contact ARELLO prior to making the submission for recertification.

427428429430431432433434435

436437438439440441442443444445446447448449450451452453454455456457458459460

461

12

1.2. Asynchronous Internet – Courses that require an Internet connection to complete. Internet or web-based courses are typically facilitated by web technologies that use a web browser as the primary means of content delivery and providing interactivity. Typically, Internet-based courses are asynchronous and store student progress information in a database housed on the provider’s server. Internet courses can typically be run from any computer having an Internet connection and a web browser. In an Internet based course, the content is delivered to the student in logical increments and assessments are performed at appropriate intervals using Internet or web technologies. Asynchronous Internet courses make up the majority of courses submitted to ARELLO for certification.

1.3. Synchronous – Courses that require students to be engaged at a particular time via any number of different technologies including the Internet, Satellite or other telecommunications device. Synchronous courses typically require a third party technology facilitator such as WebEx™ or GoToMeeting™ among others. Synchronous courses offer unique capabilities and challenges for educators when compared with asynchronous delivery methods. Synchronous delivery platforms can provide educational experiences similar to those in a classroom environment. For more information synchronous Internet courses, see the Addendum for Synchronous Internet Courses.

1.4. Considerations Regarding Course Delivery Method DefinitionsStatic computer files like Adobe® PDF or other word processing files consisting of course content cannot be placed on a web site for download and called an “Internet based” course.

Courses that appear to have identical interfaces but utilize different technological infrastructures should be considered two different courses when submitted to ARELLO. This is often the case for providers who wish to offer both CBT and Internet based delivery methods for a single course to accommodate students regardless of whether or not they have an Internet connection.

Delivery method definitions take into account how courses are “delivered” as opposed to how they are “distributed.” For example, many providers may make their CBT courses available for download on a web site. This does not make the course Internet based in that the “delivery” typically does not require an Internet connection even though the method of “distribution” may or may not.

Typically, when a provider replicates course features using different technologies to accommodate market demand for a single course, this equates to utilization of different delivery methods. Therefore, in regard to ARELLO Certification, two different delivery methods would require a unique


462463464465466467468469470471472473

474475476477478479480481482483484485486487488489

490491492493494495496497498499500501502503504505506507

13

certification for each “set” of technologies used to offer a particular course. For example, a course provider creates an Internet course that a student runs from a centralized web server at the provider’s office. The provider then creates a course that is identical in the way the course looks but utilizes different technologies so the course can be run from a CD-ROM and without an Internet connection. In this case, the provider has used two different base technologies for offering the same course content. Therefore, since ARELLO also examines the technological layer of each delivery method, the provider would have to certify both delivery methods. In this case, one would be CBT and the other Internet.

2. Course Design RequirementsDepending upon what type of distance education course you are developing there are several requirements concerning course design. This section examines several requirements for typical asynchronous Internet and CBT based courses. If you have questions concerning a particular course design, contact ARELLO to find out specifically how the standards will be applied in the review process for certification.

2.1. Course Navigation (Applicable to CBT and Internet Based Courses)When regulators examine the credibility of distance education courses, one of the first items they consider is the completion time of the course. If a course offered for eight clock hours takes 30 minutes for the typical student to complete, the course essentially has little credibility with ARELLO or regulatory agencies.

Therefore, Internet and CBT courses should control navigation through course content to ensure the student masters the material on a unit-by-unit basis. For example, if a student begins unit one of a course, he should not be able to navigate to unit two until he has successfully demonstrated mastery of unit one by some means of assessment like a quiz.

Designing courses within these parameters accomplishes several things: 1) helps ensure the student masters the material in the course. 2) Helps ensure the completion time is appropriate for the number of clock hours the course is offered for.

2.2. Course Design Criteria (Correspondence Courses)Correspondence courses in their traditional form have inherent issues that cause concern for regulators. In some cases, regulatory agencies do not accept traditional correspondence courses due to the lack of required interactivity. While providers are required to implement incremental assessments within correspondence courses, these assessments are often not


508509510511512513514515516517518

519

520521522523524525

527528529530531532533

534535536537538539540541542543544545

546547548549550551

14

presented in a form that requires the student to complete them or provides mandatory remediation. Despite certain inherent issues with navigating though a correspondence (paper and pencil) course, many regulatory agencies feel these courses do provide a legitimate learning potential and therefore approve courses via this delivery method for credit.

Regarding courses offered via “correspondence” and/or “web enhanced correspondence” delivery methods:

a. ARELLO will cease to certify new courses using these delivery methods after September 30, 2009.

b. Recertification of courses certified via these delivery methods will be continued after September 30, 2009, but shall not have an expiration date that surpasses September 30, 2012. ARELLO will prorate recertification fees for courses not being allowed the full three year certification term. Providers that need to have recertification fees prorated should contact ARELLO prior to making the submission for recertification.

3. InteractivityEducational research indicates that interactivity is a critical part of the learning process. Therefore, ARELLO requires courses have learning strategies that provide interactivity and that those strategies are measurable in terms of their success. There are three types of interaction: a) student-content, b) student-student, and c) student-instructor.

3.1ARELLO requires the availability of student-instructor interactivity and also requires acceptable levels of interactivity in regard to student-content.

3.2While student-student interactivity is welcomed and recognized as a valuable learning strategy, it is not required.

3.3ARELLO requires reasonable levels of student-content interaction. Student content interaction is evaluated on a course-by-course basis by reviewers. Examples of student content interaction include but are not limited to, hyperlinks to vocabulary words, links to supplemental reference material, quizzes, final exams, and remediation exercises.

4. Evaluation and Assessment4.1. Pretests

Pretests are administered to measure the student’s knowledge prior to taking the course. By the end of the course, a clear measurement of what


552553554555556557558

559560

561

562563564565566567568569

571572573574575576577578579

580

581582583

15

has been learned can be provided to the student by comparing pre and post assessments. Pretests are not required for ARELLO Certification. However, pretests do serve as interactive elements that can be attributed to a course. A pretest may also be a worthy learning strategy to implement in order to extend the time it takes to complete a course and meet clock hour requirements. Implementing pretests solely as a way to extend course time is not recommended. Any implementation of pretest should be integrated with other learning strategies such as post-tests to ensure they are purposeful and accurately provide some measurement of student performance.

4.2. Incremental AssessmentsIncremental assessments are typically considered to be quizzes given after each logical unit of instruction. Incremental assessments are required at appropriate intervals throughout the course to obtain certification. These assessments should be properly designed to ensure questions are not too easy and are not repetitive. Incremental assessments should be designed to properly measure whether or not mastery of the material has been achieved. ARELLO will not quantify a preferred number of assessment items. ARELLO will look for a “reasonable number” of questions to reflect the material presented and more importantly that the objectives have been appropriately measured. For example, if a particular unit takes the typical student thirty minutes to complete and is fairly complex in nature, it is doubtful five true or false questions would adequately measure competency. Instead the provider may more appropriately choose a higher number of multiple choice questions for the incremental assessment. ARELLO will examine the number and type of assessment items to ensure the instrument appropriately measures the course objectives.

4.3. Summative AssessmentsSummative assessments are typically considered final exams to be administered at the end of a course to ensure mastery over all the material presented in the course. As with incremental assessments, summative assessments should be properly designed to ensure questions are not too easy and are not repetitive. Summative assessments shall not consist of identical questions posed in the incremental assessments. In other words, final exam questions shall not consist of questions previously posed in a quiz. Summative assessments should be designed to properly measure whether or not mastery of the material as been achieved. ARELLO does not quantify a preferred number of assessment items. ARELLO will look for a reasonable number of questions to reflect the material presented and more importantly that the course objectives are appropriately assessed.

Summative assessments are highly recommended but not required for ARELLO Certification because a number of regulatory agencies do not require final exams for course approval.


584585586587588589590591592593

595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629

16

4.4. RemediationRemediation takes place during the process of an assessment. Remediation involves providing the student, when a question is answered incorrectly, with some indication of why their answer was incorrect and what the appropriate answer should have been. Remediation should be implemented as a part of the required incremental assessments. Remediation may not be considered vital on summative assessments or final exams. Course developers should ensure remediation strategies do not over simplify the course to where student are simply “given the answers.” ARELLO reviewers will evaluate remediation strategies on a course-by-course basis.

4.5. Assessment BanksAssessment banks allow assessment (quiz or test) questions to be presented from an appropriately sized “pool” of potential questions. For example, for every 10 quiz questions, there may be 30 potential questions within the bank. Implementation of an assessment bank within a course provides an excellent instructional strategy to ensure students do not simply memorize questions and answers. When using an assessment bank, no assessment is identical to one a student previously completed. Providers should ensure that questions in the bank provide adequate distribution across objectives being measured. For example, if there are 10 course objectives for a unit and 30 questions in the bank for the unit quiz, there should be 3 questions developed per objective. Questions should be proportionally implemented for the number of objectives being measured. Providers shall submit all assessment banks with answers to ARELLO for the review process.

ARELLO recommends that in addition to assessment banks, if the student could theoretically experience the same questions twice, that when multiple choice options are employed, the answer options appear in random order.

Required Implementation of Assessment Banks or Similar ProcessEffective after September 30, 2009, incremental (quizzes) and summative (final exam) assessments shall be developed using technologies that better ensure objectives have been achieved. This will include the appropriate use of “assessment bank” systems or similar process that better ensure integrity of the assessment. For example, if a student is asked a question in an assessment and fails to answer the question appropriately, the system should reassess using different assessment items that cover the same objective. ARELLO recommends, using a “three to one” ratio of questions stored to questions presented. Providers may also choose other strategies such as implementing multiple versions of an assessment where the entire assessment is replaced with a new one in the event the student does not succeed on initial attempts.


630631632633634635636637638639640641

642643644645646647648649650651652653654655656657658659660

661

17

4.6. Assessment QualityCourse developers and instructional designers should be familiar with Benjamin Bloom’s taxonomy for categorizing the level of abstraction of questions that commonly occur in educational settings. Bloom’s taxonomy provides a useful framework in which to categorize assessment items. Ideally, and unless course objectives state otherwise, instructional designers should design assessment items that incorporate a variety of questions that would range from the knowledge level on through the application, analysis, synthesis, and evaluation levels. ARELLO reviewers will evaluate the quality of assessment items in light of the course objectives and other learning strategies found in the course.

5. Student Support Services5.1. Course Syllabus and Orientation

Providers must give students a course syllabus or student manual during the orientation for the course. Items in syllabus or student manual should include but be not limited to:

Instructor contact information for instructional support. How credit will be reported to regulatory agencies if applicable. Technology requirements or prerequisite skills. Links to the schools policies on the cheating or misrepresenting

student identity. How much time the student has to complete the course. Refund polices.

6. Other Requirements for Primary Providers6.1. Manual for Secondary Providers

ARELLO requires primary providers or course developers that use secondary providers as a means of distributing and instructionally supporting courses to develop a reference manual for secondary providers.

The secondary provider’s manual should contain but not be limited to the following when applicable:

Instructional strategies for the secondary provider that may enhance the course.

Information on how to access administrative features of a course used in monitoring student progress.


662

663664665666667668669670671672673674

675

676677678679680681682683684685686

688

689

690691692693694

696697

699700

702703

18

Suggestions on how to offer timely and quality instructional support.

Information pertaining to course design and delivery that may be necessary for secondary providers to obtain prior to receiving ARELLO Certification. For example, a manual might include comments on the course instructional design that would help secondary providers understand how to answer the necessary question on the secondary provider application form for ARELLO Certification.

Technical support information.

6.2. Time StudiesNote: time studies are not necessary for courses taught live or completely by synchronous delivery methods. These delivery methods include but are not limited to live satellite broadcasts, live video conferencing, or live web casts using streaming media. Even though time studies are not required for courses using these types of delivery methods, a timing analysis of the course syllabus or schedule of what is taught when and for how long is necessary to document that the course will take the appropriate amount of time to complete.

Time studies are required to help ensure the average completion time of a course is appropriate for the number of clock hours the course is being offered for. Two methods are available for providers to meet the time study requirement.

6.2.1. The provider can have 10 individuals of an appropriate sample audience complete the course and fill out the appropriate Time Study Participant form available at ARELLO.NET. Please note students must complete the affidavits provided by ARELLO. These reports received in any other manner will not be accepted.

6.2.2. The provider may have two qualified professionals, which are experienced with course content and instructional design, complete the course and attest to the appropriate time. These professionals must complete the Time Study Participant form located at ARELLO.NET.

6.2.3. The primary difference between these two options is obviously the number of participants that would be involved in the time study. ARELLO prefers “10 individuals of an appropriate sample audience” because it represents a broader sample that would better verify the appropriate number of clock hours. It also represents the best method a provider can assure the course time is appropriate.


705

707708709710711712

714715

716717718719720721722723724725726727728729730

732733734735736737

738739740741742

744745746747748749

19

However, ARELLO realizes having ten individuals taking lengthier courses, like prelicense or other qualifying education courses, can be cost prohibitive and therefore allows for the “two professionals” option.

During the course review, ARELLO reviewers conduct their own time analysis to ensure the integrity of the provider’s time study. If ARELLO finds that its time study substantially differs from the one provided by the course provider, one of two actions will occur: a) The course will be certified with a reduced number of clock hours.b) Certification for the course will be denied.

6.2.4. ARELLO will not allow providers to resubmit courses that are lacking the appropriate clock hours multiple times. ARELLO does attempt to allow providers with instructional design issues identified during the review process to address those issues and resubmit the course for no additional fee. This is not the case for courses that do not take the appropriate amount of time to complete. ARELLO course reviewers reserve the right to reject time studies not validated by their own and require a provider to resubmit time studies using the “10 individuals of an appropriate sample audience” approach.

6.2.5. ARELLO as most regulatory agencies recognize an “instructional hour” as 50 minutes. So a “3 hour” course should take the average student 150 minutes to complete.

6.3. Required Learning Management Time Tracking Functionality

6.3.1. Primary providers of courses offered via asynchronous Internet or CBT delivery methods should have a learning management system in place that quantifies the amount of time students spent in a course. This includes time reading content, taking assessments, and participating in other instructional activities. This is not “mandating seat time” for each student but documenting students, on average, are spending the appropriate amount of time in a course. Providers will be required to show evidence that their LMS has time tracking and reporting capability during the (re)certification process. Student affidavits or “personal testimony” affirming course time will no longer be accepted at recertification as a means of documenting that a course is taking students the appropriate amount of time to complete. ARELLO shall accept student testimony on the “Time Study Participants Form” for ensuring time as a part of the “time study” process during the initial review for certification only.


750751752753754755756757758759760761762763764765766767768769770771772773774

775776777778

20

6.3.2. If the course is offered via secondary providers, the secondary provider will need to quantify the time students are spending in the course for recertification. Primary providers shall ensure secondary providers have access to reports in the learning management system that allows them to view how long their students are taking to complete a course.

7. Course ContentARELLO does not as a matter of practice review course content but only instructional design and delivery. Content is often unique to individual jurisdictions and it would be cost prohibitive for ARELLO to maintain content experts for all the jurisdictions that recognize ARELLO Certification. However, during the course review for instructional design and delivery, if content issues become obvious to members of the review team, ARELLO will address those as part of the review process. ARELLO may also take into consideration the difficulty level of the course objectives and assessments as they pertain to course time. Courses with higher level objectives and assessment items may be certified for more time that those that are offered at a lower level. ARELLO reviewers use Bloom’s Taxonomy as a gauge for evaluating the difficulty of objectives and assessment items.

8. Secondary ProvidersA secondary provider obtains a course from primary provider. They also have approved instructors and are responsible for monitoring student progress and verifying course completion and student identity. Secondary providers provide opportunities for interaction, support, administrative communications, and assessment or evaluation feedback. During the process of reviewing an application for secondary certification, ARELLO reviewers will evaluate and document the secondary provider’s plan to perform the associated responsibilities. Secondary providers may also be the subject of audits by ARELLO to ensure the course is taught as certified. More information about requirements for secondary providers can be found in the Secondary Provider’s Application for Certification available in the Documents area of www.arello.net.

9. Instructors9.1. All ARELLO Certified courses must have an instructor regardless of

the delivery method. Students taking distance education courses require instructional support that can only be provided by a qualified course instructor. Instructors may also play an important role in monitoring student progress.


779780781

782

783784785786787788789790791792793794795

796

797798799800801802803804805806807808

809

810811812813814

21

9.2. As of January 1, 2009, instructor(s) associated with distance education courses being submitted for ARELLO (re)certification will be required to have the CDEI™ designation. Instructors that have not earned the CDEI™ designation will not be permitted to be listed as the instructor for an ARELLO certified courses. ARELLO requires a qualified instructor to be associated with every certified course. Education providers should double check with their course instructors to ensure they are maintaining the CDEI™ designation. The CDEI™ requires renewal every three years.

9.3. Instructor credentials must be submitted with the application and reflect experience and expertise in the subject matter and the distance delivery. If instructors are changed or added, the credentials must be submitted and approved by ARELLO before they can officially teach or instructionally support the course.

9.4. A part of the instructor’s responsibility is to monitor student progress in the course. On-line courses should have a course/learning management system where the instructor monitors student progress. In the case of other delivery methods such as correspondence and CBT, the school and the instructor must have a system in place to check on student progress at a minimum of every 30-days. This can be done via mail, e-mail or telephone. See the ARELLO-IDECC Standards and Resource Guide for more information.

Summary

This policy document is subject to change and does not pertain to all delivery methods. Delivery methods such as satellite broadcasting, or other delivery methods not commonly used by course providers seeking certification may have additional requirements not mentioned here. All courses are subject to the general ARELLO Distance Education Standards.

If you have questions pertaining to this policy document, please contact ARELLO at 334-260-2928 or email [email protected] for more information.


815816817818819820821822823824825826827828829830831832833834835836837838839840

841

842843844845846847848849850851852

22

mailto:[email protected]

Addendum for Synchronous Internet Courses

Introduction

The purpose of this addendum is to clarify how ARELLO’s Distance Education Standards apply to synchronous Internet or “Webinar” courses. It is important to clarify that most of the policies provided in this document also apply to synchronous courses. For example, synchronous Internet courses like asynchronous ones must:

Provide an appropriate orientation to the student. Assess the learning at appropriate intervals throughout the course. Use assessments that measure objectives etc. Provide remediation strategies where necessary to ensure mastery. Provide adequate amounts of interactivity. Accommodate the availability of timely instructional support by a

credentialed instructor. Etc.

Synchronous Internet courses offer unique capabilities and challenges for educators when compared with asynchronous delivery methods. With the cost of facilitating synchronous courses decreasing, more education providers than ever are considering synchronous delivery methods. Education providers that have strong classroom programs are especially considering the advantages of synchronous education because they can extend their live classroom programs to students at a distance. Of all distance education delivery methods, synchronous distance education is often viewed as being closest to the live classroom experience.

How ARELLO Standards Apply To Synchronous Internet Courses

Listed below are some specific issues with ARELLO certification and how synchronous Internet courses differ from asynchronous.

1. Time studies – are not required for ARELLO Certification in synchronous Internet courses. A synchronous Internet course is offered live requiring the student to be online at a particular date and time. ARELLO will require a detailed timed outline of the course content, course materials such as books, handouts, and PowerPoint® presentations to ensure that there is enough content to justify the clock hours requested. As with a live classroom course, instructors must document how they will use the allotted time to help students achieve the course objectives.

2. Assessments – are required at appropriate intervals of the course. The course provider must submit the assessments and a detailed plan on how


853

855856

857858859860861

862863864865866867868869870

871872873874875876877878879880

881882

883884885

886887888889890891892893

895896

23

they will use the technology to administer them. The plan submitted must meet minimum standards. ARELLO reviewers are familiar with almost every major platform for offering synchronous education. The review process for certification will document that the provider is capable and familiar with the features of the technology they intend to use.

Also in synchronous courses, course instructors can assess and remediate the class or cohort as opposed to solely the individual. For example, in asynchronous courses, when a student is given a quiz, the quiz score is attributed to the individual student. In an asynchronous environment, it is far more difficult to provide individualized instruction because of the constraints of time. In the synchronous environment, an instructor can “quiz” or “poll” the individuals in a class with a specific question. The instructor can then examine the results of the quiz and determine where remediation, if any, may be needed in context of the whole class. If a class of 30 students answers a quiz question and the instructor sees through the technology used that 10 students answered the question incorrectly, the instructor would know to remediate appropriately and would do so with the whole class. In contrast, if the instructor sees that only one student answered the question incorrectly, the instructor may bypass remediation at their discretion and/or give the student that answered the question incorrectly the ability to remediate outside of “class.” Other approaches may be used in assessing students with synchronous technologies. The example listed here represents a common methodology for assessing students but course developers are not limited to this approach.

Note: Regulatory agencies charged with approving professional education often require summative assessments (final exams) at the end of a course and require the student achieve a minimum score. This can be one of the major challenges of using a synchronous delivery platform. If the education provider is required to implement a final exam as a part of the course, then they will have to work out the details of measuring individual performance on a final exam with their application vendor. Some vendors have this capability while others do not. Care should be given in choosing a technology vendor that can accommodate measurement of individual performance on a final exam. In some cases, course providers have implemented “fax in” final exams or other technology systems of their own outside of the synchronous delivery environment to accommodate individualized testing, scoring, and reporting.

3. Instructor Training – is vital in a synchronous Internet course. Instructors are responsible for using technology to present content, while assessing, remediating, and completing the course within time constraints. ARELLO requires that course instructors and facilitators document the training they have completed that has prepared them to use a specific synchronous technology platform. ARELLO reviewers may also request the education


897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935

937938939940941942

24

provider verify competency by demonstrating the features of the course with an ARELLO reviewer.

A course “instructor” will always be required in synchronous Internet courses. In some cases, ARELLO may require or recommend an additional course “facilitator.” The “facilitator” would focus on the technology and helping the instructor where necessary. The instructor would focus on content delivery and answering questions. Whether the “facilitator” is required for ARELLO Certification will be considered on a case-by-case basis. A good rule of thumb to consider is, as the number of students in a course increase, so does the justification for a facilitator in addition to the instructor. Facilitators (or teaching assistants) do not have to be content experts as long as their role is limited to administrative functions.

4. Interactivity – is required in all certified courses. Quizzes, polls, chat, whiteboard annotations, document sharing, cooperative learning exercises with other students are all strategies that are available in most synchronous learning platforms. While certification is not contingent upon the use of all these strategies, course designers are encouraged to think beyond rote lecture and quizzes.

Instructors are also required to document how they will use interactivity to ensure students are present and engaged in the course. With most quality synchronous platforms, it is simple to include interactive strategies that make it very difficult for a student to login and “walk away.” Instructors should have procedures in place to deal with students that do not participate in course activities. These procedures should be communicated in the course orientation.

Summary

Synchronous Internet courses can be very similar to the classroom experience if:

The technological platform has features that allow instructors to interact with and assess students in different ways.

The instructor is well trained on the delivery platform, and has clear objectives on how and when to use its features.

Synchronous courses that have historically not met ARELLO standards include those that were:

Rote lecture with no assessment or with only a final exam. For ARELLO certification, incremental assessments are required, final (summative) assessments are recommended but not required.

Not designed to meet or appropriately measure the objectives.


943944945946947948949950951952953954955

957958959960961962963964965966967968969970971972

973974

975

977978979980981982983984985986987988

25

Finally, it is often the aspiration of education providers to record a synchronous course and allow students to access it at a later date asynchronously for credit. In this case, ARELLO would consider the live and recorded versions to be two different delivery methods and would thus require two different certifications. The instructional design of the course and procedures for assessment and remediation dramatically change between synchronous and asynchronous delivery methods. It is recommended that course providers that have questions regarding synchronous delivery methods consult ARELLO prior to developing the course.


989990991992993994995996997

26