Introduction

Based on the SRTO, an auditor can audit a site only by looking at paperwork and evaluating it for compliance. However, that is a poor indicator of program quality.

A better approach also includes:

• Onsite audits wherever possible. (The fee structure was changed to acommodate it.) The exception is online studies, where a physical site is not relevant but full online access is essential.
• Training people in the requirements, which might also count as their PD. For example:
  • explaining the purpose of relevant parts of the SRTO,
  • explaining key forms, some of which are quite complex,
  • discussing implementation, and
  • overseeing people check (or write) documents and fill in forms.
• Interviewing stakeholders. These are staff and students, and may include industry if the program is in-house training. This gives a picture of program effectiveness and whether implementation matches the paperwork.
• Observing some lessons and looking at students' work to get an idea of the effectiveness of the teaching-learning process.
• Checking whether each program fits its qualification level.
• Evaluating the effectiveness of the program for achieving its goals.
• Asking staff whether their systems work.
• Asking for their ideas for improvements.
• Pointing out changes that will be necessary in the foreseeable future, even if they are not needed now.
• Meeting some students.

This particular audit approach is designed specifically for ACAS and its series of forms. Processes might look quite different In a more centralised RTO or an all-on-one-site institution.

 

Learning the system

ACAS central office appoints auditors. (ACAS program directors may not do so.)

To be able to act as auditor, you will need to do the following:

1. Be familiar enough with the standards to be able to explain them to somebody else. The standards are simple, but sometimes need interpreting for a particular context. (You will also find the standards have many shortcomings and some unnecessary redundancies.)
2. Be familiar with current ACAS procedures.

You will also need to develop several particular attitudes:

1. Willing to learn and improve. The more you use the process and standards, the more you will understand them.
2. You can be as tough as you like as long as you are fair and honest.
3. You need to be flexible enough to accept that programs find different ways of complying. In fact, fitness for context is often required, and one size fits all is often noncompliant.

As the auditor, you still need to wear the policeman role to ascertain compliance, but you can also act in a helping role so that people will feel that you have suppported and helped them. In fact, part of an auditor's role is to promote improvement. Be warned that people will probably ask you to do it for them. Think of your role as a teacher. You can explain, ask questions, and give examples, but the final work must be the students' own work.

As auditor, the purpose of the audit is to reach accurate, defensible conclusions based on the evidence. Consider the two extremes, both equally unsatisfactory. The fault-finding expedition means that you feel you've failed if you haven't found something wrong. At the the other extreme is the encouragement tour, where the auditor is willing to forgive anything so people will feel good about the audit.

Encourage auditees to aspire to excellence, within the limits of practicality. They should not feel that it is enough to do the minimum amount of work to just fall over the line.

Some terminology

Principal auditee: The person representing the program being audited, whether it is a qualification, a site, or a department. The person is usually the head of the program, but may be someone else authorized to be principal auditee.
Training and Assessment Strategy (TAS): A long form that covers program planning and related compliance requirements.
Approval to deliver: The staff form that documents which units that each staff member has the skills and qualifications to instruct.
Evidence: Information, materials, or products that show whether or not a student has met the competencies of the unit. For example, an instructor could observe a student, make observation notes, and record an assessment result.

The role of policies and procedures

ACAS policies and procedures (P & Ps) are intended to ensure compliance with the standards, so simply following ACAS policies and procedures should be enough to ensure compliance. This creates the following tasks for auditors:

1. Is the audited program complying with ACAS P & Ps?
2. Are P & Ps efficient in ensuring compiance? For example:
1. Do they keep the amount of time, effort, and record-keeping to a reasonable minimum?
3. Are ACAS P & Ps effective in ensuring compiance? For example:
1. Do they ensure compliance or do they leave gaps and unacceptable risks?
2. Do they work in all situations?

About the mapping

The guide contains mapping to the SRTO. The mapping has various meanings:

• Demonstrates compliance at the site for that requirement.
• Only partially relevant, e.g. the compliance burden is shared with ACAS centre.
• Might or might not apply to the site, e.g. Is anybody teaching under supervision? Are there cases of transfer credit (national recognition?)
• Collects evidence of implementation.

Some parts of the SRTO do not apply to ACAS at all, such as teaching TAE qualifications. Other parts are done completely by ACAS centre and do not apply to sites.

 

Planning

The first step is to contact the head of the program and set an audit date. Brief them on what will be involved and give them a link to What's an audit?.

If you will need an interpreter, make sure that they will provide to be present for the entire audit.

Can they all come?

What will be done to make sure everybody comes and who will do it? What will you do if some people can't come? At what point will you cancel the whole meeting because not enough people can come?

Suggestion: Prepare an attendance roll with spaces for name, basic contact details and signatures. Get people to fill it in as they come in. You can also use it as a Professional Development record and issue attendance certificates to attenders.

Plan your meetings

Plan your meetings beforehand. For example, you might want to re-arrange the sequence of items so that it is easier for people to follow and understand.

Which parts of the whole audit do all teaching staff need to attend? Which parts don’t they? This will affect the order of reviewing items. The recommended approach at this stage is that all of them come for everything.

However, you might decide on another approach, for example:

• Principal auditee only:
  • Signed form for the latest version of the auspice agreement
  • All advertising literature, including micro-ads and documents for anybody appearing in ads.
  • Admission checklists
• All instructors and assessors:
  • Training and Assessment Strategy (TAS)
  • Course information for students
  • Staff expertise
  • Delivery materials
  • Equipment list
  • Assessment materials
  • RPL assessment tools
  • Validations
  • Program review and improvement

It also depends on the staff. Even if some items are not relevant to most staff, they might need to learn the compliance requirements or how the audit process works. Most staff are either unaware of them or have only learned one particular compliance system. If the site is subject to a franchise agreement, you might speak with the principal auditee alone to sign the latest version of the agreement. Then again, you might want staff to understand the terms of the agreement.

You might find that, at some sites, they only want to send administrators. This is not a good option. At some points, their input is absolutely necessary. More than that, the purpose of this approach is partly to educate more widely on compliance obligations.

About assessment validations

The validation meeting might require personnel other than regular instructors. The SRTO says that validation is to be done by one or more persons who are not directly involved in the particular instance of delivery and assessment of the training product being validated, and who collectively have ... vocational competencies and current industry skills relevant to the assessment being validated; ... current knowledge and skills in vocational teaching and learning; and ... the training and assessment credential .... Your ideal solution is to do it as part of the whole workshop if you can.

About industry consultation

Industry consultation can be done at different levels:

• Done at each site for each qualification. This is ACAS default, because most sites have their own constituency.
• Done by the centre and applies to all ACAS sites. This works well if the purpose is that graduates will be employable anywhere in the industry.
• Done on a client-by-client basis. For example, management qualifications might address different needs for different clients.

The meeting will cover the report on industry consultation. If you are doing the consultation for that site, you have several options:

• Invite the industry person to the meeting and discuss expectations.
• Do the consultation beforehand and discuss how it will be reported.
• Discuss how you will do and report the consultation, and do the actual consultation afterward.

 

Preparation: New programs

In a new program or a program with many new staff, decide whether giving them a set of policies and forms as pre-reading would confuse them rather than help them.

It will help if you don’t have to plan everything from scratch during the audit. Although a new site should already have Cert IV trained personnel, you cannot assume that they know enough to plan a course. However, they at least they should have a clear idea of what they want to do, so get them to discuss and decide on these things before the meeting:

1. What qualification do you want to offer?
2. What elective units do you want to offer in that qualification?
3. The trainer/assessors
4. How it will run:
   1. Which units will you offer?
   2. Who will teach and assess each unit?
   3. How will the schedule work? (E.g. How often will you meet each week, and how long will meetings be?)
   4. Who will be their students? How many students do they expect?
   5. How will they market it? What marketing materials are they planning?
   6. Discuss the job description(s) that you want to train people to do.
   7. What facilities and equipment do you have? What facilities will you need?
   8. Will you use ACAS materials for teaching and assessment?
   9. Pre-read the main ACAS forms and write down any questions that you might have.:
      1. Read the Training and assessment strategy form.
      2. Read the staff from (called “Approval to deliver”)
      3. Get all core staff to read ACAS policies, as much as they are interested, and write down any questions that they might have.

Go through the risk management page. If they are high risk for items colored red, they probably will not pass the audit. However, they might still want guidance on the way ahead.

Other preparation:

1. Arrange an interpreter if the meeting will not all be done in English.
2. Get people to bring original documents of their qualifications and experience.

 

Preparation: Continuing programs

In an ongoing program, staff should already be familar with ACAS systems, and the site will already have most or all the required documents. The task of the audit will then mostly be checking and updating them. Inform people what documents to bring:

• Hard copy originals and photocopies of qualifications and other credentials
• All advertising literature, including micro-ads
• Admission checklists
• Course information for students: qualification level and unit level
• Delivery materials
• Equipment list
• Assessment materials
• RPL assessment tools
• A sample of assessment records
• Signed permission statements from any people depicted in publicity documents or website.

 

The room

I suggest sitting people around tables, probably with coffee and tea available in the room. One table might be enough for a small group.

Use a data projector to put the main points and key questions on a screen. That means you'll have to prepare your screen content.

It’s common to appoint a scribe and a discussion leader for each table, but watch the dynamics. Some scribes write everyone else’s thoughts down, while others dictate all answers. Groups might need time to reflect and edit responses so they are not just a list of brainstorm ideas.

Collate contents on the day and copy them into the relevant forms. Don’t let people go home with a promise to email them later because some people will forget. They could email items at the time or put all notes on a thumb drive for you.

Prepare each item

You will need to explain the purpose of the item (Why are we doing this?), state what you want them to do, answer or discuss their questions, and then get them to produce answers..

For each item during the audit below, explain the SRTO requirements and rationale, and answer any questions. For each item, you will probably need to clarify who is the intended audience. Some of them are for instructors/assessors, some are for students, and some are for administrators. And a few things are only for ASQA auditors to read, and serve no other known purpose.

It is helpful to discuss possible examples, but, if you provide the examples or your examples are too few, beware that people might simply copy them.

About compliance requirements

Decide how much you will orient people to the text of the Standards. Many trainers and assessors are are unaware of compliance requirements and reasons behind them, so it could be a good opportunity to teach them.

• If your group is new, then people will probably be unable to absorb extra information about the Standards in the time available; it might make them feel overloaded and switch off from everything.
• If your group is not new and they are only updating documents, then it could be very helpful for them to see the actual standards upon which ACAS procedures are based. Some might find it boring, while others will ask questions like: Are you sure that's what it means?, Why do you do it the way you do? and Why don't you do it this way ...? Beware that you might get so many questions that your schedule will go overtime and you will have difficulty keeping up.

 

Schedule

This schedule is intended to put most of the audit into a one-day workshop, and might be too rushed for many sites. Most sessions are only 45 minutes, and there is no time for interviews on implementation. It might be most difficult for schools as their schedules are often quite rigid.

9.00 – 9.30 a.m.	Session 1.	Introductions, goals for the workshop, any housekeeping announcements, entry meeting, signed form for the latest version of the auspice agreement, grand tour. .
9.30 – 10.15 a.m.	Session 2.	Training and Assessment Strategy (TAS).
10.15 – 10.30 a.m.	Break.	☕.
10.30 – 11.15 a.m.	Session 3.	Information for the public, Course information for students.
11.15 a.m. – 12.00 noon.	Session 4.	Staff expertise.
12.00 noon – 12.30 p.m.	Lunch.	🍴.
12.30 – 1.15 p.m.	Session 5.	Delivery materials, Equipment list, Admission checklists.
1.15 – 2.00 p.m.	Session 6.	Assessment materials, RPL assessment tools.
2.00 – 2.45 p.m.	Session 7.	Assessment validations, sample of assessment records, retention of assessment evidence.
2.45 – 3.00 p.m.	Break.	☕.
3.30 – 4.15 p.m.	Session 8.	National recognition, Program review and improvements.
4.15 – 5.00 p.m.	Session 9.	Exit meeting & conclusion.

 

The audit

Opening

Meet the key person (the principal auditee) and any other key people.

Do the audit entry meeting.

Do the grand tour. Get an overall picture of what the organization does, how they do it, and who their clients are. Inspect the site and its facilities. This overall picture will probably answer many of your questions before you get to ask them.

---

Start the audit

---

1. Signed form for the latest version of the auspice agreement

View latest version of agreement | Form

---

2. Training and Assessment Strategy (TAS)

This must be updated annually for each qualification, although some courses may be allowed to go over multiple years without change. The form includes many different compliance requirements. The task isn't finished until it is done well, that is, you are convinced that the program is well planned and will run effectively.

If they already have a Training and Assessment Strategy (TAS) document, check that it is the current form and update anything so that it is correct and up to the minute.

If it's a new program and they don't already have a TAS, work with them to write a new one. As there are so many items, I suggest that you delegate items to separate tables, so different questions get answered simultaneously.

---

3. Information for the public

If the ACAS website information is correct, they need no other publicity materials to be compliant.

Much of this information is usually advertising literature and the site's own website. Some other forms of information for prospective students are not ads, such as school handbooks and newsletter articles. Even small ads and micro-ads are included, because the standards do not set a minumum size ad for compliance. Some sites do not issue ads at all and refer only to the ACAS website information. If the ACAS website information is accurate, their other information may be incomplete, as long as it is accurate and not misleading.

• Check the ACAS website information. It is based on their TAS. Is it up to date and accurate?
• Check their website information and brochures, handbooks, etc.
• Check that they comply with the ACAS policy Auspicees: Student information. (Link opens new window.)
• If they use photos of people anywhere, check they have signed permission statements from the people depicted.

---

4. Course information for students

 
Course description

This is a written description for students informing them how the whole course will be taught and assessed:
• It is the version of the training and assesment strategy given to students that informs them about the program.
• It may be the same as the advertising literature or prospectus.
• The ACAS website gives the minimum amount of information, unless electives offered vary between groups, or unless locations and schedules vary according to employer.
• It is good practice to incude the purpose of the program and a job description of the role for which students will be trained.
• Some programs issue only separate unit descriptions for each unit. This only works where each unit is a standalone. Students still need information about their whole program.
 

Unit descriptions

• Look at the unit descriptions. Do they inform students about expectations (e.g. what's in the unit, what's the schedule, how they'll be assessed, etc.?
• Some programs combine all unit descriptions into a handbook for the whole course. This is very effective if the units are integrated into a rather seamless whole.

(The ACAS website gives the minimum amount of information to be compliant but it is good practice to give detailed information.)

---

5. Staff expertise

Each instructor needs to provide a current Approval to deliver form, and copies of all relevant credentials.
• Use this procedure to categorize staff and see what credentials are required. (Link opens new window.)
• Note: The Approval to deliver form now includes the staff form.

Check qualifications and any other credentials of all teaching and assessing staff. Get new paper copies and check that they are complete. Verify your copies: E.g. Original sighted, 31 Feb, 2020, [signed] J. Bloggs.

If they already have "Approval to deliver" forms, check them and update anything so that they are up to the minute on the current form. If they don't, check them (or oversee them filling them in).

• Approve forms that are all correct. If not, get corrections.
• Check that they fit the units listed on the TAS.

---

6. Delivery materials

The point is to show that the site has sufficient delivery materials for all units offered. Materials will vary greatly according to how the program is delivered, e.g. online, practicum, classroom, outsourced. Delivery materials must cover all units taught and all specific unit requirements.

It's not possible to include all training materials in the audit report, but auditors should describe them and go through a sample of at least two units and check that materials cover all requirements. The evidence may vary, for example:
• copies of local materials,
• links to online materials,
• list of contents of commercial materials,
• photograph of the library.

---

7. Equipment list

The point is to show that the site has all equipment prescribed as necessary for all units offered, and enough for the number of students.
• If units specify equipment (and most do), make a checklist of all required equipment and do an inspection. Sign the checklist showing what equipment is present and what is missing. To pass the audit, all required items must be present. It must also be sufficient for the number of students, so you might need more than the amount specified in the units.
 
Some units and some whole qualifications do not specify any required equipment, and this will also need to be recorded.

---

8. Assessment materials

Check assessment materials for all units of every qual to be offered.

• Assessment materials must be appropriate for context. It is possible to write good generic tools, although some generic tools might be unsuited to your particular context.
• Assessment materials will vary greatly according to the kind of delivery, e.g. online, practicum, classroom, assignments.
• Assessment materials must cover all units offered and all specific unit requirements.
• Any items for students should use ordinary language or workplace language, not TrainingSpeak gobbledygook.
• Enclose copies of assessment materials for a sample of two units in the audit report.

See the footnote for a working definition of assessment tools.

---

9. RPL assessment tools

A site must be able to offer RPL for all programs it has in scope for all units listed in the Training and Assessment Strategy form:
• If the above tools are unsuitable for RPL, the site needs a separate set of RPL tools.
• ACAS recommends that, where possible, assessment tools should be written to be equally suitable for both taught and RPL students.
• The ACAS website has generic RPL instructions but they might not be specific enough for the course being audited.

---

10. Assessment validations

Check that validations have been done for every unit of every qual. Ideally these are already done, but not always. There are two kinds, which are often done at different stages of the program:
• Validate assessment tools every year.
• Validate assessment judgements every year, unless no assessments were done in a course.

What to require:

• If they are using ACAS tools for which full validations have already been done in the last audit year, your audit report simply needs to report that.
• If they are using ACAS tools for which full validations are unchanged but no audit done during the last audit year, check that no improvements are required, update the form if needed, and produce a new version with today's date.
• If assessment tools are new or have never before been used, then they only need to validate assessment tools. (You cannot validate assessment judgements because you have not yet used them.)
• If assessment tools have been used, validate both the tools and the assessment judgements:
  • Validating tools should be only a check if they have not been changed. Check them anyway, do any updates if needed, put a new date on the form, and save it as the new version.
  • Validating assessment judgements should be the full process and a completely new form. For that, you will need a sample of assessment records.

---

12. Admission checklists

These checklists record compliance with admission procedures.

---

13. Student feedback

Overall, what has student feedback been like?

---

14. Program review and improvements

A list of glitches, risks, and program weaknesses and a plan to make improvements for each one. (The purpose is to show that the program is continually improving cf. SRTO 2.2 b.)

---

Other: Implementation

If the program is already running, interview some students and some instructors to evaluate actual implementation. (See separate page on audit questions.)

---

Other: National recognition

Ask if they have examples of accepting credit from other RTOs or AQF institutions. They might not have, but if they have, it is evidence that they practice national recognition.

---

Other: Retention of assessment evidence

Ask how they store assessment evidence.

---

Exit meeting

Hold the exit meeting.

Conclusion

At the end, ask for ideas and improvements:

• “How can we improve this process?”
• We want to make documents useful, not just compliance documents that only an auditor will read. For example, what could be usefully moved from the TAS to the student course description?
• Should the centre provide assessment tools? Should all assessment tools follow the same layout? What else should the centre provide?
• If you used ACAS materials, what kinds of improvements do you suggest?

 

Edit

In consultation with the principal auditee, you might need to tidy up the documentation. For example:

1. Are there gaps?
2. Clarify anything that looks to be no more than a list of suggestions or brainstorm ideas.
3. Check that documents are internally consistent.
4. Check that documents would be accurately understood by an outside person (e.g. no in-house jargon, no unusual terms used without explanation).
5. Check the language grammar, punctuation, etc.

 

Risk analysis

As part of ACAS quality management, audits include a risk analysis key aspects of a program. This means “Is the program viable and healthy, with little to go wrong?” ACAS needs to know whether programs are strong and healthy or at risk. This approach uses the criteria listed below. Scores of 5 represent an excellent program. Some scores of 0 represent a “non-compliant” audit outcome, while others represent a new program that is unproven but compliant. New, unproven programs will normally be 0 for some criteria, even if they have some very strong points.

To the auditor

In your report, give the program a star rating of zero to five for each criterion, based on the descriptions of weakest and strongest ratings, explain your reasons, and mention any obvious opportunities for improvement. Beware also the possible effects of the hype cycle when evaluating new programs. (See below.)

Programs are expected to improve. If a program does not improve or is high risk, the auditor will refer it to the board for further evaluation.

Items written in dark red like this indicate an automatic fail in an audit of compliance and quality standards.

	Weakest ☆ ☆ ☆ ☆ ☆	Strongest ★ ★ ★ ★ ★	Rating
1.	Has no clear purpose, or has no clear planning that ensures it will achieve its purpose. If it has a plan, it might be more like a wish list.	Has a clear purpose and has clear, concrete planning that ensures it will achieve its purpose. The program is already running well.
2.	New or has a very small number of students. It hopes to recruit students from the general public because it does not have a well-defined constituency from which it can draw new students.	Has either growing student enrolments, or as many students as it can handle well. It has a well-defined constituency, which is a reliable source of new students for the foreseeable future.
3.	Instructors are not teaching effectively and/or students are not learning effectively.	Instructors are teaching effectively, and students are learning everything they are supposed to. (This is not just a review of training and assessment documents.)
4.	Has no professional development program for instructors.	Professional development for instructors is effective in improving the quality of instruction.
5.	Materials are new and untested, and might be incomplete.	Training and assessment materials work well and need only minor adjustments from time to time.
6.	Has difficulty demonstrating compliance.	Has no difficulty demonstrating compliance.
7.	Often does not comply with ACAS policies and procedures.	Consistently complies with ACAS policies and procedures.
8.	Income currently does not cover costs. For example, it tries to offer courses at very low fees, or even for free, and has no income from other sources. It might have to cross-subsidise courses from other sources in order to survive.	Income covers costs with a safety margin. For example, it charges reasonable fees or has stable, assured income from other sources such as employers. It has few bad or slow debts receivable.
9.	Either has no clear budget or its estimates and projections are unrealistic.	Has a clear budget with realistic estimates and projections.
10.	Would be in trouble if key people were to drop out. For example, it depends heavily on volunteers.	Would still be healthy if key people were to drop out. Has enough staff and does not depend heavily on volunteers.
11.	Record-keeping is poor. Required documents were never created or are now missing.	Keeps good records and can easily demonstrate compliance.
12.	Satisfied to meet minimum requirements.	Trying to achieve excellence and best practice.
13.	Unaware of its areas of weakness and/or future improvements.	Understands its areas of weakness and/or future improvements.
14.	Delivery in another language	No delivery in another language
15.	Has only one qualified assessor if delivering in another language	Has multiple qualified assessors if delivering in another language
16.	Plagiarism, especially if delivering in other languages	Has good methods to prevent or detect plagiarism

 

The "hype cycle"

Take-up of new programs might follow the so-called "hype cycle," a pattern that warps perceptions. At some points, perceptions are overly optimistic, while they are overly pessimistic at others. The stages are below:

(Source: Wikipedia, s.v. Massive open online course) .

A new product creates unrealistic expectations in both providers and consumers. Those expectations generate rapid growth. When delivery fails to meet expectations, sales go into rapid decline. The product appears to have failed. Both providers and consumers develop a set of realistic expectations. The providers delivers proficiently based on a set of realistic expectations. .

 

The final report

Audits must be documented and the final full report comprises several elements: the auditor's report, the auditor's evaluation of processes, and a collection of evidence. These then need to be collated into one document of the whole audit.

The auditor's report

Keep good notes as you go and write your final report as soon as possible after the audit is finished.

Inside the report, give the date of each meeting and the names of everybody who attended. If there were more than one meeting, list them as separate meetings and mention what each meeting was about. (If you used an attendance roll, simply photocopy it for the report. This will make it easier to write your report.)

Go through all the audit items, and for each one:

1. List the evidence reviewed (documents, interviews, etc.).
2. State whether the site is compliant on non-compliant for that item, and give your reasons.
3. Copy and paste the SRTO reference.
4. Mention any particular strengths. Give credit where it is due.
5. Mention any particular weaknesses of the program(s) and opportunities for improvement.
6. Did the personnel at the audit site comply with ACAS procedures and policies? (I.e. not just SRTO requirements).
7. Did the personnel at the audit site recommend any improvements in the following (clearer, easier to comply with, better assurance of compliance, etc.):
   1. ACAS policies and procedures
   2. ACAS website information
   3. ACAS forms
8. Mention any follow-up work that needs to be done, and when and to whom it must be submitted.
9. Mention any residual risks and your evaluation of each one.

State what was done in the exit meeting.

Sign and date the report, and give your name as the writer of the report.

Review of audit processes

On a separate sheet, give your answers to the following questions:

1. What difficulties did you encounter in the audit process?
2. What are your suggestions for improving the audit process, including reporting forms?
3. Could any information be moved from the TAS to the website?
4. What are your suggestions for improving this ebook?

Forward a copy of the review to the ACAS office.

Soft or hard copy (or both)?

The best answer is probably both. Considerations include:

1. Soft and hard copy are both easy to store, although they are stored in different ways.
   1. Soft copies can suffer file corruption.
   2. If the number of hard copies builds up, storage becomes cumbersome.
   3. Soft copies are easy to send by email.
2. The forms are most useful in soft copy.
3. Anything that needs an original signature is best in hard copy.
4. Navigation of the contents is usually better in hard copy.

Assembing the final full report

The goal is to present the audit in a way that readers can easily understand and confirm your conclusions, and can easily find any document in it. To achieve this goal, present the whole audit inside one set of covers, complete with any evidence that you collected. Being well-organised is even more essential in larger programs that produce a fairly large collection of staff documents or represent multiple qualifications.

Arrange the full report like a book:

1. Front title page:
   1. Name of the site
   2. Name of the qualification(s)
   3. Date(s) of audit
   4. Name of auditor
2. Table of contents
3. Audit entry meeting form
4. Your audit report.
5. Exit meeting report.
6. Your audit review.
7. Enclosures: Evidence for each audit item.

Send a copy of the report to the ACAS office.

 

What are assessment tools?

The Standards for Registered Training Organisations 2015 mentions Assessement tools but does not contain a definition. Consequently, it provides no criteria for what must be included and what may be excluded.

For the purpose of ACAS audits, assessment tools are defined as follows:

1. A set of written instructions to students.
   1. The unit title and code
   2. Conditions (e.g "in the workplace")
   3. Tasks for assessment / evidence
   4. Criteria for making the assessment.
   5. Anything else specified in the unit as a requirement.
2. A separate set of instructions for trainers/assessors if procedures and specific assessment tasks are not fully covered in the instructions to students.
3. Assessment forms for reporting the assessment. (They may be electronic, or part of a software package.)
4. Some kind of written mapping to show how assessments meet all unit requirements.
5. A means of ensuring reliability. (This is often a weak point.)
   1. ASQA prefers a separate marking guide.
   2. A description of proficient performance.
   3. The default in ACAS assessment instructions is to use a second person in the assessment.
   4. Detailed performance criteria (more detailed that than those in the original units).

An official ASQA definition (not in the standards) of assessment tools is as follows:

• Assessment tools include the following components—context and conditions of assessment, tasks to be administered to the student, an outline of the evidence to be gathered from the candidate and evidence criteria used to judge the quality of performance (i.e. the assessment decision-making rules). ... This term also takes in the administration, recording and reporting requirements, and may address a cluster of competencies as applicable for holistic assessment.
  (General Direction – Retention requirements for completed student assessment items, updated 12 May 2016

 

Future

It would be helpful to also incude the following:

1. A WHS inspection
2. A check of Working With Children cards
3. Greater itemization of risks, and risk management
4. Check on conflicts of interest
5. Itemisation of industry consultation, especially
   1. Selection of qualification
   2. Job description for which persons are trained
   3. Editing assessment tools

 

Risk analysis

SRTO Standard 2, esp. 2.2

As part of ACAS quality management, audits include a risk analysis key aspects of a program. This means “Is the program viable and healthy, with little to go wrong?” ACAS needs to know whether programs are strong and healthy or at risk. This approach uses the criteria listed below. Scores of 5 represent an excellent program. Some scores of 0 represent a “non-compliant” audit outcome, while others represent a new program that is unproven but compliant. New, unproven programs will normally be 0 for some criteria, even if they have some very strong points.

To the auditor

In your report, give the program a star rating of zero to five for each criterion, based on the descriptions of weakest and strongest ratings, explain your reasons, and mention any obvious opportunities for improvement. Beware also the possible effects of the hype cycle when evaluating new programs. (See below.)

Programs are expected to improve. If a program does not improve or is high risk, the auditor will refer it to the board for further evaluation.

Items written in dark red like this indicate an automatic fail in an audit of compliance and quality standards.

	Weakest ☆ ☆ ☆ ☆ ☆	Strongest ★ ★ ★ ★ ★	Rating
1.	Has no clear purpose, or has no clear planning that ensures it will achieve its purpose. If it has a plan, it might be more like a wish list.	Has a clear purpose and has clear, concrete planning that ensures it will achieve its purpose. The program is already running well.
2.	New or has a very small number of students. It hopes to recruit students from the general public because it does not have a well-defined constituency from which it can draw new students.	Has either growing student enrolments, or as many students as it can handle well. It has a well-defined constituency, which is a reliable source of new students for the foreseeable future.
3.	Instructors are not teaching effectively and/or students are not learning effectively.	Instructors are teaching effectively, and students are learning everything they are supposed to. (This is not just a review of training and assessment documents.)
4.	Has no professional development program for instructors.	Professional development for instructors is effective in improving the quality of instruction.
5.	Materials are new and untested, and might be incomplete.	Training and assessment materials work well and need only minor adjustments from time to time.
6.	Has difficulty demonstrating compliance.	Has no difficulty demonstrating compliance.
7.	Often does not comply with ACAS policies and procedures.	Consistently complies with ACAS policies and procedures.
8.	Income currently does not cover costs. For example, it tries to offer courses at very low fees, or even for free, and has no income from other sources. It might have to cross-subsidise courses from other sources in order to survive.	Income covers costs with a safety margin. For example, it charges reasonable fees or has stable, assured income from other sources such as employers. It has few bad or slow debts receivable.
9.	Either has no clear budget or its estimates and projections are unrealistic.	Has a clear budget with realistic estimates and projections.
10.	Would be in trouble if key people were to drop out. For example, it depends heavily on volunteers.	Would still be healthy if key people were to drop out. Has enough staff and does not depend heavily on volunteers.
11.	Record-keeping is poor. Required documents were never created or are now missing.	Keeps good records and can easily demonstrate compliance.
12.	Satisfied to meet minimum requirements.	Trying to achieve excellence and best practice.
13.	Unaware of its areas of weakness and/or future improvements.	Understands its areas of weakness and/or future improvements.

 

The "hype cycle"

Take-up of new programs might follow the so-called "hype cycle," a pattern that warps perceptions. At some points, perceptions are overly optimistic, while they are overly pessimistic at others. The stages are below:

(Source: Wikipedia, s.v. Massive open online course) .

A new product creates unrealistic expectations in both providers and consumers. Those expectations generate rapid growth. When delivery fails to meet expectations, sales go into rapid decline. The product appears to have failed. Both providers and consumers develop a set of realistic expectations. The providers delivers proficiently based on a set of realistic expectations. .

 

Risk assessment: New programs

This risk assessment is intended only for new programs wanting to run as Nationally Recognised Training, although some criteria might also apply to established programs. It is generally unlikely that ACAS would now accept a new program unless it is almost all low risk.

There are easy and difficult approaches to starting a new program. Low risks represent programs that are relatively easier to implement. Some higher risks might not be any better at the first stage of a new program; they generally represent programs that are very difficult, or perhaps impossible, to implement in an RTO. Items written in dark red like this indicate an automatic fail in an audit of compliance and quality standards.

	Highest risk ☆ ☆ ☆ ☆ ☆	Lowest risk ★ ★ ★ ★ ★	Rating
Qualifications
1.	Offer multiple qualifications.	Offer only one qualification.
Trainers & assessors
2.	Recruit new staff to teach.	Use your current staff as supervisors.
3.	Proposed trainers-assessors would have to learn new skills.	Offer programs that match the core skills & expertise of proposed trainers-assessors.
4.	All proposed trainers & assessors still need to get all formal qualifications.	All trainers & assessors already have all formal qualifications and experience.
5.	Only one main trainer-assessor.	Multiple trainer-assessors.
Structures
6.	Establish a new school structure and ignore your current organizational structure.	Use your current organizational structures.
7.	Not clear about the role students would be trained to meet.	Clear statement of the role students would be trained to meet and already have people in that role.
Recruitment
8.	Needs to advertise the program to recruit students.	Already has people who want to do the program.
9.	Ignore your current constituency and pioneer a new one from scratch.	Recruit students from your current constituency.
10.	Recruit full-time class-room students who will get Austudy.	Have only part-time students.
Delivery
11.	Teach regular classes in a classroom atmosphere.	Train on the job.
12.	Needs to buy new equipment for classrooms.	Use existing workplace equipment.
13.	Teach everything in the classroom.	Only conduct classes and training weekends as you need them.
14.	Needs to write or gather all teaching and assessment materials.	Already has all teaching and assessment materials.
Programs already offered
15.	No recognised programs already offered.	Already offers recognised qualifications (e.g. NRT courses, higher education.)

 

Questions for audits

Plan your key questions, which should cover all requirements listed for the audit. You will also need to aaks follow-up questions to explore particular topics as they arise.

Start with an overview

1. Please show me around; give a general tour of what you do.
1. What is the purpose of this whole program?
2. Why did you choose to offer this qualification?
3. What kinds of people do you get (or want) as students?
4. Why do students choose this qualification?
2. How do the fees work?
3. How do your training materials work?

Information

1. How do you inform students about the course before they enroll?
2. The ACAS website has lots of information for your staff and students.
1. Did you read the information on the ACAS website?
2. Did your staff and students read any of that information? How do you know?
3. Was information for your staff accurate, helpful, and easy to understand?
4. Was information for your students accurate, helpful, and easy to understand?
3. Many ACAS programs produce their own information for their staff and students.
1. What information does your program give your staff and students?
2. Did your staff and students read any of that information? How do you know?
3. Was information for your staff accurate, helpful, and easy to understand?
4. Was information for your students accurate, helpful, and easy to understand?
4. Some of the information on the ACAS website explained requirements (e.g. procedures).
1. Did students do as instructed?
2. Did staff do as instructed?
5. You want to put out a new advert for your program. Do you know what the requirements are?

Advertising

1. What kinds of adverts do you use? (E.g. website, flyers, micro ads)
2. You want to put out a new advert for your program. Do you know what the requirements are?

Student support

1. What do you do to identify students’ support needs:
1. during admission?
2. during the course?
2. What kinds of extra support do students need?
3. What kinds of extra support do you give?

Assessment

1. How does your assessment system work?
2. Show me your assessment tools. How do they work?
3. You must have a set of RPL assessment tools for each qual. These can be your regualr tools or a separate set of RPL tools. If your regular tools work for RPL, how do they work? Do you have a separate set of assessment tools for RPL?

Improving

1. Have you had any complaints or appeals, either formal and informal?
2. What kinds of improvement have you made during the last year?
3. What do you think are the strengths and weaknesses of your programs?
4. What kinds of ways would you like it to improve?

If the program is already running, interview some students and instructors

1. Interview some students:
1. How well were you informed about the course and fees before you applied?
2. How well were you informed about how course expectations after you applied?
3. Did they deliberately show you the ACAS Handbook when you enrolled?
4. Is the program running as advertised?
5. Did you fill in the ACAS online form yourself? (Some sites have an admi. Do it for them, so you might not have done it yourself.) If you did it yourself, did you do it at the very beginning of the program or later on?
6. What do you think are its weaknesses and strengths?
7. What improvements would you like to see?
8. What kinds of support did you need? Did you get it?
9. Were you given enough information before you enrolled? What else would you like to have been told that you weren't?
10. Was there enough time in training sessions?
11. Were training sessions helpful? What went well? What didn't?
12. What materials did you have (e.g. textbooks)? Were they helpful and adequate?
13. Did you like the way that units were assessed?
2. Interview some instructors:
1. What do you think are the program's weaknesses and strengths?
2. What improvements would you like to see?
3. Did the selection critera and processes for admission ensure that the right applicants were accepted?
4. Is the program running as advertised? If not, why?
5. Is the program running as planned? If not, why?
6. Did students need extra suppport? What did you do to help students get it? (This might include giving extra support yourself.)
3. Sit in on one or two classes.

Note: The current standards say very little about the teaching-learning experience of instructors and students.

You will also need to ask enough questions to verify any documents.

 

Title

Content 14

 

Title

Content 15

 

Title

Content 16

 

Title

Content 17

 

Title

Content 18

 

Title

Content 19

 

Title

Content 20