FASEA exam doesn’t get passing grade

With four Financial Adviser Standards and Ethics Authority (FASEA) exams now completed, 5,257 advisers have sat the first three exams, with another 2,200 who sat the February exam.

Despite the anxiety in the industry over sitting the exam, the lowest pass rate was 86% – which was in the December exam.

However, despite the high pass rate, many advisers were dissatisfied with the conditions of the exam.

In an attempt to gauge the sentiment for the exam and find out why advisers were unhappy with the exam despite passing it, Money Management published an open survey to gather feedback, and received over 250 responses. 

Some 89.6% of the respondents said they had taken the exam, 75.4% had passed it and the overwhelming majority (97.9%) of respondents said they had only needed to sit it once, but the feedback was universally negative regardless.

Advisers were particularly unhappy with how the exam was conducted, particularly when it came to how they were treated on the day and access to personal comforts like water, tissues, or using the bathroom.

They were also critical of the questions – how they were worded, the ambiguity of the questions, and whether they were relevant to their field of expertise.

Money Management contacted FASEA to find out why the exam was conducted this way, how they felt about adviser concerns, and if they planned to make any changes.

Stephen Glenfield, FASEA chief executive, defended the exam’s methods and said FASEA relied on the Australia Council for Education Research (ACER) to conduct the exams.

“In conjunction with our exam provider ACER, the Standards Authority takes candidate feedback seriously,” Glenfield said.

“Together with ACER and its venue providers, we discuss all feedback to determine improvements.”

The pass mark was one of the biggest queries advisers had as they did not know what grade they received whether they passed or failed.

Glenfield said they used a method of ‘scaled scoring’ which was the best practice for ‘high stakes’ exam scores like accountancy, law and medicine.

“The scaled score for a given exam takes into account the difficulty level of the questions in that exam to ensure that a consistent standard is applied across all exams,” Glenfield said.

“The scaled score is determined once all results have gone through extensive psychometric analysis.  

“The ‘pass mark’ for each exam is set by an expert independent review panel using formalised and internationally accepted standard setting procedures.”

This meant the starting point was 65%-74%, which was the equivalent of a university credit grade.

“To ensure equity and fairness for all candidates, the ‘pass mark’ will be reviewed for each exam cycle and may be adjusted to account for differences in exam difficulty and to maintain standards,” Glenfield said.

CONDITIONS

When it came to exam conditions being fair, 52.9% said ‘yes’, 34.4% said ‘no’ and 12.7% said it was ‘hard to say’.

Source: Money Management

Among the biggest complaints about the exam was that, despite being open-book, it had intense conditions that matched a closed-book exam.

“Aggressive, antagonistic instructors. It was not really an open-book exam either and the indexing and search capability in the software was a hindrance, not an aid,” a respondent said.

“I am 47, and have an undergrad and masters. I have sat numerous exams and I would say this has been the worst exam experience,” another respondent said.

Advisers were unhappy with the restrictions on their desk, including access to simple items like water and tissues.

“It was way, way overboard. No water bottles, tissues etc for a three-and-a-half-hour exam. This just meant that if one person wanted a drink of water it interrupted everyone,” a respondent said.

“To go to the toilet, you had to raise your hand and await the supervisor. It seemed as though they thought we were all going to cheat and it was their job to catch us.”

“[We were] treated like school children, followed to the toilet, and water left at the front table,” another respondent said.

Glenfield said as a result of previous feedback, together with ACER and its venue providers, they had discussed changes to this issue which would take effect from the April exam.

“As a result of candidate requests to have water at their desks during the exam, FASEA has agreed with venue providers that future exams will allow candidates to have clear water bottles at their workstation,” Glenfield said.

QUESTION?

Only 19.7% said the exam questions were fair, with a significant majority of advisers finding the questions unfair.

Many in the industry already had experience in tertiary education, including exams, and felt the FASEA exam did not compare in fairness.

“I have never failed an exam, but despite having 20 years’ experience, and swotting beforehand I could make no guess at how I went,” a respondent said.

“A complete waste of time, and a detrimental effect on one’s mental health, knowing failure can mean the end of your career,” another respondent said. 

“This was the most convoluted and poorly-worded exam I have ever sat, in terms of making someone a better adviser, it did absolutely nothing,” another respondent said.

Source: Money Management

The exam is three-and-a-half-hours (210 minutes) including 15 minutes reading time – for perspective, that’s longer than the longest theatrical release in The Lord of the Rings film series which was 201 minutes (The Return of the King, if you were wondering).

Despite the epic length of the exam, many advisers still complained of there being too many questions to complete in the allotted time.

“Three hours, 15 minutes for 78 questions does not leave enough time to think about a question let alone answer the full range, under time pressure, throughout the exam,” a respondent said.

Glenfield said the length of the exam was consistent with other ‘high stakes’ exams.

“ACER’s data analysis has not identified any issues with the length of the exam as the majority of candidates are completing all questions in the exam within the allocated time,” Glenfield said.

Another complaint was learning disabilities and those who spoke English as a second language were not accounted for, which becomes a greater issue when combined with the ambiguous questioning line.

“If English isn’t your first language it would be very hard as the questions are ambiguous,” a respondent said.

Given the multicultural nature of our country, this could have the effect of reducing advisers who were able to provide advice to certain communities.

“Results for each exam are subject to ACER’s comprehensive marking approach which includes analysis for differences between candidates with English as a second language, as well as other demographic characteristics,” Glenfield said.

When it came to accommodating learning disabilities, Glenfield said candidates had the option to apply for reasonable adjustments prior to sitting the exam.

“The application process for this is set out in the candidate information booklet available when a candidate registers for the exam,” Glenfield said.

“ACER will assess each candidate based on their application and consider advice from their medical practitioner. Appropriate, reasonable adjustments would be provided.”

Another adviser said they had injured themselves two weeks before the exam – when the deadline for special provisions was a month earlier.

They had the option for deferment of the exam, which the respondent noted was fine in a capital city, but rural advisers lacked this luxury as there were not as many exam dates available.

Beau Riley, TAL head of licensees and partnerships, said a three-and-a-half-hour exam was not a straightforward task and it had not been made clear on what the pass mark was or whether there will be an extension on the completion date.

TAL ran the Risk Academy, which was one of the many provider courses on the market to help advisers prepare for the exam.

“When you consider how busy advisers are with their business, family and new education requirements, finding time to prepare for the exam is not easy,” Riley said.

UMM, WHAT?

The keyword throughout all the survey responses was ‘ambiguous’, as advisers felt the questions were subjective, unclear or poorly worded.

Joel Ronchi, principal at myIntegrity in Practice, had extensive feedback from advisers who had sat the June, September, December and February exams.

He started myIntegrity in Practice to help financial advisers and licensees understand and implement the FASEA standards.

“There are key phrases that keep coming up, one is ambiguous, a lot of advisers feel the wording in the questions can be ambiguous and that means different things to different people,” Ronchi said.

“For example, in some of the multiple-choice questions, advisers have said there’s two options so close it’s hard to pick and it revolves around wording.”

“I walked out of the exam not knowing if I would have passed or failed. I did not walk out of the exam thinking my clients will benefit from me sitting this exam,” a respondent said.

Riley said feedback from Risk Academy attendees was that they felt the questions were ambiguous, especially the multiple-choice ones.

“For advisers that specialise, there is often concern around how broad the questions may be in the exam,” Riley said.

Glenfield said candidates would be familiar with exams that asked questions based on knowledge recall, but the FASEA exam was different as it required candidates to analyse a scenario and apply their knowledge.

“Why this method? Because ACER has used its extensive global experience in managing exams to develop the questions and the most effective methodology,” Glenfield said.

“ACER is also using experts in the field to write and mark the questions. The exam is testing the practical application of the relevant knowledge areas using complex questioning techniques consistent with other high stakes exams.

“In addition, detailed psychometric analysis is undertaken after every exam cycle. It should be noted that where an individual question does not perform satisfactorily, it is removed from scoring.”

ETHICS

Only 29.7% of respondents found the ethics component relevant, while 40.6% said ‘no’ and 29.7% said ‘it’s hard to say’.

Source: Money Management

“The questions did not really test if an adviser is ‘ethical’, rather, it tested random knowledge of different areas in a poorly-worded way,” a respondent said.

“Questions and answers could easily be misinterpreted. [It] feels like this exam is a revenue raising exercise – slightly unethical perhaps?”

FEEDBACK

Advisers had said they lacked feedback from results, leaving them unaware if they had performed poorly in a certain area.

“Unsuccessful candidates are able to re-sit the exam and these advisers will receive guidance on which knowledge areas they need to improve to enhance their ability to pass at a future sitting,” Glenfield said after FASEA announced the September exam results on 8 November, 2019.

“Not receiving a percentage on the fail, we are given the opportunity to remark the written questions at a cost,” a respondent said.

“We need to know the mark to gauge if we should pay the funds to re-mark otherwise it is pointless.”

The exam costs $540 plus GST to sit, but to re-mark the exam it would cost $198, including GST.

Glenfield said the exam tested the application of three ‘knowledge areas’: financial advice regulatory and legal requirements, financial advice construction, and applied ethical and professional reasoning and communication (incorporating the code of ethics).

“Should a candidate fail the exam, analysis of their performance is undertaken to determine whether they underperformed in a particular knowledge area, or across all knowledge areas,” Glenfield said.

“Based on this analysis, candidates who have failed the exam are provided the following feedback:

1. If a candidate had clearly underperformed on one knowledge area versus others, they receive advice to revise that particular area. If a candidate has underperformed across all areas consistently, they will be advised to revise all knowledge areas.
2. Advice regarding the re-marking process. If a candidate’s result does not mathematically allow for a pass (if their exam is re-marked), the candidate will be advised accordingly.”

Although there was no shortage of exam preparation resources available in the market, older advisers still felt the pressure of sitting an exam for the first time in decades, if at all.

“Supervisors were rude and grumpy adding more stress to an already stressful situation, desks were too close to each other and people tapping loudly on keyboards was distracting, but thankfully they had earplugs we could use,” a respondent said.

Unsurprisingly, it’s older advisers who felt the hardest done by, as the investment into further education doesn’t have the same return on investment given they have less time left in their career.

“It’s embarrassing to have to sit an exam for your own profession to confirm I know what I have been doing correctly for the last 25 years,” a respondent said.

“Age discrimination for those that are not comfortable with tech – the guy next to me really struggled with the typing and the control-F concept,” another respondent said.

SPECIALISATION

The exam covers multiple areas of specialisation, which meant advisers who specialised in one area were expected to accurately answer questions on areas they don’t specialise in to pass the exam.

Many advisers questioned if the exam was appropriate for professionals who specialised in a specific field like stockbroking, life insurance or self-managed superannuation funds (SMSFs).

“If you were a risk writer you would have no idea of the answers to some questions,” a respondent said. 

“Some of the questions were around portfolio construction, some were not clear in what they were asking, some do not relate to financial planning, some were not in the reading.”

“I would have liked to see more relevant questions where it is testing my knowledge of legislation and ethics, not some obscure area of financial planning that we will never be dealing with on a day-to-day basis,” another respondent said.

“This exam proves how out of touch the question makers are with this profession.”
Glenfield said the exam wasn’t meant to test technical knowledge in those areas, only the application of the three ‘knowledge areas’.

“Although there may be references to specific advice areas in the client scenario provided, the exam is testing the application of, for example, the Corporations Act or the code of ethics, not technical specialist areas,” Glenfield said.

GETTING THE MOST OUT OF PREPARATION

The most popular study provider was Kaplan, which 48.7% of respondents said they used, followed by TAL (42.3%), myIntegrity (15.5%), Cram4Exam (3.2%), TAFE (2.1%), Blue Path (1.6%), and AIA (1.1%).

Knowledge Shop, BT and OnePath were also popular responses in the ‘other’ category. 

Apart from the study providers available, there was important information about the exam available online for free.

Riley said the materials FASEA provided were beneficial, which included practice questions and the candidate information video which explained how the exam worked.

“In addition, some of the recommended readings available to advisers are invaluable for their exam preparation, including but not limited to the code of ethics explanatory statement and guidance document, and the ASIC [Australian Securities and Investments Commission] guide RG 175,” Riley said.

“Knowing how to navigate the legislative instruments provided is an essential component of exam preparation.”

Ronchi said it helps to have a strategy going into the exam and there’s no one size fits all approach when it came to preparing for the exam.

“Take the time to go to the FASEA website, have a look at the video they put out that walks through the actual platform,” Ronchi said.

“It’s an eight-minute video, not only does it walk you through and show you how it works, but it gives a visual on what to expect on the day.

“Be aware about what’s going to happen on the day, you can’t take in anything with you except your ID, you get a scratch piece of paper and a pen provided, and you get a desktop with one screen.

“Just read the questions carefully, read the wording, read what the intent of the question is and answer the question as it’s being asked.”

Ben Marshan, head of policy and standards at the Financial Planning Association (FPA), said most of its members had adapted and would just get on with preparing for the exam. 

“They’re telling us they need put in a little bit of work to do the study and get themselves up to speed with the material,” Marshan said.

“They’re providing us the feedback that the exam is challenging, not something they can just walk into and think they will be able to pass without preparation.”

The FPA has created a learning portal called FPA Return to Learn which provided information from universities to help members getting back into tertiary study and prepare for the exam.

“Rather than give them big slabs of legislation to study, we’re able to summarise and put it in a more useful format for them,” Marshan said.

“We’ve done analysis on the code and we’re able to map how it works, so they can get their head around a little more easily and understand it.”

Marshan said some of the feedback they had received was the shock from advisers on the length and breadth of financial planning regulation.

“Some members have said leaving that amount of material to the last weekend before you sit the exam has been a mistake, even though they passed,” Marshan said.

“What was a little surprising was that some advisers thought just could rely on understanding their compliance manual their licensee put in place.”
Riley said most advisers have found managing the volume of reading as one of their biggest struggles.

“While suggested readings including the Corporations Act (Chapter 7) and the Privacy Act 1988 are important, and knowing how to navigate the document is required for the exam, reading these in their entirety may not be the best way to achieve the desired outcomes for the exam,” Riley said.