The continuing story of AI’s evolution

Research by Goldman Sachs claims AI will replace 300 million jobs in the next 15 years. The international Labour organisation says AI will enhance jobs rather than eliminate them. Time will tell.

My particular interest is financial planning and I ask the following question regularly to test its progress. Here is the latest result.

“I have $300,000 in my superannuation and will retire in 10 years’ time with only employer contributions on a salary of $120,000 pa. I will need $60,000 a year in retirement. How long will my superannuation last?”

This question demands AI establish employer contributions, make reasonable assumptions on growth and inflation, and apply them to establish a value at retirement. It must then see how long this value will last, again making reasonable assumptions. A complex task.

ChatGPT3-5 continues to do a pretty good job making very reasonable assumptions on growth and inflation and applying its own (correct) future value formula. Inexplicably, it applied simple interest to contributions and the 2020 SG contribution rate. This caused it to be about 32% wrong in its final answer.

ChatGPT-4 was hopeless at the last check but has improved dramatically. It makes very reasonable assumptions and is totally transparent. It got the employer contribution slightly wrong (10.5% instead of 11%) but everything else was 100% and the final answer was almost correct. As a bonus it provided the Python code if you wanted to create your own calculator. An adviser could copy and paste the answer (if correct) into an advice document saving a lot of time and cost.

Microsoft’s Co-pilot hasn’t changed from the last test. It made reasonable assumptions as to growth and inflation (slightly different to ChatGPT), then visited the MoneySmart superannuation calculator to find the answer. As before, it identifies the correct values to input but then “hallucinated” the final answer which is about 50% wrong.

What was interesting was firstly how much ChatGPT-4 has improved and secondly how each of them adopted a completely different method. ChatGPT-3.5 used a simple future vale formula, ChatGPT-4 used Python code and Co-pilot used the MoneySmart superannuation calculator. They each took their own road.

ChatGPT-4 is very close and if it improves at this rate there may be little reason to use expensive financial planning software and paraplanners.

Sarah Abood, chief executive officer of the Financial Advice Association Australia, said the proposed amendments to statements of advice (SOAs) could alleviate the scarcity of affordable advice by allowing businesses to service this segment of the market with inexpensive options. On its current trajectory, ChatGPT-4 could allow advisers to construct SOAs quickly and cheaply. It costs $22 per month.

Its not there yet though.