Microsoft’s co-pilot is not ready to take the controls.
There is little doubt generative AI like ChatGPT and Co-pilot are the future and I have been monitoring their ability to provide financial planning calculations and information. To do this I asked them the same question;
I have $300,000 in my superannuation and will retire in 10 years’ time with only employer contributions on a salary of $120,000 pa. I will need $60,000 a year in retirement. How long will my superannuation last?
ChatGPT3-5 did a pretty good job making very reasonable assumptions on growth and inflation and applying its own (correct) future value formula. Inexplicably, it applied simple interest to contributions and the 2020 SG contribution rate. This caused it to be about 32% wrong in its final answer.
ChatGPT-4 didn’t even try and find what the SG contributions are or apply a future value formula. It simply divided $300,000/ $60,000 which is 100% wrong. It recommended the user find a financial adviser which was the only sound advice it gave.
Microsoft’s Co-pilot has a different approach. You can choose whether you want the response to be “more creative”, “more balanced” or “more precise”. “More creative” provided the most comprehensive answer with “more precise” simply giving a list of superannuation calculators.
“More creative” had a real crack at it. It made reasonable assumptions as to growth and inflation (slightly different to ChatGPT), then visited the MoneySmart, Australian Super, Canstar and Australian Retirement Trust websites, using the MoneySmart superannuation calculator to find the answer. It never mentions SG contributions as ChatGPT3-5 did but may have applied them.
Co-pilot’s result was $1,016,000 using the MoneySmart calculator which would apparently last 22 years after retirement. I entered the question’s values into the MoneySmart calculator, assumed a 7% net return and got $554,560 which I verified on an Excel sheet. Co-pilot appears to be around 50% wrong.
I was always sure AI would get more accurate and supplant existing software and human services but am no longer as confident. How can you trust it?Activate to view larger image,
