Not sure what these guys are studying but can tell you in the real world - essentially zero AI rollout in accounting world for anything serious.

We've got access to some fancy enterprise copilot version, deep research, MS office integration and all that jazz. I use it diligently every day...to make me a summary of today's global news.

When I try to apply it to actual accounting work. It hallucinates left, right & center on stuff that can't be wrong. Millions and millions off. That's how you get the taxman to kick down your door. Even simple "are these two numbers the same" get false positives so often that it's impossible to trust. So now I've got a review tool that I can't trust the output of? It's like a programming language where the equality (==) symbol has a built in 20% random number generator and you're supposed to write mission critical code with it.

I keep trying to get it to review my personal credit card statements. I have my own budget tracking app that I made, and sometimes there's discrepancies. Resolving this by hand is annoying, and an LM should be able to do it: scrape the PDF, compare the records to mine, find the delta.

I've tried multiple models over the course of 6 months. Yesterday it told me I made a brilliant observation, but it hasn't managed to successfully pin down a single real anomaly. Once it told me the charges were Starbucks, when I had not been to a Starbucks—it's just that Starbucks is a probable output when analyzing credit card statements.

And I'm only dealing with a list of 40 records that I can check by hand, with zero consequences if I get it wrong beyond my personal budgeting being off by 1%.

I can't imagine trusting any business that leans on this for inappropriate jobs.

There's a very fun video about accounting by Dan Toomey [0] that I think really drives home the point that accounting is:

1) Extremely important

2) Not that glamorous

I always think of accountants as the "nerds" of the finance world. I say this lovingly - I think in another life I would have become an accountant. I find it very fascinating. I worked at a company that worked with auditing datasets, so I knew much more about accounting that I would have otherwise.

Nobody ever wants to listen to accountants because they either are giving you bad news, or telling you the things that you should be doing. No one can deny how important they are, despite how much it seems like everyone wants to get rid of them.

An accounting story I love is how my old company got a lot of business because of Enron. Part of the reason that Enron was caught was due to their audit fees.

Their audit fees were reporting that Arthur Andersen was charging for a huge percentage of non-auditing work (audit fees report what percentage was auditing related and not). This was a huge red flag.

My company was the only one at the time that kept track of audit fees, and so a huge number of people paid to access that data stream.

If one day I quit programming, maybe I'll get my CPA.

[0]: https://youtu.be/vL4INHaK-sA?si=jIvFQVtrXU6tjh-1

"...can tell you in the real world - essentially zero AI rollout in accounitng world for anything serious."

The jobs the reseearchers concluded were affected were "unregulated" ones where there are no college education or professional certification requirements, e.g.,

   receptionists
   translators
   software "engineers"

Apparently, they studied payroll data from ADP on age, job title and headcount together with, who would have guessed, data from an AI company (Anthropic)

https://digitaleconomy.stanford.edu/publications/canaries-in...

This study has not been peer-reviewed

Yup, using AI for any serious tax calculation or even advice is a REALLY BAD idea.

A close relative is a top expert in US Trust & Estate Tax law working at a well-known BigLaw firm. Of course they have substantial AI initiatives, integration with their system, mandatory training, etc.

She finds tha AI marginally useful for some things, but overall not very much and there are serious errors, particularly the types of errors only a top expert would catch.

One of the big examples is that in the world of T&E law, there are a lot of mediocre (to be kind) attorneys who claim expertise but are very bad at it (causing a lot of work for the more serious firms and a lot of costs & losses for the intended heirs). The mediocre-minus attorneys of course also write blogs and papers to market themselves, often in greater volume than the top experts. Many of these blogs/papers are seriously WRONG, as in giving the exact opposite of the right advice.

Everyone here sees where this is going. The AI has zero ability to reason or figure out which parts of its training input are from actual top experts and which are dreck. The AI can not reason, and can not even validly check their 'thinking' against existing tax code (which is massive), or the regulations and rulings (which are orders of magnitude more massive). So, the AI gives advice that is confident, cheerful, and WRONG.

Worse yet, the LLM's advice is wrong in ways only a top expert would know, and in ways that will massively screw the heirs. But the errors will likely only be discovered decades later, when it is too late to fix.

Seriously, do NOT use LLMs for tax advice, unless you are also consulting a TOP professional. And skipping the LLM part is best.

My relative is quite frustrated and annoyed by the whole thing, which should be more helpful with these massive code/regs/rulings, but finds it often more work than just using the standard WestLaw/Lexis legal database searches.

> It's like a programming language where the equality (==) symbol has a built in 20% random number generator and you're supposed to write mission critical code with it.

<bad joke> Why are we talking about JavaScript in a thread about AI? </bad joke>

There seems to be this dream of Tax AI Software that will just do all of the taxes. But other than using AI as a fancy text search, I don't see it happening for a long long time. LLMs can't do arithmetic or count.

It's not about using LLM for calculation, it's about automation and agentic. The integration and deployment in enterprise will be a matter of time. Hiring fresh out of school used to be labor and training, but knowing in 5 years the industry will shrink while the senior staff will keep the boat floating aided by AI, the cutoff time is now. There is no need to keep sending young generation to industries that're bound to be automated enmass. And these are in the immediate near term.

Other industries are yet to see how AI will impact, it may or may not ever. But in some science fields, new graduated PhDs are seen the same hiring freeze. The complete outsourcing of school knowledge to LLM is coming to our life real soon, the only factor not making it faster is the data center and energy, which are being worked on to resolve in a couple years. These are the reason AI is not yet as cheap as search and ready for consumer market. But it's cheap or will be cheaper enough in two years for enterprise at a cost lower than human resource. The answer is obvious when looking at it on a 5 year horizon.

There are a lot of jobs which don't require meticulous accuracy - coming up with marketing plans, press releases, writing HR policies, reading and summarising reports, etc.

Even in accounting I am sure you could use AI at times to help write or read your emails or summarise legislation or IRS rulings. Have it drive Excel or your financial systems directly? No, not yet.

It is profoundly bad at accounting. But with a calculator tool, it works okay for math.

Aisde: Hey, whats the prompt you're using for a summary of the news events?

LLMs basically can't do arithmetic directly, trying to get them to do so is a skill issue. Most models can and will happily write and execute code to do that work instead.

[dead]

[dead]

In fairness, a "20% random number generator" on "mission critical code" is something they literally do at NASA