I wonder if telling (or somehow architecturally coaxing) the LLM it has 'skin in the game' will make it more risk-averse? I imagine it does.

This makes me wonder too about the entire premise and worthiness of these evals. They orient themselves around normal one-shot interactions with a likely non-sys-prompted model with no built up context or memory of the person. I doubt the mentioned 'job loss' scenario is even contextually seen as a 'loss'; it is only a circumstance descriptor, a single snapshot without a history. Maybe to get the best advice we actually need to tell the LLM our entire story, not just a narrow request for a question; a question that - itself - is biased to our own imaginings of what problem we perceive ourselves as having, which humans are often bad at.

The problem I see with this approach is threefold.

First, from a technical standpoint the required context window would be massive if you're looking at a person's career/life holistically. Probably solvable, but definitely something to be aware of.

Second, privacy goes completely out the window since you're sharing everything. You don't know what's relevant and what's not up front so you need to provide everything.

Third, you would need a training dataset of all those input variables and their outcomes to be able to provide any sort of useful output. The first set of people to share everything wouldn't be able to derive any value from the tool, and I think you'd be hard pressed to convince enough people to do it to get a useful dataset.

You wouldn't need history. You can probably build a flowchart choose-your-own-adventure that gives good financial advice, and then have an LLM hold the user's hand through that flowchart.

Like, do you have $1,000 in an emergency fund? No? Start there.