| ▲ | xgk 4 hours ago | |||||||||||||
The paper does not seem to define what Energy-Based means. The paper says: "EBMs learn a scalar function that assigns an energy to a candidate prediction given an input." What codomain do those learned scalar function have? I guess real numbers? But is "Energy" just a synonym for the reals? | ||||||||||||||
| ▲ | program_whiz 3 hours ago | parent | next [-] | |||||||||||||
See my comment below, I explain what it means. Energy is just a number that comes out of matrix which learns the relationship between different states of the system being searched. For example, suppose we're talking about sudoku. You would have a matrix that has W_ij (weights between spaces). For each of those, there are 9x9 numbers (energy between two of the digits). A combination like W[0, 1](4, 4) is high because you would have a 4 in the first space, and a 4 in the 2nd space (which breaks the rules). Most of the values are low energy (e.g. 4 and 2 is energy 0, no conflict). However, as different numbers are placed, they are checking for the lowest energy number across the whole board, and the "clues" are locked in. So if you had a "clue" like 2 in a space, then the energy for 2 on that row / column would be high, but other numbers low. You just sum this up and pick the lowest energy number. As you add numbers, you can hit conflicts (e.g. two 7s in the row), changing one of them will be the next lowest energy move. The probability of changing a number is inversely proportional to its energy. So changing the conflicting numbers is the usual most likely move. So in the best case it works like this: For each empty cell, pick a number at random, but its weighted by the other numbers that affect it (e.g. if 7 is in the row, the chance of 7 is 0.1%). Continue doing this on empty cells, unless there is a conflict in solved cells (higher prob of mutation), in which case change that number first. So its a slightly fuzzified version of a very fast concrete algo. "Energy" is just a number from training, its defining by the task at hand (in this case, "energy" is how much conflict numbers have on the board, 0 meaning no conflict and solved if every cell is filled in). | ||||||||||||||
| ||||||||||||||
| ▲ | CuriouslyC 3 hours ago | parent | prev | next [-] | |||||||||||||
To simplify the explanation further, the main difference in practice between energy based and more traditional models is that they throw out normalization. | ||||||||||||||
| ||||||||||||||
| ▲ | usernametaken29 4 hours ago | parent | prev [-] | |||||||||||||
I find this particularly concerning because in their related works they did not refer to the free energy principle, which is the underlying principle of energy based cognition... so eh... what are they referring to? If its not the energy as defined by FEP and active inference, which energy is it? Did they mean entropy? I didn't get it from the paper, really | ||||||||||||||