Add laplace_latent_solve and laplace_latent_solve_tol functions
I'm not certain about the names, but the first word is common for alla laplace_ functions. The second word is common with laplace_latent_rng(_tol) which provides random draws of the latent values. The third word is based on the output being the Newton solver result and is connected to Stan having other solvers.
These functions would return a tuple with elements being mean and cholesky of covariance found by Newton solver.
We have laplace_marginal and laplace_latent_rng (and corresponding _tol versions. The first one computes the normalization term of the normal approximation (often marginal likelihood) and the second one draws from the normal approximation. There are some needs where it would be useful to have the actual solution. One example, is importance sampling corrected Laplace marginal, where we draw from the normal approximation $q$ and compute expectation $E[f/q]$ for which we need to evaluate density of the draws with respect to $q$ (normal approximation) and $f$ (the true distribution). Given latent solution mean and Cholesky of the covariance we can do both drawing from the approximation (which laplace_latent_rng could do, too) but also evaluating the density with respect to that normal approximation.
laplace_marginal and laplace_latent_rng already compute mean and I guess also Cholesky (or LDL) of the covariance, so we can start by copying one of these functions and just changing what is the output.
@florence-bockting was interested in implementing this, but pinging @charlesm93, @SteveBronder and @WardBrian for feedback on the name and other aspects. I assume this is small enough update that actual design-doc is not needed, but I can also make one if more details are needed
Add
laplace_latent_solveandlaplace_latent_solve_tolfunctionsI'm not certain about the names, but the first word is common for alla
laplace_functions. The second word is common withlaplace_latent_rng(_tol)which provides random draws of the latent values. The third word is based on the output being the Newton solver result and is connected to Stan having other solvers.These functions would return a tuple with elements being mean and cholesky of covariance found by Newton solver.
We have$q$ and compute expectation $E[f/q]$ for which we need to evaluate density of the draws with respect to $q$ (normal approximation) and $f$ (the true distribution). Given latent solution mean and Cholesky of the covariance we can do both drawing from the approximation (which
laplace_marginalandlaplace_latent_rng(and corresponding_tolversions. The first one computes the normalization term of the normal approximation (often marginal likelihood) and the second one draws from the normal approximation. There are some needs where it would be useful to have the actual solution. One example, is importance sampling corrected Laplace marginal, where we draw from the normal approximationlaplace_latent_rngcould do, too) but also evaluating the density with respect to that normal approximation.laplace_marginalandlaplace_latent_rngalready compute mean and I guess also Cholesky (or LDL) of the covariance, so we can start by copying one of these functions and just changing what is the output.@florence-bockting was interested in implementing this, but pinging @charlesm93, @SteveBronder and @WardBrian for feedback on the name and other aspects. I assume this is small enough update that actual design-doc is not needed, but I can also make one if more details are needed