Fisher Information metric

[Nimrais]

Fisher Information Metric and Second-Order Retractions for Exponential Family Manifolds

Key Features

1. Fisher Information Metric

The FisherInformationMetric type integrates the Fisher information matrix as a Riemannian metric for exponential family manifolds. This creates a proper geometric structure that respects the natural parameterization of exponential families.

2. Retraction Methods

Two levels of retraction accuracy are implemented:

First-Order Retraction

A simple linear retraction that applies a tangent vector directly in parameter space. This is equivalent to a standard Euclidean step in the natural parameters.

Second-Order Retraction

A more sophisticated retraction that accounts for manifold curvature using Christoffel symbols. This produces a much better approximation to the true exponential map, especially for highly curved distributions.

Usage Example

using ExponentialFamilyManifolds
using Manifolds

# Create a manifold for Beta distribution parameters
M = get_natural_manifold(Beta, ())

# Generate points and tangent vectors
p = rand(M)
X = rand(M; vector_at=p)

# Use first-order retraction
q1 = retract(M, p, X, FirstOrderRetraction())

# Use second-order retraction with ForwardDiff backend
using ADTypes: AutoForwardDiff
q2 = retract(M, p, X, SecondOrderRetraction(backend=AutoForwardDiff()))

[kellertuer]

This is just a very short first impression – but it looks nice!

[kellertuer]

Is this a retraction type? Then it could subtype https://github.com/JuliaManifolds/ManifoldsBase.jl/blob/85b42907c26df0463f3fc91ba7dafd3fa534f800/src/retractions.jl#L8-L13 ?

[kellertuer]

is this maybe similar to https://juliamanifolds.github.io/Manifolds.jl/stable/manifolds/metric/#Manifolds.metric-Tuple{MetricManifold} ?

[kellertuer]

I personally would prefer get_X methods, since Julia is often snake_case (ok also with counter examples like isapprox)

[Nimrais]

is this maybe similar to https://juliamanifolds.github.io/Manifolds.jl/stable/manifolds/metric/#Manifolds.metric-Tuple{MetricManifold} ?

I think it’s exactly the same, thanks

[kellertuer]

This seems to indicate your retraction type atop should indeed be an AbstractRetraction subtype – and in the long run this could be documented a bit more. For me natural coordinates seemed a bit magic in the beginning ;)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.31%. Comparing base (bd8eff2) to head (cb1b998).

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #33      +/-   ##
==========================================
+ Coverage   96.81%   97.31%   +0.49%     
==========================================
  Files          25       25              
  Lines         345      409      +64     
==========================================
+ Hits          334      398      +64     
  Misses         11       11              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.