step_functions module

Full Documentation for hippynn.experiment.step_functions module. Click here for a summary page.

This file implements various stepping protocols used by different optimizer APIs.

In particular:

• The “standard” step function which only requires that backwards has been called.

• The “closure” step function for when line search is required (currently only active on LBFGS)

• The “two step” style of Sharpness Aware Minimization algorithms

The main output function here is get_step_function(optimizer)-> callable.

The various step functions are provided as classes that act with staticmethods. This is to provide for the possibility of extension, for example, to schemes with stepping schemes that require additional state, or for the possibility to specifiy the step function explicitly within the controller.

class ClosureStep

Bases: StepFn

static step(optimizer, model, loss, batch_inputs, batch_targets)

class StandardStep

Bases: StepFn

static step(optimizer, model, loss, batch_inputs, batch_targets)

class StepFn

Bases: object

step = NotImplemented

class TwoStep

Bases: StepFn

static step(optimizer, model, loss, batch_inputs, batch_targets)

closure_step_fn(optimizer, model, loss, batch_inputs, batch_targets)

get_step_function(optimizer: Optimizer) → callable

standard_step_fn(optimizer, model, loss, batch_inputs, batch_targets)

twostep_step_fn(optimizer, model, loss, batch_inputs, batch_targets)