metric_tracker module

Full Documentation for hippynn.experiment.metric_tracker module. Click here for a summary page.

Keep track of training metrics over the experiment

class MetricTracker(metric_names, stopping_key, quiet=False)

Bases: object

MetricTracker instances keep track of metrics and models for an experiment

Variables:

• metric_names – the tracked metrics

• stopping_key – the metric used to determine what model is best

• best_metric_values – dictionary of metric keys to the best metric values observed so far. lower is assumed to be better.

• best_model – state dict for the best model so far.

• epoch_metric_values – list (index epoch) of dictionary (key split, value of dictionary (key metric, value metric value)

• other_metric_values – dictionary of metric values at other times than after an epoch, for example the final test values.

• epoch_times – timing info for each epoch

• quiet – whether to print the values registered.

evaluation_print(evaluation_dict, quiet=None, _print=<built-in function print>)

evaluation_print_better(evaluation_dict, better_dict, quiet=None, _print=<built-in function print>)

classmethod from_evaluator(evaluator)

plot_over_time()

register_metrics(metric_info, when)

Parameters:

• metric_info – dictionary of metric names: metric values

• when – string or integer specifying epoch number.

Returns:

property current_epoch

plot_all_over_time(*args)

table_evaluation_print(evaluation_dict, metric_names, n_columns, _print=<built-in function print>)

Print metric results as a table.

Parameters:

• evaluation_dict – dict[eval type]->dict[metric]->value

• metric_names – Names

• n_columns – Number of columns for name fields.

Returns:

None

table_evaluation_print_better(evaluation_dict, better_dict, metric_names, n_columns, _print=<built-in function print>)

Print metric results as a table, add a ‘*’ character for metrics in better_dict.

Parameters:

• evaluation_dict – dict[eval type]->dict[metric]->value

• better_dict – dict[eval type]->dict[metric]->bool

• metric_names – Names

• n_columns – Number of columns for name fields.

Returns:

None