How to Export Evaluation Results in Tensorflow

In TensorFlow if you’re using a tf.estimator model, for instance tf.estimator.DNNLinearCombinedClassifier, and as part of your automated training infrastructure you want to save the evaluation results as a JSON file, it’s not super straightforward, so here’s how to do it.

Let’s say you define your EvalSpec like this:

eval_spec = tf.estimator.EvalSpec(eval_input_fn,

You’ll need to write a new exporter class that will take the eval_result from your evaluation step and save it to a file using the GFile API.

class EvalResultsExporter(tf.estimator.Exporter):
  """Passed into an EvalSpec for saving the result of the final evaluation
  step locally or in Google Cloud Storage.

  def __init__(self, name):
    assert name, '"name" argument is required.'
    self._name = name

  def name(self):
    return self._name

  def export(self, estimator, export_path, checkpoint_path,
    eval_result, is_the_final_export):

    if not is_the_final_export:
      return None'EvalResultsExporter (name: %s) '
      'running after final evaluation.') % self._name)'export_path: %s' % export_path)'eval_result: %s' % eval_result)

    for key, value in eval_result.iteritems():
      if isinstance(value, np.float32):
        eval_result[key] = value.item()


    with tf.gfile.GFile('%s/eval_results.json' % export_path, 'w') as f:

And then add that exporter to the list of exporters in your EvalSpec.

eval_spec = tf.estimator.EvalSpec(eval_input_fn,
  exporters=[exporter, EvalResultsExporter('eval_results')],

Contents (top)