torchdistill.misc

---

torchdistill.misc.log

torchdistill.misc.log.set_basic_log_config()

Sets a default basic configuration for logging.

torchdistill.misc.log.setup_log_file(log_file_path)

Sets a file handler with log_file_path to write a log file.

Parameters:

log_file_path (str) – log file path.

class torchdistill.misc.log.TrainingTracker(engine, **kwargs)

A thin wrapper around an experiment tracking library (trackio or wandb). The library is imported lazily so that it stays an optional dependency.

Parameters:

• engine (str) – tracking library name (‘trackio’ or ‘wandb’).

• kwargs (dict) – keyword arguments passed to the library’s init function (e.g., project, name, config). See the external references below for available arguments.

An example to instantiate TrainingTracker and log metrics with it.

 tracker = TrainingTracker(
     'trackio',
     project='torchdistill-cifar10',
     name='resnet18-kd-run1',
     config={'train': {'num_epochs': 182}, 'student_model': 'resnet18'}
 )
 tracker.log({'train/loss': 0.512, 'train/lr': 0.1}, step=100)
 tracker.log({'val/acc1': 92.3, 'epoch': 0}, step=391)
 tracker.finish()

See also

• Trackio documentation (trackio.init) for engine='trackio'

• wandb.init reference for engine='wandb'

log(metrics, step=None)

Logs a metric dict.

Parameters:

• metrics (dict) – metric names and values.

• step (int or None) – global step to associate the metrics with.

finish()

Finishes the tracking run.

torchdistill.misc.log.setup_tracker(tracker_config, run_config=None)

Sets up a TrainingTracker from tracker_config.

Parameters:

• tracker_config (dict or None) – tracker configuration with ‘engine’ (‘trackio’ or ‘wandb’) and optional ‘kwargs’ passed to the library’s init function. If None or its ‘engine’ is None, no tracker is set up.

• run_config (dict or None) – run configuration (e.g., loaded yaml config) to be logged as the run’s config.

Returns:

training tracker if configured and this is the main process, None otherwise.

Return type:

TrainingTracker or None

An example (partial) YAML config whose tracker entry is passed to setup_tracker() as tracker_config. kwargs is passed as-is to trackio.init / wandb.init.

 tracker:
   engine: 'trackio'
   kwargs:
     project: 'torchdistill-cifar10'
     name: 'resnet18-kd-run1'

An example to set up a TrainingTracker with the YAML config above.

 config = yaml_util.load_yaml_file('/path/to/the/yaml/config/above.yaml')
 tracker = setup_tracker(config.get('tracker', None), run_config=config)

 # Equivalent dict-based setup without a YAML file
 tracker = setup_tracker(
     {'engine': 'trackio', 'kwargs': {'project': 'torchdistill-cifar10', 'name': 'resnet18-kd-run1'}},
     run_config=config
 )

See also

• Trackio documentation (trackio.init) for engine: 'trackio'

• wandb.init reference for engine: 'wandb'

class torchdistill.misc.log.TrainingTrackerReader(engine, wandb_entity=None)

A read-side companion to TrainingTracker that loads metrics logged with trackio or wandb back as pandas.DataFrame. The libraries are imported lazily so that they stay optional dependencies.

Parameters:

• engine (str) – tracking library name (‘trackio’ or ‘wandb’).

• wandb_entity (str or None) – wandb entity (user or team name). Used only if engine = ‘wandb’. If None, the default entity of the wandb API key is used.

An example to load the run history logged with the TrainingTracker example.

 reader = TrainingTrackerReader('trackio')
 history = reader.load_run_history('torchdistill-cifar10', 'resnet18-kd-run1')
 # `history` is a pandas.DataFrame with 'step', 'relative_time', and metric columns
 # such as 'train/loss' and 'val/acc1'
 print(history[history['val/acc1'].notna()][['epoch', 'val/acc1']])

See also

• Trackio documentation for engine='trackio'

• wandb public API reference for engine='wandb'

load_run_history(project, run_name)

Loads the metric history of a run as a pandas.DataFrame, one row per logged step. Engine-specific columns are normalized so that ‘step’ and ‘relative_time’ (seconds since the first log) are available for both engines, in addition to the logged metric columns.

Parameters:

• project (str) – project name used at logging time.

• run_name (str) – run name used at logging time.

Returns:

metric history of the run.

Return type:

pandas.DataFrame

class torchdistill.misc.log.SmoothedValue(window_size=20, fmt=None)

A deque-based value object tracks a series of values and provides access to smoothed values over a window or the global series average. The original implementation is https://github.com/pytorch/vision/blob/main/references/classification/utils.py

Parameters:

• window_size (int) – window size.

• fmt (str or None) – text format.

update(value, n=1)

Appends value.

Parameters:

• value (float or int) – value to be added.

• n (int) – sample count.

synchronize_between_processes()

Synchronizes between processes.

Warning

It does not synchronize the deque.

class torchdistill.misc.log.MetricLogger(delimiter='\t', tracker=None, tracker_prefix='', tracker_start_step=0)

A metric logger with SmoothedValue. The original implementation is https://github.com/pytorch/vision/blob/main/references/classification/utils.py

Parameters:

• delimiter (str) – delimiter in a log message.

• tracker (TrainingTracker or None) – training tracker to log metrics with. If None, no tracking is done.

• tracker_prefix (str) – prefix prepended to metric names when logging with tracker (e.g., ‘train/’).

• tracker_start_step (int) – global step at which this logger’s iterations start.

update(**kwargs)

Updates a metric dict whose values are SmoothedValue.

Parameters:

kwargs (dict) – keys and values.

synchronize_between_processes()

Synchronizes between processes.

add_meter(name, meter)

Add a new metric name and value.

Parameters:

• name (str) – metric name.

• meter (SmoothedValue) – smoothed value.

log_every(iterable, log_freq, header=None)

Add a new metric name and value.

Parameters:

• iterable (Iterable) – iterable object (e.g., data loader).

• log_freq (int) – log frequency.

• header (str) – log message header.

Returns:

item in iterative.

Return type:

Any