sc2bench.analysis

sc2bench.analysis.register_analysis_class(cls)

Registers an analyzer class.

Parameters:

cls (class) – analyzer class to be registered

Returns:

registered analyzer class

Return type:

class

class sc2bench.analysis.AnalyzableModule(analyzer_configs=None)

A base module to analyze and summarize the wrapped modules and intermediate representations.

Parameters:

analyzer_configs (list[dict] or None) – list of analysis configurations

forward(*args, **kwargs)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

activate_analysis()

Makes internal analyzers ready to run.

deactivate_analysis()

Turns internal analyzers off.

analyze(compressed_obj)

Analyzes a compressed object using internal analyzers.

Parameters:

compressed_obj (Any) – compressed object

summarize()

Shows each of internal analyzers’ summary of results.

clear_analysis()

Clears each of internal analyzers’ results.

class sc2bench.analysis.BaseAnalyzer

A base analyzer to analyze and summarize the wrapped modules and intermediate representations.

analyze(*args, **kwargs)

Analyzes a compressed object.

summarize()

Shows the summary of results.

This should be overridden by all subclasses.

clear()

Clears the results.

This should be overridden by all subclasses.

class sc2bench.analysis.FileSizeAnalyzer(unit='KB', **kwargs)

An analyzer to measure file size of compressed object in the designated unit.

Parameters:

unit (str) – unit of data size in bytes (‘B’, ‘KB’, ‘MB’)

analyze(compressed_obj)

Computes and appends binary object size of the compressed object.

Parameters:

compressed_obj (Any) – compressed object

summarize()

Computes and shows mean and std of the stored file sizes and the number of samples .

clear()

Clears the file size list.

class sc2bench.analysis.FileSizeAccumulator(unit='KB', **kwargs)

An accumulator to store pre-computed file size in the designated unit.

Parameters:

unit (str) – unit of data size in bytes (‘B’, ‘KB’, ‘MB’)

analyze(file_size)

Appends a file size.

Parameters:

file_size (int or float) – pre-computed file size

sc2bench.analysis.get_analyzer(cls_name, **kwargs)

Gets an analyzer module.

Parameters:

• cls_name (str) – analyzer class name

• kwargs (dict) – kwargs for the analyzer class

Returns:

analyzer module

Return type:

BaseAnalyzer or None

sc2bench.analysis.check_if_analyzable(module)

Checks if a module is an instance of AnalyzableModule.

Parameters:

module (Any) – module

Returns:

True if the module is an instance of AnalyzableModule. False otherwise

Return type:

bool

sc2bench.analysis.analyze_model_size(model, encoder_paths=None, additional_rest_paths=None, ignores_dtype_error=True)

Approximates numbers of bits used for parameters of the whole model, encoder, and the rest of the model.

Parameters:

• model (nn.Module) – model

• encoder_paths (list[str] or None) – list of module paths for the model to be considered as part of encoder’s parameters

• additional_rest_paths (list[str] or None) – list of additional rest module paths whose parameters should be shared with encoder e.g., module path of entropy bottleneck in the model if applied

• ignores_dtype_error (bool) – if False, raise an error when any unexpected dtypes are found

Returns:

model size (sum of param x num_bits) with three keys: model (whole model), encoder, and the rest

Return type:

dict