Configuration¶
The configuration schema is a set of dataclasses: SONNConfig holds the
model and train sections and a few top-level flags. Their fields are
documented by comments in the source and, on this site, in
Configuration keys; this page shows the classes.
Configuration shows how to build a
configuration from them.
dataclass
¶
SONNConfig(model: ModelConfig = ModelConfig(), train: TrainConfig = TrainConfig(), resume: bool = False, finetune_end_to_end: bool = False, finetune_drop_head: bool = False, finetune_prune_first: bool = False, tutorial: Dict[str, Any] = dict())
Root config schema: the model: and train: sections plus top-level flags.
Registered with Hydra as default; SONN merges the user config into it,
so unknown keys are rejected. The comments next to the fields document
them.
dataclass
¶
ModelConfig(type: str = 'multi-class', soft_binner: bool = True, soft_binner_scale: float = 100.0, num_classes: int = 3, ref_functions: List[Any] = (lambda: ['linear_cov'])(), shortcut: Any = (lambda: {'raw_features': True, 'prev_layers': None})(), squash_method: str = 'sigma', squash_n_sigma: float = 2.0, squash_core_range: float = 0.75, nbest_neurons: Optional[int] = None, max_neuron_models: Optional[int] = None, use_output_projection: bool = False, num_out_neurons: Optional[int] = None, use_neuron_proj: bool = False, output_clamp_value: float = 1000.0, use_layer_norm: bool = False)
The model: section: task type, neuron families and network structure.
The comments next to the fields document them.
dataclass
¶
TrainConfig(seed: int = 10, use_deterministic_algorithms: bool = False, criterion_type: str = 'validate', bias_ce_type: str = 'js', error_alpha: float = 0.5, error_normalization: str = 'variance', max_layer_count: int = 999, criterion_minimum_width: int = 5, stop_train_epsilon_condition: float = 0.001, stop_train_min_delta: float = 0.0, layer_err_criterion: str = 'top', layer_err_source: str = 'neuron', neuron_selection_method: str = 'plain', neuron_selection_orth_threshold: float = 0.3, eval_step_interval: int = 1000, eval_display_interval: int = 1, eval_smoothing_factor: float = 0.2, save_interval: int = 1000, save_last_layer: bool = True, keep_last_n: int = 10, skip_saving_at_epoch_end: bool = True, checkpoint_dir: str = 'checkpoints', early_stop_completion_percentage: int = 100, early_stop_patience: float = 0.0001, early_stop_tolerance_steps: int = 10, early_stop_source: str = 'dev', stop_source: str = 'dev', shared_proj_lr_multiplier: float = 0.1, precompute_features: bool = False, log_layer_diagnostics: bool = False, divergence_threshold: float = float('inf'), ridge_alpha: float = 0.0, censor_target_at: Optional[float] = None, input_sample_rows: int = 65536, rbf_kmeans_mode: str = 'auto', rbf_kmeans_iters: int = 20, rbf_kmeans_passes: int = 1, optimizer: OptimizerConfig = OptimizerConfig(), out_proj_train: OutProjTrainConfig = OutProjTrainConfig(), finetune_train: OutProjTrainConfig = OutProjTrainConfig(), layer_finetune: bool = False, device: str = 'cpu', batch_size: int = 1, steps: int = 1000, shuffle: bool = False, verbose: bool = True)
The train: section: fitting, selection, stopping and checkpointing.
The comments next to the fields document them.
dataclass
¶
OptimizerConfig(name: str = 'adam', verbose: bool = True, optimizer_params: Dict[str, Any] = _default_optimizer_params())
The train.optimizer: section: the optimizer for candidate neuron fits.
The comments next to the fields document them.
dataclass
¶
OutProjTrainConfig(max_steps: int = 5000, lr: float = 0.001, weight_decay: float = 0.0, optimizer: str = 'adam', eval_interval: int = 100, lr_patience: int = 5, lr_factor: float = 0.5, lr_min: float = 1e-05, early_stop_patience: int = 10, early_stop_min_delta: float = 0.0001, keep_best_weights: bool = False, best_weights_copy: str = 'device', lbfgs_history_size: int = 20, lbfgs_max_iter: int = 20, lbfgs_line_search: str = 'strong_wolfe', cuda_graph: bool = True, data_on_device: str = 'auto')
Optimizer settings for a head or fine-tune pass.
Used twice in TrainConfig: train.out_proj_train (the output head fit
and the per-layer fine-tune) and train.finetune_train (the end-to-end
pass). The comments next to the fields document them.