Evaluation Strategy¶

class substrafl.evaluation_strategy.EvaluationStrategy(test_data_nodes: List[substrafl.nodes.test_data_node.TestDataNode], rounds: Union[int, List[int]])¶

Creates an iterator which returns True or False depending on the defined strategy.

Parameters

test_data_nodes (List[TestDataNode]) – nodes on which the model is to be tested.
rounds (Union[int, List[int]]) – on which round the model is to be tested. If rounds is an int the model will be tested every rounds rounds starting from the first round. It will also be tested on the last round. If rounds is a list the model will be tested on the index of a round given in the rounds list. Note that the first round starts at 1.

Raises

ValueError – test_data_nodes cannot be an empty list
TypeError – test_data_nodes must be filled with instances of TestDataNode
TypeError – rounds must be a list or an int
ValueError – if rounds is an int it must be strictly positive
ValueError – if rounds is a list it cannot be empty
TypeError – if rounds is a list it must be filled with variables of type int
ValueError – if rounds is a list it must be filled only with positive integers

Return type

None

Example

Evaluation strategy which returns True every 2 rounds

my_evaluation_strategy = EvaluationStrategy(
        test_data_nodes = list_of_test_data_nodes,
        rounds = 2,
    )

every next next(my_evaluation_strategy) will return:

True
False
True
True
StopIteration Error

Example

Evaluation strategy which returns True on rounds 1 and 2

my_evaluation_strategy = EvaluationStrategy(
        test_data_nodes = list_of_test_data_nodes,
        rounds = [1, 2],
    )

every next next(my_evaluation_strategy) will return

True
True
False
False
False
StopIteration Error

__next__()¶: returns True if this round matches the defined strategy and False otherwise. required for iterators

property num_rounds¶

Property to get the num_rounds.

restart_rounds()¶: reinitializes current round to 0 (generator will start from the beginning)