Metric

framework3.base.base_clases.BaseMetric

Bases: BasePlugin

Base class for implementing metric calculations in the framework.

This abstract class defines the interface for metric evaluation and provides a structure for implementing various performance metrics. It extends BasePlugin to inherit core functionality for attribute management and serialization.

Key Features

• Abstract evaluate method for implementing specific metric calculations
• higher_better attribute to indicate if higher metric values are better
• Inherits BasePlugin functionality for attribute management and serialization

Usage

To create a new metric, inherit from this class and implement the evaluate method:

from framework3.base.base_clases import BaseMetric
from framework3.base.base_types import XYData
import numpy as np

class MeanSquaredError(BaseMetric):
    higher_better = False

    def evaluate(self, x_data: XYData, y_true: XYData, y_pred: XYData) -> float:
        return np.mean((y_true.value - y_pred.value) ** 2)

Attributes:

Name	Type	Description
higher_better	bool	Indicates whether higher values of the metric are better. Defaults to True.

Methods:

Name	Description
evaluate	XYData, y_true: XYData | None, y_pred: XYData) -> Float | np.ndarray: Abstract method to be implemented by subclasses for specific metric calculations.

Source code in framework3/base/base_clases.py

class BaseMetric(BasePlugin):
    """
    Base class for implementing metric calculations in the framework.

    This abstract class defines the interface for metric evaluation and provides
    a structure for implementing various performance metrics. It extends BasePlugin
    to inherit core functionality for attribute management and serialization.

    Key Features:
        - Abstract evaluate method for implementing specific metric calculations
        - higher_better attribute to indicate if higher metric values are better
        - Inherits BasePlugin functionality for attribute management and serialization

    Usage:
        To create a new metric, inherit from this class and implement the evaluate method:

        ```python
        from framework3.base.base_clases import BaseMetric
        from framework3.base.base_types import XYData
        import numpy as np

        class MeanSquaredError(BaseMetric):
            higher_better = False

            def evaluate(self, x_data: XYData, y_true: XYData, y_pred: XYData) -> float:
                return np.mean((y_true.value - y_pred.value) ** 2)
        ```

    Attributes:
        higher_better (bool): Indicates whether higher values of the metric are better.
                              Defaults to True.

    Methods:
        evaluate(x_data: XYData, y_true: XYData | None, y_pred: XYData) -> Float | np.ndarray:
            Abstract method to be implemented by subclasses for specific metric calculations.
    """

    higher_better: bool = True

    @abstractmethod
    def evaluate(
        self, x_data: XYData, y_true: XYData | None, y_pred: XYData
    ) -> Float | np.ndarray:
        """
        Evaluate the metric based on the provided data.

        This abstract method should be implemented by subclasses to calculate
        the specific metric. It provides a standardized interface for all metrics
        in the framework.

        Args:
            x_data (XYData): The input data used for the prediction.
            y_true (XYData | None): The ground truth or actual values. Can be None for some metrics.
            y_pred (XYData): The predicted values.

        Returns:
            Float | np.ndarray: The calculated metric value. This can be a single float
                                or a numpy array, depending on the specific metric implementation.

        Raises:
            NotImplementedError: If the subclass does not implement this method.

        Note:
            Subclasses must override this method to provide the specific metric calculation logic.
        """

        ...

higher_better = True class-attribute instance-attribute

evaluate(x_data, y_true, y_pred) abstractmethod

Evaluate the metric based on the provided data.

This abstract method should be implemented by subclasses to calculate the specific metric. It provides a standardized interface for all metrics in the framework.

Parameters:

Name	Type	Description	Default
x_data	XYData	The input data used for the prediction.	required
y_true	XYData | None	The ground truth or actual values. Can be None for some metrics.	required
y_pred	XYData	The predicted values.	required

Returns:

Type	Description
Float | ndarray	Float | np.ndarray: The calculated metric value. This can be a single float or a numpy array, depending on the specific metric implementation.

Raises:

Type	Description
NotImplementedError	If the subclass does not implement this method.

Note

Subclasses must override this method to provide the specific metric calculation logic.

Source code in framework3/base/base_clases.py

@abstractmethod
def evaluate(
    self, x_data: XYData, y_true: XYData | None, y_pred: XYData
) -> Float | np.ndarray:
    """
    Evaluate the metric based on the provided data.

    This abstract method should be implemented by subclasses to calculate
    the specific metric. It provides a standardized interface for all metrics
    in the framework.

    Args:
        x_data (XYData): The input data used for the prediction.
        y_true (XYData | None): The ground truth or actual values. Can be None for some metrics.
        y_pred (XYData): The predicted values.

    Returns:
        Float | np.ndarray: The calculated metric value. This can be a single float
                            or a numpy array, depending on the specific metric implementation.

    Raises:
        NotImplementedError: If the subclass does not implement this method.

    Note:
        Subclasses must override this method to provide the specific metric calculation logic.
    """

    ...