Clustering Metrics¶

`ARI` ¶

Bases: BaseMetric

Adjusted Rand Index (ARI) metric for clustering evaluation.

This class calculates the ARI score, which is the corrected-for-chance version of the Rand Index. It measures similarity between two clusterings, adjusted for chance.

Key Features

Measures the similarity of the cluster assignments, ignoring permutations and with chance normalization
Suitable for comparing clusterings of different sizes
Symmetric: switching argument order will not change the score

Usage

The ARI metric can be used to evaluate clustering algorithms:

from framework3.plugins.metrics.clustering import ARI
from framework3.base.base_types import XYData
import numpy as np

# Create sample data
x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
y_true = np.array([0, 0, 0, 1, 1, 1])
y_pred = np.array([0, 0, 1, 1, 1, 1])

# Create and use the ARI metric
ari_metric = ARI()
score = ari_metric.evaluate(x_data, y_true, y_pred)
print(f"ARI Score: {score}")

Methods:

Name	Description
`evaluate`	XYData, y_true: Any, y_pred: Any, **kwargs) -> Float \| np.ndarray: Calculate the Adjusted Rand Index score.

Note

This metric uses scikit-learn's adjusted_rand_score function internally. Ensure that scikit-learn is properly installed and compatible with your environment.

Source code in framework3/plugins/metrics/clustering.py

@Container.bind()
class ARI(BaseMetric):
    """
    Adjusted Rand Index (ARI) metric for clustering evaluation.

    This class calculates the ARI score, which is the corrected-for-chance version of the Rand Index.
    It measures similarity between two clusterings, adjusted for chance.

    Key Features:
        - Measures the similarity of the cluster assignments, ignoring permutations and with chance normalization
        - Suitable for comparing clusterings of different sizes
        - Symmetric: switching argument order will not change the score

    Usage:
        The ARI metric can be used to evaluate clustering algorithms:

        ```python
        from framework3.plugins.metrics.clustering import ARI
        from framework3.base.base_types import XYData
        import numpy as np

        # Create sample data
        x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
        y_true = np.array([0, 0, 0, 1, 1, 1])
        y_pred = np.array([0, 0, 1, 1, 1, 1])

        # Create and use the ARI metric
        ari_metric = ARI()
        score = ari_metric.evaluate(x_data, y_true, y_pred)
        print(f"ARI Score: {score}")
        ```

    Methods:
        evaluate(x_data: XYData, y_true: Any, y_pred: Any, **kwargs) -> Float | np.ndarray:
            Calculate the Adjusted Rand Index score.

    Note:
        This metric uses scikit-learn's adjusted_rand_score function internally. Ensure that scikit-learn
        is properly installed and compatible with your environment.
    """

    def evaluate(
        self, x_data: XYData, y_true: Any, y_pred: Any, **kwargs: Dict[str, Any]
    ) -> Float | np.ndarray:
        """
        Calculate the Adjusted Rand Index score.

        This method computes the ARI score between two clusterings.

        Args:
            x_data (XYData): The input data (not used in this metric, but required by the interface).
            y_true (Any): The ground truth labels.
            y_pred (Any): The predicted cluster labels.
            **kwargs (Dict[str,Any]): Additional keyword arguments passed to sklearn's adjusted_rand_score.

        Returns:
            Float | np.ndarray: The ARI score.

        Note:
            This method uses scikit-learn's adjusted_rand_score function internally.
        """
        return adjusted_rand_score(y_true, y_pred)

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

Calculate the Adjusted Rand Index score.

This method computes the ARI score between two clusterings.

Parameters:

Name	Type	Description	Default
`x_data`	`XYData`	The input data (not used in this metric, but required by the interface).	required
`y_true`	`Any`	The ground truth labels.	required
`y_pred`	`Any`	The predicted cluster labels.	required
`**kwargs`	`Dict[str, Any]`	Additional keyword arguments passed to sklearn's adjusted_rand_score.	`{}`

Returns:

Type	Description
`Float \| ndarray`	Float \| np.ndarray: The ARI score.

Note

This method uses scikit-learn's adjusted_rand_score function internally.

Source code in framework3/plugins/metrics/clustering.py

def evaluate(
    self, x_data: XYData, y_true: Any, y_pred: Any, **kwargs: Dict[str, Any]
) -> Float | np.ndarray:
    """
    Calculate the Adjusted Rand Index score.

    This method computes the ARI score between two clusterings.

    Args:
        x_data (XYData): The input data (not used in this metric, but required by the interface).
        y_true (Any): The ground truth labels.
        y_pred (Any): The predicted cluster labels.
        **kwargs (Dict[str,Any]): Additional keyword arguments passed to sklearn's adjusted_rand_score.

    Returns:
        Float | np.ndarray: The ARI score.

    Note:
        This method uses scikit-learn's adjusted_rand_score function internally.
    """
    return adjusted_rand_score(y_true, y_pred)

`CalinskiHarabasz` ¶

Bases: BaseMetric

Calinski-Harabasz Index metric for clustering evaluation.

This class calculates the Calinski-Harabasz Index, which is the ratio of the sum of between-clusters dispersion and of inter-cluster dispersion for all clusters.

Key Features

Measures the ratio of between-cluster variance to within-cluster variance
Higher values indicate better-defined clusters
Can be used to determine the optimal number of clusters

Usage

The Calinski-Harabasz metric can be used to evaluate clustering algorithms:

from framework3.plugins.metrics.clustering import CalinskiHarabasz
from framework3.base.base_types import XYData
import numpy as np

# Create sample data
x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
y_pred = np.array([0, 0, 0, 1, 1, 1])

# Create and use the Calinski-Harabasz metric
ch_metric = CalinskiHarabasz()
score = ch_metric.evaluate(x_data, None, y_pred)
print(f"Calinski-Harabasz Score: {score}")

Methods:

Name	Description
`evaluate`	XYData, y_true: Any, y_pred: Any, **kwargs) -> Float \| np.ndarray: Calculate the Calinski-Harabasz Index.

Note

This metric uses scikit-learn's calinski_harabasz_score function internally. Ensure that scikit-learn is properly installed and compatible with your environment.

Source code in framework3/plugins/metrics/clustering.py

@Container.bind()
class CalinskiHarabasz(BaseMetric):
    """
    Calinski-Harabasz Index metric for clustering evaluation.

    This class calculates the Calinski-Harabasz Index, which is the ratio of the sum of
    between-clusters dispersion and of inter-cluster dispersion for all clusters.

    Key Features:
        - Measures the ratio of between-cluster variance to within-cluster variance
        - Higher values indicate better-defined clusters
        - Can be used to determine the optimal number of clusters

    Usage:
        The Calinski-Harabasz metric can be used to evaluate clustering algorithms:

        ```python
        from framework3.plugins.metrics.clustering import CalinskiHarabasz
        from framework3.base.base_types import XYData
        import numpy as np

        # Create sample data
        x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
        y_pred = np.array([0, 0, 0, 1, 1, 1])

        # Create and use the Calinski-Harabasz metric
        ch_metric = CalinskiHarabasz()
        score = ch_metric.evaluate(x_data, None, y_pred)
        print(f"Calinski-Harabasz Score: {score}")
        ```

    Methods:
        evaluate(x_data: XYData, y_true: Any, y_pred: Any, **kwargs) -> Float | np.ndarray:
            Calculate the Calinski-Harabasz Index.

    Note:
        This metric uses scikit-learn's calinski_harabasz_score function internally. Ensure that scikit-learn
        is properly installed and compatible with your environment.
    """

    def evaluate(
        self, x_data: XYData, y_true: Any, y_pred: Any, **kwargs: Dict[str, Any]
    ) -> Float | np.ndarray:
        """
        Calculate the Calinski-Harabasz Index.

        This method computes the Calinski-Harabasz Index for the clustering.

        Args:
            x_data (XYData): The input data.
            y_true (Any): Not used for this metric, but required by the interface.
            y_pred (Any): The predicted cluster labels.
            **kwargs (Dict[str,Any]): Additional keyword arguments passed to sklearn's calinski_harabasz_score.

        Returns:
            Float | np.ndarray: The Calinski-Harabasz Index.

        Note:
            This method uses scikit-learn's calinski_harabasz_score function internally.
        """
        return calinski_harabasz_score(x_data.value, y_pred)

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

Calculate the Calinski-Harabasz Index.

This method computes the Calinski-Harabasz Index for the clustering.

Parameters:

Name	Type	Description	Default
`x_data`	`XYData`	The input data.	required
`y_true`	`Any`	Not used for this metric, but required by the interface.	required
`y_pred`	`Any`	The predicted cluster labels.	required
`**kwargs`	`Dict[str, Any]`	Additional keyword arguments passed to sklearn's calinski_harabasz_score.	`{}`

Returns:

Type	Description
`Float \| ndarray`	Float \| np.ndarray: The Calinski-Harabasz Index.

Note

This method uses scikit-learn's calinski_harabasz_score function internally.

Source code in framework3/plugins/metrics/clustering.py

def evaluate(
    self, x_data: XYData, y_true: Any, y_pred: Any, **kwargs: Dict[str, Any]
) -> Float | np.ndarray:
    """
    Calculate the Calinski-Harabasz Index.

    This method computes the Calinski-Harabasz Index for the clustering.

    Args:
        x_data (XYData): The input data.
        y_true (Any): Not used for this metric, but required by the interface.
        y_pred (Any): The predicted cluster labels.
        **kwargs (Dict[str,Any]): Additional keyword arguments passed to sklearn's calinski_harabasz_score.

    Returns:
        Float | np.ndarray: The Calinski-Harabasz Index.

    Note:
        This method uses scikit-learn's calinski_harabasz_score function internally.
    """
    return calinski_harabasz_score(x_data.value, y_pred)

`Completeness` ¶

Bases: BaseMetric

Completeness metric for clustering evaluation.

This class calculates the Completeness score, which measures whether all members of a given class are assigned to the same cluster.

Key Features

Measures the extent to which all members of a given class are assigned to the same cluster
Ranges from 0 to 1, where 1 indicates perfectly complete clustering
Invariant to label switching

Usage

The Completeness metric can be used to evaluate clustering algorithms:

from framework3.plugins.metrics.clustering import Completeness
from framework3.base.base_types import XYData
import numpy as np

# Create sample data
x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
y_true = np.array([0, 0, 0, 1, 1, 1])
y_pred = np.array([0, 0, 1, 1, 1, 1])

# Create and use the Completeness metric
completeness_metric = Completeness()
score = completeness_metric.evaluate(x_data, y_true, y_pred)
print(f"Completeness Score: {score}")

Methods:

Name	Description
`evaluate`	XYData, y_true: Any, y_pred: Any, **kwargs) -> Float \| np.ndarray: Calculate the Completeness score.

Note

This metric uses scikit-learn's completeness_score function internally. Ensure that scikit-learn is properly installed and compatible with your environment.

Source code in framework3/plugins/metrics/clustering.py

@Container.bind()
class Completeness(BaseMetric):
    """
    Completeness metric for clustering evaluation.

    This class calculates the Completeness score, which measures whether all members of
    a given class are assigned to the same cluster.

    Key Features:
        - Measures the extent to which all members of a given class are assigned to the same cluster
        - Ranges from 0 to 1, where 1 indicates perfectly complete clustering
        - Invariant to label switching

    Usage:
        The Completeness metric can be used to evaluate clustering algorithms:

        ```python
        from framework3.plugins.metrics.clustering import Completeness
        from framework3.base.base_types import XYData
        import numpy as np

        # Create sample data
        x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
        y_true = np.array([0, 0, 0, 1, 1, 1])
        y_pred = np.array([0, 0, 1, 1, 1, 1])

        # Create and use the Completeness metric
        completeness_metric = Completeness()
        score = completeness_metric.evaluate(x_data, y_true, y_pred)
        print(f"Completeness Score: {score}")
        ```

    Methods:
        evaluate (x_data: XYData, y_true: Any, y_pred: Any, **kwargs) -> Float | np.ndarray:
            Calculate the Completeness score.

    Note:
        This metric uses scikit-learn's completeness_score function internally. Ensure that scikit-learn
        is properly installed and compatible with your environment.
    """

    def evaluate(
        self, x_data: XYData, y_true: Any, y_pred: Any, **kwargs: Dict[str, Any]
    ) -> Float | np.ndarray:
        """
        Calculate the Completeness score.

        This method computes the Completeness score for the clustering.

        Args:
            x_data (XYData): The input data (not used in this metric, but required by the interface).
            y_true (Any): The ground truth labels.
            y_pred (Any): The predicted cluster labels.
            **kwargs (Dict[str,Any]): Additional keyword arguments passed to sklearn's completeness_score.

        Returns:
            Float | np.ndarray: The Completeness score.

        Note:
            This method uses scikit-learn's completeness_score function internally.
        """
        return completeness_score(y_true, y_pred)

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

Calculate the Completeness score.

This method computes the Completeness score for the clustering.

Parameters:

Name	Type	Description	Default
`x_data`	`XYData`	The input data (not used in this metric, but required by the interface).	required
`y_true`	`Any`	The ground truth labels.	required
`y_pred`	`Any`	The predicted cluster labels.	required
`**kwargs`	`Dict[str, Any]`	Additional keyword arguments passed to sklearn's completeness_score.	`{}`

Returns:

Type	Description
`Float \| ndarray`	Float \| np.ndarray: The Completeness score.

Note

This method uses scikit-learn's completeness_score function internally.

Source code in framework3/plugins/metrics/clustering.py

def evaluate(
    self, x_data: XYData, y_true: Any, y_pred: Any, **kwargs: Dict[str, Any]
) -> Float | np.ndarray:
    """
    Calculate the Completeness score.

    This method computes the Completeness score for the clustering.

    Args:
        x_data (XYData): The input data (not used in this metric, but required by the interface).
        y_true (Any): The ground truth labels.
        y_pred (Any): The predicted cluster labels.
        **kwargs (Dict[str,Any]): Additional keyword arguments passed to sklearn's completeness_score.

    Returns:
        Float | np.ndarray: The Completeness score.

    Note:
        This method uses scikit-learn's completeness_score function internally.
    """
    return completeness_score(y_true, y_pred)

`Homogeneity` ¶

Bases: BaseMetric

Homogeneity metric for clustering evaluation.

This class calculates the Homogeneity score, which measures whether all of its clusters contain only data points which are members of a single class.

Key Features

Measures the extent to which each cluster contains only members of a single class
Ranges from 0 to 1, where 1 indicates perfectly homogeneous clustering
Invariant to label switching

Usage

The Homogeneity metric can be used to evaluate clustering algorithms:

from framework3.plugins.metrics.clustering import Homogeneity
from framework3.base.base_types import XYData
import numpy as np

# Create sample data
x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
y_true = np.array([0, 0, 0, 1, 1, 1])
y_pred = np.array([0, 0, 1, 1, 1, 1])

# Create and use the Homogeneity metric
homogeneity_metric = Homogeneity()
score = homogeneity_metric.evaluate(x_data, y_true, y_pred)
print(f"Homogeneity Score: {score}")

Methods:

Name	Description
`evaluate`	XYData, y_true: Any, y_pred: Any, **kwargs) -> Float \| np.ndarray: Calculate the Homogeneity score.

Note

This metric uses scikit-learn's homogeneity_score function internally. Ensure that scikit-learn is properly installed and compatible with your environment.

Source code in framework3/plugins/metrics/clustering.py

@Container.bind()
class Homogeneity(BaseMetric):
    """
    Homogeneity metric for clustering evaluation.

    This class calculates the Homogeneity score, which measures whether all of its clusters
    contain only data points which are members of a single class.

    Key Features:
        - Measures the extent to which each cluster contains only members of a single class
        - Ranges from 0 to 1, where 1 indicates perfectly homogeneous clustering
        - Invariant to label switching

    Usage:
        The Homogeneity metric can be used to evaluate clustering algorithms:

        ```python
        from framework3.plugins.metrics.clustering import Homogeneity
        from framework3.base.base_types import XYData
        import numpy as np

        # Create sample data
        x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
        y_true = np.array([0, 0, 0, 1, 1, 1])
        y_pred = np.array([0, 0, 1, 1, 1, 1])

        # Create and use the Homogeneity metric
        homogeneity_metric = Homogeneity()
        score = homogeneity_metric.evaluate(x_data, y_true, y_pred)
        print(f"Homogeneity Score: {score}")
        ```

    Methods:
        evaluate(x_data: XYData, y_true: Any, y_pred: Any, **kwargs) -> Float | np.ndarray:
            Calculate the Homogeneity score.

    Note:
        This metric uses scikit-learn's homogeneity_score function internally. Ensure that scikit-learn
        is properly installed and compatible with your environment.
    """

    def evaluate(
        self, x_data: XYData, y_true: Any, y_pred: Any, **kwargs: Dict[str, Any]
    ) -> Float | np.ndarray:
        """
        Calculate the Homogeneity score.

        This method computes the Homogeneity score for the clustering.

        Args:
            x_data (XYData): The input data (not used in this metric, but required by the interface).
            y_true (Any): The ground truth labels.
            y_pred (Any): The predicted cluster labels.
            **kwargs (Dict[str,Any]): Additional keyword arguments passed to sklearn's homogeneity_score.

        Returns:
            Float | np.ndarray: The Homogeneity score.

        Note:
            This method uses scikit-learn's homogeneity_score function internally.
        """
        return homogeneity_score(y_true, y_pred)

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

Calculate the Homogeneity score.

This method computes the Homogeneity score for the clustering.

Parameters:

Name	Type	Description	Default
`x_data`	`XYData`	The input data (not used in this metric, but required by the interface).	required
`y_true`	`Any`	The ground truth labels.	required
`y_pred`	`Any`	The predicted cluster labels.	required
`**kwargs`	`Dict[str, Any]`	Additional keyword arguments passed to sklearn's homogeneity_score.	`{}`

Returns:

Type	Description
`Float \| ndarray`	Float \| np.ndarray: The Homogeneity score.

Note

This method uses scikit-learn's homogeneity_score function internally.

Source code in framework3/plugins/metrics/clustering.py

def evaluate(
    self, x_data: XYData, y_true: Any, y_pred: Any, **kwargs: Dict[str, Any]
) -> Float | np.ndarray:
    """
    Calculate the Homogeneity score.

    This method computes the Homogeneity score for the clustering.

    Args:
        x_data (XYData): The input data (not used in this metric, but required by the interface).
        y_true (Any): The ground truth labels.
        y_pred (Any): The predicted cluster labels.
        **kwargs (Dict[str,Any]): Additional keyword arguments passed to sklearn's homogeneity_score.

    Returns:
        Float | np.ndarray: The Homogeneity score.

    Note:
        This method uses scikit-learn's homogeneity_score function internally.
    """
    return homogeneity_score(y_true, y_pred)

`NMI` ¶

Bases: BaseMetric

Normalized Mutual Information (NMI) metric for clustering evaluation.

This class calculates the NMI score, which is a normalization of the Mutual Information (MI) score to scale the results between 0 (no mutual information) and 1 (perfect correlation).

Key Features

Measures the agreement of the true labels and predicted clusters, ignoring permutations
Normalized to output values between 0 and 1
Suitable for comparing clusterings of different sizes

Usage

The NMI metric can be used to evaluate clustering algorithms:

from framework3.plugins.metrics.clustering import NMI
from framework3.base.base_types import XYData
import numpy as np

# Create sample data
x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
y_true = np.array([0, 0, 0, 1, 1, 1])
y_pred = np.array([0, 0, 1, 1, 1, 1])

# Create and use the NMI metric
nmi_metric = NMI()
score = nmi_metric.evaluate(x_data, y_true, y_pred)
print(f"NMI Score: {score}")

Methods:

Name	Description
`evaluate`	XYData, y_true: Any, y_pred: Any, **kwargs) -> Float \| np.ndarray: Calculate the Normalized Mutual Information score.

Note

This metric uses scikit-learn's normalized_mutual_info_score function internally. Ensure that scikit-learn is properly installed and compatible with your environment.

Source code in framework3/plugins/metrics/clustering.py

@Container.bind()
class NMI(BaseMetric):
    """
    Normalized Mutual Information (NMI) metric for clustering evaluation.

    This class calculates the NMI score, which is a normalization of the Mutual Information (MI) score
    to scale the results between 0 (no mutual information) and 1 (perfect correlation).

    Key Features:
        - Measures the agreement of the true labels and predicted clusters, ignoring permutations
        - Normalized to output values between 0 and 1
        - Suitable for comparing clusterings of different sizes

    Usage:
        The NMI metric can be used to evaluate clustering algorithms:

        ```python
        from framework3.plugins.metrics.clustering import NMI
        from framework3.base.base_types import XYData
        import numpy as np

        # Create sample data
        x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
        y_true = np.array([0, 0, 0, 1, 1, 1])
        y_pred = np.array([0, 0, 1, 1, 1, 1])

        # Create and use the NMI metric
        nmi_metric = NMI()
        score = nmi_metric.evaluate(x_data, y_true, y_pred)
        print(f"NMI Score: {score}")
        ```

    Methods:
        evaluate(x_data: XYData, y_true: Any, y_pred: Any, **kwargs) -> Float | np.ndarray:
            Calculate the Normalized Mutual Information score.

    Note:
        This metric uses scikit-learn's normalized_mutual_info_score function internally. Ensure that scikit-learn
        is properly installed and compatible with your environment.
    """

    def evaluate(
        self, x_data: XYData, y_true: Any, y_pred: Any, **kwargs: Unpack[NMIClustKwargs]
    ) -> Float | np.ndarray:
        """
        Calculate the Normalized Mutual Information score.

        This method computes the NMI score between two clusterings.

        Args:
            x_data (XYData): The input data (not used in this metric, but required by the interface).
            y_true (Any): The ground truth labels.
            y_pred (Any): The predicted cluster labels.
            **kwargs (Unpack[NMIClustKwargs]): Additional keyword arguments passed to sklearn's normalized_mutual_info_score.

        Returns:
            Float | np.ndarray: The NMI score.

        Note:
            This method uses scikit-learn's normalized_mutual_info_score function internally.
        """
        return normalized_mutual_info_score(y_true, y_pred, **kwargs)

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

Calculate the Normalized Mutual Information score.

This method computes the NMI score between two clusterings.

Parameters:

Name	Type	Description	Default
`x_data`	`XYData`	The input data (not used in this metric, but required by the interface).	required
`y_true`	`Any`	The ground truth labels.	required
`y_pred`	`Any`	The predicted cluster labels.	required
`**kwargs`	`Unpack[NMIClustKwargs]`	Additional keyword arguments passed to sklearn's normalized_mutual_info_score.	`{}`

Returns:

Type	Description
`Float \| ndarray`	Float \| np.ndarray: The NMI score.

Note

This method uses scikit-learn's normalized_mutual_info_score function internally.

Source code in framework3/plugins/metrics/clustering.py

def evaluate(
    self, x_data: XYData, y_true: Any, y_pred: Any, **kwargs: Unpack[NMIClustKwargs]
) -> Float | np.ndarray:
    """
    Calculate the Normalized Mutual Information score.

    This method computes the NMI score between two clusterings.

    Args:
        x_data (XYData): The input data (not used in this metric, but required by the interface).
        y_true (Any): The ground truth labels.
        y_pred (Any): The predicted cluster labels.
        **kwargs (Unpack[NMIClustKwargs]): Additional keyword arguments passed to sklearn's normalized_mutual_info_score.

    Returns:
        Float | np.ndarray: The NMI score.

    Note:
        This method uses scikit-learn's normalized_mutual_info_score function internally.
    """
    return normalized_mutual_info_score(y_true, y_pred, **kwargs)

`Silhouette` ¶

Bases: BaseMetric

Silhouette Coefficient metric for clustering evaluation.

This class calculates the Silhouette Coefficient, which is calculated using the mean intra-cluster distance and the mean nearest-cluster distance for each sample.

Key Features

Measures how similar an object is to its own cluster compared to other clusters
Ranges from -1 to 1, where higher values indicate better-defined clusters
Can be used to determine the optimal number of clusters

Usage

The Silhouette metric can be used to evaluate clustering algorithms:

from framework3.plugins.metrics.clustering import Silhouette
from framework3.base.base_types import XYData
import numpy as np

# Create sample data
x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
y_pred = np.array([0, 0, 0, 1, 1, 1])

# Create and use the Silhouette metric
silhouette_metric = Silhouette()
score = silhouette_metric.evaluate(x_data, None, y_pred)
print(f"Silhouette Score: {score}")

Methods:

Name	Description
`evaluate`	XYData, y_true: Any, y_pred: Any, **kwargs) -> Float \| np.ndarray: Calculate the Silhouette Coefficient.

Note

This metric uses scikit-learn's silhouette_score function internally. Ensure that scikit-learn is properly installed and compatible with your environment.

Source code in framework3/plugins/metrics/clustering.py

@Container.bind()
class Silhouette(BaseMetric):
    """
    Silhouette Coefficient metric for clustering evaluation.

    This class calculates the Silhouette Coefficient, which is calculated using the mean
    intra-cluster distance and the mean nearest-cluster distance for each sample.

    Key Features:
        - Measures how similar an object is to its own cluster compared to other clusters
        - Ranges from -1 to 1, where higher values indicate better-defined clusters
        - Can be used to determine the optimal number of clusters

    Usage:
        The Silhouette metric can be used to evaluate clustering algorithms:

        ```python
        from framework3.plugins.metrics.clustering import Silhouette
        from framework3.base.base_types import XYData
        import numpy as np

        # Create sample data
        x_data = XYData(value=np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]))
        y_pred = np.array([0, 0, 0, 1, 1, 1])

        # Create and use the Silhouette metric
        silhouette_metric = Silhouette()
        score = silhouette_metric.evaluate(x_data, None, y_pred)
        print(f"Silhouette Score: {score}")
        ```

    Methods:
        evaluate(x_data: XYData, y_true: Any, y_pred: Any, **kwargs) -> Float | np.ndarray:
            Calculate the Silhouette Coefficient.

    Note:
        This metric uses scikit-learn's silhouette_score function internally. Ensure that scikit-learn
        is properly installed and compatible with your environment.
    """

    def evaluate(
        self,
        x_data: XYData,
        y_true: Any,
        y_pred: Any,
        **kwargs: Unpack[SilhouetteKwargs],
    ) -> Float | np.ndarray:
        """
        Calculate the Silhouette Coefficient.

        This method computes the Silhouette Coefficient for each sample.

        Args:
            x_data (XYData): The input data.
            y_true (Any): Not used for this metric, but required by the interface.
            y_pred (Any): The predicted cluster labels.
            **kwargs (Unpack[SilhouetteKwargs]): Additional keyword arguments passed to sklearn's silhouette_score.

        Returns:
            Float | np.ndarray: The Silhouette Coefficient.

        Note:
            This method uses scikit-learn's silhouette_score function internally.
        """
        return silhouette_score(x_data.value, y_pred, **kwargs)

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

Calculate the Silhouette Coefficient.

This method computes the Silhouette Coefficient for each sample.

Parameters:

Name	Type	Description	Default
`x_data`	`XYData`	The input data.	required
`y_true`	`Any`	Not used for this metric, but required by the interface.	required
`y_pred`	`Any`	The predicted cluster labels.	required
`**kwargs`	`Unpack[SilhouetteKwargs]`	Additional keyword arguments passed to sklearn's silhouette_score.	`{}`

Returns:

Type	Description
`Float \| ndarray`	Float \| np.ndarray: The Silhouette Coefficient.

Note

This method uses scikit-learn's silhouette_score function internally.

Source code in framework3/plugins/metrics/clustering.py

def evaluate(
    self,
    x_data: XYData,
    y_true: Any,
    y_pred: Any,
    **kwargs: Unpack[SilhouetteKwargs],
) -> Float | np.ndarray:
    """
    Calculate the Silhouette Coefficient.

    This method computes the Silhouette Coefficient for each sample.

    Args:
        x_data (XYData): The input data.
        y_true (Any): Not used for this metric, but required by the interface.
        y_pred (Any): The predicted cluster labels.
        **kwargs (Unpack[SilhouetteKwargs]): Additional keyword arguments passed to sklearn's silhouette_score.

    Returns:
        Float | np.ndarray: The Silhouette Coefficient.

    Note:
        This method uses scikit-learn's silhouette_score function internally.
    """
    return silhouette_score(x_data.value, y_pred, **kwargs)

Overview¶

The Clustering Metrics module in framework3 provides a set of evaluation metrics specifically designed for assessing the performance of clustering algorithms. These metrics help in understanding various aspects of a clustering model's performance, such as cluster homogeneity, completeness, and overall quality.

Available Clustering Metrics¶

Normalized Mutual Information (NMI)¶

The Normalized Mutual Information score is implemented in the NMI class. It measures the mutual information between the true labels and the predicted clusters, normalized by the arithmetic mean of the labels' and clusters' entropy.

Usage¶

from framework3.plugins.metrics.clustering import NMI
from framework3.base.base_types import XYData

nmi_metric = NMI()
score = nmi_metric.evaluate(x_data, y_true, y_pred)

Adjusted Rand Index (ARI)¶

The Adjusted Rand Index is implemented in the ARI class. It measures the similarity between two clusterings, adjusted for chance. It has a value close to 0 for random labeling and 1 for perfect clustering.

Usage¶

from framework3.plugins.metrics.clustering import ARI
from framework3.base.base_types import XYData

ari_metric = ARI()
score = ari_metric.evaluate(x_data, y_true, y_pred)

Silhouette Score¶

The Silhouette Score is implemented in the Silhouette class. It measures how similar an object is to its own cluster compared to other clusters. The silhouette value ranges from -1 to 1, where a high value indicates that the object is well matched to its own cluster and poorly matched to neighboring clusters.

Usage¶

from framework3.plugins.metrics.clustering import Silhouette
from framework3.base.base_types import XYData

silhouette_metric = Silhouette()
score = silhouette_metric.evaluate(x_data, y_true, y_pred)

Calinski-Harabasz Index¶

The Calinski-Harabasz Index is implemented in the CalinskiHarabasz class. It's also known as the Variance Ratio Criterion. The score is defined as the ratio of the sum of between-clusters dispersion and of inter-cluster dispersion for all clusters. A higher Calinski-Harabasz score relates to a model with better defined clusters.

Usage¶

from framework3.plugins.metrics.clustering import CalinskiHarabasz
from framework3.base.base_types import XYData

ch_metric = CalinskiHarabasz()
score = ch_metric.evaluate(x_data, y_true, y_pred)

Homogeneity Score¶

The Homogeneity Score is implemented in the Homogeneity class. It measures whether all of its clusters contain only data points which are members of a single class. The score is bounded below by 0 and above by 1. A higher value indicates better homogeneity.

Usage¶

from framework3.plugins.metrics.clustering import Homogeneity
from framework3.base.base_types import XYData

homogeneity_metric = Homogeneity()
score = homogeneity_metric.evaluate(x_data, y_true, y_pred)

Completeness Score¶

The Completeness Score is implemented in the Completeness class. It measures whether all members of a given class are assigned to the same cluster. The score is bounded below by 0 and above by 1. A higher value indicates better completeness.

Usage¶

from framework3.plugins.metrics.clustering import Completeness
from framework3.base.base_types import XYData

completeness_metric = Completeness()
score = completeness_metric.evaluate(x_data, y_true, y_pred)

Comprehensive Example: Evaluating a Clustering Model¶

In this example, we'll demonstrate how to use the Clustering Metrics to evaluate the performance of a clustering model.

from framework3.plugins.filters.clustering.kmeans import KMeansPlugin
from framework3.plugins.metrics.clustering import NMI, ARI, Silhouette, CalinskiHarabasz, Homogeneity, Completeness
from framework3.base.base_types import XYData
from sklearn.datasets import make_blobs
import numpy as np

# Generate sample data
X, y_true = make_blobs(n_samples=300, centers=4, cluster_std=0.60, random_state=0)

# Create XYData object
X_data = XYData(_hash='X_data', _path='/tmp', _value=X)

# Create and fit the clustering model
kmeans = KMeansPlugin(n_clusters=4, random_state=0)
kmeans.fit(X_data)

# Get cluster predictions
y_pred = kmeans.predict(X_data)

# Initialize metrics
nmi_metric = NMI()
ari_metric = ARI()
silhouette_metric = Silhouette()
ch_metric = CalinskiHarabasz()
homogeneity_metric = Homogeneity()
completeness_metric = Completeness()

# Compute metrics
nmi_score = nmi_metric.evaluate(X_data, y_true, y_pred.value)
ari_score = ari_metric.evaluate(X_data, y_true, y_pred.value)
silhouette_score = silhouette_metric.evaluate(X_data, y_true, y_pred.value)
ch_score = ch_metric.evaluate(X_data, y_true, y_pred.value)
homogeneity_score = homogeneity_metric.evaluate(X_data, y_true, y_pred.value)
completeness_score = completeness_metric.evaluate(X_data, y_true, y_pred.value)

# Print results
print(f"Normalized Mutual Information: {nmi_score}")
print(f"Adjusted Rand Index: {ari_score}")
print(f"Silhouette Score: {silhouette_score}")
print(f"Calinski-Harabasz Index: {ch_score}")
print(f"Homogeneity Score: {homogeneity_score}")
print(f"Completeness Score: {completeness_score}")

This example demonstrates how to:

Generate sample clustering data
Create XYData objects for use with framework3
Train a KMeans clustering model
Make predictions on the dataset
Initialize and compute various clustering metrics
Print the evaluation results

Best Practices¶

Multiple Metrics: Use multiple metrics to get a comprehensive view of your clustering model's performance. Different metrics capture different aspects of clustering quality.
Ground Truth: When available, use metrics that compare against ground truth labels (like NMI, ARI) for a more robust evaluation.
Internal Metrics: When ground truth is not available, rely on internal metrics like Silhouette Score and Calinski-Harabasz Index.
Interpretation: Remember that the interpretation of these metrics can depend on the specific characteristics of your dataset and the clustering algorithm used.
Visualization: Complement these metrics with visualization techniques to get a better understanding of your clustering results.
Parameter Tuning: Use these metrics to guide the tuning of your clustering algorithm's parameters (e.g., number of clusters).
Stability: Consider evaluating the stability of your clustering results by running the algorithm multiple times with different initializations.
Domain Knowledge: Always interpret these metrics in the context of your domain knowledge and the specific goals of your clustering task.

Conclusion¶

The Clustering Metrics module in framework3 provides essential tools for evaluating the performance of clustering models. By using these metrics in combination with other framework3 components, you can gain valuable insights into your model's strengths and weaknesses. The example demonstrates how easy it is to compute and interpret these metrics within the framework3 ecosystem, enabling you to make informed decisions about your clustering models.

Clustering Metrics¶

ARI ¶

evaluate(x_data, y_true, y_pred, **kwargs) ¶

CalinskiHarabasz ¶

evaluate(x_data, y_true, y_pred, **kwargs) ¶

Completeness ¶

evaluate(x_data, y_true, y_pred, **kwargs) ¶

Homogeneity ¶

evaluate(x_data, y_true, y_pred, **kwargs) ¶

NMI ¶

evaluate(x_data, y_true, y_pred, **kwargs) ¶

Silhouette ¶

evaluate(x_data, y_true, y_pred, **kwargs) ¶

Overview¶

Available Clustering Metrics¶

Normalized Mutual Information (NMI)¶

Usage¶

Adjusted Rand Index (ARI)¶

Usage¶

Silhouette Score¶

Usage¶

Calinski-Harabasz Index¶

Usage¶

Homogeneity Score¶

Usage¶

Completeness Score¶

Usage¶

Comprehensive Example: Evaluating a Clustering Model¶

Best Practices¶

Conclusion¶

`ARI` ¶

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

`CalinskiHarabasz` ¶

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

`Completeness` ¶

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

`Homogeneity` ¶

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

`NMI` ¶

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶

`Silhouette` ¶

`evaluate(x_data, y_true, y_pred, **kwargs)` ¶