Classification Metrics

Functional API

SeqMetrics also provides a functional API for all the performance metrics.

SeqMetrics.f1_score(true, predicted, average=None) → ndarray | float

Calculates f1 score according to following formula

\[F1 = 2 \cdot \frac{\text{precision} \cdot \text{recall}}{\text{precision} + \text{recall}}\]

Parameters:

• average (str, optional) – It can be macro or weighted. or micro

• true (ture/observed/actual/target values. It must be a numpy array, or pandas series/DataFrame or a list.) –

• predicted (simulated values) –

Return type:

array or float

Examples

>>> import numpy as np
>>> from SeqMetrics import f1_score
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(metrics = f1_score(true, pred, average="macro"))
>>> print(metrics = f1_score(true, pred, average="weighted"))

SeqMetrics.accuracy(true, predicted, normalize: bool = True) → float

calculates accuracy

\[\text{Accuracy} = \frac{\sum_{i=1}^{N} \mathbb{1}(true_i = predicted_i)}{N}\]

Parameters:

• normalize (bool) –

• true (ture/observed/actual/target values. It must be a numpy array, or pandas series/DataFrame or a list.) –

• predicted (simulated values) –

Return type:

float

Examples

>>> import numpy as np
>>> from SeqMetrics import accuracy
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> accuracy(true, pred)

SeqMetrics.precision(true, predicted, average=None)

Returns precision score, also called positive predictive value. It is number of correct positive predictions divided by the total number of positive predictions. TP/(TP+FP)

\[\text{Precision}_{\text{micro}} = \frac{\sum TP}{\sum (TP + FP)}\]

\[\text{Precision}_{\text{macro}} = \frac{1}{N} \sum_{i=1}^{N} \frac{TP_i}{TP_i + FP_i}\]

\[\text{Precision}_{\text{weighted}} = \frac{\sum_{i=1}^{N} (TP_i + FN_i) \cdot \frac{TP_i}{TP_i + FP_i}}{\sum_{i=1}^{N} (TP_i + FN_i)}\]

Parameters:

• average (string, [None, macro, weighted, micro]) –

• true (ture/observed/actual/target values. It must be a numpy array,) – or pandas series/DataFrame or a list.

• predicted (simulated values) –

Examples

>>> import numpy as np
>>> from SeqMetrics import precision
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = precision(true, pred, average="macro")
>>> metrics = precision(true, pred, average="weighted")

SeqMetrics.recall(true, predicted, average=None)

It is also called sensitivity or true positive rate. It is number of correct positive predictions divided by the total number of positives

\[\text{Recall} = \frac{\text{True Positive}}{\text{True Positive} + \text{False Negative}}\]

\[\text{Recall}_{\text{micro}} = \frac{\sum_{i=1}^{n} \text{TP}_i}{\sum_{i=1}^{n} (\text{TP}_i + \text{FN}_i)}\]

\[\text{Recall}_{\text{macro}} = \frac{1}{n} \sum_{i=1}^{n} \frac{\text{TP}_i}{\text{TP}_i + \text{FN}_i}\]

\[\text{Recall}_{\text{weighted}} = \sum_{i=1}^{n} w_i \cdot \frac{\text{TP}_i}{\text{TP}_i + \text{FN}_i}\]

Parameters:

• average (str (default=None)) – one of None, macro, weighted, or micro

• true (ture/observed/actual/target values. It must be a numpy array,) – or pandas series/DataFrame or a list.

• predicted (simulated values) –

Examples

>>> import numpy as np
>>> from SeqMetrics import recall
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = recall(true, pred, average="macro")
>>> metrics = recall(true, pred, average="weighted")

SeqMetrics.balanced_accuracy(true, predicted, average=None) → float

balanced accuracy. It performs better on imbalanced datasets.

\[\text{Balanced Accuracy} = \frac{1}{C} \sum_{i=1}^{C} \frac{TP_i}{TP_i + FN_i}\]

Parameters:

• true (ture/observed/actual/target values. It must be a numpy array,) – or pandas series/DataFrame or a list.

• predicted (simulated values) –

Examples

>>> import numpy as np
>>> from SeqMetrics import balanced_accuracy
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = balanced_accuracy(true, pred)

SeqMetrics.confusion_matrix(true, pred, num_classes=None, normalize=False)

Computes a confusion matrix using numpy for two np.arrays true and pred.

Results are identical (and similar in computation time) to: “from sklearn.metrics import confusion_matrix”

However, this function avoids the dependency on sklearn.

SeqMetrics.cross_entropy(true, predicted, epsilon=1e-12) → float

Computes cross entropy between targets (encoded as one-hot vectors) and predictions.

\[CE = - \frac{1}{N} \sum_{i=1}^{N} \left[ y_i \log(\hat{y}_i + \epsilon) \right]\]

Return type:

scalar

Parameters:

• true – ture/observed/actual/target values. It must be a numpy array, or pandas series/DataFrame or a list.

• predicted – simulated values

Examples

>>> import numpy as np
>>> from SeqMetrics import cross_entropy
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = cross_entropy(true, pred)

SeqMetrics.specificity(true, predicted, average=None)

It is also called true negative rate or selectivity. It is the probability that the predictions are negative when the true labels are also negative. It is number of correct negative predictions divided by the total number of negatives.

\[\text{Specificity} = \frac{TN}{TN + FP}\]

Parameters:

• true (ture/observed/actual/target values. It must be a numpy array,) – or pandas series/DataFrame or a list.

• predicted (simulated values) –

Examples

>>> import numpy as np
>>> from SeqMetrics import specificity
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(metrics = specificity(true, pred, average="macro"))
>>> print(metrics = specificity(true, pred, average="weighted"))

SeqMetrics.f2_score(true, predicted, average=None)

f2 score

\[F2 = \left(1 + 2^2\right) \cdot \frac{\text{Precision} \cdot \text{Recall}}{(2^2 \cdot \text{Precision}) + \text{Recall}}\]

averagestr (default=None)

one of None, macro, weighted, or micro

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(f2_score(true, pred, average="macro"))
>>> print(f2_score(true, pred, average="weighted"))

SeqMetrics.false_positive_rate(true, predicted)

False positive rate is the number of incorrect positive predictions divided by the total number of negatives. Its best value is 0.0 and worst value is 1.0. It is also called probability of false alarm or fall-out.

\[\text{FPR} = \frac{\text{FP}}{\text{FP} + \text{TN}}S\]

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(false_positive_rate(true, pred))

SeqMetrics.false_discovery_rate(true, predicted)

False discovery rate

\[FDR = \frac{FP}{TP + FP}\]

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(false_discovery_rate(true, pred))

SeqMetrics.negative_predictive_value(true, predicted)

Negative Predictive Value

\[\text{NPV} = \frac{TN}{TN + FN}\]

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(negative_predictive_value(true, pred))

SeqMetrics.error_rate(true, predicted)

Error rate is the number of all incorrect predictions divided by the total number of samples in data.

\[\text{Error Rate} = \frac{\text{FP} + \text{FN}}{n}\]

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(error_rate(true, pred))

SeqMetrics.mathews_corr_coeff(true, predicted)

Methew’s correlation coefficient

\[\text{MCC} = \frac{TP \cdot TN - FP \cdot FN}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}}\]

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(mathews_corr_coeff(true, pred))

SeqMetrics.positive_likelihood_ratio(true, predicted, average=None)

Positive likelihood ratio

\[LR+ = \frac{\text{Sensitivity}}{1 - \text{Specificity}}\]

averagestr (default=None)

one of None, macro, weighted, or micro

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(positive_likelihood_ratio(true, pred, average="macro"))
>>> print(positive_likelihood_ratio(true, pred, average="weighted"))

SeqMetrics.negative_likelihood_ratio(true, predicted, average=None)

Negative likelihood ratio

\[\text{NLR} = 1 - \frac{\text{Sensitivity}}{\text{Specificity}}\]

averagestr (default=None)

one of None, macro, weighted, or micro

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(negative_likelihood_ratio(true, pred, average="macro"))
>>> print(negative_likelihood_ratio(true, pred, average="weighted"))

SeqMetrics.youden_index(true, predicted, average=None)

Youden index, also known as informedness

\[J = \text{TPR} + \text{TNR} - 1 = \text{sensitivity} + \text{specificity} - 1\]

averagestr (default=None)

one of None, macro, weighted, or micro

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(youden_index(true, pred, average="macro"))
>>> print(youden_index(true, pred, average="weighted"))

SeqMetrics.fowlkes_mallows_index(true, predicted, average=None)

Fowlkes–Mallows index

\[\text{FMI} = \sqrt{\text{PPV} \times \text{TPR}}\]

PPV is positive predictive value or precision. TPR is true positive rate or recall or sensitivity

fowlkes_mallows_score

averagestr (default=None)

one of None, macro, weighted, or micro

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(fowlkes_mallows_index(true, pred, average="macro"))
>>> print(fowlkes_mallows_index(true, pred, average="weighted"))

SeqMetrics.prevalence_threshold(true, predicted, average=None)

Prevalence threshold

\[PT = \frac{\sqrt{FPR}}{\sqrt{TPR} + \sqrt{FPR}}\]

TPR is true positive rate or recall

averagestr (default=None)

one of None, macro, weighted, or micro

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(prevalence_threshold(true, pred, average="macro"))
>>> print(prevalence_threshold(true, pred, average="weighted"))

SeqMetrics.false_omission_rate(true, predicted)

False omission rate

\[\text{FOR} = \frac{\text{FN}}{\text{FN} + \text{TN}}\]

trueture/observed/actual/target values. It must be a numpy array,

or pandas series/DataFrame or a list.

predicted : simulated values

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> print(false_omission_rate(true, pred))

SeqMetrics.ClassificationMetrics(true, predicted, multiclass: bool = False, *args, **kwargs)

Calculates classification metrics.

Parameters:

• true (array-like of shape = [n_samples] or [n_samples, n_classes]) – True class labels.

• predicted (array-like of shape = [n_samples] or [n_samples, n_classes]) – Predicted class labels.

• multiclass (boolean, optional) – If true, it is assumed that the true labels are multiclass.

• **kwargs (optional) – Additional arguments to be passed to the Metrics class.

Examples

>>> import numpy as np
>>> from SeqMetrics import ClassificationMetrics

boolean array

>>> t = np.array([True, False, False, False])
>>> p = np.array([True, True, True, True])
>>> metrics = ClassificationMetrics(t, p)
>>> accuracy = metrics.accuracy()

binary classification with numerical labels

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> accuracy = metrics.accuracy()

multiclass classification with numerical labels

>>> true = np.random.randint(1, 4, 100)
>>> pred = np.random.randint(1, 4, 100)
>>> metrics = ClassificationMetrics(true, pred)
>>> accuracy = metrics.accuracy()

You can also provide logits instead of labels.

>>> predictions = np.array([[0.25, 0.25, 0.25, 0.25],
>>>                        [0.01, 0.01, 0.01, 0.96]])
>>> targets = np.array([[0, 0, 0, 1],
>>>                     [0, 0, 0, 1]])
>>> metrics = ClassificationMetrics(targets, predictions, multiclass=True)
>>> metrics.cross_entropy()
...  0.71355817782

Working with categorical values is seamless

>>> true = np.array(['a', 'b', 'b', 'b'])
>>> pred = np.array(['a', 'a', 'a', 'a'])
>>> metrics = ClassificationMetrics(true, pred)
>>> accuracy = metrics.accuracy()

same goes for multiclass categorical labels

>>> t = np.array(['car', 'truck', 'truck', 'car', 'bike', 'truck'])
>>> p = np.array(['car', 'car',   'bike',  'car', 'bike', 'truck'])
>>> metrics = ClassificationMetrics(targets, predictions, multiclass=True)
>>> print(metrics.calculate_all())

Class-Based API

class SeqMetrics.ClassificationMetrics(true, predicted, multiclass: bool = False, *args, **kwargs)

Bases: Metrics

Calculates classification metrics.

Parameters:

• true (array-like of shape = [n_samples] or [n_samples, n_classes]) – True class labels.

• predicted (array-like of shape = [n_samples] or [n_samples, n_classes]) – Predicted class labels.

• multiclass (boolean, optional) – If true, it is assumed that the true labels are multiclass.

• **kwargs (optional) – Additional arguments to be passed to the Metrics class.

Examples

>>> import numpy as np
>>> from SeqMetrics import ClassificationMetrics

boolean array

>>> t = np.array([True, False, False, False])
>>> p = np.array([True, True, True, True])
>>> metrics = ClassificationMetrics(t, p)
>>> accuracy = metrics.accuracy()

binary classification with numerical labels

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> accuracy = metrics.accuracy()

multiclass classification with numerical labels

>>> true = np.random.randint(1, 4, 100)
>>> pred = np.random.randint(1, 4, 100)
>>> metrics = ClassificationMetrics(true, pred)
>>> accuracy = metrics.accuracy()

You can also provide logits instead of labels.

>>> predictions = np.array([[0.25, 0.25, 0.25, 0.25],
>>>                        [0.01, 0.01, 0.01, 0.96]])
>>> targets = np.array([[0, 0, 0, 1],
>>>                     [0, 0, 0, 1]])
>>> metrics = ClassificationMetrics(targets, predictions, multiclass=True)
>>> metrics.cross_entropy()
...  0.71355817782

Working with categorical values is seamless

>>> true = np.array(['a', 'b', 'b', 'b'])
>>> pred = np.array(['a', 'a', 'a', 'a'])
>>> metrics = ClassificationMetrics(true, pred)
>>> accuracy = metrics.accuracy()

same goes for multiclass categorical labels

>>> t = np.array(['car', 'truck', 'truck', 'car', 'bike', 'truck'])
>>> p = np.array(['car', 'car',   'bike',  'car', 'bike', 'truck'])
>>> metrics = ClassificationMetrics(targets, predictions, multiclass=True)
>>> print(metrics.calculate_all())

__init__(true, predicted, multiclass: bool = False, *args, **kwargs)

Parameters:

• true (array like,) – ture/observed/actual/target values. This can be anything which can be converted to numpy array.

• predicted (array like,) – predicted/simulated values. This can be anything which can be converted to numpy array.

• replace_nan (default None. if not None, then NaNs in true) – and predicted will be replaced by this value.

• replace_inf (default None, if not None, then inf vlaues in true and) – predicted will be replaced by this value.

• remove_zero (default False, if True, the zero values in true) – or predicted arrays will be removed. If a zero is found in one array, the corresponding value in the other array will also be removed.

• remove_neg (default False, if True, the negative values in true) – or predicted arrays will be removed.

• remove_inf (bool (default=True)) – whether to remove infitinity (np.inf) values from true and predicted arrays or not.

• remove_nan (bool (default=True)) – whether to remove nan (np.nan) values from true and predicted arrays or not.

• metric_type (type of metric.) –

• np_errstate (dict) – any keyword options for np.errstate() to calculate np.log1p

accuracy(normalize: bool = True) → float

calculates accuracy

\[\text{Accuracy} = \frac{\sum_{i=1}^{N} \mathbb{1}(true_i = predicted_i)}{N}\]

Examples

>>> import numpy as np
>>> from SeqMetrics import ClassificationMetrics
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.accuracy())

balanced_accuracy(average=None) → float

balanced accuracy It performs better on imbalanced datasets.

\[\text{Balanced Accuracy} = \frac{1}{C} \sum_{i=1}^{C} \frac{TP_i}{TP_i + FN_i}\]

Examples

>>> import numpy as np
>>> from SeqMetrics import ClassificationMetrics
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> metrics.balanced_accuracy()

confusion_matrix(normalize=False)

calculates confusion matrix

Examples

>>> import numpy as np
>>> from SeqMetrics import ClassificationMetrics
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> metrics.confusion_matrix()

multiclass classification

>>> true = np.random.randint(1, 4, 100)
>>> pred = np.random.randint(1, 4, 100)
>>> metrics = ClassificationMetrics(true, pred)
>>> metrics.confusion_matrix()

cross_entropy(epsilon=1e-12) → float

calculates cross entropy

Examples

>>> import numpy as np
>>> from SeqMetrics import ClassificationMetrics
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.cross_entropy())
Computes cross entropy between targets (encoded as one-hot vectors)
and predictions.

error_rate()

Error rate is the number of all incorrect predictions divided by the total number of samples in data.

\[\text{Error Rate} = \frac{\text{FP} + \text{FN}}{n}\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.error_rate())

f1_score(average=None) → ndarray | float

Calculates f1 score according to following formula

\[F1 = 2 \cdot \frac{\text{precision} \cdot \text{recall}}{\text{precision} + \text{recall}}\]

Examples

>>> import numpy as np
>>> from SeqMetrics import ClassificationMetrics
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> calc_f1_score = metrics.f1_score()
...
>>> print(metrics.f1_score(average="macro"))
>>> print(metrics.f1_score(average="weighted"))

f2_score(average=None)

f2 score

\[F2 = \left(1 + 2^2\right) \cdot \frac{\text{Precision} \cdot \text{Recall}}{(2^2 \cdot \text{Precision}) + \text{Recall}}\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> calc_f2_score = metrics.f2_score()
...
>>> print(metrics.f2_score(average="macro"))
>>> print(metrics.f2_score(average="weighted"))

false_discovery_rate()

False discovery rate

\[FDR = \frac{FP}{TP + FP}\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.false_discovery_rate())

false_negative_rate()

False Negative Rate or miss rate.

\[\text{FNR} = \frac{\text{FN}}{\text{FN} + \text{TP}}\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.false_negative_rate())

false_omission_rate()

False omission rate

\[\text{FOR} = \frac{\text{FN}}{\text{FN} + \text{TN}}\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.false_omission_rate())

false_positive_rate()

False positive rate is the number of incorrect positive predictions divided by the total number of negatives. Its best value is 0.0 and worst value is 1.0. It is also called probability of false alarm or fall-out.

\[\text{FPR} = \frac{\text{FP}}{\text{FP} + \text{TN}}S\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.false_positive_rate())

fowlkes_mallows_index(average=None)

Fowlkes–Mallows index

\[\text{FMI} = \sqrt{\text{PPV} \times \text{TPR}}\]

PPV is positive predictive value or precision. TPR is true positive rate or recall or sensitivity

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.fowlkes_mallows_index(average="macro"))
>>> print(metrics.fowlkes_mallows_index(average="weighted"))

mathews_corr_coeff()

Methew’s correlation coefficient

\[\text{MCC} = \frac{TP \cdot TN - FP \cdot FN}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}}\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.mathews_corr_coeff())

negative_likelihood_ratio(average=None)

Negative likelihood ratio

\[\text{NLR} = 1 - \frac{\text{Sensitivity}}{\text{Specificity}}\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.negative_likelihood_ratio(average="macro"))
>>> print(metrics.negative_likelihood_ratio(average="weighted"))

negative_predictive_value()

Negative Predictive Value

\[\text{NPV} = \frac{TN}{TN + FN}\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.negative_predictive_value())

positive_likelihood_ratio(average=None)

Positive likelihood ratio

\[LR+ = \frac{\text{Sensitivity}}{1 - \text{Specificity}}\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.positive_likelihood_ratio(average="macro"))
>>> print(metrics.positive_likelihood_ratio(average="weighted"))

precision(average=None)

Returns precision score, also called positive predictive value. It is number of correct positive predictions divided by the total number of positive predictions. .. math:

\text{Precision}_{\text{micro}} = \frac{\sum TP}{\sum (TP + FP)}

\[\text{Precision}_{\text{macro}} = \frac{1}{N} \sum_{i=1}^{N} \frac{TP_i}{TP_i + FP_i}\]

\[\text{Precision}_{\text{weighted}} = \frac{\sum_{i=1}^{N} (TP_i + FN_i) \cdot \frac{TP_i}{TP_i + FP_i}}{\sum_{i=1}^{N} (TP_i + FN_i)}\]

Examples

>>> import numpy as np
>>> from SeqMetrics import ClassificationMetrics
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.precision())
...
>>> print(metrics.precision(average="macro"))
>>> print(metrics.precision(average="weighted"))

prevalence_threshold(average=None)

Prevalence threshold

\[PT = \frac{\sqrt{FPR}}{\sqrt{TPR} + \sqrt{FPR}}\]

TPR is true positive rate or recall

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.prevalence_threshold(average="macro"))
>>> print(metrics.prevalence_threshold(average="weighted"))

recall(average=None)

It is also called sensitivity or true positive rate. It is number of correct positive predictions divided by the total number of positives

\[\text{Recall} = \frac{\text{True Positive}}{\text{True Positive} + \text{False Negative}}\]

\[\text{Recall}_{\text{micro}} = \frac{\sum_{i=1}^{n} \text{TP}_i}{\sum_{i=1}^{n} (\text{TP}_i + \text{FN}_i)}\]

\[\text{Recall}_{\text{macro}} = \frac{1}{n} \sum_{i=1}^{n} \frac{\text{TP}_i}{\text{TP}_i + \text{FN}_i}\]

\[\text{Recall}_{\text{weighted}} = \sum_{i=1}^{n} w_i \cdot \frac{\text{TP}_i}{\text{TP}_i + \text{FN}_i}\]

Examples

>>> import numpy as np
>>> from SeqMetrics import ClassificationMetrics
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> metrics.recall()

specificity(average=None)

It is also called true negative rate or selectivity. It is the probability that the predictions are negative when the true labels are also negative. It is number of correct negative predictions divided by the total number of negatives.

\[\text{Specificity} = \frac{TN}{TN + FP}\]

Examples

>>> import numpy as np
>>> from SeqMetrics import ClassificationMetrics
>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.specificity())
...
>>> print(metrics.specificity(average="macro"))
>>> print(metrics.specificity(average="weighted"))

youden_index(average=None)

Youden index, also known as informedness

\[J = \text{TPR} + \text{TNR} - 1 = \text{sensitivity} + \text{specificity} - 1\]

Examples

>>> true = np.array([1, 0, 0, 0])
>>> pred = np.array([1, 1, 1, 1])
>>> metrics = ClassificationMetrics(true, pred)
>>> print(metrics.youden_index(average="macro"))
>>> print(metrics.youden_index(average="weighted"))