torchmetrics：支持80+任务评价指标

torchmetrics是由pytorch-lightning开源的评估指标计算库，内置了常见的80+指标计算逻辑，可以直接使用，也可以与pytorch-lightning结合一起使用。

安装 & 使用

pip install torchmetrics

基础案例：

# import 三方包
import torchmetrics
 
# initialize metric
metric = torchmetrics.Accuracy()
 
# move the metric to device you want computations to take place
device = "cuda" if torch.cuda.is_available() else "cpu"
metric.to(device)
 
n_batches = 10
for i in range(n_batches):
    # simulate a classification problem
    preds = torch.randn(10, 5).softmax(dim=-1).to(device)
    target = torch.randint(5, (10,)).to(device)
 
    # metric on current batch
    acc = metric(preds, target)
    print(f"Accuracy on batch {i}: {acc}")
 
# metric on all batches using custom accumulation
acc = metric.compute()
print(f"Accuracy on all data: {acc}")

Auido 任务指标

• Perceptual Evaluation of Speech Quality (PESQ)

• Permutation Invariant Training (PIT)

• Scale-Invariant Signal-to-Distortion Ratio (SI-SDR)

• Scale-Invariant Signal-to-Noise Ratio (SI-SNR)

• Short-Time Objective Intelligibility (STOI)

• Signal to Distortion Ratio (SDR)

• Signal-to-Noise Ratio (SNR)

分类 任务指标

• Accuracy

• AUC

• AUROC

• Average Precision

• Binned Average Precision

• Binned Precision Recall Curve

• Binned Recall At Fixed Precision

• Calibration Error

• Cohen Kappa

• Confusion Matrix

• Coverage Error

• Dice Score

• F1 Score

• FBeta Score

• Hamming Distance

• Hinge Loss

• Jaccard Index

• KL Divergence

• Label Ranking Average Precision

• Label Ranking Loss

• Matthews Corr. Coef.

• Precision

• Precision Recall

• Precision Recall Curve

• Recall

• ROC

• Specificity

• Stat Scores

图像 任务指标

• Error Relative Global Dim. Synthesis (ERGAS)

• Frechet Inception Distance (FID)

• Image Gradients

• Inception Score

• Kernel Inception Distance

• Learned Perceptual Image Patch Similarity (LPIPS)

• Multi-Scale SSIM

• Peak Signal-to-Noise Ratio (PSNR)

• Spectral Angle Mapper

• Spectral Distortion Index

• Structural Similarity Index Measure (SSIM)

• Universal Image Quality Index

检测 任务指标

• Mean-Average-Precision (mAP)

Pairwise 任务指标

• Cosine Similarity

• Euclidean Distance

• Linear Similarity

• Manhattan Distance

Regression 任务指标

• Cosine Similarity

• Explained Variance

• Mean Absolute Error (MAE)

• Mean Absolute Percentage Error (MAPE)

• Mean Squared Error (MSE)

• Mean Squared Log Error (MSLE)

• Pearson Corr. Coef.

• R2 Score

• Spearman Corr. Coef.

• Symmetric Mean Absolute Percentage Error (SMAPE)

• Tweedie Deviance Score

• Weighted MAPE

Retrieval 任务指标

• Retrieval Fall-Out

• Retrieval Hit Rate

• Retrieval Mean Average Precision (MAP)

• Retrieval Mean Reciprocal Rank (MRR)

• Retrieval Normalized DCG

• Retrieval Precision

• Retrieval R-Precision

• Retrieval Recall

Text 任务指标

• BERT Score

• BLEU Score

• Char Error Rate

• ChrF Score

• Extended Edit Distance

• Match Error Rate

• ROUGE Score

• Sacre BLEU Score

• SQuAD

• Translation Edit Rate (TER)

• Word Error Rate

• Word Info. Lost

• Word Info. Preserved