Example-4 (File)

In [1]:
from pycm import ConfusionMatrix
import numpy as np
y_test = np.array([600, 200, 200, 200, 200, 200, 200, 200, 500, 500, 500, 200, 200, 200, 200, 200, 200, 200, 200, 200])
y_pred = np.array([100, 200, 200, 100, 100, 200, 200, 200, 100, 200, 500, 100, 100, 100, 100, 100, 100, 100, 500, 200])
In [2]:
cm=ConfusionMatrix(y_test, y_pred)
cm
Out[2]:
pycm.ConfusionMatrix(classes: [100, 200, 500, 600])
In [3]:
print(cm)
Predict          100    200    500    600    
Actual
100              0      0      0      0      

200              9      6      1      0      

500              1      1      1      0      

600              1      0      0      0      





Overall Statistics : 

95% CI                                                           (0.14096,0.55904)
AUNP                                                             None
AUNU                                                             None
Bennett S                                                        0.13333
CBA                                                              0.17708
Chi-Squared                                                      None
Chi-Squared DF                                                   9
Conditional Entropy                                              1.23579
Cramer V                                                         None
Cross Entropy                                                    1.70995
Gwet AC1                                                         0.19505
Hamming Loss                                                     0.65
Joint Entropy                                                    2.11997
KL Divergence                                                    None
Kappa                                                            0.07801
Kappa 95% CI                                                     (-0.2185,0.37453)
Kappa No Prevalence                                              -0.3
Kappa Standard Error                                             0.15128
Kappa Unbiased                                                   -0.12554
Lambda A                                                         0.0
Lambda B                                                         0.0
Mutual Information                                               0.10088
NIR                                                              0.8
Overall ACC                                                      0.35
Overall CEN                                                      0.3648
Overall J                                                        (0.60294,0.15074)
Overall MCC                                                      0.12642
Overall MCEN                                                     0.37463
Overall RACC                                                     0.295
Overall RACCU                                                    0.4225
P-Value                                                          1.0
PPV Macro                                                        None
PPV Micro                                                        0.35
Phi-Squared                                                      None
RCI                                                              0.11409
RR                                                               5.0
Reference Entropy                                                0.88418
Response Entropy                                                 1.33667
SOA1(Landis & Koch)                                              Slight
SOA2(Fleiss)                                                     Poor
SOA3(Altman)                                                     Poor
SOA4(Cicchetti)                                                  Poor
Scott PI                                                         -0.12554
Standard Error                                                   0.10665
TPR Macro                                                        None
TPR Micro                                                        0.35
Zero-one Loss                                                    13

Class Statistics :

Classes                                                          100                     200                     500                     600                     
ACC(Accuracy)                                                    0.45                    0.45                    0.85                    0.95                    
AUC(Area under the roc curve)                                    None                    0.5625                  0.63725                 0.5                     
AUCI(Auc value interpretation)                                   None                    Poor                    Fair                    Poor                    
BM(Informedness or bookmaker informedness)                       None                    0.125                   0.27451                 0.0                     
CEN(Confusion entropy)                                           0.33496                 0.35708                 0.53895                 0.0                     
DOR(Diagnostic odds ratio)                                       None                    1.8                     8.0                     None                    
DP(Discriminant power)                                           None                    0.14074                 0.4979                  None                    
DPI(Discriminant power interpretation)                           None                    Poor                    Poor                    None                    
ERR(Error rate)                                                  0.55                    0.55                    0.15                    0.05                    
F0.5(F0.5 score)                                                 0.0                     0.68182                 0.45455                 0.0                     
F1(F1 score - harmonic mean of precision and sensitivity)        0.0                     0.52174                 0.4                     0.0                     
F2(F2 score)                                                     0.0                     0.42254                 0.35714                 0.0                     
FDR(False discovery rate)                                        1.0                     0.14286                 0.5                     None                    
FN(False negative/miss/type 2 error)                             0                       10                      2                       1                       
FNR(Miss rate or false negative rate)                            None                    0.625                   0.66667                 1.0                     
FOR(False omission rate)                                         0.0                     0.76923                 0.11111                 0.05                    
FP(False positive/type 1 error/false alarm)                      11                      1                       1                       0                       
FPR(Fall-out or false positive rate)                             0.55                    0.25                    0.05882                 0.0                     
G(G-measure geometric mean of precision and sensitivity)         None                    0.56695                 0.40825                 None                    
IS(Information score)                                            None                    0.09954                 1.73697                 None                    
J(Jaccard index)                                                 0.0                     0.35294                 0.25                    0.0                     
MCC(Matthews correlation coefficient)                            None                    0.10483                 0.32673                 None                    
MCEN(Modified confusion entropy)                                 0.33496                 0.37394                 0.58028                 0.0                     
MK(Markedness)                                                   0.0                     0.08791                 0.38889                 None                    
N(Condition negative)                                            20                      4                       17                      19                      
NLR(Negative likelihood ratio)                                   None                    0.83333                 0.70833                 1.0                     
NPV(Negative predictive value)                                   1.0                     0.23077                 0.88889                 0.95                    
P(Condition positive or support)                                 0                       16                      3                       1                       
PLR(Positive likelihood ratio)                                   None                    1.5                     5.66667                 None                    
PLRI(Positive likelihood ratio interpretation)                   None                    Poor                    Fair                    None                    
POP(Population)                                                  20                      20                      20                      20                      
PPV(Precision or positive predictive value)                      0.0                     0.85714                 0.5                     None                    
PRE(Prevalence)                                                  0.0                     0.8                     0.15                    0.05                    
RACC(Random accuracy)                                            0.0                     0.28                    0.015                   0.0                     
RACCU(Random accuracy unbiased)                                  0.07563                 0.33062                 0.01562                 0.00063                 
TN(True negative/correct rejection)                              9                       3                       16                      19                      
TNR(Specificity or true negative rate)                           0.45                    0.75                    0.94118                 1.0                     
TON(Test outcome negative)                                       9                       13                      18                      20                      
TOP(Test outcome positive)                                       11                      7                       2                       0                       
TP(True positive/hit)                                            0                       6                       1                       0                       
TPR(Sensitivity, recall, hit rate, or true positive rate)        None                    0.375                   0.33333                 0.0                     
Y(Youden index)                                                  None                    0.125                   0.27451                 0.0                     
dInd(Distance index)                                             None                    0.67315                 0.66926                 1.0                     
sInd(Similarity index)                                           None                    0.52401                 0.52676                 0.29289                 

Save

In [4]:
cm.save_obj("cm")
Out[4]:
{'Message': 'D:\\For Asus Laptop\\projects\\pycm\\Document\\cm.obj',
 'Status': True}

Load

In [5]:
cm_load = ConfusionMatrix(file=open("cm.obj","r"))
cm
Out[5]:
pycm.ConfusionMatrix(classes: [100, 200, 500, 600])
In [6]:
print(cm)
Predict          100    200    500    600    
Actual
100              0      0      0      0      

200              9      6      1      0      

500              1      1      1      0      

600              1      0      0      0      





Overall Statistics : 

95% CI                                                           (0.14096,0.55904)
AUNP                                                             None
AUNU                                                             None
Bennett S                                                        0.13333
CBA                                                              0.17708
Chi-Squared                                                      None
Chi-Squared DF                                                   9
Conditional Entropy                                              1.23579
Cramer V                                                         None
Cross Entropy                                                    1.70995
Gwet AC1                                                         0.19505
Hamming Loss                                                     0.65
Joint Entropy                                                    2.11997
KL Divergence                                                    None
Kappa                                                            0.07801
Kappa 95% CI                                                     (-0.2185,0.37453)
Kappa No Prevalence                                              -0.3
Kappa Standard Error                                             0.15128
Kappa Unbiased                                                   -0.12554
Lambda A                                                         0.0
Lambda B                                                         0.0
Mutual Information                                               0.10088
NIR                                                              0.8
Overall ACC                                                      0.35
Overall CEN                                                      0.3648
Overall J                                                        (0.60294,0.15074)
Overall MCC                                                      0.12642
Overall MCEN                                                     0.37463
Overall RACC                                                     0.295
Overall RACCU                                                    0.4225
P-Value                                                          1.0
PPV Macro                                                        None
PPV Micro                                                        0.35
Phi-Squared                                                      None
RCI                                                              0.11409
RR                                                               5.0
Reference Entropy                                                0.88418
Response Entropy                                                 1.33667
SOA1(Landis & Koch)                                              Slight
SOA2(Fleiss)                                                     Poor
SOA3(Altman)                                                     Poor
SOA4(Cicchetti)                                                  Poor
Scott PI                                                         -0.12554
Standard Error                                                   0.10665
TPR Macro                                                        None
TPR Micro                                                        0.35
Zero-one Loss                                                    13

Class Statistics :

Classes                                                          100                     200                     500                     600                     
ACC(Accuracy)                                                    0.45                    0.45                    0.85                    0.95                    
AUC(Area under the roc curve)                                    None                    0.5625                  0.63725                 0.5                     
AUCI(Auc value interpretation)                                   None                    Poor                    Fair                    Poor                    
BM(Informedness or bookmaker informedness)                       None                    0.125                   0.27451                 0.0                     
CEN(Confusion entropy)                                           0.33496                 0.35708                 0.53895                 0.0                     
DOR(Diagnostic odds ratio)                                       None                    1.8                     8.0                     None                    
DP(Discriminant power)                                           None                    0.14074                 0.4979                  None                    
DPI(Discriminant power interpretation)                           None                    Poor                    Poor                    None                    
ERR(Error rate)                                                  0.55                    0.55                    0.15                    0.05                    
F0.5(F0.5 score)                                                 0.0                     0.68182                 0.45455                 0.0                     
F1(F1 score - harmonic mean of precision and sensitivity)        0.0                     0.52174                 0.4                     0.0                     
F2(F2 score)                                                     0.0                     0.42254                 0.35714                 0.0                     
FDR(False discovery rate)                                        1.0                     0.14286                 0.5                     None                    
FN(False negative/miss/type 2 error)                             0                       10                      2                       1                       
FNR(Miss rate or false negative rate)                            None                    0.625                   0.66667                 1.0                     
FOR(False omission rate)                                         0.0                     0.76923                 0.11111                 0.05                    
FP(False positive/type 1 error/false alarm)                      11                      1                       1                       0                       
FPR(Fall-out or false positive rate)                             0.55                    0.25                    0.05882                 0.0                     
G(G-measure geometric mean of precision and sensitivity)         None                    0.56695                 0.40825                 None                    
IS(Information score)                                            None                    0.09954                 1.73697                 None                    
J(Jaccard index)                                                 0.0                     0.35294                 0.25                    0.0                     
MCC(Matthews correlation coefficient)                            None                    0.10483                 0.32673                 None                    
MCEN(Modified confusion entropy)                                 0.33496                 0.37394                 0.58028                 0.0                     
MK(Markedness)                                                   0.0                     0.08791                 0.38889                 None                    
N(Condition negative)                                            20                      4                       17                      19                      
NLR(Negative likelihood ratio)                                   None                    0.83333                 0.70833                 1.0                     
NPV(Negative predictive value)                                   1.0                     0.23077                 0.88889                 0.95                    
P(Condition positive or support)                                 0                       16                      3                       1                       
PLR(Positive likelihood ratio)                                   None                    1.5                     5.66667                 None                    
PLRI(Positive likelihood ratio interpretation)                   None                    Poor                    Fair                    None                    
POP(Population)                                                  20                      20                      20                      20                      
PPV(Precision or positive predictive value)                      0.0                     0.85714                 0.5                     None                    
PRE(Prevalence)                                                  0.0                     0.8                     0.15                    0.05                    
RACC(Random accuracy)                                            0.0                     0.28                    0.015                   0.0                     
RACCU(Random accuracy unbiased)                                  0.07563                 0.33062                 0.01562                 0.00063                 
TN(True negative/correct rejection)                              9                       3                       16                      19                      
TNR(Specificity or true negative rate)                           0.45                    0.75                    0.94118                 1.0                     
TON(Test outcome negative)                                       9                       13                      18                      20                      
TOP(Test outcome positive)                                       11                      7                       2                       0                       
TP(True positive/hit)                                            0                       6                       1                       0                       
TPR(Sensitivity, recall, hit rate, or true positive rate)        None                    0.375                   0.33333                 0.0                     
Y(Youden index)                                                  None                    0.125                   0.27451                 0.0                     
dInd(Distance index)                                             None                    0.67315                 0.66926                 1.0                     
sInd(Similarity index)                                           None                    0.52401                 0.52676                 0.29289                 

Obj File

In [7]:
print(open("cm.obj","r").read())
{"Transpose": false, "Matrix": [[100, [[200, 0], [500, 0], [100, 0], [600, 0]]], [200, [[200, 6], [500, 1], [100, 9], [600, 0]]], [500, [[200, 1], [500, 1], [100, 1], [600, 0]]], [600, [[200, 0], [500, 0], [100, 1], [600, 0]]]], "Actual-Vector": [600, 200, 200, 200, 200, 200, 200, 200, 500, 500, 500, 200, 200, 200, 200, 200, 200, 200, 200, 200], "Predict-Vector": [100, 200, 200, 100, 100, 200, 200, 200, 100, 200, 500, 100, 100, 100, 100, 100, 100, 100, 500, 200], "Digit": 5, "Sample-Weight": null}
  • Notice : `Matrix` save method changed in version 1.5