Example-3 (Activation threshold)

In [1]:
from pycm import ConfusionMatrix

def activation_1(i):
    if i<0.75:
        return 1
    else:
        return 0
In [2]:
y_actual = [1,1,0,1,1,0]
y_pred = [0.65,0.34,0.80,0.54,0.32,0.12]
In [3]:
cm = ConfusionMatrix(y_actual,y_pred,threshold=activation_1)
cm.classes
Out[3]:
[0, 1]
In [4]:
print(cm)
Predict          0    1    
Actual
0                1    1    

1                0    4    





Overall Statistics : 

95% CI                                                           (0.53513,1.13154)
AUNP                                                             0.75
AUNU                                                             0.75
Bennett S                                                        0.66667
CBA                                                              0.65
Chi-Squared                                                      2.4
Chi-Squared DF                                                   1
Conditional Entropy                                              0.33333
Cramer V                                                         0.63246
Cross Entropy                                                    1.03701
Gwet AC1                                                         0.73333
Hamming Loss                                                     0.16667
Joint Entropy                                                    1.25163
KL Divergence                                                    0.11871
Kappa                                                            0.57143
Kappa 95% CI                                                     (-0.19538,1.33824)
Kappa No Prevalence                                              0.66667
Kappa Standard Error                                             0.39123
Kappa Unbiased                                                   0.55556
Lambda A                                                         0.5
Lambda B                                                         0.0
Mutual Information                                               0.31669
NIR                                                              0.66667
Overall ACC                                                      0.83333
Overall CEN                                                      0.39624
Overall J                                                        (1.3,0.65)
Overall MCC                                                      0.63246
Overall MCEN                                                     0.27683
Overall RACC                                                     0.61111
Overall RACCU                                                    0.625
P-Value                                                          0.35117
PPV Macro                                                        0.9
PPV Micro                                                        0.83333
Phi-Squared                                                      0.4
RCI                                                              0.34487
RR                                                               3.0
Reference Entropy                                                0.9183
Response Entropy                                                 0.65002
SOA1(Landis & Koch)                                              Moderate
SOA2(Fleiss)                                                     Intermediate to Good
SOA3(Altman)                                                     Moderate
SOA4(Cicchetti)                                                  Fair
Scott PI                                                         0.55556
Standard Error                                                   0.15215
TPR Macro                                                        0.75
TPR Micro                                                        0.83333
Zero-one Loss                                                    1

Class Statistics :

Classes                                                          0                       1                       
ACC(Accuracy)                                                    0.83333                 0.83333                 
AUC(Area under the roc curve)                                    0.75                    0.75                    
AUCI(Auc value interpretation)                                   Good                    Good                    
BM(Informedness or bookmaker informedness)                       0.5                     0.5                     
CEN(Confusion entropy)                                           0.52832                 0.35221                 
DOR(Diagnostic odds ratio)                                       None                    None                    
DP(Discriminant power)                                           None                    None                    
DPI(Discriminant power interpretation)                           None                    None                    
ERR(Error rate)                                                  0.16667                 0.16667                 
F0.5(F0.5 score)                                                 0.83333                 0.83333                 
F1(F1 score - harmonic mean of precision and sensitivity)        0.66667                 0.88889                 
F2(F2 score)                                                     0.55556                 0.95238                 
FDR(False discovery rate)                                        0.0                     0.2                     
FN(False negative/miss/type 2 error)                             1                       0                       
FNR(Miss rate or false negative rate)                            0.5                     0.0                     
FOR(False omission rate)                                         0.2                     0.0                     
FP(False positive/type 1 error/false alarm)                      0                       1                       
FPR(Fall-out or false positive rate)                             0.0                     0.5                     
G(G-measure geometric mean of precision and sensitivity)         0.70711                 0.89443                 
IS(Information score)                                            1.58496                 0.26303                 
J(Jaccard index)                                                 0.5                     0.8                     
MCC(Matthews correlation coefficient)                            0.63246                 0.63246                 
MCEN(Modified confusion entropy)                                 0.5                     0.46439                 
MK(Markedness)                                                   0.8                     0.8                     
N(Condition negative)                                            4                       2                       
NLR(Negative likelihood ratio)                                   0.5                     0.0                     
NPV(Negative predictive value)                                   0.8                     1.0                     
P(Condition positive or support)                                 2                       4                       
PLR(Positive likelihood ratio)                                   None                    2.0                     
PLRI(Positive likelihood ratio interpretation)                   None                    Poor                    
POP(Population)                                                  6                       6                       
PPV(Precision or positive predictive value)                      1.0                     0.8                     
PRE(Prevalence)                                                  0.33333                 0.66667                 
RACC(Random accuracy)                                            0.05556                 0.55556                 
RACCU(Random accuracy unbiased)                                  0.0625                  0.5625                  
TN(True negative/correct rejection)                              4                       1                       
TNR(Specificity or true negative rate)                           1.0                     0.5                     
TON(Test outcome negative)                                       5                       1                       
TOP(Test outcome positive)                                       1                       5                       
TP(True positive/hit)                                            1                       4                       
TPR(Sensitivity, recall, hit rate, or true positive rate)        0.5                     1.0                     
Y(Youden index)                                                  0.5                     0.5                     
dInd(Distance index)                                             0.5                     0.5                     
sInd(Similarity index)                                           0.64645                 0.64645