Task 1(Project: CS674) Mishra A.(Y6 Kumar D.(Y6152) Venkat(Y

Introduction:
The Pattern classification for given problem is posed with serious challenge as on one side the data set is highly imbalanced in favour of B+E against B(So we have to avoid over fitting for generalization) & on the other side wrong classification can have serious consequences in diplomatic relationship between nations.
So Our thrust has been to choose between various methods , one with sound justification towards our results & showing how was it better than others .
Based on comparative study of various methods we have finally chosen Biased Minimax Probability Machine [1] & we would be proving superiority of our methods over SVM classifier with different parameters which we tried. Besides authors [1] have shown superiority of BMPM over DT, Naive Bayesian Classifier, K-nn classification, & other under/over Sampling methods.

Methodology of BMPM:
For two class Classification:
Let Family {x}, {y} with mean vector & Covariance matrices {x, ∑x }, {y, ∑y} belong to class1 & class2 respectively.
Let α be the worst-case accuracy for future data points from family of {x}, and β be the worst-case accuracy for future data points from family of {y}.
Depending upon severity of the false positive & true positive rates α, β(Policy variables) it tries to find a maximal hyper plane to separate the two classes
[pic]
[pic]
We can also have Non Linear Classifier by mapping the feature space into suitable higher dimension.
The above optimization is changed according to needs, so we would be doing for our data sets.

Reasons for Selection of BMPM:
In words of authors [1]:
“Traditional methods have tried to utilize some intermediate factors, e.g., the distribution of the training set, the decision thresholds or the cost matrices, to influence the bias of the classification.”
But this is imperfect as
Breiman et al. [2] establishes the connection among the distribution of the training data, the prior probability of each class, the costs of misclassification of each class, and the setup of the decision threshold. Changing one of these factors is equivalent to changing other factors. Thus, simply changing thresholds or adjust the weight for each class lacks the systematic foundation in the same sense as the sampling method.”

So We would be following this line of approach & give our results for this method in a comparison to other classifiers we tried.

Results & Comparisons;

References:
[1] Kaizhu Huang, Haiqin Yang, Irwin King, Michael R. Lyu, and Laiwan Chan. Learning Classifiers from Imbalanced Data Based on Biased Minimax Probability Machine IEEE 2004

[2]L. Breiman. Arcing classifiers.Technical Report 460, Statistics Department, University of California, 1997.