A Comparative Study of the Fisher’s Discriminant Analysis and Support Vector Machines



Abstract Views
268

Downloads
371

Citations

Share

##plugins.themes.bootstrap3.article.sidebar##

Submitted Aug 17, 2017
Published Aug 31, 2017
10.24018/ejeng.2017.2.8.448

Abstract viewed = 268 times

Table of Contents

##plugins.themes.bootstrap3.article.main##

The relative merits of Fisher’s Discriminant Analysis (FDA) over Support Vector Machines or vice versa, will remain a bone of contention among statisticians and the machine learning community. This line of thought may be owed to the fact that FDA is due to Fishers R. A., a statistician, whereas SVM is a credit to Vanik and his team of the machine learning. In order to give a clearer picture on the strength and weakness of both classifiers, they are compared in terms of the different theories behind each one. We also look into the ways regularization is carried out by each classifier, and further examine how FDA and SVM respond to linear
transformations. We conclude with examination of the behaviour of FDA and SVM on data, given different scenarios, and in high dimensions too. In the end, we clearly draw out the differences and similarities between the two classifiers, and further highlight features that make each classifier ideal for a given classification problem.

Keywords: Binary Classification Fisher’s Discriminant Analysis Support Vector Machines Classification in High Dimensions

Downloads

Download data is not yet available.

References

  1. R. A. Fisher, “The use of Multiple Measurements in Taxonomic Problems”, Ann. Eugen., vol. 7, no. 2, pp. 179–188, 1936.
     Google Scholar
  2. C. Cortes and V. Vapnik, “Support-Vector Networks”, Mach. Learn., vol. 20, no. 3, pp. 273–297, 1995.
     Google Scholar
  3. Y. Li, S. Gong, and H. Liddell, “Recognising Trajectories of Facial Identities using Kernel Discriminant Analysis”, Image Vis. Comput., vol. 21, no. 13, pp. 1077–1086, 2003.
     Google Scholar
  4. Q. Liu, H. Lu, and S. Ma, “Improving Kernel Fisher Discriminant Analysis for Face Recognition”, IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 1, pp.42–49, 2004.
     Google Scholar
  5. T. Kurita and T. Taguchi, “A Modification of Kernel-Based Fisher Discriminant Analysis for Face Detection, in Automatic Face and Gesture Recognition”, Proceedings Fifth IEEE International Conference, pp. 300–305, 2002.
     Google Scholar
  6. Y. Feng and P. Shi, “Face Detection Based on Kernel Fisher Discriminant Analysis, in Automatic Face and Gesture Recognition”, Proceedings Sixth IEEE International Conference, pp. 381–384, 2004.
     Google Scholar
  7. L. Wei, Y. Yang, R. M. Nishikawa, and Y. Jiang, “A Study on Several Machine-Learning Methods for Classification of Malignant and Benign Clustered Microcalcifications”, IEEE Trans. Med. Imaging, vol. 24, no. 3, pp. 371–380, 2005.
     Google Scholar
  8. G. Baudat and F. Anouar, “Generalized Discriminant Analysis Using a Kernel Approach”, Neural Comput., vol. 12, no. 10, pp. 2385–2404, 2000.
     Google Scholar
  9. J. J. Liu, M. H. Bharati, K. G. Dunn, and J. F. MacGregor, “Automatic Masking in Multivariate Image Analysis using Support Vector Machines”, Chemom. Intell. Lab. Syst., vol. 79, no. 1, pp. 42–54, 2005.
     Google Scholar
  10. R. Burbidge, M. Trotter, B. Buxton, and S. Holden, “Drug Design by Machine Learning: Support Vector Machines for Pharmaceutical Data Analysis”, Comput Chem., vol. 26, no. 1, pp. 5–14, 2001.
     Google Scholar
  11. U. Norinder, “Support Vector Machine Models in Drug Design: Applications to Drug Transport Processes and Qsar using Simplex Optimisations and Variable Selection”, Neurocomputing, vol. 55, no. 1, pp. 337–346, 2003.
     Google Scholar
  12. L. Bao and Z. Sun, “Identifying Genes Related to Drug Anticancer Mechanisms Using Support Vector Machine”, FEBS Lett., vol. 521, no. 1–3, pp. 109–114, 2002.
     Google Scholar
  13. K. Kyoung-jae, “Financial Time Series Forecasting using Support Vector Machines”, Neurocomputing, vol. 55, no. 1, pp. 307–319, 2003.
     Google Scholar
  14. U. Thissen, R. Van Brakel, A. De Weijer, W. Melssen, and L. Buydens, “Using Support Vector Machines for Time Series Prediction”, Chemom. Intell. Lab. Syst., vol. 69, no. 1, pp. 35–49, 2003.
     Google Scholar
  15. L. Xie, Y. Ying, and T. Ying, “Classification of Tomatoes with Different Genotypes by Visible and Short-Wave Near-Infrared Spectroscopy with Least-Squares Support Vector Machines and other Chemometrics”, J. Food Eng., vol. 94, no. 1, pp. 34–39, 2009.
     Google Scholar
  16. N. Gilardi, M. Kanevski, M. Maignan, and E. Mayoraz, “Environmental and Pollution Spatial Data Classification with Support Vector Machines and Geostatistics”, Greece ACAI, vol. 99, pp. 43–51, 1999.
     Google Scholar
  17. M. Kanevski, M. Maignan, and A. Pozdnukhov, “Active Learning of Environmental Data Using Support Vector Machines”, Conference of the International Association for Mathematical Geology, 2005.
     Google Scholar
  18. I. Gokcen and J. Peng, “Comparing Linear Discriminant Analysis and Support Vector Machines”, International Conference on Advances in Information Systems, 2002, pp. 104–113.
     Google Scholar
  19. M. Khondoker, R. Dobson, C. Skirrow, A. Simmons, and D. Stahl, “A Comparison of Machine Learning Methods for Classification using Simulation with Multiple Real Data Examples from Mental Health Studies”, Stat. Methods Med. Res., vol. 25, no. 5, pp. 1804–1823, 2016.
     Google Scholar
  20. [20] C. W. Hsu, C. C. Chang, C. J. Lin, and others, “A Practical Guide to Support Vector Classification”, Taipe, Taiwan, 2003.
     Google Scholar
  21. Wikipedia, “Discriminant Function Analysis” — Wikipedia, The Free Encyclopedia. 2016.
     Google Scholar
  22. J. H. Friedman, “Regularized Discriminant Analysis”, J. Am. Stat. Assoc., vol. 84, no. 405, pp. 165–175, 1989.
     Google Scholar
  23. D. M. Witten and R. Tibshirani, “Penalized Classification using Fisher’s Linear Discriminant”, J.R. Stat. Soc. Ser. B Stat. Methodol., vol. 73, no. 5, pp. 753–772, 2011.
     Google Scholar
  24. P. J. Bickel and E. Levina, “Some Theory for Fisher’s Linear Discriminant Function,Naive Bayes, and Some Alternatives When There Are Many More Variables than Observations”, Bernoulli, pp. 989–1010, 2004.
     Google Scholar
  25. J. Taylor, “Data Mining”, http://statweb.stanford.edu/~jtaylo/courses/stats20/restricted/notes/outlier.pdf, Dec-2012.
     Google Scholar
  26. H. Schütze, “An Introduction to Information Retrieval”. Cambridge, England: Cambridge University Press, 2009
     Google Scholar