Voting-based SVM Ensemble with Map Reduce and Stochastic Gradient Descent

Voting-based SVM Ensemble with Map Reduce and Stochastic Gradient Descent

  IJRES-book-cover  International Journal of Recent Engineering Science (IJRES)          
  
© 2016 by IJRES Journal
Volume-3 Issue-6
Year of Publication : 2016
Authors : Shuxia Lu, Zhao Jin
DOI : 10.14445/23497157/IJRES-V3I6P105

How to Cite?

Shuxia Lu, Zhao Jin, "Voting-based SVM Ensemble with Map Reduce and Stochastic Gradient Descent," International Journal of Recent Engineering Science, vol. 3, no. 6, pp. 31-34, 2016. Crossref, https://doi.org/10.14445/23497157/IJRES-V3I6P105

Abstract
Stochastic Gradient Descent (SGD) is an attractive choice for SVM training. In order to deal with the large-scale data linear classification problems, a method named Voting-based SVM Ensemble with MapReduce and Stochastic Gradient Descent (MRSGD) is proposed. Firstly, to deal with the large-scale data classification problems, we use the MapReduce technique. Secondly, SVM optimization problem can be solved by stochastic gradient descent algorithm. Finally, the voting mechanism is used to ensemble several SVMs classifiers. Experimental results on datasets show that the proposed method is effective.

Keywords
Stochastic gradient descent, Large-scale learning, Support vector machines, MapReduce, Voting Mechanism.

Reference
[1] W. Tsang, J. T. Kwok, and P. M. Cheung, Core vector machines, fast SVM training on very large data sets. Journal of Machine Learning Research. 6, 2005, 363-392.
[2] Shalev-Shwartz, Y. Singer, N. Srebro, et al, Pegasos: Primal Estimated sub-Gradient Solver for SVM, Mathematical Programming, 127(1), 2011, 3-30.
[3] Krzysztof Sopyla, Pawel Drozda, Stochastic Gradient Descent with Barzilai-Borwein update step for SVM, Information Sciences, 316, 2015, 218-233.
[4] Zhuang Wang, Koby Crammer, Slobodan Vucetic, Breaking the Curse of Kernelization: Budgeted Stochastic Gradient Descent for Large-Scale SVM Training, Journal of Machine Learning Research, 13, 2013, 3103-3131.
[5] Nicolas Couellan, Wenjuan Wang, Bi-level stochastic gradient for large scale support vector machine, Neurocomputing, 153, 2015, 300-308.
[6] R. Johnson and T. Zhang, Accelerating Stochastic Gradient Descent using predictive variance reduction. In Advances in Neural Information Processing Systems, 2013, 315-323.
[7] A. Bordes, L. Bottou, P. Gallinari, SGD-QN: careful quasiNewton stochastic gradient descent, J. Mach. Learn, 10, 2009, 1737-1754.
[8] A. Bordes, L. Bottou, P. Gallinari, et al, Sgdqn is less careful than expected, J. Mach. Learn, 11, 2010, 2229-2240.
[9] Shai Shalev-Shwartz, Tong Zhang, Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization, Math. Program., 155, 2016, 105-145.
[10] Shalev-Shwartz, Zhang, et al, Stochastic dual coordinate ascent methods for regularized losss minimization, J. Mach. Learn, 14, 2013, 567-599.
[11] Stephan Clemencon, Aurelien Bellet, Ons Jelassi, et al, Scalability of Stochastic Gradient Descent based on Smart Sampling Techniques, Procedia Computer Science, 53, 2015, 308–315.
[12] Elad Hazan, Satyen Kale, Beyond the Regret Minimization Barrier: Optimal Algorithms for Stochastic Strongly Convex Optimization, Journal of Machine Learning Research, 15, 2014, 2489-2512.
[13] Z. Lei, Y. Yang, Z. Wu, Ensemble of support vector machine for text-independent speaker recognition, International Journal Computer Science and Network Security, 6 (1), 2006, 163–167.
[14] Nasullah Khalid Alham, Maozhen Li, Yang Liu, Man Qi, A MapReduce-based distributed SVM ensemble for scalable image classification and annotation. Computers and Mathematics with Applications, 66, 2013, 1920-1934.
[15] Ferhat Ozgur CATAK, Mehmet Erdal BALABAN, A MapReduce-based distributed SVM algorithm for binary classification, Turkish Journal of Electrical & Computer Science, 2013, 863-873
[16] Apache Hadoop, http://hadoop.apache.org/
[17] K Shvachko, H Kuang, S Radia, R Chansler, The Hadoop Distributed File System, IEEE Symposium on Mass Storage System & Technologies, 11, 2010, 1-10.
[18] J. Dean and S. Ghemawat, Mapreduce: simplified data processing on large clusters, Communications of the ACM, 51(1), 2008, 107–113.
[19] S. Sonnenburg, V. Franc, E.Y. Tov, M. Sebag, PASCAL largescale learning challenge, 2008.
[20] Chih-Chung Chang and Chih-Jen Lin, LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2016.