Comparing Imperialist Competitive Algorithm with Backpropagation Algorithms for Training Feedforward Neural Network
-
3306
Downloads
-
4628
Views
Authors
Maryam Zanganeh
- Department of computer engineering, Central Tehran Branch, Islamic Azad University, Tehran, Iran.
Seyed Javad Mirabedini
- Department of computer engineering, Central Tehran Branch, Islamic Azad University, Tehran, Iran.
Abstract
Artificial Neural Networks (ANN) and evolutionary algorithms are two relatively young research areas that were subject to a steadily growing interest during the past years. The use of ANN has been proved to be a cost-effective technique. It is very important to choose a suitable algorithm for training a neural network. Mostly Back Propagation (BP) algorithm is a gradient descent algorithm (a first-order optimization algorithm) on the error space, which most likely gets trapped into a local minimum and has very slow convergence. This shortcoming can be removed by global searching ability of the evolutionary algorithms such as Imperialist Competitive Algorithm (ICA) which is a new evolutionary algorithm based on the human's socio-political evolution. This investigation provides a comparison between training a neural network with BP algorithms used for training Feed-forward Neural Networks (FNN) and ICA. Among the BP algorithms, Gradient descent, Levenberg–Marquardt, Conjugate gradient descent, Resilient, BFGS Quasi-newton, and One-step secant algorithm are tested then the obtained results will be compared with the results of training the neural network with ICA. Also, Accuracy and Mean Squared Error (MSE) are the main measures selected to assess both models. Also the MSE was used as a criterion to specify optimum number of neurons in the hidden layer. The results showed that ICA approach outperforms the BP for training neural network models.
Share and Cite
ISRP Style
Maryam Zanganeh, Seyed Javad Mirabedini, Comparing Imperialist Competitive Algorithm with Backpropagation Algorithms for Training Feedforward Neural Network, Journal of Mathematics and Computer Science, 14 (2015), no. 3, 193-204
AMA Style
Zanganeh Maryam, Mirabedini Seyed Javad, Comparing Imperialist Competitive Algorithm with Backpropagation Algorithms for Training Feedforward Neural Network. J Math Comput SCI-JM. (2015); 14(3):193-204
Chicago/Turabian Style
Zanganeh, Maryam, Mirabedini, Seyed Javad. "Comparing Imperialist Competitive Algorithm with Backpropagation Algorithms for Training Feedforward Neural Network." Journal of Mathematics and Computer Science, 14, no. 3 (2015): 193-204
Keywords
- Imperialist Competitive Algorithm
- Backpropagation
- Artificial Neural Network.
MSC
References
-
[1]
S. M. Aquil Burney, Tahseen A. Jilani, time series forecasting using artificial neural network methods for karachi stock exchange, A project at department of computer science, university of Karachi (2002)
-
[2]
Y. Shang, B.W. Wah, Global Optimization for Neural Networks Training, IEEE Computer, 29 (1996), 45-54.
-
[3]
A. Abraham, Meta learning evolutionary artificial neural networks, Neuro Computing. , 56 (2004), 1-38.
-
[4]
Jiancheng Fang, Zhongyu Wang, The combination of immune evolution and neural network for nonlinear time series forecasting Sixth International Symposium on Instrumentation and Control Technology: Sensors, Automatic Measurement, Control and Computer Simulation, Beijing, China (2006)
-
[5]
Haibin Duan, Linzhi Huang, Imperialist competitive algorithm optimized artificial neural networks for UCAV global path planning, Neurocomputing, (2014)
-
[6]
A. Iftikhar, M. A. Ansari, S. Mohsin, Performance Comparison between Back propagation Algorithms Applied to Intrusion Detection in Computer Network Systems, 9th WSEAS International Conference on Neural Networks., (2008), 231-236.
-
[7]
R. Noori, M. S. Sabahi, A. R. Karbassi, Evaluation of PCA and Gamma test techniques on ANN operation for weekly solid waste predicting, J. Environ. Manage. , 91 (2010), 767-771.
-
[8]
Mohammad Ali Ahmadi, Mohammad Ebadi, Amin Shokrollahi, Seyed Mohammad Javad Majidi., Evolving artificial neural network and imperialist competitive algorithm for prediction oil flow rate of the reservoir, Applied Soft Computin., 13 (2013), 1085-1095.
-
[9]
Shahram Mollaiy Berneti, Mehdi Shahbazian, an imperialist competitive algorithm-artificial neural network method to predict oil flow rate of the wells, International Journal of Computer Applications. , 26 (2011), 975-8887.
-
[10]
S. M. Hosseini, A. Al Khaed., A survey on the Imperialist Competitive Algorithm metaheuristic, Implementation in engineering domain and directions for future research., 24 (2014), 1078-1094.
-
[11]
S. Mirjalili, A. Safa Sadiq, Magnetic optimization algorithm for training multi-layer perceptron, in: IEEE International Conference on Industrial and Intelligent Information, Indonesia., 2 (2011), 42-46.
-
[12]
M. C. P. de Souto, A. Yamazaki, T. B. Ludernir , Optimization of neural network weights and architecture for odor recognition using simulated annealing, Proceedings of the 2002 International Joint Conference on Neural Networks. , 1 (2002), 547–552.
-
[13]
K. Hornick, M. Stinchcombe, H. White, Multilayer feed forward networks are universal approximators, Neural Networks , 2 (1989), 359–366.
-
[14]
N. García-Pedrajas, C. Hervás-Martínez, J. Munoz-Perez, A cooperative co-evolutionary model for evolving artificial neural networks, IEEE Transactions on Neural Networks. , 14 (2003), 575–596.
-
[15]
T. Ganesan, I. Elamvazuthi, P. Vasant, Solving engineering optimization problems with the Karush–Kuhn–Tucker Hopfield neural networks, International Review of Mechanical Engineering., 5 (2011), 1333-1339.
-
[16]
S. Verma, B. D. Huey, D. J. Burgess, Scanning probe microscopy method for nanosuspension stabilizer selection, Langmuir, 25 (2009), 12481-12487.
-
[17]
K. I. Funahashi, On the Approximate Realization of Continuous Mappings by Neural Networks, Neural Networks., 2 (1989), 183-192.
-
[18]
K. Hornik, Approximation Capabilities of Multilayer Feedforward Networks, Neural Networks. , 4 (1991), 251-257.
-
[19]
G. Cybenko, Approximation by Super positions of Sigmoidal Function, Math. Contr. Signals Syst. , 2 (1989), 303-314.
-
[20]
K. Levenberg, A Method for the Solution of Certain Nonlinear Problems in Least Squares, Quarterly of Applied Mathematics. , 2 (1944), 164-168.
-
[21]
M. S. Mirtalaei, M. Saberi, O. K. Hussain, B. Ashjari, F. K. Hussain, A trust-based bio-inspired approach for credit lending decisions, Computing, 94 (2012), 541-577.
-
[22]
Xinying Miao, Jinkui Chu, Linghan Zhang, Jing Qiao., An Evolutionary Neural Network Approach to Simple Prediction of Dam Deformation, Journal of Information & Computational Science. , 10 (2013), 1315–1324.
-
[23]
Lian Kunlei, Zhang Chaoyong, Gaoa Liang, Single row facility layout problem using an imperialist competitive algorithm, Proceedings of the \(41^{st}\) International Conference on Computers & Industrial Engineering. , (),
-
[24]
E. Atashpaz-Gargari, C. Lucas, Imperialist competitive algorithm: an algorithm for optimization inspired by imperialistic competition, IEEE Congress on Evolutionary Computation., (2007), 4661-4667.