Observing Pre-Trained Convolutional Neural Network (CNN) Layers as Feature Extractor for Detecting Bias in Image Classification Data


  • Amadea Claire Isabel Ardison Bina Nusantara University
  • Mikhaya Josheba Rumondang Hutagalung Bina Nusantara University
  • Reynaldi Chernando Bina Nusantara University
  • Tjeng Wawan Cenggoro Bina Nusantara University




Pre-Trained Convolutional Neural Network (CNN), Features Extractor, Data Bias, Image Classification


Detecting bias in data is crucial since it can pose serious problems when developing an AI algorithm. The research aims to propose a novel study design to detect bias in image classification data by using pretrained Convolutional Neural Network (CNN) layers as a feature extractor. There are three datasets used in the research with varying degrees of complexity, those are low, medium, and high complexity. There are Modified National Institute of Standards and Technology (MNIST) Digits, batik collections (Parang, Megamendung, and Kawung), and Canadian Institute for Advanced Research (CIFAR-10) datasets. Then, the researchers make a baseline workflow and substitute a step-in feature extraction with a convolution using the first pre-trained CNN layer and each of its kernels. Then, the researchers evaluate the effect of the experiments using accuracy. By observing the effect of the individual kernel, the research can better make sense of what happens inside a CNN layer. The research finds that color in the image is an essential factor when working with CNN. Furthermore, the proposed study design can detect bias in image classification data where it is related to the color of the image. Detecting this bias early is important in helping developers to improve AI algorithms.


Plum Analytics

Author Biographies

Amadea Claire Isabel Ardison, Bina Nusantara University

Computer Science Department, School of Computer Science

Mikhaya Josheba Rumondang Hutagalung, Bina Nusantara University

Computer Science Department, School of Computer Science

Reynaldi Chernando, Bina Nusantara University

Computer Science Department, School of Computer Science

Tjeng Wawan Cenggoro, Bina Nusantara University

Computer Science Department, School of Computer Science

Bioinformatics and Data Science Research Center


S. Leavy, “Gender bias in artificial intelligence: The need for diversity and gender theory in machine learning,” in Proceedings of the 1st International Workshop On Gender Equality in Software Engineering, 2018, pp. 14–16.

D. J. Fuchs, “The dangers of human-like bias in machine-learning algorithms,” Missouri S&T’s Peer to Peer, vol. 2, no. 1, pp. 1–14, 2018.

J. Dastin, “Amazon scraps secret AI recruiting tool that showed bias against women,” 2018. [Online]. Available: https://reut.rs/2UghQQS

A. Chouldechova, D. Benavides-Prado, O. Fialko, and R. Vaithianathan, “A case study of algorithmassisted decision making in child maltreatment hotline screening decisions,” in Proceedings of the 1st Conference on Fairness, Accountability and Transparency. PMLR, 2018, pp. 134–148.

N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM Computing Surveys (CSUR), vol. 54, no. 6, pp. 1–35, 2021.

S. Alelyani, “Detection and evaluation of machine learning bias,” Applied Sciences, vol. 11, no. 14, pp. 1–17, 2021.

H. Jiang and O. Nachum, “Identifying and correcting label bias in machine learning,” arXiv Preprint arXiv:1901.04966, 2019.

W. Sun, O. Nasraoui, and P. Shafto, “Evolution and impact of bias in human and machine learning algorithm interaction,” PLOS ONE, vol. 15, no. 8, pp. 1–39, 2020.

S. Albawi, T. A. Mohammed, and S. Al-Zawi, “Understanding of a convolutional neural network,” in 2017 International Conference on Engineering and Technology (ICET). Antalya, Turkey: IEEE, Aug. 21–23, 2017, pp. 1–6.

B. B. Traore, B. Kamsu-Foguem, and F. Tangara, “Deep convolution neural network for image recognition,” Ecological Informatics, vol. 48, pp. 257–268, 2018.

L. Shang, Q. Yang, J. Wang, S. Li, and W. Lei, “Detection of rail surface defects based on CNN image recognition and classification,” in 2018 20th International Conference on Advanced Communication Technology (ICACT). Chuncheon, South Korea: IEEE, Feb. 11–14, 2018, pp. 45–51.

R. Chauhan, K. K. Ghanshala, and R. C. Joshi, “Convolutional Neural Network (CNN) for image detection and recognition,” in 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC). Jalandhar, India: IEEE, Dec. 15–17, 2018, pp. 278–282.

B. P. Gyires-T´oth, M. Osv´ath, D. Papp, and G. Sz˝ucs, “Deep learning for plant classification and content-based image retrieval,” Cybernetics and Information Technologies, vol. 19, no. 1, pp. 88–100, 2019.

Y. Wang, H. Liu, M. Guo, X. Shen, B. Han, and Y. Zhou, “Image recognition model based on deep learning for remaining oil recognition from visualization experiment,” Fuel, vol. 291, pp. 1–14, 2021.

X. Yang, Y. Zhang, W. Lv, and D. Wang, “Image recognition of wind turbine blade damage based on a deep learning model with transfer learning and an ensemble learning classifier,” Renewable Energy, vol. 163, pp. 386–397, 2021.

S. Hicks, M. Riegler, K. Pogorelov, K. V. Anonsen, T. de Lange, D. Johansen, M. Jeppsson, K. R. Randel, S. L. Eskeland, and P. Halvorsen, “Dissecting deep neural networks for better medical image classification and classification understanding,” in 2018 IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS). Karlstad, Sweden: IEEE, June 18–21, 2018, pp. 363–368.

R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi, “A survey of methods for explaining black box models,” ACM Computing Surveys (CSUR), vol. 51, no. 5, pp.

–42, 2019.

N. O’Mahony, S. Campbell, A. Carvalho, S. Harapanahalli, G. V. Hernandez, L. Krpalkova, D. Riordan, and J. Walsh, “Deep learning vs. traditional computer vision,” in Science and Information Conference. Las Vegas, USA: Springer, April 25–26, 2019, pp. 128–144.

S. T. Krishna and H. K. Kalluri, “Deep learning and transfer learning approaches for image classification,” International Journal of Recent Technology and Engineering (IJRTE), vol. 7, no. 5S4, pp. 427–432, 2019.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.

J. Wang, Y. Ma, L. Zhang, R. X. Gao, and D. Wu, “Deep learning for smart manufacturing: Methods and applications,” Journal of Manufacturing Systems, vol. 48, pp. 144–156, 2018.

A. Krizhevsky, Learning multiple layers of features from tiny images. University of Toronto, 2009.

Y. LeCun, C. Cortes, and C. J. C. Burges, “The MNIST database of handwritten digits,” 1998. [Online]. Available: http://yann.lecun.com/exdb/mnist/

M. K. Hu, “Visual pattern recognition by moment invariants,” IRE Transactions on Information Theory, vol. 8, no. 2, pp. 179–187, 1962.

C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995.



Abstract 171  .
PDF downloaded 69  .