p–ISSN: 2723 - 6609 e-ISSN: 2745-5254

Vol. 4, No. 8, August 2023 http://jist.publikasiindonesia.id/

Doi : 10.59141/jist.v4i8.669 930

E-COMMERCE PRODUCT IMAGE-BASED RECOMMENDATION SYSTEM

KALCARE.COM USING DEEP LEARNING

Trenggana Natadirja

, Habibullah Akbar

, Gerry Firmansyah

, Budi Tjahjono

Esa Unggul University Jakarta, Indonesia

Email: [email protected]

, [email protected]

[email protected]

, [email protected]

*Correspondence

ARTICLE INFO

ABSTRACT

Accepted

: 04-08-2023

Revised

: 08-08-2023

Approved

: 09-08-2023

Recommendation systems have now become an important part of a

digital service, one example is e-commerce. The facts show that the

COVID-19 pandemic has had a significant impact on customers by

making them spend more time surfing online to get daily necessities

products by shopping on e-commerce sites. With the rapid development

of deep learning technology today, of course, it can be used to help in

terms of the process of producing a product image-based

recommendation system that has a fairly high level of similarity. This

study will discuss how to produce an image-based product

recommendation system architecture by comparing the results of the

application of algorithms 8 pre-trained models that are available and

have also been widely used in various studies and the information

technology industry. The dataset used is product images sourced from

the kalcare.com website. After testing the pre-trained model, then an

application prototype was made to be tested then in the final stage of

this research a poll was conducted to determine the response and

opinion of users to the protopine recommendation system made for e-

commerce kalcare.com using deep learning.

Keywords: recommendation

system; e-commerce; picture;

deep learning.

Attribution-ShareAlike 4.0 International

Introduction

The massive development of internet technology today and also the COVID-19

pandemic has an impact on the fact that customers are increasingly spending more time

surfing online to get their daily necessities products including healthy food, nutrition,

and medicine (Bhatti et al., 2020). This was also explained by Orinaldi (Laming, 2020)

that during the Covid-19 pandemic, the trend of using e-commerce increased quite

drastically. Coupled with restrictions on the movement of people, the choice of

shopping online is one of the main choices (Hanifah & Rahadi, 2020).

Kalcare.com is an online buying and selling site based in Indonesia that presents

the best quality health products from various brands, making it easier for Indonesian

people to be able to meet their needs in nutrition and medicine. Currently, kalcare.com

websites have text-based product search features, but everyone's ability to describe the

desired product in text form is certainly different, even often users do not get the

appropriate search results. Image-based search has several potential advantages when

compared to text-based (Goel, 2017). First, it can be faster and more intuitive, just by

uploading or taking a photo of an image and then doing a search. Second, language

E-Commerce Product Image-Based Recommendation System Kalcare.Com Using Deep

Learning

Jurnal Indonesia Sosial Teknologi, Vol. 4, No. 8, Agustus 2023 931

agnostic, which is becoming increasingly important as online shopping goes global, and

also doesn't require customers to get acquainted with the terminology used by e-

commerce sites for the type of merchandise they're looking for.

Based on the conditions mentioned above, this study made an architectural design

of an image-based product recommendation system, with input queries in the form of

images, which will search and display the most similar images, where users can use

images of the products they want to buy or also to search for similar products visually,

to find the products they want more quickly and interactively. Then to get a direct

response from users, it is necessary to make a prototype recommendation system, and

then conduct a poll.

A. Recommendation System

Recommender Systems, hereinafter abbreviated to RSS, is a subclass of

information filtering systems that generally aim to predict the rating or preference that

will be given to users of an item or product (Sulthana, Gupta, Subramanian, & Mirza,

2020).

B. Content-Based Image Retrieval

Content-Based Image Retrieval (Content-Based Image Retrieval) hereinafter can

be called CBIR known as queries against image content and content-based visual

information retrieval, namely the application of computer vision techniques to image

capture problems, namely the problem of searching for digital images in large

databases. CBIR is used to search in an image database to retrieve images of visual

content similar to the specified query image (Jenni, Mandala, & Sunar, 2015).

C. Deep Learning

Deep learning, also known as deep structured learning, is part of a broader

machine learning method based on artificial neural networks with representation

learning (Appalaraju & Chaoji, 2017). Deep learning architectures such as deep neural

networks, deep belief networks, deep reinforcement learning, recurrent neural networks,

and convolutional neural networks have been applied to areas including computer

vision, speech recognition, natural language processing, machine translation,

bioinformatics, drug design, medical image analysis, material inspection, and board

game programs, which can produce output comparable to and in some cases exceed

performance human experts (Schmidhuber, 2015).

D. Cosine Similarity

To produce the similarity value between one image and another, calculations are

carried out using the cosine similarity formula, which is the size of the similarity

between two nonzero vectors from the product space in the input image and the image

dataset that has been prepared. It is defined as equal to the cosine of the angles between

images, which is also equal to the inner product of the same vector normalized for both

having a length of 1 (Patra et al., 2020), as described in Figure 1 below.

Trenggana Natadirja, Habibullah Akbar, Gerry Firmansyah, Budi Tjahjono

Jurnal Indonesia Sosial Teknologi, Vol. 4, No. 8, August 2023 932

Picture. 1 Cosine Similarity Formula

Research Methods

The stages/steps and methods carried out in this study are as follows:

Picture. 2 Research Steps

Picture. 2 explain the steps taken in this study, starting from identifying problems,

determining solutions to be carried out, preparing product drawings, and preparing pre-

trained models (ResNet50, VGG19, VGG16, InceptionV3, NasnetMobile, MobileNet,

MobileNet, MobileNetV2, Xception) that will be used, creating algorithms to generate

image recommendations, conducting the Transfer Learning process, getting pre-trained

models that produce image recommendations with The highest level of similarity,

prototyping a web-based recommendation system, conducting polls to users, to

documenting conclusions.

1. Dataset

This study used an image dataset sourced from the kalcare.com website with a

product count of around 5,256 items (as of 2021). The dataset taken is an image of

products that are still actively sold on the e-commerce with JPG/JPEG (Joint

Photographic Experts Group) image format.

2. Pre-processing Dataset

Before processing calculations to produce the level of similarity to the image, an

initial stage or commonly called pre-processing datasets is needed which aims to reduce

the amount of work done by the machine, speed up time and increase accuracy. The

initial stage is to create and implement a SQL (Structured Query Language) query

against a database of e-commerce products kalcare.com that have an active status of sale

and have product images to get the URL location (Uniform Resource Locator) of the

E-Commerce Product Image-Based Recommendation System Kalcare.Com Using Deep

Learning

Jurnal Indonesia Sosial Teknologi, Vol. 4, No. 8, Agustus 2023 933

image. The query results obtained were then exported into CSV (Comma Separated

Values) file format, then scripted using Python programming language which serves to

download product images from the server to the local computer. In the script, in

addition to functioning to download images, there is also a function to resize images to

have the same pixel size, which is 240x240 pixels.

3. Training Process

The training process in this study uses eight (8) pre-trained models, then utilizes

transfer learning (Transfer Learning) to adjust the model to the task given (Mira,

Sembiring, & Purnomo, 2022).

Picture. 3 Ideas from Transfer Learning

Picture. 3 Explain the idea and process overview of Transfer Learning. To

simplify and speed up this process, the Keras library is used, a Deep Learning

Application Programming Interface (API) written in Python programming language that

runs on a machine learning platform called TensorFlow (Sanchez, Romero, & Morales,

2020). It was developed with a focus on allowing experiments to be carried out quickly.

To predict the final feature on the last layer of the predicted 1,000 labels, it is necessary

to redesign the model to remove the prediction layer (Dagan, Guy, & Novgorodov,

2021).

Trenggana Natadirja, Habibullah Akbar, Gerry Firmansyah, Budi Tjahjono

Jurnal Indonesia Sosial Teknologi, Vol. 4, No. 8, August 2023 934

Picture. 4 Cosine similarity calculation algorithm based on CNN feature extraction approach

Figure. 4 describes a cosine similarity calculation algorithm based on the CNN

feature extraction approach for object recognition, freezes the initial convolution layer

of the network and trains only the last few layers that make predictions. The idea is that

convolution layers extract common low-level features that apply throughout the image

(such as edges, patterns, gradients) and further layer identify specific features in the

image such as eyes or wheels. Thus, we can use networks trained on unrelated

categories in large data sets (for example: Imagenet) and apply them to the problems

discussed in this study because there are low-level universal features shared among

images, therefore the Imagenet data set and the knowledge that models learn in

Imagenet can be easily transferred into the processing of product images kalcare.com

(Alamdari, Navimipour, Hosseinzadeh, Safaei, &; Darwesh, 2022).

Results and Discussion

In this study, tests were carried out using pre-trained models that have been

determined and prepared before, namely: resnet50, vgg19, vgg16, mobv2,

nasnetmobile, mobile net, incepv3, and exception. The selection of images used in this

test is random from a pre-prepared corpus. The test aims to display 5 images that have

the highest degree of similarity. The testing process was run in an online Google Collab

environment with considerations such as being able to harness the full power of popular

Python libraries to analyze and visualize data, simply run in the browser without

performing complex configurations, get free CPU access, and be easy to share code

E-Commerce Product Image-Based Recommendation System Kalcare.Com Using Deep

Learning

Jurnal Indonesia Sosial Teknologi, Vol. 4, No. 8, Agustus 2023 935

with colleagues and colleagues. Picture. 5 Displays examples of products tested to

produce the most similar product recommendations. Picture. The 6-13 displays product

recommendations resulting from each pre-trained model.

Picture. 5 Examples of tested products

Picture. 6 Test results of pre-trained model incepv3

Picture. 7 Mobilenet model pre-trained test results

Picture. 8 Pre-trained test results of mobv2 model

Picture. 9 nasnetmobile model pre-trained test results

Picture. 10 Resnet50 model pre-trained test results

Picture. 11 vgg16 model pre-trained test results

Trenggana Natadirja, Habibullah Akbar, Gerry Firmansyah, Budi Tjahjono

Jurnal Indonesia Sosial Teknologi, Vol. 4, No. 8, August 2023 936

Picture. 12 vgg19 model pre-trained test results

Picture. 13 Xception model pre-trained test results

From the test results, it can be seen and compared visually that the one that

produces product recommendations with the best level of similarity is the resnet50 pre-

trained model. Furthermore, to get responses and responses from users, a prototype

image-based recommendation system was made that was applied to web browsers, as

seen in the image. 14.

Picture. 14 Prototypes of Image-Based Recommendation Systems

E-Commerce Product Image-Based Recommendation System Kalcare.Com Using Deep

Learning

Jurnal Indonesia Sosial Teknologi, Vol. 4, No. 8, Agustus 2023 937

The poll is conducted as a preliminary reference before the research results will be

implemented in the production environment of the kalcare.com website. This was

followed by 52 respondents, all of whom were employees working at kalcare.com. They

fill out a poll through the Google form facility at https://forms.gle/J5LufG4Z1oKz11jg7

address with the condition that they must have tried the prototype of the

recommendation system. The number of questions asked is 5, namely:

a. Prefer to search for products by typing text or image-based?

b. Is it easier to search for image-based products?

c. Is image-based product search more intuitive?

d. Is the image-based product search stage faster?

e. Does the existence of image-based product search make you come back to shop

again?

The results of the poll can be seen in the picture. The following 15-19:

Picture. 15 First-question survey results

Picture. 16 Second question survey results

Picture. 17 Third-question survey results

Trenggana Natadirja, Habibullah Akbar, Gerry Firmansyah, Budi Tjahjono

Jurnal Indonesia Sosial Teknologi, Vol. 4, No. 8, August 2023 938

Picture. 18 Survey results fourth question

Picture. 19 Fifth question survey results

Conclusion

In this section, it can be concluded that from the results of trials on eight (8) pre-

trained models that have been selected, subjectively ResNet50 is the one that can

produce visual product image recommendations with the most relevant level of

similarity. For the results of the poll on the prototype of the image-based

recommendation system, the majority of respondents are still accustomed to searching

for images by giving input in the form of text input rather than images (51.9%).

Searching for image-based products is easier, so it is very helpful for respondents in the

process of finding the product they want (75%). The majority of respondents feel that

image-based product search is more intuitive (76.9%). According to respondents,

image-based product searches are perceived to be faster than text-based (73.1%).

Image-based product search is very positive, so it can make respondents want to return

to shopping (80.8%).

E-Commerce Product Image-Based Recommendation System Kalcare.Com Using Deep

Learning

Jurnal Indonesia Sosial Teknologi, Vol. 4, No. 8, Agustus 2023 939

Bibliography

Alamdari, Pegah Malekpour, Navimipour, Nima Jafari, Hosseinzadeh, Mehdi, Safaei,

Ali Asghar, & Darwesh, Aso. (2022). An image-based product recommendation

for E-commerce applications using convolutional neural networks. Acta

Informatica Pragensia, 11(1), 15–35.

Appalaraju, Srikar, & Chaoji, Vineet. (2017). Image similarity using deep CNN and

curriculum learning. ArXiv Preprint ArXiv:1709.08761.

Bhatti, Anam, Akram, Hamza, Basit, Hafiz Muhammad, Khan, Ahmed Usman, Raza,

Syeda Mahwish, & Naqvi, Muhammad Bilal. (2020). E-commerce trends during

COVID-19 Pandemic. International Journal of Future Generation

Communication and Networking, 13(2), 1449–1452.

Dagan, Arnon, Guy, Ido, & Novgorodov, Slava. (2021). An image is worth a thousand

terms? analysis of visual e-commerce search. Proceedings of the 44th

International ACM SIGIR Conference on Research and Development in

Information Retrieval, 102–112.

Goel, Nishant. (2017). Shopbot: an image-based search application for e-commerce

domain. https://doi.org/10.31979/etd.r7a5-6dzf

Hanifah, Nurul, & Rahadi, Dedi Rianto. (2020). Analisis perilaku konsumen dalam

memutuskan pembelian secara online pada masa pandemi COVID-19.

SULTANIST: Jurnal Manajemen Dan Keuangan, 8(2), 112–122.

https://doi.org/10.37403/sultanist.v8i2.206

Jenni, Kommineni, Mandala, Satria, & Sunar, Mohd Shahrizal. (2015). Content-based

image retrieval using colour strings comparison. Procedia Computer Science, 50,

374–379. https://doi.org/10.1016/j.procs.2015.04.032

Laming, Syamsidarti. (2020). Tren E-Commerce Pada Era Pandemi COVID-19.

Humano: Jurnal Penelitian, 11(2), 55–63.

https://doi.org/10.33387/hjp.v11i2.2323

Mira, Mira, Sembiring, Irwan, & Purnomo, Hindriyanto Dwi. (2022). Implementasi

Transfer Learning Pada Algoritma Convolutional Neural Network untuk

Mengklasifikasikan Image Objek Wisata. Building of Informatics, Technology,

and Science (BITS), 4(1), 209–216. https://doi.org/10.47065/bits.v4i1.1764

Patra, Braja Gopal, Maroufy, Vahed, Soltanalizadeh, Babak, Deng, Nan, Zheng, W.

Jim, Roberts, Kirk, & Wu, Hulin. (2020). A content-based literature

recommendation system for datasets to improve data reusability–A case study on

Gene Expression Omnibus (GEO) datasets. Journal of Biomedical Informatics,

104, 103399. https://doi.org/10.1016/j.jbi.2020.103399

Sanchez, S. A., Romero, H. J., & Morales, A. D. (2020). A review: Comparison of

performance metrics of pretrained models for object detection using the

Trenggana Natadirja, Habibullah Akbar, Gerry Firmansyah, Budi Tjahjono

Jurnal Indonesia Sosial Teknologi, Vol. 4, No. 8, August 2023 940

TensorFlow framework. IOP Conference Series: Materials Science and

Engineering, 844(1), 12024. https://doi.org/10.1088/1757-899X/844/1/012024

Schmidhuber, Jürgen. (2015). Deep learning in neural networks: An overview. Neural

Networks, 61, 85–117. https://doi.org/10.1016/j.neunet.2014.09.003

Sulthana, A. Razia, Gupta, Maulika, Subramanian, Shruthi, & Mirza, Sakina. (2020).

Improvising the performance of image-based recommendation systems using

convolution neural networks and deep learning. Soft Computing, 24(19), 14531–

14544.