p–ISSN: 2723 - 6609 e-ISSN: 2745-5254
Vol. 5, No. 4 April 2024 http://jist.publikasiindonesia.id/
Indonesian Journal of Social Technology, Vol. 5, No. 4 April, 2024 1771
Implementation of The Kirkpatrick Model In
Evaluating The Use of Digital Technology In Counseling
Guidance Programs at SMP Negeri 1 Matan Hilir Utara
Rahmi Mayarani
1*
, Urai Salam
2
, Luhur Wicaksono
3
Universitas Tanjungpura, Indonesia
Email: [email protected]
*Correspondence
ABSTRACT
Keywords:
Counseling Guidance
Services; Use Of
Technology In
Counseling Guidance;
Kirkpatrick Evaluation
Model.
This research aimed to evaluate the Guidance Counseling
(BK) service program using digital technology at SMP
Negeri 1 Matan Hilir Utara using the Kirkpatrick Evaluation
Model. This research method was quantitative with a
descriptive approach. Data is technically collected using
questionnaire techniques, the primary data source. The
subjects in the research were 60 class VIII students who
were determined based on a saturated sampling technique
where all members of the population were used as research
samples. In the context of this research, students act as
evaluators or beneficiaries. At the same time, the guidance
and counseling teachers and school principals are the parties
who evaluate the implemented guidance and counseling
program services. The results of the research show that: 1)
At the reaction level, BK services are perceived as showing
a scale using digital technology that is on a medium scale
with an average of 3.35, 2) At the learning level, BK services
in terms of technology use obtain an average of 3. 57 in the
high category, 3) At the behavior level with indicators of
application of values in the program it is in the medium
category with an average of 3.38 in aspects of technology
use that need special attention to be optimized, 4) At the
results level, students assume that the use of technology is in
the medium category with an average of 3.35 which needs
to be maximized to improve the quality of the BK program
at the research location.
Introduction
The existence of schools as educational institutions is oriented to develop the
potential and shape students' character in a better direction. This is strengthened by the
facilitation of counseling guidance service (BK) programs in the school environment to
encourage accelerating the achievement of potential development and character building
(Nurmala, Pertiwi, Devi, & Rachmayanti, 2021). Batubara et al. (2022) stated that the
scope of the role of guidance and counseling is not only related to academics but can also
Rahmi Mayarani, Urai Salam, Luhur Wicaksono
Indonesian Journal of Social Technology, Vol. 5, No. 4 April, 2024 1772
improve personality, value-giving, social and intellectual (Guyot-Staal, Da Silva, van der
Merwe, & Hattingh, 2022).
Technically, improving the quality of BK services can be done by developing the
professionalism of BK teachers to be able to work professionally; another fundamental
demand is also for the Principal to optimize BK service management, which can have
implications for the realization of the success of BK service programs in general (Churnia,
Firman, & Ahmad, 2021). There are various obstacles faced in implementing BK services
to reach optimal points, which are also identified in implementing BK service programs
at SMP Negeri 1 Matan Hilir Utara, Ketapang Regency.
Some gaps occur in implementing BK services at SMP Negeri 1 Matan Hilir Utara,
which researchers identified during the pre-research period. In this case, the researcher
interviewed the technical implementation and obstacles faced in implementing the BK
service program to BK teachers in the school designated by the researcher as the location
of the study. The informant stated that in the last period, counseling guidance was not
involved in the KBM process or was not included as a subject based on the informant's
perspective; this was one of the factors that hindered the optimization of the BK service
program at SMP Negeri 1 Matan Hilir.
Another obstacle identified based on the results of previous research related to the
implementation of BK service programs is the understanding and direction of career
orientation for students, which is also the purpose of holding BK service programs is not
optimized, the same obstacle is related to not being given space for BK teachers to provide
teaching materials about BK in class accompanied by the lack of BK teachers who cannot
accommodate services to all students (Sihite, 2023). rs regarding the identification of
children's problems. In this momentum, data were obtained that there were various
essential cases of children such as there were still many children who dropped out of
school because they married early or helped their parents' economy, the number of
violations committed by students, and the low ethics and courtesy of students towards
their teachers which showed that in each period there was no downward trend but an
increase in the number of cases. This is another trigger point for researchers to evaluate
the program against existing BK services. Researchers believe that evaluating the BK
service program at SMP 1 Matan Hilir Utara will help reduce the percentage of student
problems.
Based on the results of the pre-research, problems related to the reactions and
behavior of students in the use of counseling service facilities in the school environment
can be identified by BK teachers. In addition, gaps also arise regarding the limitation of
meetings related to the presentation of BK material in class, which has implications for
the non-optimal quality of BK service output at SMP Negeri 1 Matan Hilir. The
emergence of gaps in aspects of reaction, behavior, and learning context, which has
implications for the quality of BK service output, is why a comprehensive evaluation is
needed.
Kirkpatrick's (2006) evaluation model accommodates the identified aspects of gaps
in the service context. The Kirkpatrick model includes an evaluation model that has
Implementation of The Kirkpatrick Model In Evaluating The Use of Digital Technology
In Counseling Guidance Programs at SMP Negeri 1 Matan Hilir Utara
Indonesian Journal of Social Technology, Vol. 5, No. 4 April, 2024 1773
advantages in evaluating a program thoroughly and flexibly in the sense that it can be
applied in various conditions (Ramadani & Herdi, 2021). This evaluation model
accommodates assessments on various aspects, namely regarding participants'
perceptions of the program, improving aspects of knowledge and skills and attitudes of
participants, aspects of transferring knowledge, skills, and attitudes of participants
positively and effectively from one level to another, to the final result. When compared
with other models, Kirkpatrick's model is classified as easy to use; this is illustrated by
the indicators contained in it, which are easily adapted; this adds a particular argument
for researchers considering that this study is more focused on assessment based on student
perspectives, thus facilitating the process of understanding evaluation indicators.
One of the things that can be optimized to streamline BK's service program is
integrating technological resources into it. According to (Pama et al., 2023), using
technology in guidance and counseling services can increase material absorption in
services in realizing counselors' efforts as effective facilitators in fostering students'
independence in schools and presenting innovative, creative, adaptive, and flexible BK
services in the era of disruption.
The orientation of using Kirkpatrick's evaluation model in this case in the context
of BK services is expected to optimize essential aspects of services that can minimize
existing gaps to optimize the implementation and achievement of counseling quality,
including efforts to improve the quality of students. In this regard, researchers conducted
empirical research by conducting interviews with one of the BK teachers and observing
service programs both in the BK room and the classroom. Based on the interviews
conducted, information was obtained that the application of technological elements in the
BK service program was minimal; it was stated that guidance and counseling services in
the BK room did not apply technological elements other than communication using
WhatsApp to connect personal communication carried out with specific students, while
in BK services manifested in classroom learning was dominated by applying an approach
Conventional is the method of lecture.
Based on this background, researchers will conduct a study entitled
"Implementation of the Kirkpatrick Model in Evaluating the Use of Digital Technology
in the Counseling Guidance Program at SMP Negeri 1 Matan Hilir Utara".
Research Methods
In order to achieve the research objectives, researchers apply a type of descriptive
quantitative research. According to (Gata, Surohman, & Nawawi, 2023), descriptive
research is a procedure for solving problems in the analyzed case by describing the
circumstances or objects of research. The source of data in this study is the output of
questionnaires by students in evaluating the use of digital technology in BK service
programs at SMP Negeri 1 Matan Hilir Utara by referring to aspects and stages of
evaluation based on the Kirkpatrick Four Levels Evaluation Model model, while the
interview results of BK Principals and Teachers as secondary to explore the obstacles
faced in optimizing the use of digital technology in BK service programs.
Rahmi Mayarani, Urai Salam, Luhur Wicaksono
Indonesian Journal of Social Technology, Vol. 5, No. 4 April, 2024 1774
The subjects in this study were class VIII students totaling 60 people who were
determined based on a saturated sampling technique where all members of the population
were used as research samples and acted as evaluators or beneficiaries, and also involved
BK teachers and principals of SMP Negeri 1 Matan Hilir Utara as program organizers.
The following will describe the essential components of the research method that
contain Kirkpatrick evaluation indicators and the techniques and data collection tools
applied in this study.
Table 1.
Research Methods
Evaluation
Model
Evaluatio
n Aspect
Indicators
Operational
Data
Collectio
n
Techniqu
es
Tool
Collection
Data
Evaluation
of the Four
Levels of
the
Kirkpatric
Model
Reaction
● Satisfaction with
the use of
technology in
BK service
programs
● Opinion on the
use of
technology in
BK's service
program
● Response to the
use of
technology in
BK service
programs
Questionn
aire,
Face-to-
face
interview
Questionnair
es, Interview
Guidelines
Learning
● Mastery of
knowledge
through the use
of technology in
BK service
programs
● Skill
improvement
through the use
of technology in
BK service
programs
● Attitude change
through the use
of technology in
Implementation of The Kirkpatrick Model In Evaluating The Use of Digital Technology
In Counseling Guidance Programs at SMP Negeri 1 Matan Hilir Utara
Indonesian Journal of Social Technology, Vol. 5, No. 4 April, 2024 1775
BK service
programs
Behavior
● Behavior change
in implementing
learning program
outcomes
through the use
of technology
Result
● The result that
occurs as a result
of participants
participating in
the service
program through
the use of
technology
Results and Discussion
During the period of distribution of questionnaires conducted to respondents
consisting of 60 students in grade VIII SMPN 01 Matan Hilir Utara, two questionnaires
were given, namely related to the evaluation of students as beneficiaries related to BK
service programs in general and regarding the use of technology in it. In this case, the
researcher provides questionnaires to students related to each of these indicators both in
the program in general and about the use of technology in the program, which then the
average will be adjusted based on the interpretation of the average score on the Likert
scale with an average score of 1.00 – 2.49 is in the low category, the average of 2.50 –
3.49 is in the medium category, and an average of 3.50 – 5.00 is in the high category.
The following will be published: the average evaluation of BK service programs
and technology integration based on students' perspectives at each level in the Kirkpatrick
evaluation model.
Evaluate the Use of Digital Technology at the Reaction Level
The following will be presented, along with the average and achievement level,
based on the evaluation questionnaire on the BK service program and the use of
technology in the program at the reaction level. Indicators in this reaction level include
participant satisfaction, opinions, and student responses.
Table 2
Questionnaire Results at the Reaction Level Based on Student Perspective
Quality of Technology Use in the
BK Program
Mean
Category
Rahmi Mayarani, Urai Salam, Luhur Wicaksono
Indonesian Journal of Social Technology, Vol. 5, No. 4 April, 2024 1776
3,38
Keep
3,35
Keep
3,32
Keep
3,35
Keep
In general, students have an excellent reaction to the BK program at school, but
regarding the use of technology in the program, students consider that this is not done
optimally. According to Kirkpatrick (2006), evaluation at the reaction level is aimed at
determining the extent of student satisfaction in terms of programs set by the school. In
the indicator of student satisfaction as beneficiaries, an average of 3.83 was obtained in
the high category and became the lowest indicator compared to other indicators at this
reaction level. This shows that efforts are needed to increase student satisfaction with BK
services.
The principal and teachers are aware of the lack of technology exploration in BK
services provided to students, and this is reinforced by the results of questionnaires that
show the non-optimal use of technology based on student perspectives. One of the factors
is the non-permissive use of mobile phones by students, which are designed
instructionally by BK teachers to carry out services to program beneficiaries. However,
the Principal has an argument in the name of justice because there are students who have
these devices, not to mention the problem of internet quota, which is an obstacle and can
trigger unsmooth program implementation.
(Pranowo & Santoso, 2021) states that guidance is an aid obtained by individuals
to obtain adequate personality so that specific individuals can organize, develop life,
make decisions, and be independent of others. In detail, it can be explained that students
assess BK services as helping them form mature personalities, arrangements, self-
development, and the construction of personal decisions. In addition, it needs to be
optimized from the aspect of using technology in BK services that have not reached the
optimal point, with an average value of 3.35 in the medium category.
Evaluate the Use of Digital Technology at the Learning Level
The following will be presented, along with the average and achievement level,
based on the evaluation questionnaire on the BK service program and the use of
technology in the program at the learning level. Indicators in this level of learning include
knowledge acquisition, skill improvement, and attitude change.
Table 3.
Questionnaire Results at the Learning Level Based on Student Perspectives
Indicators
Quality of Use of Technology
In BK Program
Implementation of The Kirkpatrick Model In Evaluating The Use of Digital Technology
In Counseling Guidance Programs at SMP Negeri 1 Matan Hilir Utara
Indonesian Journal of Social Technology, Vol. 5, No. 4 April, 2024 1777
Mean
Category
Knowledge mastery
3,97
Tall
Upskilling
3,42
Keep
Attitude change
3,33
Keep
Average of All Indicators
3,57
Tall
According to (Sanusi, Gustini, Suryana, & Almafahir, 2022), learning evaluation is
aimed at determining the understanding of students in obtaining targeted knowledge,
skills, and attitudes so that the evaluation process can be done using written tests,
performance, or daily tasks. Based on the results of questionnaires at the learning level,
students perceive that BK services in schools on the use of technology in general are also
in the high category. Based on students' perspective, it shows that the indicator of mastery
of knowledge is higher than other indicators, namely 3.97 in the medium category, in the
indicator of skill improvement with an average of 3.42 in the medium category. In
contrast, in the student response indicator, an average of 3.33 in the medium category is
obtained.
This is triggered by the use of PowerPoint applications as presentation media in
the momentum of service, especially in class guidance. The principal and BK teachers
conveyed this as program implementers, who stated that the presentation media was
applied to improve students' knowledge of BK and its services. However, it was also
conveyed that there is a need for exploration in other digital technology-based media,
such as BK service systems such as e-counseling, and so on, which currently have not
been implemented.
Batubara et al. (2022) stated that the scope of the role of guidance and counseling
is not only related to academics but can also improve personality, value-giving, social and
intellectual. The study results show that, in general, BK service students in schools are
practical not only in terms of strengthening intellectuality, social values, and skills. In this
level of learning, students also assess the use of technology well because of the element
of utilization in the form of presentation media when presenting material at a certain
momentum.
Evaluate the Use of Digital Technology at the Behavior Level
At this behavior level, it contains indicators of the application of values in the
program obtained the following results:
Table 4.
Results of Questionnaires at the Behavior Level Based on Student Perspectives
Indicators
Quality of Use of Technology
In BK Program
Mean
Category
Rahmi Mayarani, Urai Salam, Luhur Wicaksono
Indonesian Journal of Social Technology, Vol. 5, No. 4 April, 2024 1778
Application of values in the program
3,38
Keep
According to Kirkpatrick (2006), the behavioral level (behavior) is concentrated on
behavior changes that occur, or it can also be explained that the process of identifying the
extent to which students practice material is manifested in the form of behavior or
attitudes both before and after the educator teaches. Based on these outputs, it shows that
from the perspective of students, BK services at the research location have not been able
to optimize changes in their behavior from before experiencing services to after getting
services, especially from BK teachers through the use of digital technology in it.
The researcher then conducted an interview with BK teachers regarding this matter;
the statement conveyed that changes in students' attitudes to be subsequently applied in
daily life are difficult to realize because, in its implementation, the BK program is not
carried out systemically with clear indicators, including in terms of attitudes that can be
a particular target for the program and teachers or counselors in running the program.
This substance affects the use of technology as a medium in building the effectiveness of
strengthening the application of BK values in students' daily lives.
Counseling can be interpreted as a relationship between clients and counselors in
helping specific individuals adjust effectively to their environment (Veronica, Purwanta,
& Astuti, 2020). Based on the research results, which connect with the theory above,
students assess that BK services in schools have implications for applying values in
everyday life. Optimizing the use of technology in services needs to be emphasized to
increase the effectiveness of efforts to adapt students to their environment.
Evaluation of the Use of Digital Technology at the Resul Level
At this result, level contains indicators of program results, the average value is
obtained as follows:
Table 5
Questionnaire Results at the Result Level Based on Student Perspectives
Indicators
Quality of Use of Technology
In BK Program
Mean
Category
Program results
3,35
Keep
According to Kirkpatrick (2006), evaluation of results is intended to evaluate final
results such as improvement, reduction, level of program quality, and so on. These results
show that students assess that the BK service program at the research location notably
improves students' quality after participating in services with BK teachers.
Based on the results of interviews with the Principal and BK teachers regarding this
matter, the statement conveyed that the optimization of the BK service program is
difficult to realize because, in its implementation, the BK program is not carried out in
proportional conditions both from the level of accessibility of students benefiting from
Implementation of The Kirkpatrick Model In Evaluating The Use of Digital Technology
In Counseling Guidance Programs at SMP Negeri 1 Matan Hilir Utara
Indonesian Journal of Social Technology, Vol. 5, No. 4 April, 2024 1779
counseling guidance in improving knowledge, skills, and attitudes, including to present
technology-based service media for the lack of suitable infrastructure from schools and
students personally, some of whom are in the lower middle economy, thus affecting the
difficulty of creating digital nuances in BK services at the research location.
Another thing identified by the researcher is the finding that the discussion room
related to the essence of BK and its function to students and schools is not realized
concretely. There is no exceptional learning BK, which can be a momentum for
teachers/counselors in providing material related to the essence of BK itself; this is
increasingly triggered by the lack of participation of students discussing their
psychological conditions to the BK room which is supporting data for the lack of
discussion space or distribution of related information BK to students, this then has
particular implications in efforts to improve the skills and attitudes of students which are
then not appropriately optimized.
Improving the quality of BK services can be done by developing the
professionalism of BK teachers to be able to work professionally; another fundamental
demand is also for the Principal in order to optimize BK service management, which can
have implications for the realization of the success of BK service programs in general
(Churnia et al., 2021). In this case, students consider that the quality of BK services,
which contains elements of BK teacher professionalism, the contribution of the Principal
in services, to service management in general, even though the use of technology needs
special attention to improve the scale of the quality of BK service programs in general
and based explicitly on Kirkpatrick indicators (1996).
Conclusion
Based on the findings in the study, it can be concluded that: 1) At the reaction level,
the use of technology in the BK program is on a medium scale, with an average of 3.35.
Student response indicators are categorized as the lowest indicators compared to other
indicators, with an average of 3.32 in the medium category, and need special attention to
be optimized; 2) At the learning level, obtaining an average of 3.57 in the high category
while the attitude change indicator is the lowest compared to other indicators, namely
with an average of 3.33 in the medium category that needs to be maximized, 3) At the
behavior level with indicators of the application of values in the program obtained an
average of 3.38 in aspects of technology use that need special attention to be optimized,
and 4) At the result level, students consider that the quality/results of BK service programs
are in the medium category with an average of 3.35 which needs to be maximized to
improve the quality of BK programs at the research location.
Rahmi Mayarani, Urai Salam, Luhur Wicaksono
Indonesian Journal of Social Technology, Vol. 5, No. 4 April, 2024 1780
Bibliography
Churnia, Elfi, Firman, Firman, & Ahmad, Riska. (2021). Professionalism Of Counseling
Teachers In A Centralized Education System. Citra: International Journal of
Community Service, Informatics, Technology, Research in Education, Art and
Humanities, 1(2), 31–44. https://doi.org/orcid.org/0000-0002-8384-1061
Gata, Windu, Surohman, Surohman, & Nawawi, Hendri Mahmud. (2023). Twitter in
analysis of policy sentiments of the omnibus law work creative design. AIP
Conference Proceedings, 2714(1). AIP Publishing.
Guyot-Staal, Amélie M. L., Da Silva, Janavi, van der Merwe, Monika, & Hattingh, Elize.
(2022). Mapping Academic to Entrepreneurial and Intrapreneurial Transition
Pathways for Water Graduates to Support Water Sector Innovation in South Africa.
Nurmala, Ira, Pertiwi, Elisa Dwi, Devi, Yuli Puspita, & Rachmayanti, Riris Diana. (2021).
Perception of roles as peer educators in high schools to prevent drug abuse among
adolescents. Indian Journal of Forensic Medicine & Toxicology, 14(1), 1351–1354.
Pama, Sindi Ayudia, Purnama, Diana Septi, Nurjanah, Anisa Siti, Adilah, Naufal Alwan,
Zatrahadi, Muhammad Fahli, & Roza, Ellya. (2023). Community counseling
strategies to improve mental health literacy. Jurnal Konseling Dan Pendidikan,
11(1), 12–20.
Pranowo, Taufik Agung, & Santoso, Ardhian Rohmad. (2021). The implementation of
guidance and counseling services in overcoming the cheating habit of high school
students. KONSELI: Jurnal Bimbingan Dan Konseling (E-Journal), 8(2), 127–134.
https://doi.org/10.24042/kons.v8i2.7196
Ramadani, Dewita, & Herdi, Herdi. (2021). Studi Kepustakaan Mengenai Kinerja Guru
Bimbingan Dan Konseling Dalam Perencanaan Program Bimbingan Dan
Konseling Di Sekolah. JURNAL EDUKASI: Jurnal Bimbingan Konseling, 7(1),
42–52.
Sanusi, Hary Priatna, Gustini, Neng, Suryana, Yaya, & Almafahir, Ahmad. (2022).
Management of Guidance and Counseling Service Based Cybercounseling in
Madrasah in The New Normal Era. AL-ISHLAH: Jurnal Pendidikan, 14(4), 7123–
7130.
Sihite, Julius Martunas. (2023). The effectiveness of guidance and counseling services
implementation to improve students’ competency standards. Journal of Advanced
Guidance and Counseling, 4(1), 41–48.
Veronica, Nia, Purwanta, Edi, & Astuti, Budi. (2020). Design and development of a
mobile learning for career planning in senior high school. International Journal of
Scientific and Technology Research, 9(1), 908–913.