Implementation of The Kirkpatrick Model In Evaluating The Use of Digital Technology In Counseling Guidance Programs at SMP Negeri 1 Matan Hilir Utara

ABSTRACT


Introduction
The existence of schools as educational institutions is oriented to develop the potential and shape students' character in a better direction.This is strengthened by the facilitation of counseling guidance service (BK) programs in the school environment to encourage accelerating the achievement of potential development and character building (Nurmala, Pertiwi, Devi, & Rachmayanti, 2021).Batubara et al. (2022) stated that the scope of the role of guidance and counseling is not only related to academics but can also improve personality, value-giving, social and intellectual (Guyot-Staal, Da Silva, van der Merwe, & Hattingh, 2022).
Technically, improving the quality of BK services can be done by developing the professionalism of BK teachers to be able to work professionally; another fundamental demand is also for the Principal to optimize BK service management, which can have implications for the realization of the success of BK service programs in general (Churnia, Firman, & Ahmad, 2021).There are various obstacles faced in implementing BK services to reach optimal points, which are also identified in implementing BK service programs at SMP Negeri 1 Matan Hilir Utara, Ketapang Regency.Some gaps occur in implementing BK services at SMP Negeri 1 Matan Hilir Utara, which researchers identified during the pre-research period.In this case, the researcher interviewed the technical implementation and obstacles faced in implementing the BK service program to BK teachers in the school designated by the researcher as the location of the study.The informant stated that in the last period, counseling guidance was not involved in the KBM process or was not included as a subject based on the informant's perspective; this was one of the factors that hindered the optimization of the BK service program at SMP Negeri 1 Matan Hilir.
Another obstacle identified based on the results of previous research related to the implementation of BK service programs is the understanding and direction of career orientation for students, which is also the purpose of holding BK service programs is not optimized, the same obstacle is related to not being given space for BK teachers to provide teaching materials about BK in class accompanied by the lack of BK teachers who cannot accommodate services to all students (Sihite, 2023).rs regarding the identification of children's problems.In this momentum, data were obtained that there were various essential cases of children such as there were still many children who dropped out of school because they married early or helped their parents' economy, the number of violations committed by students, and the low ethics and courtesy of students towards their teachers which showed that in each period there was no downward trend but an increase in the number of cases.This is another trigger point for researchers to evaluate the program against existing BK services.Researchers believe that evaluating the BK service program at SMP 1 Matan Hilir Utara will help reduce the percentage of student problems.
Based on the results of the pre-research, problems related to the reactions and behavior of students in the use of counseling service facilities in the school environment can be identified by BK teachers.In addition, gaps also arise regarding the limitation of meetings related to the presentation of BK material in class, which has implications for the non-optimal quality of BK service output at SMP Negeri 1 Matan Hilir.The emergence of gaps in aspects of reaction, behavior, and learning context, which has implications for the quality of BK service output, is why a comprehensive evaluation is needed.Kirkpatrick's (2006) evaluation model accommodates the identified aspects of gaps in the service context.The Kirkpatrick model includes an evaluation model that has advantages in evaluating a program thoroughly and flexibly in the sense that it can be applied in various conditions (Ramadani & Herdi, 2021).This evaluation model accommodates assessments on various aspects, namely regarding participants' perceptions of the program, improving aspects of knowledge and skills and attitudes of participants, aspects of transferring knowledge, skills, and attitudes of participants positively and effectively from one level to another, to the final result.When compared with other models, Kirkpatrick's model is classified as easy to use; this is illustrated by the indicators contained in it, which are easily adapted; this adds a particular argument for researchers considering that this study is more focused on assessment based on student perspectives, thus facilitating the process of understanding evaluation indicators.
One of the things that can be optimized to streamline BK's service program is integrating technological resources into it.According to (Pama et al., 2023), using technology in guidance and counseling services can increase material absorption in services in realizing counselors' efforts as effective facilitators in fostering students' independence in schools and presenting innovative, creative, adaptive, and flexible BK services in the era of disruption.
The orientation of using Kirkpatrick's evaluation model in this case in the context of BK services is expected to optimize essential aspects of services that can minimize existing gaps to optimize the implementation and achievement of counseling quality, including efforts to improve the quality of students.In this regard, researchers conducted empirical research by conducting interviews with one of the BK teachers and observing service programs both in the BK room and the classroom.Based on the interviews conducted, information was obtained that the application of technological elements in the BK service program was minimal; it was stated that guidance and counseling services in the BK room did not apply technological elements other than communication using WhatsApp to connect personal communication carried out with specific students, while in BK services manifested in classroom learning was dominated by applying an approach Conventional is the method of lecture.
Based on this background, researchers will conduct a study entitled "Implementation of the Kirkpatrick Model in Evaluating the Use of Digital Technology in the Counseling Guidance Program at SMP Negeri 1 Matan Hilir Utara".

Research Methods
In order to achieve the research objectives, researchers apply a type of descriptive quantitative research.According to (Gata, Surohman, & Nawawi, 2023), descriptive research is a procedure for solving problems in the analyzed case by describing the circumstances or objects of research.The source of data in this study is the output of questionnaires by students in evaluating the use of digital technology in BK service programs at SMP Negeri 1 Matan Hilir Utara by referring to aspects and stages of evaluation based on the Kirkpatrick Four Levels Evaluation Model model, while the interview results of BK Principals and Teachers as secondary to explore the obstacles faced in optimizing the use of digital technology in BK service programs.
The subjects in this study were class VIII students totaling 60 people who were determined based on a saturated sampling technique where all members of the population were used as research samples and acted as evaluators or beneficiaries, and also involved BK teachers and principals of SMP Negeri 1 Matan Hilir Utara as program organizers.
The following will describe the essential components of the research method that contain Kirkpatrick evaluation indicators and the techniques and data collection tools applied in this study.

Results and Discussion
During the period of distribution of questionnaires conducted to respondents consisting of 60 students in grade VIII SMPN 01 Matan Hilir Utara, two questionnaires were given, namely related to the evaluation of students as beneficiaries related to BK service programs in general and regarding the use of technology in it.In this case, the researcher provides questionnaires to students related to each of these indicators both in the program in general and about the use of technology in the program, which then the average will be adjusted based on the interpretation of the average score on the Likert scale with an average score of 1.00 -2.49 is in the low category, the average of 2.50 -3.49 is in the medium category, and an average of 3.50 -5.00 is in the high category.
The following will be published: the average evaluation of BK service programs and technology integration based on students' perspectives at each level in the Kirkpatrick evaluation model.

Evaluate the Use of Digital Technology at the Reaction Level
The following will be presented, along with the average and achievement level, based on the evaluation questionnaire on the BK service program and the use of technology in the program at the reaction level.Indicators in this reaction level include participant satisfaction, opinions, and student responses.In general, students have an excellent reaction to the BK program at school, but regarding the use of technology in the program, students consider that this is not done optimally.According to Kirkpatrick (2006), evaluation at the reaction level is aimed at determining the extent of student satisfaction in terms of programs set by the school.In the indicator of student satisfaction as beneficiaries, an average of 3.83 was obtained in the high category and became the lowest indicator compared to other indicators at this reaction level.This shows that efforts are needed to increase student satisfaction with BK services.
The principal and teachers are aware of the lack of technology exploration in BK services provided to students, and this is reinforced by the results of questionnaires that show the non-optimal use of technology based on student perspectives.One of the factors is the non-permissive use of mobile phones by students, which are designed instructionally by BK teachers to carry out services to program beneficiaries.However, the Principal has an argument in the name of justice because there are students who have these devices, not to mention the problem of internet quota, which is an obstacle and can trigger unsmooth program implementation.(Pranowo & Santoso, 2021) states that guidance is an aid obtained by individuals to obtain adequate personality so that specific individuals can organize, develop life, make decisions, and be independent of others.In detail, it can be explained that students assess BK services as helping them form mature personalities, arrangements, selfdevelopment, and the construction of personal decisions.In addition, it needs to be optimized from the aspect of using technology in BK services that have not reached the optimal point, with an average value of 3.35 in the medium category.

Evaluate the Use of Digital Technology at the Learning Level
The following will be presented, along with the average and achievement level, based on the evaluation questionnaire on the BK service program and the use of technology in the program at the learning level.Indicators in this level of learning include knowledge acquisition, skill improvement, and attitude change.According to (Sanusi, Gustini, Suryana, & Almafahir, 2022), learning evaluation is aimed at determining the understanding of students in obtaining targeted knowledge, skills, and attitudes so that the evaluation process can be done using written tests, performance, or daily tasks.Based on the results of questionnaires at the learning level, students perceive that BK services in schools on the use of technology in general are also in the high category.Based on students' perspective, it shows that the indicator of mastery of knowledge is higher than other indicators, namely 3.97 in the medium category, in the indicator of skill improvement with an average of 3.42 in the medium category.In contrast, in the student response indicator, an average of 3.33 in the medium category is obtained.
This is triggered by the use of PowerPoint applications as presentation media in the momentum of service, especially in class guidance.The principal and BK teachers conveyed this as program implementers, who stated that the presentation media was applied to improve students' knowledge of BK and its services.However, it was also conveyed that there is a need for exploration in other digital technology-based media, such as BK service systems such as e-counseling, and so on, which currently have not been implemented.Batubara et al. (2022) stated that the scope of the role of guidance and counseling is not only related to academics but can also improve personality, value-giving, social and intellectual.The study results show that, in general, BK service students in schools are practical not only in terms of strengthening intellectuality, social values, and skills.In this level of learning, students also assess the use of technology well because of the element of utilization in the form of presentation media when presenting material at a certain momentum.

Evaluate the Use of Digital Technology at the Behavior Level
At this behavior level, it contains indicators of the application of values in the program obtained the following results: According to Kirkpatrick (2006), the behavioral level (behavior) is concentrated on behavior changes that occur, or it can also be explained that the process of identifying the extent to which students practice material is manifested in the form of behavior or attitudes both before and after the educator teaches.Based on these outputs, it shows that from the perspective of students, BK services at the research location have not been able to optimize changes in their behavior from before experiencing services to after getting services, especially from BK teachers through the use of digital technology in it.
The researcher then conducted an interview with BK teachers regarding this matter; the statement conveyed that changes in students' attitudes to be subsequently applied in daily life are difficult to realize because, in its implementation, the BK program is not carried out systemically with clear indicators, including in terms of attitudes that can be a particular target for the program and teachers or counselors in running the program.This substance affects the use of technology as a medium in building the effectiveness of strengthening the application of BK values in students' daily lives.
Counseling can be interpreted as a relationship between clients and counselors in helping specific individuals adjust effectively to their environment (Veronica, Purwanta, & Astuti, 2020).Based on the research results, which connect with the theory above, students assess that BK services in schools have implications for applying values in everyday life.Optimizing the use of technology in services needs to be emphasized to increase the effectiveness of efforts to adapt students to their environment.

Evaluation of the Use of Digital Technology at the Resul Level
At this result, level contains indicators of program results, the average value is obtained as follows: According to Kirkpatrick (2006), evaluation of results is intended to evaluate final results such as improvement, reduction, level of program quality, and so on.These results show that students assess that the BK service program at the research location notably improves students' quality after participating in services with BK teachers.
Based on the results of interviews with the Principal and BK teachers regarding this matter, the statement conveyed that the optimization of the BK service program is difficult to realize because, in its implementation, the BK program is not carried out in proportional conditions both from the level of accessibility of students benefiting from counseling guidance in improving knowledge, skills, and attitudes, including to present technology-based service media for the lack of suitable infrastructure from schools and students personally, some of whom are in the lower middle economy, thus affecting the difficulty of creating digital nuances in BK services at the research location.
Another thing identified by the researcher is the finding that the discussion room related to the essence of BK and its function to students and schools is not realized concretely.There is no exceptional learning BK, can be a momentum for teachers/counselors in providing material related to the essence of BK itself; this is increasingly triggered by the lack of participation of students discussing their psychological conditions to the BK room which is supporting data for the lack of discussion space or distribution of related information BK to students, this then has particular implications in efforts to improve the skills and attitudes of students which are then not appropriately optimized.
Improving the quality of BK services can be done by developing the professionalism of BK teachers to be able to work professionally; another fundamental demand is also for the Principal in order to optimize BK service management, which can have implications for the realization of the success of BK service programs in general (Churnia et al., 2021).In this case, students consider that the quality of BK services, which contains elements of BK teacher professionalism, the contribution of the Principal in services, to service management in general, even though the use of technology needs special attention to improve the scale of the quality of BK service programs in general and based explicitly on Kirkpatrick indicators (1996).

Conclusion
Based on the findings in the study, it can be concluded that: 1) At the reaction level, the use of technology in the BK program is on a medium scale, with an average of 3.35.Student response indicators are categorized as the lowest indicators compared to other indicators, with an average of 3.32 in the medium category, and need special attention to be optimized; 2) At the learning level, obtaining an average of 3.57 in the high category while the attitude change indicator is the lowest compared to other indicators, namely with an average of 3.33 in the medium category that needs to be maximized, 3) At the behavior level with indicators of the application of values in the program obtained an average of 3.38 in aspects of technology use that need special attention to be optimized, and 4) At the result level, students consider that the quality/results of BK service programs are in the medium category with an average of 3.35 which needs to be maximized to improve the quality of BK programs at the research location.