Hot-Fit Model to Measure the Effectiveness and Efficiency of Information System in Public Sector

The research endeavored to determine the effectiveness and efficiency of the information systems’ implementation. The research used the modified Human Organization Technology-Fit (HOT-Fit) as the model to measure information systems’ effectiveness and efficiency. Each indicator of the HOT-Fit model is mapped into two categories, namely effectiveness and efficiency. Data collection was conducted through interviews with the users of population administration information system (SIAK), and observation of activities at the public office. The data was analyzed qualitatively by assessing the achievement of each indicator of effectiveness and efficiency found in the interview and observation. It is found that effectiveness indicators are level of use, attitude, perceived usefulness, communication, availability, security, completeness, accuracy, timeliness, empathy, and transparency. Meanwhile, indicators to measure efficiency are users, training, staff support, ease of use, ease of learning, response time, access to technical support, and quick responsiveness. the HOT-Fit model in evaluating information systems. Practically, the research can be a reference for evaluating the effectiveness and efficiency of information system implementation in the public sector. It also can be used to improve the achievement of indicators that have not achieved maximum results to improve services to the community.


I. INTRODUCTION
The use of technology has covered all aspects of human life. Information systems are one of these technologies. In government, the term e-government is known for using information technology in government activities. Today every country needs to deal with ongoing changes in digital technology and focus on the right set of technology for the governments' progress in delivering the public services (Mittal, 2020). There are various kinds of information systems used according to each government agency's needs. To implement information system, the government has spent an extensive amount of funds. It is hoped that by using the system, the data and services will be better for the community, provide complete and accurate data, and fast and satisfying services. The significant investment of procurement and implementation of government programs requires that an assessment be carried out after the performance. Evaluation is a way to evaluate the system to determine if the system is running as expected and useful for users. Evaluation has three main objectives, namely: 1) to assess the extent and accessibility of the functions of the system, 2) to assess the user experience to interact with the system, and 3) to identify specific problems with the system (Hendarti et al., 2018). Sistem Informasi Administrasi Kependudukan (SIAK), an information system that utilizes information and communication technology to facilitate the management of population administration information at every level of the government administration area. The system has been used in Ende district since 2006, but there has never been an evaluation of its implementation despite some problems in giving service, especially the number of queues and the length of waiting time to process population documents. Therefore, it is considered necessary to measure the effectiveness and efficiency of implementing information systems to ensure its implementation. Information system evaluation is a process to explore and find out, about the sustainability of an information system implementation activities, both from the user's perception and organization perspective, also in terms of information systems technology (Puspita, Supriyantoro, & Hasyim, 2020). The information system evaluation is known as the Human Organization Technology-Fit (HOT-Fit) model. Yusof, Paul, and Stergioulas (2006) suggest this model assign important components in information systems which are humans, organization, technology, as well their relationships' suitability. Various studies have been carried out by implementing the HOT-Fit model. Following the initial idea of making this model evaluate information systems in the health sector, more research using the HOT-Fit model is carried out in the health sector. Kodarisman and Nugroho (2013) state that over time, studies with this model began to be carried out in another type of organization, such as the government. HOT-Fit model is advantageous in evaluating an information system because the aspects assessed are three main aspects in the evaluation process. However, to expand its implementation scope, the HOT-Fit model needs to be developed. One of the development directions is using the HOT-Fit model domains as the tools for measuring the effectiveness and efficiency of information systems. Measuring the effectiveness and efficiency of SIAK is essential because the result can show whether the implementation of SIAK in Ende has met the objectives of using SIAK initiated by the government. Hamilton and Chervany (1981) state that there are two general views regarding the definition of system effectiveness. Firstly, a goal-centered theory states that the system's effectiveness is determined by comparing performance against goals. An example of a viewpoint centered on system effectiveness objectives is comparing real costs and benefits to budgeted expenses and benefits.
Secondly, according to the system resource view, system effectiveness is determined by achievement at the mutual stage, for example, standards for "good" practice. Conceptualization of effectiveness is about the resource's feasibility in specific task objectives. For example, communication and conflict between MIS and user personnel, user participation in system development, and user job satisfaction can demonstrate the point of a system in terms of human resources. In terms of technological resources, its quality or service level can show the effectiveness. As stated by Subriadi, Herdiyanti, and Ayundari (2015), product effectiveness is defined as the user's achievement's accuracy and completeness. Other literature describes it as the suitability of the output and the objectives set.
On the other hand, Subriadi et al. (2015) suggest efficiency as the minimum use of resources to achieve optimum results. In quality management (QM), efficiency refers to doing things right. For example, whatever is done, is done most appropriately, given the availability of resources (high efficiency). On the other hand, effectiveness refers to doing the right thing, namely selecting and focusing on producing the output on demand. The word efficiency implies that one thing is better than others.
In information systems, effectiveness and efficiency are often not defined explicitly but are implied in several criteria to fulfill information system objectives. Based on the above definition, information systems' effectiveness and efficiency can be said as information systems with various qualities to achieve information system goals that support organizational goals using appropriate resources.
HOT-Fit is an evaluation model developed to ensure effective implementation and positively impact implementing the Health Information (Yusof et al., 2006). HOT-Fit is built on the existing information system evaluation model, namely the Information System (IS) Success Model (DeLone & McLean, 2003) and the IT-Organization Fit Model by Scott Morton. IS Success Model is considered not to summarize the organizational factors, so it is shown in the IT-Organization Fit Model, as well as IS Success Model provide specific evaluation dimensions and measures that are not appropriate for IT-Organization Fit. Therefore, after knowing the strengths and limitations of the two models, HOT-Fit is developed to complement each other in presenting a comprehensive evaluation framework. The HOT-Fit model and the relationship between dimensions as seen in Figure 1.
The research aims to expand the scope of Figure 1 HOT-Fit Model the HOT-Fit model into a model for measuring the effectiveness and efficiency of information systems. This measurement model is then used in SIAK to determine the level of effectiveness and efficiency of its implementation.

II. METHODS
There are six stages in conducting the research: 1) literature study stage, 2) modeling and indicators classification stage, 3) mapping stage, 4) data collection and analysis stage, 5) scoring stage, and 6) measurement and categorization stage. In the literature study stage, the HOT-Fit model and its domain are studied, including its background and implementation over the years. This stage allows to learn the meaning of the information system's effectiveness and efficiency and what standards can be categorized into effectiveness and efficiency.
The modeling and indicators classification stage is to determine the HOT-Fit domains and indicators mapped to the effectiveness and efficiency categories. The research conceptual model is designed based on the HOT-Fit Model (Yusof et al., 2006), which is developed referring to the HOT variable from Kodarisman's research because of the suitability of the research object. The domains are the basis of research, namely: 1) human, 2) organization, 3) technology, 4) effectiveness, and 5) efficiency. The model is seen in Figure 2.
The first sub-domain, system use, is also described as affecting efficiency. Wibiksana (2019) state that the advantage of using information systems is to increase efficiency. Buana and Wirawati (2018) suggest that companies' use of accounting information systems helps workers perform tasks more efficiently.
Four indicators used to represent the system use subdomain, namely training, user, attitude, and levels of use.
Secondly, human-user satisfaction as stated in Yusof et al. (2006), is the overall evaluation of user experience using information systems and the potential impact of information systems. User satisfaction is declared to affect effectiveness. Munap et al. (2018) state that user satisfaction has been widely used to measure the effectiveness of information system. User satisfaction is related to perceived usefulness, therefore, personal characteristics affect user satisfaction with information systems (Yusof et al., 2006).
The third sub-domain, organization structure, is the organization's nature, hierarchy, and functional division (Yusof et al., 2006). Top management and management information systems are organizational structure elements. Therefore, organizational structure can be said to affect the effectiveness of information systems as well as efficiency. Cahyaningsih (2017) finds the efficiency indicators are described by sub-indicators related to organizational structure, including employee placement according to specialization, synchronization between fields, and bureaucratic mechanisms. The indicator structure for communication and staff support is based on the research object.
Fourth, technology-system quality of information systems is defined as the desired characteristics of an information system with information quality according to the desired characteristics of information products. According to Yusof et al. (2006), measuring system quality means to measure information processing systems. System quality reflects the effectiveness of information systems. The statement is supported by Pradipta, Setiawan, and Lusanjaya (2017), which states that system quality is one factor that can potentially affect the effectiveness of an information system. Likewise, Handoko and Dharmadiaksa (2017) state that the effectiveness of information is effected significant positive by the quality of information system.
Apart from being a factor affecting information Figure 2 The Proposed Model I n P r e s s systems' effectiveness, Buana and Wirawati (2018) suggest that the efficiency domain can measure information system quality. Based on previous research and conformity with the research object, the system quality indicators used in the research are ease of use, ease of learning, access to technical support, availability, response time, and security. The fifth sub-domain, technology-information quality, as suggested in Yusof et al. (2006), focuses on information systems' information. Weber in Pradipta et al. (2017) regards information quality as one factor that can potentially affect an information system's effectiveness. Information quality also determines efficiency, Wardhana and Astuti (2018) suggest that information systems are expected to provide accurate, fast, and precise information. In the end, this process can shorten time without reducing the quality of data and show efficient results in presenting information for all parties. Yusof et al. (2006) suggest, and Wirapraja (2020) finds that the information quality indicators used in this study are completeness, accuracy, and timeliness.
Technology-service quality, as stated by Yusof et al. (2006), that all the support of the system or technology service provider is the focus of subdomain service quality. In-Law Number 25 of 2009 concerning Public Services, there is a statement regarding increasing services' effectiveness, with cooperation between providers, that service quality is related to effectiveness. Vellayati and Dwihartanti (2018) state that service is directed at achieving work effectiveness and efficiency. The research will use quick responsiveness, empathy, and transparency indicators to assess service quality (Yusof et al., 2006). In brief, the HOT-Fit model elements proposed as indicators for measuring the effectiveness and efficiency of the population administration information system are summarized, as shown in Table 1.
The next sub-domain is effectiveness, which has various criteria to define the effectiveness of information systems. Hamilton and Chervany (1981) suggest the effectiveness perspective in three levels of objectives, which is: (1) information provided by information systems and support obtained from the benefits of information systems to users; (2) use of information systems and their effect on user processes and performance in the organization; (3) the effect of information systems on organizational performance.
Effectiveness is closely related to efforts to achieve goals, so the point of SIAK in question is following the objectives of SIAK in Government Regulation Number 37 of 2007, improving the service quality as well as the availability of data and information that is accurate, complete, up-to-date, and easily accessible. Effectiveness in the research is measured by giving a score to each indicator representing the category of effectiveness. The calculation results are percentages to categorize the current system's level of effectiveness.
Lastly, Hamilton and Chervany (1981) suggest efficiency through: 1) definition of the need for information systems; 2) consumption of resources needed to produce information systems; 3) production capability or resource capacity; 4) the level of investment in resources. In general, the word efficiency is associated with the proper use of resources to achieve maximum results. The resources in question can be in the form of time, cost, people, and others. Another literature said that the efficiency criteria are contact handling, resource utilization, and cost-efficiency. In the research, information systems' efficiency is related to time and resources. Measurement of efficiency is carried out by grouping indicators related to the appropriate use of time and resources. A score is given to each hand based on the informant's statement about the indicator's achievement. The results of the total score are categorized into efficiency levels.
Mapping stage is the third in the research. After defining the domains, sub-domains, and elements of the research, the HOT-Fit domain mapping is carried out on effectiveness and efficiency. The effectiveness criteria refer to the objectives of the SIAK: 1) increasing the quality of service and 2) availability of being accurate, complete, up-to-date, and easily accessible.
Meanwhile, the criteria for efficiency are time and resource. Based on the criteria, the grouping for the HOT-Fit model indicator mapping on effectiveness and efficiency is shown in Table 2. The mapping results show that information system implementation's   I n P r e s s card operator, and 4) birth certificate operator. The data are analyzed and interpreted the meaning of each informant's statement describing each indicator. The fifth is scoring stage, in which the results of the analysis that explain each indicator are given a score, from 1-4 according to their achievement, including statements with definite values. Formulas and scores for effectiveness are shown in Table 3, while efficiency is in Table 4.
The final stage is measuring and categorization. The total score obtained from the scoring stage is calculated using the standard Key Performance Indicator (KPI) formula: total score earned x 100% maximum score The calculation produces the value of effectiveness and efficiency value. Based on these values, the effectiveness and efficiency levels are categorized, referring to Table 7.

Average time to get technical support
The researcher's assessment, based on the results of interviews and observations. 0 > 1 year 1 > 1 month -1 year 2 1 week -1 month 3 1 day -1 week 4 < 1 day Quick Responsiveness average time for people to input documents until they get a response The researcher's assessment, based on the results of interviews and observations. 0 > 1 year 1 > 1 month -1 year 2 1 week -1 month 3 1 day -1 week 4 < 1 day technical support, and 19) quick responsiveness. Each system officer is competent and mastering the use of the system. The expert's using the system remembers their minimum tenure of a year with work frequency every day, five days a week. Furthermore, the users are satisfied with the current system. Feelings of satisfaction present an attitude of accepting SIAK to continue operating.
Regarding perceived usefulness, three benefits are felt directly by the informants, namely making it easier to manage population data, making work complete faster, and avoiding informants from making mistakes.
In terms of communication, the activities shown by the system users indicate that there is good communication, so the process of managing population documents is smooth and quickly completed. It means the communication is frequent, continuous, and daily communication.
The system is always available unless there is an internet network problem or hardware failure. Thus, based on observations, the system is available approximately 36 hours a week (5 working days).
The network admin's statement regarding the fulfillment of system security standards as required is supported by observations of no security problems, system, data, or system environment.
However, regarding completeness, the data contained in the system is not entirely complete. There are still unfilled fields due to limited data support, and the system has not been updated. Moreover, there are still errors in inputting data and the lack of supporting data on the old version of the system, which is why the data in the system is less accurate.
The timeliness promised to the community was fulfilled, following the document's time. Regarding empathy, the action taken by the officer shows the act of listening to complaints from the community who are experiencing problems.
Although it is quite transparent in providing services, there is still a category of service transparency that has not been fulfilled, namely complaint service. Most system users have a background following their duties as system officers, but not all system officer positions are available at the research object. Hence, the level of achievement of this indicator is only 30%.
Regarding training, from the interview, information is obtained that the SIAK officers had attended training to support their duties as system officers. This indicator's level of achievement is not optimal (30%) because the number of system officers in Ende district population and civil registration office is not by the rules due to a lack of human resources.
Information from interviews and observations indicates that there is support from staff for system officers, by helping each other, working together to complete tasks. The found system is easy to use, with menus and navigation easy to understand and use. Besides, the system is easy to learn, especially for new users. Users can quickly complete tasks using the system. It appears that the system has a menu that is easy to understand, as well as straightforward navigation, and there is a warning system, making it easier for new users to learn.
The system appears to have a fast response time immediately after a user's request. Once clicked, there is a response from the system. The period needed to get technical support, mostly when a technical problem occurs, is 1-7 days.
Based on the interviews conducted, the two informants who serve as operators state that the time needed is between 1 and 2 weeks. Through observation, it is known that the time required to retrieve the queue number to enter files is a maximum of half an hour so that in a day, officers can receive more than 100 files for processing. Furthermore, the community is promised to take the documents at least four days after submitting the files. Therefore, the average quick responsiveness indicator is around 1-7 days.
In the research, the HOT-Fit model is developed to measure the effectiveness and efficiency of SIAK. Therefore, after obtaining the informants' assessment of each indicator of effectiveness and efficiency, the measurement process is carried out according to the effectiveness and efficiency measurement tools. Measurement of effectiveness uses effectiveness indicators, as presented in Table 5. Simultaneously, the measure of efficiency is provided in Table 6. The categorization of effectiveness and efficiency is shown in Table 5. Based on Table 5, the total domain value on the effectiveness measurement tool is 44, so the calculation of the SIAK effectiveness score using the standard KPI formula is: The effectiveness of SIAK = 34 x 100% 44 = 77,27% SIAK's effectiveness value of 77,27%, based on the percentage range of categories in Table 7, grouped into the very good category. This shows that the implementation of SIAK, from the SIAK management side, has met the objectives of population data collection, namely improving the quality of services and the availability of being accurate, complete, up-todate, and easily accessible data.  Based on Table 6, the total domain value on the efficiency measurement device is 32, so the calculation of the SIAK efficiency score using the standard KPI formula is: SIAK's Efficiency = 22 x 100% 32 = 68,75% The SIAK efficiency score is 68,75%, in the range of categories classified in the very good category. It shows that from the SIAK management side, the implementation of SIAK in Ende District has an efficiency impact in terms of time and resources.
Based on the stages and results of the research, the initial HOT-Fit model (Yusof et al., 2006) is expanded into a measurement model with each element of human, organization, and technology as an indicator. A significantly good results indicate services that have been effective and efficient. This is in line with previous research which finds that the effectiveness and efficiency of information systems in government improve the quality of government services (Durachman et al., 2020).

IV. CONCLUSIONS
The effectiveness and efficiency of SIAK can be a measure of the success rate of implementing SIAK in a region. Based on the research that has been done, several conclusions have been drawn. The HOT-Fit model's domains and elements have been successfully measuring the effectiveness and efficiency of SIAK implementation in the Ende District. There are 11 indicators used in measuring the effectiveness of the implementation of information systems, namely level of use, attitude, perceived usefulness, communication, availability, security, completeness, accuracy, timeliness, empathy, and transparency. Meanwhile, eight indicators to measure efficiency are users, training, staff support, ease of use, ease of learning, response time, access to technical support, and quick responsiveness.
According to the measurement results, the implementation of SIAK in Ende District reached good level for effectiveness (34 of 44), which means that the implementation of SIAK has realized the objectives of establishing this system well. Meanwhile, it is included in the very good category for efficiency (22 of 32). The result indicates that the implementation of SIAK has made the document processing process more efficient in terms of time and other resources.
The research provides a model for measuring the effectiveness and efficiency of the successful implementation of information systems, especially those used in the government service sector. Furthermore, the research expands the scope of the HOT-Fit model in evaluating information systems. Practically, the research is one of the references for evaluating services using SIAK in Ende Regency. Improving services to the community can be achieved through improving indicators that have not yet achieved maximum results.
The measurement model has only been tested on one research object in the government sector. In fact, with complete and comprehensive indicators both from the human side, as well as organization and technology, the model has the prospect of expanding its scope. Further research can implement this measurement model on research objects in other sectors. I n P r e s s