Evaluation of The Implementation of IBM IOC Using the Delone and Mclean Model

critical success factors. Six key variables that support information systems success are information quality, quality system, service quality, use, user satisfaction, and net benefits. This research uses quantitative research methods, and the data used were 80 respondents by using purposive sampling, assisted by using questionnaire data collection methods. Analysis of the data used is PLS-SEM using the SmartPLS application. The results showed a significant effect on the quality information on use, user satisfaction, and net benefits. There is no significant effect on the quality system to use, user satisfaction, and net benefits. Quality of service is significant for use and net benefits, but there is no significant effect on user satisfaction.


INTRODUCTION
Data processing through information systems has aimed to produce information that is useful for its users. Not all data and information can use because data and information have a standard that has been predetermined, as stated in Perpres Number 39 of 2019 on Indonesian Data One. So, if the standard cannot fulfill, then the data or information cannot be said to be unusable. With these standards, the data and information collected become quality and valid data and information. A collection of data and information stored in a computer device is systematically converted into a data format in the database. The database is an essential part of information systems because it is the basis for providing information. The database aims to help users and organizations obtain the data needed quickly, smoothly, and efficiently. However, to help users and organizations analyze data and information, not only collect data in the database, but the integration of each database is needed.
Data integration is the process of interconnecting data from various database sources. Data integration aims to facilitate the process of analyzing data, sharing information between organizational divisions, and avoiding data duplication. Duplication of data and information can cause problems in decision making and policy. In government, there is often duplication of data from each agency, because each agency has its data standards. So, when the data is collected, it overlaps and complicates the analysis process. Therefore, we need a centralized data integration platform to collect and process data in one place. The platform can connect all data from each agency and carry out an analysis process so that it can produce output as a basis for decision making and policy.
Batu city government cooperates with a third party, namely IBM (International Business Machines Corporation), a United States company that manufactures and sells computer hardware and software, to use its platform, the IBM IOC (Intelligent Operations Center). IBM IOC offers its capabilities in integrated data processing, monitoring, and managing every vital service in the city. Not only Batu City uses this platform, but the city of Bandung and Malang regency have also implemented it, to help in creating a smart city. Kota Batu has completed the Smart city master plan on the Movement Towards 100 Smart city in 2019. The Smart City program in Kota Batu has run from the beginning of September 2017. By recruiting a successful screening of 80 Smart city freelancers, as experts responsible for it. It decides into two fields 32 experts in the field of information technology who served in the Command Center as an information system control center under the auspices of the Office of Communication and Information, and 48 experts in the field of agriculture as a companion to farmers on agricultural land under the auspices of the Department of Agriculture. Currently, a Smart city in Batu has run for two and a half years, and no evaluation of the information system has been implemented. Thus, it is necessary to evaluate an information system that can assess an action or the implementation of an ongoing program so that it can do the develop of information systems and services, which expected to improve the government's performance in helping people in Batu City.

Research Design
The research design can be interpreted as a specific framework designed or designed by researchers who describe the research process plan. The research design used in this study is quantitative by collecting primary and secondary data to measure the research variables. The data collection method uses the questionnaire method by distributing questions that have been compiled by researchers to research respondents. Analysis of the data used in this study was PLS-SEM using the SmartPLS application.

Scope
The scope of this research is on IBM IOC information system users in the Batu City Government, especially at the Communication and Information Office as the person in charge of managing the information system.

Research Location
The location of the study conducted in the Batu City Government. Researchers chose this location because they had worked as Smart city freelancers at the Communication and Information Service, and IBM IOC information system management officer. The latter had not evaluated the system for two years of use.

Research Variable
The research variable is an attribute or nature or value of people, objects, or activities with individual variations determined by researchers to be studied and drawn conclusions (Sugiyono, 2013). In this study, researchers used six variables in the success model of Delone and Mclean (2003) to measure the successful implementation of IBM IOC information systems. Then from these variables sought indicators as benchmarks in a study. According to Sugiyono (2013), the classification of research variables on the interdependence of variables, namely:

Independent Variable
The independent variable or the independent variable is the variable that influences or is the cause of the change or the emergence of the dependent variable (Sugiyono, 2011).

Intervening Variable
Intervening variables theoretically affect the relationship between the independent and dependent variables into an indirect relationship that cannot be observed and measured. This variable is the interrupting variable between the independent and dependent variables, so the independent variable does not directly affect the change or the emergence of the dependent variable (Sugiyono, 2011).

Dependent Variable
The dependent variable is the variable affected or the result because of the independent variable (Sugiyono, 2011).

Types and Source of Data
According to Sugiyono (2015), the types of data are distinguished by their nature into two, namely qualitative and quantitative. In this study, researchers used quantitative research because of the data obtained in the form of numbers or scores. Then the number will be analyzed, which produces the research conclusions.

Sampling Technique
The sampling technique researchers used non-probability sampling with a purposive sampling technique. The technique to determine the research sample with specific considerations aimed at making the data obtained later more representative (Sugiyono, 2010). Responsible for uploading data to the information system database 2

Collecting Data Technique
In this study, researchers used a method of collecting data through a questionnaire using the help of the google form application. Questionnaire links distributed to predetermined research subjects in measuring the attitudes and opinions of research subjects used a research scale using the Likert scale.

Data Analysis
In this study, the data analysis technique used is the SEM (Structural Equation Modeling) method with quantitative analysis. According to Latan (2013), SEM is a second-generation multivariate analysis technique that combines factor and path analysis to enable researchers to test and estimate the relationship between multiple exogenous and endogenous simultaneously with many indicators. In this study, the sample used is less than 100 samples. The researchers used the Partial Least Square (PLS) method to conduct SEM testing assisted with SmartPLS software. Evaluation of the measurement model aims to find the validity and reliability of indicators. Conducting structural model evaluation aims to see the significance of the relationship between latent variables by looking at the path coefficient, which shows the relationship between latent variables in the research model.

Hypothesis Testing and Interpretation
Hypothesis testing performed by the bootstrapping sampling method developed by Geisser and Stone. Testing calculated by t-test; if a p-value ≤ 0,05 or Cronbach alpha 5% obtained, it concluded significantly.

RESULTS
The questionnaire data collected through the Google Forms link-sharing method, which distributes to the research sample via social networks, namely Whatsapp and Instagram. The questionnaire distributes to all THL Smart cities through personal contact and social networking groups. Respondents receive the shared link and fill out a questionnaire. The completed and collected questionnaires are tabulated through the Microsoft Excel application and saved with a file with a CSV extension to imports into the SmartPLS application. The data is then processed using the SmartPLS application, starting with importing the CSV questionnaire data and then drawing the research model. The research model gives an indicator that already contains CSV data by dragging and dropping it on each variable. The calculation of the PLS Algorithm, Bootstrapping, and Blindfolding calculates the data description used by the researcher. The method of describing the research results used by researchers includes the distribution of answers, structural model testing, measurement tests, path tests, and hypothesis testing.

Distribution of answers to research indicators
The data collected is classified based on the total score of the respondents' answers from each variable. Calculation of the average score of the answers can be categorized as follows:

Outer Model Test
The researcher's reflective indicator model has five stages, namely the discriminant validity, composite reliability, AVE, and Cronbach alpha. The three weighting factor calculations, average variance extracted, and discriminant validity categorizes in the validity test, and composite reliability and Cronbach alpha categorize in the reliability test. The calculation in the SmartPLS application uses the PLS Algorithm calculation.

Validity Test
Convergent validity testing is through testing the loading factor and AVE value while testing the discretionary validity through discriminant validity testing.

Outer loadings
Outer loadings value > 0,7 in the initial research step, if the value of the outer loadings of 0,50-0,60 is obtaining, it is still acceptable (Ghozali, 2014). The test results for loading factors through the SmartPLS application can see the following figure: All research indicators provide a value of > 0,5, and it can say that the use of indicators is valid. If the indicator value shows < 0,7, it cannot use in further research.

Average variance extracted
AVE value indicates the magnitude of the indicator variance contained by the construct; the value must indicate ≥ 0,5. Testing between constructs has in stock or different from other constructs. HTMT value < 0,9, then discriminant validity has been established between the two reflective constructions. Values > 0,9 indicate a lack of discriminant validity. The conclusions given according to the results of the above data calculation are:

Reliability Test
Reliability testing in the SmartPLS application uses two methods of calculating Composite reliability and Cronbach Alpha.

Composite reliability and Cronbach alpha
Then composite reliability testing is used to measure internal consistency, and the value must be > 0,6. The results of the calculation of composite reliability show a value of > 0,7 so that it shows all constructs are said to be reliable. The Cronbach alpha calculation results show a value of > 0,6, but the Y1 and Z variables show a value of ≥ 0,6. According to Hair (2014), the value of Cronbach alpha > 0,4-0,6 can categorize as quite reliable.

Inner Model Test
Structural model tests performed to see the significance of the relationship between latent variables by looking at the path coefficients that show the relationship between latent variables in the research model. Calculations on the SmartPLS application use PLS Algorithm, Bootstrapping, and Blindfolding calculations. R square results of 0,75, 0,50, and 0,25 for endogenous latent variables in the structural model indicate that the model is "good", "moderate", and "weak". The conclusions given according to the results of data processing above are:   The calculation of Q2 has a value> 0, indicating that the model has predictive relevance, or it can be said to predict its endogenous variables. If the calculation of values obtains 0,02, 0,15, and 0,35, the category of results obtained is "small," "medium," and "large." Calculations using reflective indicators on endogenous constructs. The conclusions given according to the calculation results above are: Test the hypothesis of the direct effect of an influential variable on the variable affected. The estimated value for the path relationship in the structural model must be significant. This significance value can obtain by bootstrapping procedure. If the P-Values value <is 0,05, then it is declared significant, while the P-Values > 0,05 is not significant. The conclusions given according to the above calculation results are:  Test the hypothesis of the indirect effect of an exogenous variable on an endogenous variable mediated by an intervening variable. If the P-Values value < 0,05 is declared significant, the intervening variable mediates the effect of an exogenous variable on an endogenous variable or has an indirect effect. If P-Values > 0,05, then declared not significant, intervening variables do not mediate the effect of an exogenous variable on an endogenous variable or have a direct effect. The conclusions given according to the above data calculations are:  The total effect is the effect of direct effects and indirect effects, namely the sum of the direct and indirect effects of an exogenous latent variable on endogenous latent variables. The p-value required of ≤ 0,05. The conclusions given according to the above data calculations are:

T-Statistics
Hypothesis testing calculates through the SmartPLS application by looking for t-statistical coefficients. The influence of indicators with values > 1,96 can be agreed to be significant.
Indicators can also answer a significant p-value of < 0,05. 1) The effect of information quality on use.

Original Sample T-Statistics P-Values
It knows the impact from X1 to Y1 has a path coefficient of 0,279; the t-statistic result of 2.075 is > 1,96 with a p-value of 0,041, which is < 0,05. The calculated value indicates that the information quality variable has a positive significance to the use variable. Hypothesis H1, the variable information quality impacts the use variable is accepted.
2) The effect of information quality on user satisfaction. It knows the impact of X1 on Y2 has a path coefficient of 0,360; the t-statistic result of 2.061 is > 1,96 with a p-value of 0,026, which is < 0,05. The calculated value indicates that the information quality variable has a positive significance to the user satisfaction variable. Hypothesis H2, the variable information quality, has an impact on the user satisfaction variable is accepted.
3) The effect of system quality on use.
It knows the impact of X2 on Y1 has a path coefficient of 0,277, a t-statistic result of 2.451, which is> 1,96, the p-value of 0,013 < 0,05. The Evaluation of The Implementation of IBM IOC Using the Delone and Mclean Model calculated value indicates that the system quality variable has a positive significance to the use variable. Hypothesis H3, the system quality variable, has an impact on the accept use variable. 4) The effect of system quality on user satisfaction. It knows that the impact of X2 on Y2 has a path coefficient of 0,010; the result of the tstatistic is 0,086, which is < 1,96 with a p-value of 0,932, namely > 0,05; this hypothesis cannot accept because the results of t-statistics and pvalues do not meet test terms. This calculation shows that the system quality variable does not have any significance to the user satisfaction variable. Hypothesis H4, variable system quality impacts the variable user satisfaction reject. 5) The effect of service quality on use.
It knows the impact of X3 on Y1 has a path coefficient of 0,254, a t-statistic result of 2.430 is > 1,96 with a p-value of 0,017, which is < 0,05. The calculated value indicates that the service quality variable has a positive significance to the use variable. Hypothesis H5, variable service quality impacts the variable use is accepted. 6) The effect of service quality on user satisfaction. It knows that the impact of X3 on Y2 has a path coefficient of 0,080, the result of the tstatistic is 0,711, which is < 1,96 with a p-value of 0,479, namely > 0,05; this hypothesis cannot accept because the results of t-statistics and pvalues do not meet test terms. The calculated value shows that the service quality variable does not significantly impact the user satisfaction variable. Hypothesis H6, variable service quality impacts the user satisfaction variable reject. 7) The effect of use on net benefits.
It knows that the impact of Y1 on Z has a path coefficient of 0,285; the t-statistic result of 2.824 is > 1,96, with a p-value of 0,006, which is < 0,05. The calculated value indicates that variable use has a positive significance for the net benefit variable. Hypothesis H7, variable use has an impact on the variable net benefits received. 8) The effect of user satisfaction on net benefits.
It knows that the impact of Y2 on Z has a path coefficient of -0,306, the result of the t-statistic is 2.694, namely > 1,96 with a p-value of 0,009, which is < 0,05. The calculated value shows that the user satisfaction variable has a significance to the net benefit variable. Hypothesis H8, the user satisfaction variable, has an impact on the net benefits variable received. 9) The effect of information quality, system quality, and service quality on net benefits. It knows that the impact of X1 (information quality) on Z (net benefit) has a t-statistic result of 0,200 with a p-value of 0,842, which is declared insignificant. The impact of X2 (system quality) on Z (net benefit) has a t-statistic of 2,290 with a p-value of 0,025, which is significant. The impact of X3 (service quality) on Z (net benefit) has a t-statistic result of -0,338 with a p-value of 0,008, which is significant with a negative value. Hypothesis H9, there is an impact of information quality on net benefits is rejected. There is an impact on system quality and service quality on the net benefits received.

Information quality has an impact on use
It is often the neglected aspect of research on the success of information systems. Information quality has the main purpose of information systems is to provide accuracy, timeliness, and relevance. The quality of information is an important dimension in the success of a system (Delone, 2016). Previous research has proven that the quality of information has an impact on the use of Delone (2016), which is strengthened by Dewantoro's research (2019) showing that information quality has a positive impact on usage. This study proves interactions impact of information quality on the use of the IBM IOC application users. 2. Information quality has an impact on user satisfaction Information quality is an important factor that contributes to user satisfaction (Delone, 2016). The impact of information quality on System quality is still a key variable in measuring information system performance (Delone, 2016). Previous research examining the impact of system quality on users had a positive impact on Dewantoro's research (2019). This study proves interactions impact of system quality interaction on use of the IBM IOC application. 4. System quality has an impact on user satisfaction System quality is still a key variable in measuring the success of information systems (Delone, 2016). Previous research tested the impact of the involvement of system quality on user satisfaction as having a positive impact explained in the research of Lestari (2019), Hidayatullah (2020), but there was no impact of system quality on user satisfaction shown in Panjaitan's research (2019). This study proves interactions impact of system quality involvement on user satisfaction of users of the IBM IOC application.

Service quality has an impact on use
The failure of service quality in measuring the success of information systems can lead to unfavorable results because net benefits that are not very good describe poor service quality (Delone, 2016). Previous research tested the impact of service quality involvement on usage had a positive impact explained in Dewantoro's research (2019).
The test results state that there is an impact of service quality involvement on the usage of the IBM IOC application users. 6. Service quality has an impact on user satisfaction Service quality measures the contribution of information technology organizations that are responsible for or create information systems (Delone, 2016). Previous research tested the impact of service quality involvement on user satisfaction which had a positive impact explained in research by Irfan (2019), Dewantoro (2019), Hidayatullah (2020), but in Lestari's research (2019) the involvement of service quality on user satisfaction had no significant impact. This study proves that there is no interaction impact on service quality involvement on user satisfaction of users of the IBM IOC application.

Use has an impact on net benefits
Use is the main focus of measuring the success of information systems (Delone, 2016 a) The results of the calculation of indirect effects explain that there is no impact on the involvement of system quality, service quality, and information quality on net benefits through use. b) The results of the calculation of indirect effects explain that there is no impact on the involvement of system quality, service quality, and information quality on net benefits through user satisfaction. c) The impact of system quality, service quality, and information quality on net benefits through calculating total effects proves that there is no impact of system quality involvement on net benefits, however, as in Delone's (2016) research, information quality, and service quality prove that there is an impact of involvement on net benefits.

CONCLUSION
Based on the results of the IBM IOC evaluation research through the Delone and McLean information system success model, the following conclusions are: 1) The information quality variable explains the accuracy and relevance indicators indicate the high category, while the timeliness shows the deficient category. The accuracy indicator describes accurate, precise, and complete data information, getting good responses from the IBM IOC information system users. The relevance indicator explains the navigation menu and required data, getting good responses from the IBM IOC information system users. The timeliness indicator describes up-to-date data and access speed, getting an inadequate response from the IBM IOC information system users. 2) The system quality variable explains that the security and flexibility indicators show the very high category, while the reliability indicator shows the high category. The security indicator describes the security when using and data security, getting good responses from users of the IBM IOC information system. The flexibility indicator describes access from various devices and can be accessed from anywhere, getting a good response from users of the IBM IOC information system. Reliability indicators describe data that can be accounted for, run well when accessed simultaneously, provide information when an error occurs, and get good responses from users of the IBM IOC information system.
3) The quality variable explains the responsiveness and empathy indicators showing a very high category. The responsiveness indicator explains the officer's response when an error occurs and the officer's quick reaction, getting a good response from users of the IBM IOC information system. The empathy indicator explains complaint handling and friendly handling, getting good responses from the IBM IOC information system users. 4) The use variable explains the frequency of use indicator shows the high category, while the use indicator's nature shows the medium category. The frequency of use indicates that it runs well even though it is often used and wants to reuse, getting good responses from the IBM IOC information system users. The nature of use indicator explains the ease of use and convenience when using, getting a reasonable response from users of the IBM IOC information system. 5) The user satisfaction variable explains the satisfaction indicator shows the moderate category. The satisfaction indicator describes satisfaction when using and working as expected, getting a reasonable response from the IBM IOC information system users. 6) The net benefit variable explains the learning indicator shows a deficient category, while the productivity and task performance indicators show the high category. Learning indicators explain user guides' availability and help improve user capabilities, getting an inadequate response from users of the IBM IOC information system. The productivity indicator explains that it helps complete tasks effectively and increase productivity, getting good responses from users of the IBM IOC information system. The task performance indicator explains that it helps to complete tasks faster and reduce errors in providing reports and getting good responses from the IBM IOC information system users.

LIMITATION
The PLS-SEM testing in this study had previous researchers conveyed several important points that needed to be considered to investigate information: 1. Relevance and learning indicators cannot be used in further research, because the results obtained in calculating outer loading are <0.7. 2. The timeliness indicator gets a poor response, so it needs to be paid attention again regarding data updates and access speed of IBM IOC information systems. 3. Learning indicators get a good response, so it needs to be considered again related to providing the use and assistance of the IBM IOC information system. In Delone and McLean's 2016 research, there are recommendations for measuring indicators of each variable, which can be used in future research in measuring the success of an information system.