15
- International Refereed Social Sciences Journal E-ISSN: 2229-4686 ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue4, October 2019 [43] DOI : 10.18843/rwjasc/v10i4/06 DOI URL : http://dx.doi.org/10.18843/rwjasc/v10i4/06 System Usability Scale for Usability Testing of Agriculture E-Commerce Website Joko Purwadi, Department of Informatics, Universitas Kristen Duta Wacana, Yogyakarta, Indonesia. Rosa Delima, Department of Informatics, Universitas Kristen Duta Wacana, Yogyakarta, Indonesia. Novia Indrayono Rante Toding, Department of Informatics, Universitas Kristen Duta Wacana, Yogyakarta, Indonesia. Argo Wibowo, Department of Information System, Universitas Kristen Duta Wacana, Yogyakarta, Indonesia. Halim Budi Santoso, Department of Information System, Universitas Kristen Duta Wacana, Yogyakarta, Indonesia. (Received September 21, 2019; Accepted October 25, 2019) ABSTRACT A software development project should pass many stages starting from defining needs to implementation. Before the process of system implementation, the development team will conduct a usability test and a user acceptance test for the application. This stage is very important to ensure the implementation process can be done well by minimizing problems when users interact with the system. In this study, usability and user acceptance tests will be carried out on the sale and purchase application of agricultural products (agricultural e - commerce). The method used in the study consists of 6 main stages, such as preliminary studies, participant studies, tool development, test implementation, data analysis, and result formulation. The tools to be used are such as a questionnaire for participants’ data collection, a System Usability Scale (SUS) questionnaire to measure the application acceptance and task scenarios to measure system usability. This article specifically discusses the design of task scenarios and the data taking process on the usability test on an agricultural e-commerce system. Keywords: Agriculture E-Commerce; Usability Test, User Acceptance, System Usability Scale; SUS. INTRODUCTION: The field of agriculture in Indonesia continues to experience growth both in terms of agricultural technology and information technology that supports agriculture. Various systems have been developed to support increased productivity and expansion of agricultural product markets both in Indonesia and in the world. Some applications that have been developed in Asian and African countries are e-Choupal (India), Agricultural Market Information System (Bangladesh), and Infotrade (Uganda) (Delima, Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia several information applications have been developed in agriculture (Delima & Purwadi, Analisis Situs Web Pertanian Berbahasa Indonesia, 2015) . One of the

System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

  • Upload
    others

  • View
    7

  • Download
    0

Embed Size (px)

Citation preview

Page 1: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [43]

DOI : 10.18843/rwjasc/v10i4/06

DOI URL : http://dx.doi.org/10.18843/rwjasc/v10i4/06

System Usability Scale for Usability Testing of

Agriculture E-Commerce Website

Joko Purwadi,

Department of Informatics,

Universitas Kristen Duta Wacana, Yogyakarta, Indonesia.

Rosa Delima,

Department of Informatics,

Universitas Kristen Duta Wacana,

Yogyakarta, Indonesia.

Novia Indrayono Rante Toding,

Department of Informatics,

Universitas Kristen Duta Wacana,

Yogyakarta, Indonesia.

Argo Wibowo,

Department of Information System,

Universitas Kristen Duta Wacana,

Yogyakarta, Indonesia.

Halim Budi Santoso,

Department of Information System,

Universitas Kristen Duta Wacana,

Yogyakarta, Indonesia.

(Received September 21, 2019; Accepted October 25, 2019)

ABSTRACT

A software development project should pass many stages starting from defining needs to

implementation. Before the process of system implementation, the development team will

conduct a usability test and a user acceptance test for the application. This stage is very

important to ensure the implementation process can be done well by minimizing problems

when users interact with the system. In this study, usability and user acceptance tests will be

carried out on the sale and purchase application of agricultural products (agricultural e-

commerce). The method used in the study consists of 6 main stages, such as preliminary

studies, participant studies, tool development, test implementation, data analysis, and result

formulation. The tools to be used are such as a questionnaire for participants’ data

collection, a System Usability Scale (SUS) questionnaire to measure the application

acceptance and task scenarios to measure system usability. This article specifically

discusses the design of task scenarios and the data taking process on the usability test on an

agricultural e-commerce system.

Keywords: Agriculture E-Commerce; Usability Test, User Acceptance, System Usability

Scale; SUS.

INTRODUCTION:

The field of agriculture in Indonesia continues to experience growth both in terms of agricultural

technology and information technology that supports agriculture. Various systems have been developed

to support increased productivity and expansion of agricultural product markets both in Indonesia and in

the world. Some applications that have been developed in Asian and African countries are e-Choupal

(India), Agricultural Market Information System (Bangladesh), and Infotrade (Uganda) (Delima,

Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan

Afrika, 2016). Meanwhile, in Indonesia several information applications have been developed in

agriculture (Delima & Purwadi, Analisis Situs Web Pertanian Berbahasa Indonesia, 2015). One of the

Page 2: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [44]

agricultural applications that has been developed is a system for sales and puschasing agricultural product

(Delima, Santoso, Andriyanto, & Wibowo, 2018).

The Agricultural Product Sales and Purchasing System is a system developed to support economic

development and market expansion, particularly for farmers and people in rural areas. The system was

completed in 2018. Before the system is introduced to the users, one step is needed, such as testing the

user acceptance of the application. A test aimed at determining the level of user acceptance e-commerce

application. This test is very important to minimize failures in the system implementation process. The

test is carried out through the SUS questionnaire.

The paper consists of five parts. Part I is an introduction, literature review can be seen in Part II, while

the methodology and design of test equipment can be seen in Parts III and IV, and Part V is a conclusion

and future work.

LITERATURE REVIEW

System Usability Scale (SUS):

System Usability Scale (SUS) is a questionnaire developed by John Brooke in 1986 that was used to

evaluate the usability level of Blackboard (Brooke, 1996). The SUS questionnaire consists of 10

questions, with odd sequences for positive items and even sequences for negative items. Each question is

graded on a 5 scales from “Strongly Disagree” to “Strongly Agree”. The ten list of questions on SUS can

be seen in Table 1.

Table 1: Question in SUS (Brooke, 1996)

No Questions

1. I Think that I would like to use this system frequently.

2. I found the system unnecessarily complex.

3. I thought the system was easy to use.

4. I think that I would need the support of a technical person to be able to use this system.

5. I found the various functions in this system were well integrated.

6. I thought there was too much inconsistency in this system.

7. I would imagine that most people would learn to use this system very quickly.

8. I found the system very cumbersome to use.

9. I felt very confident using the system.

10. I needed to learn a lot of things before I could get going with this system.

To measure the application’s level of acceptance using the SUS questionnaire, we can use a measurement

developed by (Bangor, Kortum, & Miller, 2009) as shown in Table 2. Through the value of SUS, the level

of application’s acceptance is evaluated from the reusability aspect.

Table 2: Application Acceptance Rates on the SUS Questionnaire (Bangor, Kortum, & Miller, 2009)

Nilai SUS Interpretation

< 50 Not Acceptable

50 - 70 Marginal

> 70 Acceptable

SUS questionnaire has been widely used in the process of system reusability testing. Some studies using

the SUS questionnaire are (Puspita, Sibarani, Muslim, & Moch., 2016) (Baharrudin, Wardani, &

Herlambang, 2018) (Devy, Wibirama, & Santosa, 2017) (Devy, Wibirama, & Santosa, 2017), and (Al-

Omar, 2018).

Page 3: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [45]

E-Commerce:

In today’s global era, E-commerce is a promising new business model. This business model utilizes

information and communication technology (ICT) support. According to Turban et al (Turban, King,

Lee, Liang, & Turban, 2015), electronic commerce (abbreviated as EC, or E-commerce) is a trading

model that includes the process of buying, selling, transferring, and exchanging products, services or

information through computer networks, including the internet.

E-commerce business is one of the most important parts of the economy in Indonesia. According to

(Gentur, 2014) “The Ministry of Trade of the Republic of Indonesia estimated that the sale and purchase

of goods transactions via the internet (E-commerce) from Indonesia will penetrate the number of US $

10,08 billion. The average value of online shopping transactions is growing 40 percent every year”.

Further, according to Agus Tjandra in the News (Berita, 2014) as vice chairman & foreign relations of

the Indonesian E-commerce Association (idEA), on the signing of E-commerce cooperation between the

Indonesian Information Technology Federation (FTII) and the Taipei Computer Association (TCA) in

Jakarta stated that Indonesia’s total E-commerce transaction value in 2013 reached US $ 8 billion or

around Rp 94 trillion and is predicted to continue to increase to reach US$ 24 billion or around Rp 283

trillion in 2016. The number of people who shop online in the country is recorded at 4,6 million people

during 2013 and is expected to increase to 8,7 million in 2016.

E-commerce also provides many benefits for both entrepreneurs and customers. Some of the

benefits of e-commerce according to (Khan, 2016) are such as increasing entrepreneur income,

reducing company’s operating costs, saving time to do shopping for consumers, and having a

certain level of convenience in shopping.

Dodolantani: An Agriculture E-Commerce:

Agricultural Product Sales and Purchasing System is a system developed to support economic

development and market expansion for farmers in particular and people in rural areas (Delima, Santoso,

Andriyanto, & Wibowo, 2018).

Sales and Purchases are two processes of trading goods and services business. At present the business

model of trading in Information and Communication Technology (ICT) is known as e-Commerce. E-

Commerce is a trading model including the sale, purchase and exchange of goods and services through

computer networks and the internet. E-Commerce has several advantages for both sellers and buyers,

such as e-commerce can reduce operational costs and buyers can save time in shopping.

Dodolantani agricultural e-commerce has 2 main modules namely the purchase module and sales

module. The purchase module functions to manage the product’s purchasing process, this module

consists of the user’s login page (Figure 1). There are 2 types of logins, such as login as a buy er

and login as an admin.To be able to login as buyer, can be done by selecting the register button on

the right side of the login button. After log in, the buyer will see the product page (Figure 2). This

page shows the latest and most popular products so ld by the seller. Through the product page,

buyers can see, search and discuss the products they want to buy. After being sure to make a

purchase transaction, the user can continue to make purchases through the buy button on the

product details (Figure 3.). Then the buyer can make a purchase transaction (Figure 4). Purchase

transactions include determining the purchase amount, purchase address, and shipping expedition .

After making a product purchase, the buyer will then be on the shopping cart page (Figure 5) and

the purchase can be proceeded to the payment process. After making payment the buyer must

upload the proof of payment. After the buyer makes a payment, the system admin must verify the

payment made by the user. Verification can be done via the validation page (Figure 6). If the

payment does not match, the system admin can send a notification to the buyer regarding the

payment incompatibility. If the payment is correct and validated, the process continues with

sending goods via the specified courier. The purchase module sitemap can be seen in Figure 7

(Delima, Santoso, Andriyanto, & Wibowo, 2018).

Page 4: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [46]

Figure 1: User Login Page

Figure 2: Product Page

Page 5: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [47]

Figure 3: Detail Product Page

Figure 4: Shopping Page

Page 6: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [48]

Figure 5: Shopping Cart Page

Figure 6: Payment Validation Page

Page 7: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [49]

Figure 7: Site Map Purchase Module in Dodolantani

The sales module in dodolantani e-commerce functions to support the seller in conducting goods sales

transactions. The sales module of agriculture is divided into 2 menu features such as the seller display

which consists of the login page, the seller main page, product listing page, product added form page,

product offer form page, product listing page, product price check page, sales page, inbox page, seller

report page, profile page. Next, there is admin display which consists of admin main page, product

category list page, product list page, product offer list page, seller list page, seller validation page, seller

data page, product validation page, transaction data page, payment validation page and admin report

page. While on the product listing page, the details of product offerings, sales transactions, product

discussions, and each sale reports can be seen in Figure 8 until Figure 12 and Site map of the sales

module can be seen in Figure 13 (Delima, Santoso, Aditya, Purwadi, & Wibowo, 2018).

Figure 8: Product List Page

Purchase

Module

Home RegistrationProduct

List

Equipment

Substance

Agriculture Product

Product Checking

Purchasing

Purchase Data Entry

Product Discussion

Product Comment

Entry

Payment

Upload Payment Receipt

Report

Sales Transection

Sales Report

Profile

Page 8: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [50]

Figure 9: Detail Sales Product Page

Figure 10: List of Sales Transactions Page

Page 9: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [51]

Figure 11: Product Discussion Page

Figure 12: Sales Report Page.

Page 10: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [52]

Figure 13: Site Map Sales Module.

RESEARCH METHODOLOGY:

This research will apply a testing method which consists of six stages, namely a preliminary study,

participant/respondent study, test tool development, implementation, data analysis, and result

formulation. The order of the stages of the research can be seen in Figure 14.

Figure 14: Research Methodology Scheme.

The Preliminary Study is carried out to make plan, identify needs, and the availability of research

resources. The test would be conducted in May to July 2019. Data collection will be carried out directly

with agricultural stakeholders including farmers, sellers of agricultural products, and farmer groups. The

data are collected in the Central Java and Yogyakarta area during May to July 2019. Researchers have

cooperation with Farmer Group in these region.

The targeted respondents consist of two groups of respondents such as respondents who are end users

such as buyers and sellers and respondents who are administrators of the system. The total number of

respondents is targeted at around 35 respondents. Criteria for buyer and seller respondents are a

minimum age of 20 years, a minimum of junior high school graduate, and able to operate a computer.

The questionnaire used to test user’s acceptance is the SUS questionnaire and to support the data retrieval

process, the Google form is used. To measure the application’s level of acceptance using the SUS

questionnaire, we use acceptance rates as shown in table II.

ANALYSIS AND DISCUSSION:

SUS Questionnaire: Questionnaire is one of the data collection tools in form of a series of questions or statements that would

be submitted to the respondents. The questionnaire is based on and refers to the SUS (System Usability

Scale) Questionnaire, which is a questionnaire consisting of 10 statements (Brooke, 1996) (Lewis &

Sauro, 2017).

Google Form for Support Usability Testing:

To support the process of data retrieval during usability test, Google form is used for data collection.

There are 2 forms developed such as a form to get the respondent’s characteristics and a form for SUS

questionnaire. A form to get the respondent’s characteristics consists of questions that include personal

identity (name, address, gender, age, education), farming experience, experience in using computer or

mobile phone, experience related to social media and internet, and experience in conducting online

transactions. Meanwhile the SUS questionnaire was made using a Likert scale from 1 to 5, with 1 for the

category of “strongly disagree” and 5 for “strongly agree”.

Sales

Module

Home RegistrationProduct

List

Equipment

Substance

Agriculture Product

Product Checking

Selling

Sales List Information

Order Checking

Status

Inbox

Product Discussion

Product

Review

Product Entry

Equipment

Substance

Agriculture

Product

Report

Sales

Transection

Sales

Report

Profit

Preliminary

Study

Perticipants

Study

Develop

Tools

Conduct

Testing

Data

AnalysisResults

Page 11: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [53]

Data Collection:

The test was conducted from May to July 2019, before conducting the test a brief interview was

conducted to explore the respondents’ experience in the use of laptops and smartphones. This process is

carried out to ensure that the respondents are qualified as required. If the respondent meets the next

requirements, then the researchers gives an explanation of the purpose of data retrieval and provide a

brief explanation related to the system to be tested. Respondents were asked to fill out the agreement

form. Then the respondents are given time to try to run the system independently. If the time is

considered as enough, then the test data would be taken to complete the task in accordance with the

scenario that had been prepared. After finishing it, the process continued with filling SUS questionnaire

to find out the level of users’ acceptance.

Participants Characteristics:

Respondents in this study are 35 persons, with a distribution of 30 respondents who were buyers or

sellers and five respondents were admin users. The respondents came from Bantul and Salatiga. The

characteristics of the respondents can be seen in Table 3.

Table 3: Participants Characteristics.

Variable Class Total Percentage

Age

20 – 30 years 9 26%

31 – 40 years 12 34%

41 – 50 years 12 34%

> 50 years 2 6%

Gender Male 16 46%

Female 19 54%

Education Level

Junior High School 5 14%

Senior High School 1 3%

Diploma 3 4 11%

Under Graduate/ Bachelor 13 37%

Post Graduate 1 3%

Based on the data in Table 3, it is known that the majority of respondents aged between 31 to 50 years

old reached 78%. Based on gender, respondents of the research were quite balanced even though the

percentage of women is more than men, but the difference is not too significant. Based on the level of

education, the largest percentage was undergraduate education at 37%. The graph of the respondents’

characteristics can be seen in Figure 15 to Figure 17.

Figure 15: Graph based on Participants Age.

0

2

4

6

8

10

12

14

20 -30 thn 31-40 thn 41-50 thn > 50 thn

Respondent Distribution Based on Age

Page 12: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [54]

Figure 16: Graph based on Participants Gender.

Figure 17: Graph based on Participants Education

Result:

System Usability Scale Questionnaire consists of 10 questions as stated on Table I. There are two groups

of respondents. The first group is buyer and seller. The second group is System Administrator.

Questionnaire uses 5 Likert Scale which has following criteria as mentioned on Table 4.

Table 4: Likert Scale for SUS Questionnaire.

Skala Categories

1 Very disagree

2 Disagree

3 Neutral

4 Agree

5 Very Agree

Table 4 describe likert scale for System Usability Scale Questionnaire (SUS Questionnaire). Respondent

should answer the questionnaire using this scale. After respondent answer the questionnaire, the next step

is to calculate the value of SUS using normalization formula (1) and (2), where SUSValue i,j is defined

as a value of question i from respondent j and RVji is respondent j’s value for question i.

SUSvalueij = RVji − 1 for i is an odd questions (1)

SUSvalueij = 5 − RVji for i is an even question (2)

Acceptance value is got by using formula (3), where j is a respondent and n is the number of respondents.

Acceptance Value = ∑ (∑ SUSvalueij

10i−1 x 2.5)n

j=1

n (3)

Men

46%

Women

54%

Men

Women

02468

101214

Junior High

School

Senior High

School

Non Degree

Diploma

Qualification

Undergraduate Graduate

Respondents Education Level

Page 13: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [55]

User acceptance level is interpreted as acceptance value and value interpretation as shown in Table 3.

Based on SUS Questionnaire, Table 5 describes the response from buyer and seller. While Table 6

describes the response from system administrator.

Table 5: Result for Buyers and Seller Respondents.

R SUS Questions

Total Total x 2.5 1 2 3 4 5 6 7 8 9 10

1 4 0 4 1 3 2 2 1 3 0 21 50

2 2 2 2 2 2 2 3 2 3 2 22 55

3 2 2 2 1 2 2 2 1 2 2 18 45

4 2 2 3 2 3 3 4 2 3 2 26 65

5 4 2 3 2 3 2 4 3 3 2 28 70

6 2 1 3 2 3 2 4 3 4 2 26 65

7 3 2 3 1 3 2 3 3 3 2 25 62.5

8 2 2 3 4 2 3 4 3 4 3 30 75

9 2 3 2 2 2 3 2 3 2 3 24 60

10 4 2 4 3 2 3 3 3 2 3 29 72.5

11 2 2 4 4 2 2 3 3 4 2 28 70

12 3 2 3 3 3 2 3 3 3 2 27 67.5

13 3 2 1 2 2 2 2 2 2 1 21 47.5

14 4 1 2 2 2 3 2 3 2 1 22 55

15 3 2 2 2 2 3 2 3 2 2 23 57.5

16 3 2 3 2 2 2 2 3 3 2 24 60

17 4 2 3 3 2 2 3 2 3 2 26 65

18 3 2 2 3 2 3 2 3 2 3 25 62.5

19 3 2 3 3 2 3 2 3 2 3 26 65

20 4 2 2 2 2 2 2 3 3 2 24 60

21 2 3 2 3 2 3 2 3 2 3 25 62.5

22 2 2 3 2 1 2 1 2 1 2 18 45

23 3 2 1 2 1 2 1 2 1 2 17 42.5

24 2 2 2 3 2 3 2 3 2 3 24 60

25 2 2 1 2 1 2 1 2 1 2 16 40

26 3 2 1 2 1 2 1 2 1 2 17 42.5

27 2 2 1 1 2 3 2 2 2 2 19 47.5

28 3 2 1 1 2 2 2 2 3 1 19 47.5

29 3 2 2 1 2 2 2 2 2 1 19 47.5

30 2 2 3 3 2 3 4 3 3 2 27 67.5

Table 6: Result for Admin System Respondents.

R SUS Questions

Total Total x 2.5 1 2 3 4 5 6 7 8 9 10

1 3 2 3 3 2 3 3 3 3 2 27 67.5

2 2 2 4 3 2 3 3 3 3 3 28 70

3 3 2 3 3 2 3 4 4 3 3 30 75

4 3 2 1 2 2 2 2 3 2 2 21 52.5

5 3 1 2 2 2 2 2 2 2 3 21 52.5

Page 14: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [56]

As mentioned before, Table 5 shows the result for buyer and seller respondents and Table 6 shows the

result for System Administrator respondents. After calculating using formula (1) and (2), we got

acceptance value for buyer and seller is 57.8 and acceptance value for system administrator is 63.5.

Based on those acceptance rate from Table 2, this acceptance value can be categorized as marginal

acceptable. Furthermore, there are some improvement should be done to the system in order to increase

the acceptance value. We also conducted another analysis to test average value for each question in SUS

Questionnaire. During the design phase, we divided 10 questions in SUS Questionnaire into two groups:

Questions which give positive contributions and questions which give negative contributions.

Positive contribution question exists for the odd number questions and negative contribution question

exists for the even number questions. After have some normalization using formula (1) and formula (2),

we gain value range from 0 to 4 with interpretation as shown on Table 7.

Table 7: Data Range and Interpretation SUS Quistionnaire.

Data Value Interpretation Impact

4 ++

3 +

2 Neutral

1 -

0 --

Table 7 shows data range and interpretation of SUS Questionnaire. Based on the value range, we conducted

calculation analysis to calculate average response for each questions. The result is shown on Table 8.

Table 8: Average Value for SUS Value.

SUS Questions Average Value

Buyer and Saller Respondents Admin System Respondents

1 2.77 2.8

2 1.93 1.8

3 2.37 2.6

4 2.2 2.6

5 2.07 2

6 2.4 2.6

7 2.4 2.8

8 2.5 3

9 2.43 2.6

10 2.2 2.6

Table 8 shows the average value for SUS Value. From this analysis, it is found that 90% of each average

value above neutral but this value has not reached positive impact value to the acceptance value of the

application. This result supports the first finding that there are some improvements should be conducted

in order to satisfy user and increase user acceptance value.

CONCLUSION:

From the analysis, it can be concluded that user acceptance level for agriculture e-Commerce dodolantani

on middle acceptable level. This result is based on SUS Questionnaires that are distributed to the users. It

means that some system improvement should be done to satisfy user acceptance level. For future work, it

needs some user task test in order to get the detail feature that should be improved. Through usability

testing, researchers are expected to find some specific feature that needs to be improved and revised.

Page 15: System Usability Scale for Usability Testing of ...Santoso, & Purwadi, Kajian Aplikasi Pertanian yang Dikembangkan di Beberapa Negara Asia dan Afrika, 2016). Meanwhile, in Indonesia

- International Refereed Social Sciences Journal

■ E-ISSN: 2229-4686 ■ ISSN: 2231-4172 ■ http://www.researchersworld.com ■ Vol.–X, Issue–4, October 2019 [57]

ACKNOWLEDGMENT:

Special thanks to Research and Community Services Institution and Faculty of Information Technology

Universitas Kristen Duta Wacana for providing facilities and funding for publishing this article.

REFERENCES:

Al-Omar, K. (2018). Evaluating the Usability and Learnability of the "Blackboard" LMS Using SUS and

Data Mining, 2nd Conference on Computing Methodologies and Communication (ICCMC) (pp.

386-390). Erode, India: IEEE.

Baharrudin, M., Wardani, N. H., & Herlambang, A. D. (2018). Analisis Usability Pada Sistem Informasi

FILKOM Apps Fakultas Ilmu Komputer Universitas Brawijaya, Jurnal Pengembangan

Teknologi Informasi dan Ilmu Komputer, Vol 2 No. 10, 4179 - 4183.

Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean : Adding an

adjective rating scale, Usability Stud, Vol. 4, No. 3, pp 114-123.

Berita, S. (2014, September Rabu). 2016, Potensi Bisnis E-Commerce Indonesia Mencapai Rp 283

Triliun. Indonesia: Berita Satu. Retrieved from Berita Satu:

http://www.beritasatu.com/iptek/207256-2016-potensi-bisnis-ecommerce-indonesia-mencapai-

rp-283-triliun.html

Brooke, J. (1996). SUS-A quick and dirty usability scale Industrial usability evaluation. 189(194): p. 4-7.

Delima, R., & Purwadi, J. (2015). Analisis Situs Web Pertanian Berbahasa Indonesia, Seminar Nasional

Komputer dan Informatika Terapan (pp. 1-5). Jember: Politeknik Negeri Jember.

Delima, R., Santoso, H. B., & Purwadi, J. (2016). Kajian Aplikasi Pertanian yang Dikembangkan di

Beberapa Negara Asia dan Afrika, Seminar Nasional Aplikasi Teknologi Informasi. 2016, pp. pp.

B19-B26. Yogyakarta: Universitas Islam Indonesia.

Delima, R., Santoso, H. B., Aditya, G. H., Purwadi, J., & Wibowo, A. (2018). Development of Sales

Modules for Agricultural E-Commerce Using Dynamic Syatem Development Method,

International Journal of New Media Technology (IJNMT), 95-103.

Delima, R., Santoso, H. B., Andriyanto, N., & Wibowo, A. (2018). Development of Purchasing Module

for Agriculture E-Commerce using Dynamic System Development, International Journal of

Advanced Computer Science and Application, 86-96.

Devy, N. I., Wibirama, S., & Santosa, P. I. (2017). Evaluating Eser Experience of English Learning

Interace using User Experience Questionnaire and System Usability Scale, 1st International

Conference on Informatics and Computational Science (ICICoS) (pp. 101 -105). Semarang,

Indonesia: IEEE.

Devy, N. R., Wibirama, S., & Santosa, P. I. (2017). Evaluating user experience of english learning

interface using User Experience Questionnaire and System Usability Scale, 1st International

Conference on Informatics and Computational Sciences (ICICoS). Semarang, Indonesia.

Gentur, P. J. (2014, September Selasa). Transaksi e-Commerce Indonesia Capai US$ 10 Miliar, Retrieved

from CNN INDONESIA: http://www.cnnindonesia.com/ekonomi/20140923142423-92-

4073/transaksi-e-commerce-indonesia-capai-us--10-miliar/

Khan, A. G. (2016). Electronic Commerce: A Study on Benefits and Challenges in an Emerging

Economy, Global Journal of Management and Business Research: Beconomics and Commerce,

3-4.

Lewis, J. R., & Sauro, J. (2017). Can I Leave This One Out? The Effect of Dropping an Item From the

SUS, Journal of Usability Studies, 13(1), 38-46.

Puspita, M. A., Sibarani, T. M., Muslim, E., & Moch., B. N. (2016). Perancangan Ulang User Interface

Mesin Parkir Meter Jakarta, Jurnal Ergonomi dan K3, Vol. 1 No. 1, 25-33.

Turban, E., King, D., Lee, J., Liang, T., & Turban, D. (2015). Electronic Commerce : A Managerial and

Social Network Perspective, Cham, Switzerland: Springer International Publishing AG.

Retrieved from E-commerce: http://staff.uns.ac.id

----