Design Thinking in EFL Context

The International Journal of

Design Education

DEsIgnprIncIplEsanDpracTIcEs.com

VOLUME 6 ISSUE 2

__________________________________________________________________________

Design Thinking in EFL ContextStudying the Potential for Language Teaching andLearningDEBOPRIYO ROY AND JOHN BRINE

THE INTERNATIONAL JOURNAL OF DESIGN EDUCATION

http://designprinciplesandpractices.com/

First published in 2013 in Champaign, Illinois, USA by Common Ground Publishing University of Illinois Research Park 2001 South First St, Suite 202 Champaign, IL 61820 USA

www.CommonGroundPublishing.com

ISSN: 2325-128X

© 2013 (individual papers), the author(s) © 2013 (selection and editorial matter) Common Ground

All rights reserved. Apart from fair dealing for the purposes of study, research, criticism or review as permitted under the applicable copyright legislation, no part of this work may be reproduced by any process without written permission from the publisher. For permissions and other inquiries, please contact <[email protected]>.

The International Journal of Design Education is a peer-reviewed scholarly journal.

Typeset in CGScholar. http://www.commongroundpublishing.com/software/

Design Thinking in EFL Context: Studying the Potential for Language Teaching and Learning

Debopriyo Roy, University of Aizu, Japan John Brine, University of Aizu, Japan

Abstract: Design thinking and incorporating design curricula in the mainstream language teaching in an EFL context is probably a new pedagogical approach, as the literature indicates. This article undertakes a case study analysis with computer science majors in Japan, suggesting that web design analysis situates language acquisition in the target language through content-based learning and higher order thinking. This design-based language learning approach has the potential to promote grammatical understanding through increased writing practice, systematic thinking, schematization, presentation and structured content authoring. As part of the analysis reported in this article, readers authored open-ended responses to a variety of design and inference-based queries, based on their analysis of a website every week (for 6 weeks). Results provide enough indication to suggest that readers largely understood most of the questions related to interface design, navigation design, information design, audience analysis, product goals, besides other design-based queries, but lacked a definite structure and pattern for analysis and thinking, as is often seen in heuristic evaluations. Data suggested that use of structured design rubrics, examples, regular feedback, and practice with web design analysis might pave the way for more systematic and higher-order thinking in the long run. Structured thinking in the process, will lead to original text and language production in the target language.

Keywords: Design, Language, Content, Web Analysis, Higher Order Thinking

Introduction

his article is an exploratory attempt to understand whether and to what extent web design analysis as a higher order thinking process could be integrated in a content-based foreign language classroom in the Japanese higher education context. Is web design analysis a

worthy exercise, benefiting students by cultivating substantial thinking skills and helping to produce original content?

Using the Internet for ESL/EFL (English as second/foreign language) writing instructions is a common practice now (Krajka, 2000). The issue of using web pages for teaching writing is raised in Tan et al., (1999). Trokeloshvili and Jost (1997) concluded that public displaying of student text on a student home page highly motivates students to conduct writing and publishing, and helps to remove mental blocks associated with publishing ordinary writing. Belisle (1996) argued that networked collaboration like using e-mail and sharing files help learners create, analyze, and produce information and ideas more easily and efficiently. This increased electronic access increases social awareness and confidence. Evidence suggests that networking frees students from the limitations of traditional writing tools that often inhibit and restrict writing and feedback processes. With networked writing and analysis, learning is transformed from a traditional passive-listening exercise into active exploration (Belisle, 1996; Krajka, 2000). However, use of different Internet resources for EFL learning contexts have gained momentum following the advent of Web 2.0.

Similarly, website evaluations or usability studies have dramatically evolved over the last decade or so. They include conceptual discussions on what should be evaluated and how to do it (Instone, 1997; Neilson, 2000). As might be anticipated, web analysis has the potential to be a beneficial exercise (Bunz, 2001; Spyridakis, 2000) and more so in an EFL context.

However, it might not be easy to implement web design and analysis practices in an EFL setting. Students with little English background have a general tendency to produce English text using translation software, copying from other sources with minimal or reasonable rephrasing (Yamada, 2003). A web design project in that sort of a EFL learning classroom has the potential to quickly turn into a mundane visual presentation and artifact through excessive and sometimes

T

The International Journal of Design Education Volume 6, 2013, wwww.designprinciplesandpractices.com, ISSN 2325-128X © Common Ground, Deborpriyo Roy, John Brine, All Rights Reserved Permissions: [email protected]


exclusive focus on page templates, style guides, text-graphic coordination, but with very little original text production for the websites. It could be safely considered from in-class observation in this EFL context that during a website analysis exercise which is more analytical in nature (as compared to web design exercise when considered as a simple text authoring process), students read the text and graphics in the web page, besides exploration of the layout and design, and comment based on what they see and understand. When reporting on website analysis, there is less possibility and opportunity for the reader to copy the text material directly from the webpages under design investigation, unlike what is possible when analyzing a text from the webpage directly. During website design analysis, the focus is relatively less on the textual content, but rather the designers spend time on the design of text formatting and visual presentation, graphics, menu and the overall screen design and navigation.

So, depending on student capability to produce design analysis, web design exercises might not work the way they are originally intended to in an EFL classroom. While usability-based web design principles (e.g. style guides, visuals, page layout etc.) are extremely important, making the designers (students in an EFL context) explain the choice of design and justify design decisions in their own words will situate the language learning process in a more authentic context, and language acquisition will become justifiably more enriched. Ideally, web design analysis through guided writing processes, logically might thus be an acceptable way to get over this problem of language production more as a superficial process of mere text production, without any substantial understanding of the content or the context of writing. An analytical focus will help writers develop a writing approach, with a sense of why a design process is chosen, what it means, how should it be implemented and with what consequences, what it means to the audience etc. Such procedures are likely to promote higher order thinking skills.

Research Focus and Significance

The major research question for this study relates to readers’ (students with computer science majors in this EFL context) ability to successfully explain design decisions by studying specific websites of interest, in an English language course. Further, the qualitative study as reported here, attempts to understand if readers are more comfortable with specific types of design queries as opposed to others, and fare consistently better or worse over time. This study attempts to understand the extent to which EFL readers are able to comprehend different aspects of web design, equipped with summative background lectures provided (at the beginning and middle of the semester) to make design decisions.

This experiment is limited in scope, given how little introductory EFL readers can successfully process, and provides indication as to whether, with more formative assessments along the way in terms of design guidance and feedback, an EFL reader could improve on his/her ability to explain design decisions more successfully, and selectively draw from the field of web design studies.

The website analysis assignment in an EFL classroom is significant for various reasons: 1. It allows for a comprehensive understanding of the extent to which readers could

understand information organization, design and layout. 2. It allows for an understanding of the extent to which readers used meta-cognitive

assessment strategies, and variety of thinking with questions that are broad in nature.

3. This assignment will help explain the extent to which readers, with their levels of English language processing ability, and assessment strategies are comfortable answering certain questions as opposed to others.

2

ROY AND BRINE: DESIGN THINKING IN EFL CONTEXT

A Review of the Literature

Lee (2000), while considering the motivational aspects of tasks that involve the Internet, describes creating and publishing web pages as one of the most potentially valuable and energizing exercises.

Problem with How EFL Learners Approach English Acquisition

The study as reported in this article is situated in a Japanese computer science environment. English instruction in Japan (and EFL context) at the secondary and university levels is devoted largely to readings taught by the translation procedure. This procedure has not been particularly successful, considering the time and energy devoted to it (Kobayashi 1975; Hino 1988; Reischauer 1978; Miller 1982). Most undergraduates who study English, including many of those who major in it, cannot use English with any facility; for example, they cannot read well enough to read English books for pleasure (Robb and Susser, 1989).

However, research in Japan suggests that different kinds of text modification facilitate different levels of comprehension (Yano et al., 1994). How does that relate to web page analysis? A web page analysis will often result from uncontrolled and selective text reading based on reader preference and need for comprehension or task completion. Also, some pages will emphasize text, while others might not. Thus, readers automatically and naturally confront linguistically variable, simple and complex text.

Also, students have other strategies to make reading and comprehension relatively more effective. Students in Japanese EFL contexts have extensively adopted meta-cognitive reading strategies like skimming, scanning, reading headlines, using dictionaries, translation etc. in analyzing technical texts and results have shown superficial and often surface level ability to comprehend, analyze and write about a technical text (Roy, 2010), while similar results have been observed in other EFL situations (Maghsudi and Talebi, 2009).

The above strategies will influence readers’ ability to analyze information organization, design and layout, grouping, navigation, audience analysis etc. Neilson (1997) has demonstrated that the website analysis task is different from any other reading task, because it requires an analytical mindset, analysis and resultant English text production in a specific design context. Also, ability to explain a design and layout might not always incorporate reading and comprehending the entire text in the web page. Readers might get away with merely understanding the headlines, the menu items, the introductory sentence of a paragraph etc. (Neilson’s Alertbox, 1997). The design analysis exercise as reported in this article does not does not make it mandatory for readers to have a comprehensive understanding of the text, but it tries to ascertain whether students are able to understand the overall textual context, and efficacy of the web design and explain it in English with reasonable success.

Importance of Analytical Approach and Higher-Order Thinking Skills

Educators have argued for the importance of promoting higher-order thinking skills in ESL and EFL classrooms (Chamot, 1995; Tarvin & Al-Arishi, 1991) and empirical evidence supports the effectiveness of teaching critical thinking skills along with English-as-a-second or -foreign language (Chapple & Curtis, 2000; Davidson, 1994, 1995). Spada & Lightbown (1993) hold that thinking skills operate effectively when students voice their analysis and take part in the learning process occurring in the classroom. Although there is little argument among theorists and educators about the interrelatedness of the development of languages and thinking skills, in typical school settings, language learning and thinking skills are often treated as independent processes (Suhor, 1984; Pica, 2000). Language as a way of thinking and learning has remained more of a theoretical construct rather than a practical strategy. The Thinking-Approach Project (2004) supported by the British Council criticizes the traditional approaches (i.e., grammar–

3


based syllabus, functional-notional syllabus, natural approach, etc.) to language education. It points out the key contradiction that language teachers spend most of their time teaching language competence rather than preparing students for real life situations. Kabilan (2000) also argues that for learners to be proficient in a language, they need to be able to think creatively and critically when using the target language.

Content-based teaching is an approach considered by many as an effective way to teach language skills while supporting the development of critical thinking. Through content-based instruction, learners develop language skills by thinking and learning in the target language (Brinton, Snow, & Wesche, 1989). Atkinson (1997) and Fox (1994) depict Japanese learners as group-oriented, harmony-seeking, hierarchical, and non-critical thinkers.

This is where a website analysis assignment as a critical thinking and content-based teaching exercise can be argued to be an important technique worth exploring. Krug (2000) points out that design needs to be intuitive. However, in an EFL context where language proficiency is limited, unless writing exercises demand that design decisions be explained, there is no way an output (a superior web page designed) in itself could explain whether designers (students in EFL or other context) have properly understood the design decisions and broader implications.

Van Hoosier-Care (1997) describes the website assignment as a rhetorical exercise in the technical communication classroom. It is important for the reader to understand the conceptual process of designing a website, include the rationale of the project, target audience, purpose of the website etc. (December and Ginsberg, 1995). The main problem with the website assignment is the time allowed for its completion. If the instructor wants the sites to have any level of sophistication, usually at the basics of scanning and a graphics program must be taught, plus lectures of conceptual issues such as web credibility, usability, audience targeted message design, visual rhetoric and visual design. Finally, class time must be allowed for students to work on their websites. However, even this level of commitment does not always lead to “good” sites (Bunz, 2001). A more practical, different and yet optimal approach might be to assign limited web design assignments with design lectures, but with more web analysis assignments.

The experimental and goal-orientated nature of web design projects involve tasks such as deciding with a partner where to place a picture on a page being constructed, or browsing, which requires active choices of where to search next. These are claimed to help promote higher order thinking skills (Mike, 1996), which include reviewing, scanning, selecting and negotiating, and particularly important for EFL students doing further studies in other disciplines, research and rhetorical skills that may be developed. Furthermore, Warschauer (1997) points out that web design skills incorporate ‘situated learning’ that which allows students: “to carry out meaningful tasks, and solve meaningful problems, in an environment that reflects their own personal interests as well as the multiple purposes to which their knowledge will be put in the future” (Collins, Brown, & Newman, 1989). With the goal of designing and publishing web pages, students can actively make use of new technologies, skills, and knowledge. Warschauer (1997) also acknowledges this, and supports the view that many skills, in particular, those that are involved in collaboratively accessing and interpreting worldwide information, and with peoples from different cultures, will be critical for success in the 21st century.

Web Design Principles

Creation of a successful web page should logically involve a proper understanding of web design principles. To ensure the best possible outcome, designers should consider a full range of user interface issues, for the best possible human performance (Koyani et al., 2004). The research questions developed for user analysis in this project are based on the following broad guidelines.

• Provide content that is engaging, relevant and appropriate to the audience (Spyridakis, 2000).

4


• Use all available resources to better understand users ‘requirements (Adkisson, 2002).

• Ensure that the website format meets user expectations, especially related to navigation, content and organization (Lynch and Horton, 2002).

• If user performance is important, make decisions about content, format, interaction, and navigation before deciding on colors and decorative graphics (Baca and Cassidy, 1999).

• Consider as many user interface issues as possible during the design process (Zimmerman et al., 2002).

Based on these relatively broader guidelines, and the different aspects of user experience design as is evident in the model proposed by Garrett (2011) and shown in Figure 1, specific questions were designed based on audience and task analysis, product goals, information design, interaction design, information architecture, etc.

Figure 1: The Elements of User Experience (Garrett, 2011)

Methods

The sample for this study is typical of any EFL context. The 28 participants in this study demonstrated pre-intermediate English language processing ability as are evidenced from regular class observations and similar writing assignments over the semesters and across different courses. The participants for the study are computer science majors without any background in website analysis and design. Students have taken other courses in English over the last two years of their academic career, but this is their first experience with information and user-centered design. This sample represents any population where individuals do not have any specific

5


background in information design, web analysis, but have experience reading English text and processing it with strategies (metacognitive approaches) mentioned above.

Structure of the Website Design Assignment

At the beginning of the course (based in the learning management system called Moodle), a group of 28 computer science students were given a trial website evaluation exercise with detailed explanation of the questions to be asked during the actual class assignment that followed over the subsequent weeks. The first couple of weeks in the course started with a detailed in-class lecture and moderate in-class discussion, with examples and demonstrations of how website evaluation is an important topic with discussions on content analysis, navigation, page layout, etc. The primary purpose was to introduce readers to preliminary concepts in document/web design. The introductory practice forum assignment ensured that every student could see what others had done and thus adopt/adjust their writing strategy accordingly. Students had over two weeks to adjust according to the questions, the lectures and to develop more ideas from what other students had written about. The graded website evaluation assignment did not start until the third week into the course.

As part of the weekly website evaluation exercise, students had eight questions to answer, and submit their assignment in moodle, on topics related to content, navigation, usability, audience analysis, marketability of the website, technical efficiency, etc. The evaluation questions were indirectly based on a website evaluation checklist designed by Anderson et al., (1998) at the University of Michigan (used as a reference), but directly based on the web design guidelines and models mentioned in the previous section. However, since this is an EFL web evaluation context, with students possessing limited English language efficiency and reading and comprehension skills, the questions were modified, trimmed, simplified, and summarized to suite a more efficient level of understanding and task completion. More emphasis was placed on whether students could understand the overall scope of the questions and had an overall idea of how to read/scan through the English website to get an answer.

Students went through evaluating one website each week for six weeks. A standard set of 8 website evaluation questions were used each week, but the website to be evaluated changed from one week to the next. Standard corporate websites were chosen for companies that might interest computer science students. Every chosen website belonged to brand name companies. Interest in the content, technology used, scope for research, laboratories, product diversity, marketing strategies etc. were preconditions on the basis of which the websites were selected for evaluation. Students had a week to consult their group partner and reach a consensus on what might be the best way to answer each question. Although the format of the assignment ensured that each participant consulted with their group partner, their individual response could differ from their partners. Class sessions were observed to ensure that group members did coordinate. However, during grading it was observed that they did differ from their group partner in their response to some degrees. The website evaluation exercise continued for six weeks with the same set of students. However, following the end of the third weekly web evaluation exercise, students were asked to present orally on what they completed for the first three weeks. The presentation focused on a summary discussion of their website evaluation strategies and how they did it. Every person’ s presentation had a short discussion session that followed. This was considered as a formative assessment strategy. The assignment ran through six weeks with repeated practice with the same set of questions (and evaluation measures) but different websites on broad related content of interest was conducted to explore whether student performance with web evaluation improves over time with practice. Weekly discussion sessions were allowed every week in this relatively autonomous learning environment with limited structured feedback. At the end of the sixth week of the website evaluation exercise, students were handed out a post-test questionnaire which was designed to test their reflective self-report of how they approached the website and the evaluation process. 6


Instruments

The assignment was posted and responded to in Moodle. Students had one week to complete and post the weekly assignment.

Each of the eight questions every week was open-ended and had a minimum word limit of around 60 words. Open-ended questions were designed so that participants are able to give voice to their opinions openly, without being influenced or constrained in any way by pre-defined options presented to them, and without being affected by the randomness of the responses inherent in multiple-choice type answers. Also, open-ended questions captured the sense of how participants interpreted each question. It is possible that for a given question, the participants ended up interpreting the question in a way not intended by the instructor. The assessment criteria and approach ensured that the subjectivity of the score with each response would be minimized, and would reflect the true quality of the answer consistently across question types and participants over time (weeks).

Figure 2 demonstrated a concept map showing the open-ended questions as posted in Moodle, and also shows the schema under which these questions fit, based on Garrett’s (2011) model. Readers had to answer these questions consistently over six weeks, but each week a different website was responded to.

Figure 2: Concept Map Showing the Design Categories and Question Classifications

Question (Based on simpler modifications) Q1. Explain whether the organization of information in the site is user friendly or not? Q2. Explain whether the presentation of content is appealing or not? Q3. Explain whether the effective use of technology is demonstrated? Q4. Who is the target audience? Is the website appropriate for the projected audience? Q5. Explain the quality of the content. Q6. Is the information accessible? Q7. Explain whether the resources use real world scenario. Q8. Here are some common reasons for building this website. Rank them order of importance.Do you have a reason that is not listed?

7


The questions are based on Garrett’s (2011) model of the elements of user experience. The questions have been generalized and modified to elicit readers’ overall understanding, and for an introductory EFL context. The responses were evaluated on the basis of whether readers have an overall understanding of what the model suggests, and/or if the responses touch upon what is intended in the designed model. It is NOT expected in this EFL context, that readers can make finer differentiations between the responses. A relatively accurate and yet surface level analysis, demonstrating an accurate representation of the design context, was considered acceptable.

The 8 questions that students were asked each week were classified into two different groups- namely the design group and the inference group. The design group was comprised of questions the answer to which could be developed and formulated directly based on observing the web page design elements like organization, information and page layout, navigation, typography, white space in the pages, etc. Responses to questions in the inference group could be developed based on readers’ understanding of the purpose of the website. There were no directly observable elements that made up the response to these questions.

Expectation from Responses

Primarily, student responses were graded based on the following criteria. Did the readers understand the question completely?

1. Did the reader make an attempt to answer the question as was asked? 2. Could the reader explain what he/she saw during interaction with the website? 3. Is there enough evidence to suggest that the reader made an attempt to understand

the given website? 4. Was the answer grammatically reasonable and of acceptable quality? 5. Was there evidence to suggest that the reader consulted his group partner? Did they

collaborate on the chosen answer? Other specific details: If it appeared that a significant portion of the text had been copied

from an earlier week, without adequate analysis of the website assigned for the week, students would be deducted two points from their overall score out of 6, for each question.

Other important criteria for assessment were as follows: • Whether students made reference to the website under study, to explicitly

demonstrate that they had taken a look at the website. • Next, readers should have been able to provide some levels of detail regarding what

they saw in the web page and what they thought it meant. General explanations could often be recycled from responses in previous weeks and that is why some details about the design were necessary in the response for each week.

Choice about Web Pages

The web pages were chosen with different uniform criteria in mind, and with an effort to maintain consistency in terms of content complexity and quality over the weekly assignment topics. The web pages were similar in terms of the following:

1. Choice of web pages ensured that the companies chosen were dedicated to computer science, electronic equipment (a company that might be construed by students as a potential employer or seller), or a more general topic of interest to all (e.g., Japan tourism).

2. Relative content complexity in terms of the topical discussions. 3. Questions were asked at a surface level with focus on navigation design, interface

design, interaction design, and overall audience analysis. Understanding the content in each web page at a deeper level was not important and not a criterion for answering the questions asked.

8


4. The web pages were chosen in a way to explore whether students paid attention and focused on the major headlines, the menu items, and the graphics in the web pages, the search criteria, the importance of topics, etc., and also whether such items in the web page were referenced to answer questions, whenever needed.

5. To what extent were readers selective in their choice of web page features, paying attention to some features more often, while ignoring some?

As mentioned before, every week students worked on a different website. The following figures (Figures 3–8) demonstrate the different design features for each of the websites used over six weeks.

Figure 3: IBM Research Tokyo Website and its Features

9


Figure 4: Oracle Argus Japan Website and its Features

Figure 5: Microsoft USA Website and its Features

10


Figure 6: Dell USA Website and its Features

Figure 7: Mitsubishi Electric Website and its Features

11


Figure 8: Japan Tourism Website and its Features Findings

This article primarily reported qualitative analysis undertaken for this study. This section reported on the various types of open-ended, retrospective and reflective responses to each question separately. Firstly, the overall response to questions in the design group has been reported.

Responses to Design Group Questions

5 questions were asked in the design group and a summary of the major responses is reported below. Q1 - Information design: Explain whether the organization of information in the site is user-friendly or not?

Overall, responses throughout the six weeks indicate that participants were comfortable making mostly detailed comments about the organization of the website each week. As observed, one popular approach towards the answer was making explicit reference to the major clickable items in the menu that appeared in the homepage. They referred to items like “popular downloads”, “latest news”, “popular searches”, etc., and then suggested these items added value to the organization. Reasoning towards the response was mostly missing. Another approach was to support reasons explaining why the organization of the website might add value to the customers.

One student responded:

I think Microsoft website is user-friendly because the basic structure is clear. The site used many photos very effectively. Users can see information that matters. However, the font size in web site is too small to be seen easily.

In some cases, references to the home page elements and menu items were used as a way to fill up the word requirement for the response, without any appropriate reasoning to suggest why

12


these elements added value. Mostly, the responses in the subsequent weeks remained similar to the first couple of weeks and there was no significant change in strategies later on. Q2 - Navigation Design/Interface Design: Explain whether the presentation of content is appealing or not?

As part of this response, readers were expected to have a strong influence from the previous response, as both Q1 and Q2, on the surface might sound similar, although lectures discussed the difference between these two design questions to a large extent, towards the beginning of the course.

Different reasons were provided in support of why one thought the website presentation to be appealing. One participant provided the following reason:

Yes, I think this website is appealing. This is because this web site writes at length about Core Research Competency. At least I can be interested in Human Computer Interaction.

Another student provided the following explanation:

No, because this site is too simple. And characters are too small to read and their color is neutral. The appeal factor is not clearly explicit in the design.

Some other explanations were of the following nature:

I think the content of website of IBM Tokyo is appealing. This web site has placed various information in details, and I think that even the people, who look at this homepage for the first time, will be satisfied with this web site content.

Most of the arguments by the same participant were consistent across weeks. In most cases, it was clear that readers were thinking along the same line, and drawing arguments based on the information they had seen on the surface, mostly the home pages. Q3 - Interface Design/Interaction Design: Explain whether the effective use of technology is demonstrated.

The main purpose of this question was to understand whether readers could explain the type of technology discussed in the website as a company product, or web-based technology used to demonstrate products in the website. The question was open and participant response was quite diverse.

One typical pattern of response discussing company product was as follows:

When I look at the website of IBM Tokyo I found various effective technologies that was developed in IBM Tokyo and are used in the real world.

For Japan tourism website, where technology is not a company product, the same participant responded as:

The site introduces a Japanese sightseeing spot, so I think the effective use of technology is not demonstrated in the site.

So, for this particular participant, effective use of technology meant any technology developed by the company itself, and if the company website mentions it. On the contrary, another type of response indicated how the website features and affordance suggest efficient use of technology. The response below provides an example.

13


This site makes effective use of web technology and there are many examples. These examples are slide pictures, search window, connect Facebook, select language; connect you tube, select category light bar, etc. For example, slide pictures have big impact and explain the tourism spot. In another example, search window is convenient to our search work. If one is searching for a specific tourism spot, typing tourism name, and search result give us the necessary information.

Q5 - Text-Graphic Content: Explain the quality of the content. Primarily, as a response to this question a large section of the participants decided to talk

about a specific topic that interested them in the web page. They didn’t necessarily talk about the design of the web page, navigation, organization of information etc. The following example shows one approach to answering this question.

I think this web site’s quality of contents is good. We can understand most content if we read this. I can understand some of the content related to “Human Computer Interaction”. I understand about the topic “Text-to-Speech”. This content discussed the development of a speech synthesis technology that generates human voices reflecting various kinds of personalities.

Some other responses indicated that quality of content for them was measured on a yardstick that explains the extent to which new technology was discussed in the website, or whether the company uses new technology to build its main products. In this specific example for the DELL USA website, one participant’s response is as follows.

This site shows products of computer. But, we cannot see to new technology. This is because this company uses other company parts to build computers. Therefore, this website does not have distinctive advantage. So, quality of the content is not good.

The following response provides clear indication that the reader might not have understood the real essence of the question. The purpose of the question is not to judge the quality of the company product, but the content and its presentation in the company web page. However, with little experience of design analysis, the participant might have confused or misunderstood the real focus of the question. These types of responses allow us to explore the range of thoughts for novice design analysts. Q6 - Navigation Design: Is the information accessible?

From the majority of responses, it appeared that respondents could get a somewhat proper sense of what this question intended. In most cases, participants graded a website on the basis of how the links worked (mostly from the home page), and the quality of information available in the linked pages. Participants made a personal judgment as to whether the linked information appealed to them in any way.

Here is an example of a response:

Yes, the information is accessible. Because, this website show list of products that is available with other companies and web pages and is linked from the DELL webpage. Customer can intuitively know how to find the page.

In the above example it is not apparently clear as to why the participant decided to use the word “intuitively”. It appears that the responder wanted to say that the text in the clickable menu is clear enough to provide an indication of the kind of information available in the linked pages. In some other cases, participants answered this question on the basis of server speed. The following text is an example.

14


No. The response of this website is too slow. Users become nervous.

There is also an indication that some readers prefer to use and value the existence of a “search” bar.

This website is easy to get information, because referenced information can be accessed in the same place in the website. In addition, this website have search bar in upper corner of the screen, and so users can search interesting words and get information. Thus, this website is accessible for people using this website.

Overall responses to this question suggested that readers were more comfortable answering this design question, rather than some of the other design queries asked before. This was probably because questions related to presentation of content, text-graphic content, etc. were rather broad and encouraged a wide range of responses.

Responses to Inference Group Questions

Three questions were asked in the inference group and a summary of the major responses is reported below. Q4 - Audience Analysis/User Experience: Who is the target audience? Is the website appropriate for the projected audience?

Readers were mostly comfortable answering this question. Post-test class interactions suggested that this question was easy to understand for them, and readers, in most cases, formed an impression about the response from their overall skimming of the webpage content. The following example about Mitsubishi TV provides an example of the clarity of their responses.

I think that this website targets general public. I understand that this website intend to introduce products of this company. For example, this website writes about introduction of TV and some accessories. I do not think that this information is useful for engineer.

Another example demonstrates thorough understanding of the scope of the question.

This site’s target audience is mainly non-Japanese nationals, because this site’s top page is not in Japanese. So, this site is not targeting Japanese tourists. The website is appropriate for the target audience because this website news is updated regularly, and supports the overall organization of the website. Japan is culturally rich; so many foreign citizens are interested in Japanese culture. Therefore the website serves the purpose.

Sometimes, the logic and language deficiency impedes the process of explanation, but such kinds of responses are acceptable for novice design analysts.

Q7 - How Practical is the Content? Explain whether the resources use real-world scenario.

Responses suggested that this question was quite broad in nature, and readers in most cases answered it, based on their own personal judgment and understanding as to what it means. That is how the question was chosen and framed, with the purpose of encouraging wide range of responses.

Some students apparently only attempted a very surface-level response, and without any explanation whatsoever. The following responses are examples:

It is written to use real-world situations. If the site is seen, the reason is understood.

15


This site’s resources use real-world situations. When I introduce my country, I only tell this site URL.

In some cases, responses centered on whether the content is for novice users or experts. So, real-world scenario was interpreted in this case on the basis of whether the information is accessible and useful for potential buyers who are non-experts.

I think mainly this website’s information could be used by experts. We don’t use this site’s information.

Alternatively, the question was also interpreted in terms of the company product like JAVA research by Oracle and its practical application in real life. The following response is an example.

The resources mentioned in the website are used in real-world situations. For example, Java is used all around. Possible applications are cell phone, PC, etc. We use applications such as games, navigators, calculators, etc. These are created by Java. Another, Java has a wide range of applications in many industries, for example construction, chemicals, aerospace, financial services, etc. We do not carry these products around. But these services make our life more comfortable.

These broad-scoped questions are geared towards making the reader think about a possible response. The question grades were not based on a specific correct response, but rather focused more on higher-order thinking. Q8 - Product Goals: Here are some common reasons for building this website. Rank them in order of importance. Do you have a reason that is not listed?

Respondents were in fact, in most cases quite comfortable answering this question and they could secure a high numerical score for this question. The example below of a response for Oracle supports our understanding that higher-order thinking could be promoted with such inference-based queries.

I think that this website is written to provide general and/or product information and I think that it is very important. It is very important that the enterprise explain their product successfully. Sales will go up if a general person learns the product. Successful product or service explanation in the webpage will also encourage people who might be interested in the company, as a customer or would like to work in the company. This can help the company to increase their profit and develop a list of qualified prospects that could be explored further. Name branding is also very important.

Similarly, other responses support various reasons for the existence of the website and what it can do to improve more participation in the company and how more visitors might find the website interesting. Some explanations border on the initial design queries and appeared to be somewhat recycled, but overall, there are enough indications to suggest that higher order thinking could be promoted with these queries. This design study is an example of how content-based learning could be successfully initiated in a foreign language classroom with systematic effort.

Summary of Quantitative Results

Quantitative analysis of the weekly web analysis data for this study suggested that reader performance, for most questions in the design and inference groups, started on a high for the first couple of weeks, but mostly for design queries there was a slump in performance for the fourth and fifth weeks, before rebounding a little during the sixth week. This indicates how lack of pedagogical feedback might impact the score (design performance) in the short run. Further, quantitative data supported the outcome that readers had better understanding of the website

16


audience, its overall design of the interface and text-graphics readability. However, overall data should still be considered as preliminary and inconclusive, and in some cases showed lesser grip of the more complicated questions related to use of technology and wider use of website information.

Discussion

Responses from different types of questions clearly indicated that readers have spent some time thinking about the possible responses. The primary reason for the decision to use open-ended questions was to understand the extent and variety with which participants could think about the responses for any given question. Expectedly, the variety of responses clearly suggests that this experiment in the foreign language classroom could be even more successful with regular feedbacks about the responses and clear structure for the questions with adequate cues suggesting how the responses should be addressed. The lack of regularly provided detailed feedback during the course of six weeks should not be considered as a limitation, but rather an opportunity to understand two major things:

1. The extent to which readers grapple with sentence-level construction issues vis-à-vis formulating content-based responses for the given queries.

2. The variety of thoughts and approach with which readers could approach a given query. This open-ended approach is typical in a usability analysis where readers might be presented with open-ended questions (or think-aloud protocol) to encourage, and subsequently explore readers’ overall thoughts about the interface and the task.

Surprisingly, although this study has been conducted in an EFL setting, there is no reason to believe that such design thinking might be typical for an EFL situation only, and might not hold true in a first language situation. Since participants have used translation software to answer the given questions each week, in more situations than not, the lack of depth in the response (wherever applicable), should not be misconstrued as a typical EFL language-level deficiency issue. Rather, the variety, or lack, of response brings to light the fact that irrespective of whether the reader is thinking and formulating the response in English or Japanese, language-level deficiency in English was not an impediment in the response building process. Rather, the quality of any response in this situation reveals lack of higher order thinking and holistic comprehension of the core design issues, which was probably expected in this introductory EFL context.

Results and examples of reader responses clearly suggested that answering the design-based queries were only a little more difficult than the inference-based queries. However, there was potential that a hypothesis for this study, based on any previous research could have suggested otherwise. This is because design-based parameters are directly observable from the website, while inference-based queries are dependent on the extent to which readers could understand the scope and overall purpose of the website. The text content might not always directly indicate such understanding of scope and purpose in the website. However, our findings suggested otherwise because of the broad nature of the design-based queries for the most part. For example, when commenting on the presentation and quality of content, readers were mostly lost as to where and how to start thinking about the response. Rather, when commenting on the navigation and technology, readers had a relative easy task, because the unit for analysis was not as broad as the “content” mentioned in the question. The word “content” might mean practically anything and that makes the analysis harder.

On a different level, readers had relative difficulty in understanding the query related to information accessibility. Were they mainly not sure about which information was referred to? Was the question referring to any information in the website and the ease with which it could be accessed? Was the information referring to the products of the company and its explanation? Readers were not provided with a detailed unit for analysis.

17


For the inference-based group, the questions were relatively more specific. Although that might not mean it is easier to respond to those questions. The query on audience analysis was very specific and responses clearly indicated that readers could understand and respond to them with ease. However, post-test class interactions suggested that the question on practicality of the content was quite hard to understand, and readers freely thought about it in terms of the website content or company products and research items.

In summary, it might not be an overstatement to suggest that the goal for this study has been accomplished. The major purpose was to explore how design pedagogy could be integrated into the language curriculum, and this exploratory analysis provides enough indications to suggest that structured rubric and feedback-based design and inference queries for website analysis might be an easy starting point towards content-based learning and developing higher-order thinking skills.

Conclusion

Web design, emails, blogs, social networking, etc. have been extensively used, but mainly for text authoring in the language classroom over the last decade or so. The major purpose for such activities has been to encourage second language learners to write more and more. However, those activities largely overlooked the process by which language learning and thinking could be integrated successfully. With rampant use of translation software and plagiarizing, the purpose for such exercises might have been lost to a large extent. We needed a platform which largely provides original context and a situation where the text content for the queries cannot be authored directly, based on the existing textual content available to the reader (e.g., cannot be copied from other web sources like online newspapers, articles, Wikipedia, etc.). This is when readers actually start thinking and formulating a response based on their independent thinking styles and thought processes. The quality of response as a starting point should not be overemphasized, firstly because design thinking can take various shapes and typically there might not be a single correct response, and secondly because higher order thinking improves through repeated practice and sustained engagement in content-level activities that create and promote such thinking contexts. Future studies should look into how readers perform with design and inference-based queries for situations where specific cues are provided for a query. Does the quality of response improve with more definite cues for thinking? Further, other studies could also explore how readers perform with multiple-choice responses to design and inference-based queries. An integrated comparative analysis with data from such studies might yield enriched understanding of the possible design curricula to be introduced in language classrooms.

18


REFERENCES

Adkisson, H. (2002). Identifying de-facto standards for e-commerce web sites (Unpublished master thesis). University of Washington, Washington: US.

Atkinson, D. (1997). A critical approach to critical thinking in TESOL. TESOL Quarterly, 31, 71–94.

Baca, B. & Cassidy, A. (1999). Intranet development and design that works. Human Factors and Ergonomics Society Annual Meeting Proceedings, 777–790.

Belisle, R. (1996). “E-mail Activities in the ESL Writing Class.” The Internet TESL Journal, 2. Retrieved November 11, 2011 from http://iteslj.org/Articles/Belisle-Email.html

Brinton, D. M., Snow, M. A., & Wesche, M. B. (1989). Content-based second language instruction. Boston, Massachusetts: Heinle & Heinle.

Bunz, U. (2001, November). Usability and gratifications–Towards a website analysis model. Presented at the National Communication Association Convention, Atlanta, GA.

Chamot, A. (1995). Creating a community of thinkers in the ESL/EFL classroom. TESOL Matters, 5, 1–16.

Chapple, L., & Curtis, A. (2000). Content-based instruction in Hong Kong: Student responses to film. System, 28, 419–433.

Collins, A., Brown, J., & Newman, S. (1989). Cognitive apprenticeship: teaching the crafts of reading, writing, and mathematics. In Resnick, L. (Ed.), Knowing, learning, and instruction (pp. 453–494). Hillsdale, HJ: Lawrence Erlbaum.

Davidson, B. (1994). Critical thinking: A perspective and prescriptions for language teachers. The Language Teacher, 18, 20–26.

Davidson, B. (1995). Critical thinking education faces the challenge of Japan. Inquiry: Critical Thinking Across the Disciplines, 14, 41–53.

December, J., & Ginsburg, M. (1995). HTML and CGI unleashed. Indianapolis, IN: Sams.net. Fox, H. (1994). Listening to the world: Cultural issues in academic writing. Urbana, Illinois:

National Council of Teachers of English. Garrett, J. J. (2011). The Elements of User Experience: User-Centered Design for the Web.

Peachpit Press. Hino, Nobuyuki. (1988). “Yakudoku”: Japan’s dominant tradition in foreign language learning.

JALT Journal, 10, 45–53. Instone, K. (1997). Usability Heuristics for the Web, [Online], http:

//webreview.com/97/10/10/usability/sidebar.html Kabilan, M. K. (2000). Creative and critical thinking in language classrooms. The Internet TESL

Journal, 6. Retrieved November 21, 2005 from http://itselj.org/Techniques/ Kabilian-CriticalThinking.html

Kobayashi, Yuko. (1975). A new look at reading in the college program. English Teaching Forum, 13, 188–195.

Koyani, S.J., Bailey, R.W. & Nall, J.R. (2004). Research-Based Web Design & Usability Guidelines. Computer Psychology.

Krajka, J. (2000). Using the Internet in ESL Writing Instruction. The Internet TESL Journal, 6 (November).

Krug, S. (2000). Don’t make me think: a common sense approach to web usability. Indianapolis, IN: Que.

Kuhn, D. & Dean, D. (2004). Metacognition: A bridge between cognitive psychology and educational practice. Theory into Practice, 43, 268–273.

Lee, K. (2000). Energizing the ESL/EFL Classroom through Internet activities. The Internet TESL Journal, 6 (April). [WWW document]. URL http://www.aitech. ac.jp/~iteslj

19


Lightbown, P. and Spada, N. (1993). How Languages are Learnt. Oxford: Oxford University Press, pp. 135. ISBN 0–19437–168–7.

Mike, D. (1996). Internet in the schools: A literacy perspective. Journal of Adolescent and Adult Literacy, 40, 1–13.

Miller, Roy Andrew. (1982). Japan’s Modern Myth. Tokyo: Weatherhill. Neilson, J. (October, 1997). How Users Read on the Web. Jacob Neilson’s Alertbox (ISSN

1548–5552). http://www.useit.com/alertbox/9710a.html Nielsen, J. (2000). Designing web usability. Indianapolis, IN: New Riders. Patrick Lynch and Sarah Horton. 2002. The Web Style Guide 2ndedition,

http://www.webstyleguide.com Pica, T. (2000). Tradition and transition in English language teaching methodology. System, 29,

1–18. Reischauer, Edwin O. (1978). The Japanese. Tokyo: Charles E. Tuttle Company. Robb, T. N., & Susser, B. (1989). Extensive reading vs. skills building in an EFL context.

Reading in a Foreign Language, 5, 239–251. Roy, D. (July, 2010). Reading strategies for procedural information in EFL business writing

environment: An exploratory analysis. Proceedings of the IEEE Professional Communication Society Conference, 143–151.

Schriver, K. A. (1997). Dynamics in document design. New York: John Wiley. Spyridakis, J.H. (2000). Guidelines for Authoring Comprehensible Web Pages and Evaluating

Their Success. Technical Communication, 47, 301–310. Suhor, C. (1984). Thinking skills in English—And across the curriculum. (ERIC Document

Reproduction Service No. ED. Tan, G., Gallo, P.B., Jacobs, G.M, Kim-Eng Lee, C. (1999). “Using Cooperative Learning to

Integrate Thinking and Information Technology in a Content-Based Writing Lesson.” The Internet TESL Journal, 5 (August).

Tarvin, W., & Al-Arishi, A. (1991). Rethinking communicative language teaching: Reflection and the EFL classroom. TESOL Quarterly, 25, 9–27.

Trokeloshvili, D. A. and Jost, N. H. (1997). The Internet and Foreign Language Instruction: Practice and Discussion. The Internet TESL Journal, 3 (August).

Van Hoosier-Carey, G. (1997). Rhetoric by design: using web development projects in the technical communication classroom. Computers and Composition, 14, 395–407.

Warschauer, M. (1997). Computer-Mediated Collaborative Learning: Theory and Practice. Modern Language Journal, 81, 470–481.

Yamada (2003). What prevents ESL/EFL writers from avoiding plagiarism? Analysis of 10 North-American college websites. Systems, 31, 247–258.

Yano, Y., Long, M., H., & Ross, S. (1994). The effects of simplified and elaborated texts on foreign language reading comprehension. Language Learning, 44, 189–219.

Zimmerman, D.E. & Akerelrea, C.A. (2002). A group card sorting methodology for developing an informational Web site. Proceedings of the IEEE International Professional Communication Conference, 437–445.

ABOUT THE AUTHORS

Dr. Debopriyo Roy: Debopriyo Roy is a Senior Associate Professor of English, specialising in Technical Communication and Usability at University of Aizu, Japan. He is the director of the Technical Communication laboratory. He holds MA degrees in Economics and Communication and a PhD in Communication and Rhetoric from Rensselaer Polytechnic Institute, NY. He specialises in information design, usability studies, technical writing, and web-based training. His research focuses on designing instructional graphics to support readers’ spatial cognition during

20


intricate procedural and problem-solving activities. He has published widely in international journals and refereed conference proceedings. He is the current chair of the IEEE Japan Chapter on Professional Communication, and a founding member of the ACM Chapter on Technical Communication and eLearning.

John Brine: John Brine (PhD OISE/Toronto) is a Professor in the Center for Language Research at the University of Aizu, Japan. He held a doctoral scholarship from the Social Sciences and Humanities Research Council of Canada (SSHRC), following which he was a Japan Foundation visiting scholar at the Tokyo Institute of Technology and a SSHRC post-doctoral fellow. His fellowship continued at the University of Waterloo, Canada where he taught courses in the Centre for Society, Technology, and Values. At the University of Waikato, he co-founded the Department of General and Applied Linguistics, and established the first computer-assisted language learning (CALL) courses in New Zealand. His current research interests include learning management systems, and the side-effects of information media. The Japan Society for the Promotion of Science has supported his eLearning research collaboration in Vietnam. He is currently a Research Associate in the Digital Library Laboratory at the University of Waikato.

21

The International Journal of Design Education is one of six thematically focused journals in the family of journals that support the Design Principles and Practices knowledge community—its journals, book series, conference and online community. It is a section of Design Principles and Practices: An International Journal.

The journal explores aspects of learning to become a designer and to develop modes of “design thinking”. It explores design strategies, methodologies and tactics. It analyzes forms of professional stance. And it examines pedagogies of engagement with design purposes, designed objects and design.

As well as papers of a traditional scholarly type, this journal invites presentations of practice—including documentation of curricular practices and exegeses analyzing the effects of those practices.

The International Journal of Design Education is a peer-reviewed scholarly journal.

ISSN 2325-128X