Document Type : Research Paper
Authors
1 English Language Department, Maragheh Branch, Islamic Azad University, Maragheh, Iran
2 Assistant Professor, English Language Department, Maragheh Branch, Islamic Azad University, Maragheh, Iran
3 Assistant professor, English language Department,Islamic Azad University, Maragheh Branch, Iran.
Abstract
Abstract
The necessity of using online education during the Coronal virus pandemic and the barriers created by the absence of face-to-face instruction has shifted the researchers’ focus to web-based instruction and assessment. In this regard, an explanatory sequential mixed-methods design was selected to consider the effects of web-based dynamic assessment (DA) on learners' grammar accuracy, autonomy, and attitudes. To this end, a convenient sample of 60 male English as a Foreign Language (EFL) learners was chosen and categorized into two groups of web-based dynamic assessment and a control group. The data collection tools were a pretest and posttest of grammar, a pretest and posttest of autonomy, and a semi-structured interview. The participants were exposed to web-based DA via a designed web based on the level and the students’ needs in grammar, whereas the learners in the control group learned the grammar through the traditional method of instruction. Based on the results of ANCOVA and Mann-Whitney U test, the learners’ grammar accuracy and autonomy mean scores in web-based DA increased compared to those of the control group. In addition, the results of interview showed that the learners had a positive attitude toward web-based dynamic assessment treatment. The results of the semi-structured interview with the experimental group verified the quantitative results. The platform and methods employed in this study suggest encouraging implications for the field of language instruction which will be discussed.
Keywords
Main Subjects
INTRODUCTION
Explaining the sociocultural theory (SCT) of Vygotsky (1978) is important to gain a clear picture about dynamic assessment (DA). Vygotsky believed that social interactions lead to cognitive development, which includes language acquisition. Furthermore, he came to the conclusion that when language learners are supported within the relevant context, they can develop their skills and knowledge more easily (Lightbown & Spada, 2021).
Since instruction and assessment are viewed as two sides of the same coin, the teacher can intervene and help the learner to reach the task objectives. DA argues that teachers can make use of assessment to make the language learners learn while taking the exams; it makes use of an approach called assessment-as-learning (Poehner, 2008). Instead of gathering information in one context and making decision at another time, which occurs in non-dynamic assessment (N-DA), in DA, the mediator supports the learners in the immediate context when the assessment and instruction are integrated. In other words, the mediator gives graded hints after assessing the learners’ answer so that they could choose the right answer. However, the challenges of applying interactionist mediation in traditional classes in an individualized and one-to-one manner have motivated the scholars to find solutions. Lantolf and Poehner (2009) suggested two ways for overcoming these two inherent problems through Computerized DA (C-DA) and Group DA (G-DA).
As Poehner (2008) mentioned, C-DA has a number of benefits, including the ability to be taken by so many students at once, the feasibility of assessing the language learners to the needed extent for the sake of optimum benefit, and the availability of the results to be drawn by a kind of automaton. Therefore, in web-based DA these distinguishing features are implemented. In the larger field of education and assessment, web-based instruction and assessment have grown significantly over the past few decades as teachers have realized the effectiveness of computer-assisted language acquisition in implementing outside-of-classroom teaching strategies (CALL). Following CALL, web-based evaluation has established itself in the field of language training. With more structured instruction and assessment techniques, web-based DA has the potential to produce a more authentic assessment environment (Khandelwal, 2006; Tzuriel & Shamir, 2002).
This study includes the broad category of DA coupled with web-based training and evaluation. As a result, assessment is not only seen as something that is intimately connected to instruction, but also as a process in which learners receive infinite and continuous mediation across the method. In fact, the examiner serves as a facilitator who adapts to the students' level and is more interested in cognitive development than in performance effectiveness (Lantolf & Poehner, 2004). It seems that web-based DA may improve EFL learners’ grammar accuracy in high schools where the focus of instruction and assessment is mostly on grammar accuracy. Additionally, the learners’ responsibility and autonomy will be expected to increase due to the fact that learners are responsible for distinguishing the grammatical correct points.
Although a large body of research has been accumulated on the effectiveness of DA in teacher–student interactions (e.g., Alemi et al., 2019; Khanahmadi & Sarkhosh, 2018; Malmeer & Zoghi, 2014; Ramazanpour et al., 2016; Safa & Jafari, 2017), the effectiveness of C-DA of grammar achievement has received scant attention in the literature. To address this gap, this study is an attempt to incorporate the broad category of DA coupled with web-based training and evaluation. In this line, this study aims to examine the effectiveness of web-based DA on improving the grammar accuracy of students, autonomy, and their attitudes. Both quantitative and qualitative methods were used in this study. Investigating the effects of changing the current teacher-oriented form of assessment and instruction, which is imposed on the current Iran’ educational system, is deemed worthwhile.
LITERATURE REVIEW
Dynamic Assessment
The concept of DA is based on the two components of SCT, namely mediation and the zone of proximal development (ZPD). Mediation, according to Vygotsky (1978), is the basis of all higher mental functioning. The ZPD, is defined as “the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem-solving under adult guidance, or in collaboration with more capable peers” (Vygotsky, 1978, p. 86). In Vygotsky's idea, the former is known as the zone of actual development (ZAD), and the latter is known as the zone of potential development (ZPD).
N-DA and DA are the two types of assessments. N-DA tests, from a Vygotskian perspective, cannot present a complete picture of the abilities they assess because such tests can only address fully internalized abilities. They ignore another significant component of the learners' abilities, namely, the abilities that are partially internalized. Both of them are said to be addressed by DA, and the former can also be explained by N-DA psychometric testing. One of the main objectives of DA is to improve the student’s performance by providing them with teaching assistance through assessment activities (Chen et al., 2022).
According to Andujar (2020), while assessment in a traditional or static approach is a type of activity for gathering information, the goal of DA is to gradually and smoothly move the students from their zone of proximal development – their state of knowledge – to the designated learning goal, or their zone of actual development. DA is the interactive form of assessment that is typically administered using the "test-teach-retest" method (Haywood & Lidz, 2007). Considering the key DA formats and the relevant study results, it was found that different researchers had different ideas about how to conduct DA (Elliott, 2003). However, they all have in common the opportunity for pupils to learn and the incorporation of education and feedback.
Web-based Dynamic Assessment
With the emergence of electronically delivering mediation, learners and scholars were drawn to C-DA to think about how technological mediation might affect several dimensions of language knowledge and the learners' development. It is common knowledge that the increased usage of technology in high schools helps the students complete their academic assignments (Haßler et al., 2016), especially in the present global conditions that the Covid-19 pandemic has influenced all sectors of the educational system. Poehner and Lantolf (2013) developed a computer software for C-DA tests where language learners can focus on listening and reading comprehension rather than production. This process generates three scores: actual, mediated, and a learning potential score. The authors can estimate the investment needed for the required promotion of future instructional activities by studying the learners' answers to computerized mediational cues. Tzuriel and Shamir (2002) created a C-DA approach for assessing the young students’ seriation thinking abilities, which is regarded as a domain that has been connected to mathematical achievement. Through "human-computer collaboration," which involves the computer containing a collection of clues structured in developing descriptiveness, the mediational part of the process mixes the interactionist and interventionist forms of DA, whereas the examiner is also able to talk with the learners, offering extra help that is more closely connected to their needs. Tzuriel and Shamir (2002) compared the improvements produced by the students who received just human guidance with those made by pupils who had access to human and computer-mediated mediations. They found that the students who received both sorts of treatments outperformed the other group.
Autonomy and Online Learning
To shift from a teacher-centered classroom to a more learner-centered one, researchers and educators are paying more attention to autonomy (Benson, 2013). According to Lazorak et al. (2021), autonomous learning entails using one's own initiative to engage in learning, locate chances and resources for learning, be persistent in learning, and be innovative. Learning is no longer seen as a passive process of receiving information; rather, it is seen as an active interpretation and processing of knowledge that students actively seek out depending on their own interests and needs. According to Madjar et al. (2013), learners' autonomy-supportive environments help them embrace online learning that results in greater academic success. The expanding potential of new technology was examined by Cole and Vanderplank (2016) and Sockett (2014) to see how it facilitated student autonomy and enhanced English proficiency. They discovered that it made it possible for a sizable number of unstructured, independent language learners to become very proficient in foreign language environments. According to Lui (2015), autonomy is a highly special variable since it entails students’ taking ownership of their education. Intrapersonal initiative, interpersonal cooperation, and learner-centered instruction all promote learner autonomy.
Review of Empirical Studies
Some empirical studies have addressed the field of computer-based or web-based DA. For instance, L2 English language learners were divided into three instructional groups by Klçkaya (2015). The three groups were computer-based, computer-based with some teacher-led instruction, and teacher-driven. The outcomes showed that the first two groups did better than the third. In another study, Ahmadi and Besharati (2017) attempted to compare the application of DA in face-to-face versus web-based modes of delivering mediation to assist the university EFL learners in writing argumentative essays. The results indicated that the two modes of delivering mediation had no significant differences in the writing ability. Merging the quantitative and qualitative analyses, the study found that the interactionist DA, in both modes, could have positive effects on the learners’ ability to write better argumentative essays.
The impact of C-DA on TOEFL IBT reading was studied by Hidri and Fekri (2020). They used question types as hints in their investigation, and the results showed statistically significant differences between the real and mediated scores with different reading levels. Malmir and Mazloom (2021) examined the effect of two DA models, namely GDA and C-DA, on Iranian EFL Learners’ pragmatic comprehension accuracy and speed. The researchers randomly assigned 52 upper-intermediate female EFL participants to three groups: (a) a dynamic assessment experimental group (GDA), (b) a C-DA, and an N-DA control group. The results of data analysis showed that the C-DA and GDA groups outperformed the N-DA group in terms of pragmatic comprehension accuracy. Furthermore, the results of C-DA group were much better than the GDA.
In another study, Jaeho Jeon (2021) examined the impact of Chatbot-Assisted Dynamic Assessment (CA-DA) on vocabulary learning and provided insights into the abilities of students as a result of its use. The EFL primary school pupils were randomly divided into three groups: The group, the control group, and the CA-NDA group. Over the course of treatment, the language learners were expected to read texts and determine the meaning of the underlined target words. The chatbots provided progressive assistance to the CA-DA group members but only focused on teaching the word meanings to the CA-NDA group. The findings suggested that CA-DA could not only promote vocabulary growth, but also offered analytical data on each student's vocabulary knowledge. This research also manifested the ability of Chatbot technology to assist the students in the process of learning.
Rassaei (2021) took into consideration the use of mobile-mediated DA in the way request forms could be taught to EFL learners, and the results showed that mobile-mediated DA increased the learners' knowledge in applying the request strategies. In a more recent study, Vakili and Ebadi (2022) examined the role of face-to-face (FTF) and computer-mediated (CM) contexts on the ways Iranian EFL learners dealt with their main developmental errors. The results highlighted the different effects of FTF and CM contexts on the learners’ pace, depth and permanency of development through not only employing different mediation forms, but also encouraging the usage of different mediational tools. The findings indicated that while FTF mediation led to an inclination toward collaborative writing, CM mediation increased the learners’ engagement with their written texts by focusing more on difficult items and transferring development to other tasks and situations which, in turn, illustrated their progression towards self-regulation.
Recently, some studies have been done on many areas of affective and social components of C-DA (e.g., Alavi, et al., 2020; Bahrami & Rahemi, 2022; Estaji & Saeedian, 2020) to name a few, nevertheless, there is a research gap regarding another mode of DA, such as web-based DA, to see its impact on grammar precision, autonomy, and the attitudes of high school EFL learners and move them from their comfort zone and leads to autonomous level, holistically. Therefore, any research or study that can close this gap is of utmost significance.
PURPOSE OF THE STUDY
Despite the bulk of research in the literature, the effects of web-based DA on grammar precision, autonomy, and the attitudes of high school EFL learners has not been addressed in the previous studies. The aim of the current research was three-folds: Firstly, it aimed to find out the effects of web-based DA on Iranian high school L2 learners’ grammatical accuracy. Secondly, the role of this strategy on the participants’ autonomy level was probed. Thirdly, the study was an attempt to explore the attitudes of EFL students in the web-based DA group to deeply investigate the influence of the treatment, its merits, and demerits. The major novelty of the current study, besides the paucity of the studies, is using a web-based platform in a study with different instruments to triangulate the research and increase its validity. Using films, pictures, sounds, text, and quizzes, in web-based DA for learners with different learning styles seem to be significant in L2 achievement. The following research questions were thus addressed:
- What is the effect of web-based DA on the grammar accuracy of high school EFL learners?
- What is the effect of web-based DA on the autonomy of high school EFL learners?
- What are the attitudes of high school EFL learners in the web-based DA on language achievement?
METHOD
Design
This study used an explanatory sequential mixed methods design located within a web-based DA framework. Creswell (2018) asserts that in explanatory sequential mixed-methods, researchers build on the findings of quantitative research and analysis to provide a more thorough explanation of them using qualitative research. After persuading these aims, the attitude of EFL learners toward the web-based DA treatment in Iran’s high school context was analyzed. Additionally, as the samples were chosen using a non-random selection procedure, the quantitative part of this study should be viewed as a quasi-experimental one. The use of web-based DA served as the independent variable, and the students' attitude, autonomy, and grammar proficiency served as the dependent variables.
Participants
A convenient sample of 63 male EFL high school students in the 11th grade who had been selected from a pool of 300 students in all grades participated in this study. Sixty students from two intact courses were divided into two groups: experimental (n = 30) and control (n = 30), and three students were considered outliers. The pupils, whose ages varied from 15 to 16, had intermediate English proficiency as determined by the Nelson test. They spent five years in school studying English as a foreign language. The participants were from Naghade, West Azerbaijan, Iran, and Azerbaijani was their native language. Their English language teacher, a TEFL PhD candidate, took part in the study by providing the necessary one-by-one explanations of the study’s objectives and methods.
Instruments
The instruments utilized in this study were the learners’ autonomy questionnaire, developed by Zhang and Li (2004) to assess learners' autonomy, the Nelson proficiency test, and pre- and post-tests (both productive and receptive) that were created to assess grammatical accuracy.
Nelson Test
One of the tools utilized in this study to identify homogeneous pupils in both groups was the Nelson test. Two weeks before the start of the trial, the exam was conducted. The exams in the book "Nelson English Language Tests" (Fowler & Coe, 1976) range in difficulty from beginner to expert. Book 1 (Elementary), Book 2 (Intermediate), and Book 3 make up the three sections of the book (Advanced). There are 50 multiple-choice questions in each test. In this study, the test 200 B from Book 2 was chosen (Intermediate). Grammar (in two parts), vocabulary, and reading comprehension made up the test's four sections. Based on the results of the proficiency test, 60 EFL students (out of a total of 63) were chosen as the study's final participants and divided into two groups. Their Nelson test scores were one standard deviation above and below the mean. Thirty pupils were thus assigned into each group. It is worth noting that since the Nelson test is a standard test, its psychometric is ensured. However, the estimated reliability for this test in the context of the study was 0.81.
Grammar Pretest and Posttest
The other instrument used, besides the Nelson test as the homogeneity test, was the pretest in grammar which was implemented before the onset of the treatment. The third instrument for gathering the data was a post-test in grammar, which was used to assess how well the study's research methods produced grammatical points. To ensure that the findings could be compared, the pre-test and post-test had the same structure and subject matter. In this study, the focus was on the grammatical points and the textbook contained different grammatical points, such as gerunds, infinitives, and conditional type one. The total number of items in the tests was 40, and the score was out of 20. The allocated time for the tests was 60 minutes. Given the significance of validity, the tests were professionally validated by three high school instructors. Cronbach's alpha was used to measure the test's reliability after piloting the test by 15 high school students from the same population, and it was found to be.83.
Autonomy Questionnaire
The questionnaire created by Zhang and Li (2004) served as the study's additional tool. It contained 21 items on a Likert scale of 1 to 5. There are five possibilities for the first 11 items of the scale ranging from never to always. The second section is a multiple-choice, and the participants were instructed to select the response that best matched their beliefs out of a range of 1 (1 point) to 5 (5 points). The estimated reliability for this instrument, using Cronbach’s alpha formula, for the context of the study was 0.86.
Semi-structured Interview
The use of web-based DA methods in language lessons was investigated from the participants' viewpoints, looking at both benefits and drawbacks, using a semi-structured interview as the other data collection tool. According to Richards and Schmidt (2010), a structured interview is one in which the format, logistics, subjects to be covered, questions to be asked, and the sequence in which they are answered have been all decided upon in advance. A semi-structured interview, which is exploratory in character and has no set format, might be opposed with this. To put it another way, the researcher asked the experimental group to discuss what had happened in their learning by expressing their opinions on the use of web-based DA in the classroom.
It can also be seen as an open-ended strategy for gathering data because it allows the respondents to freely and frankly express their thoughts on the questions the interviewer gives to them. The participants were asked to think back on their overall impression of the lesson, including whether they enjoyed it and would suggest it to others. They had to provide justifications for their responses as well. For the sections that follow, the interviews were audio-recorded, and the researcher transcribed the responses.
Low-level learners in the current study were given the Farsi translation of the test questions. To aid the interviewees in understanding the questions, the investigator translated them into Farsi. The respondents then responded in Farsi, and their remarks were later translated into English and included in the study's qualitative data. The researcher gave the interviewers the data they had provided, asked them to pay attention to each question, and then asked them to self-report their responses to see if there were any issues or discrepancies. This was done to determine the reliability of the interview questions. The participants agreed that the data were accurate, and, as a result, the interview's reliability was confirmed. Two of the researcher's colleagues who were familiar with the data analysis section double-checked 20% of the interview findings in order to ensure dependability, and the results of the inter-rater reliability were reported to be .92.
Procedure
Two weeks prior to the treatment, the Nelson test was given to 63 high school students to ensure the homogeneity of the students. Based on the findings of the homogeneity test, a sample of 60 students was chosen. One week prior to the start of the treatment, the researcher administered the Nelson test, followed by the autonomy surveys and the grammar pretest to the groups of the students. Following the phases, the treatment began, and the researcher served as the experimental group's instructor, providing the necessary explanations regarding the web-based training and assessment program, the study's objectives, and its methods one at a time.
The procedure for web-based DA classroom as treatment group represented in two facets of instruction and assessment: The content of each lesson in web-based DA was developed in the forms of visual and auditory learning style with IP of http://www.drfathii.com/vision. By web-based platform, we mean the program that has been designed by a computer and software engineer with IP of http://www.drfathii.com/vision. In the first step of instruction, each student entered the main page and did the tasks previously planned by the teacher. In the instruction phase of the DA group, the learners first watched the instructional films, read the grammar points and listened to the podcasts. In the assessment phase, the learners received preplanned quizzes and tests which were provided with hints. The phases of instruction and assessment and receiving hints are illustrated in figures 1-2.
Figure1: The instruction phase for the DA group
Figure 2: The assessment phase of the DA group
The most important part of the platform was the last image that contained filling in the blank’s questions with 4 hints (Figure 2). If the student could not do the task, he would receive the first hint in the form of feedback, but the score was dedicated by the program. Giving hints continued for four times if the student could not do the exercise and the scores were reduced as the students received more hints. As it is clear, teaching was not done by the teacher, but by the program. The students were needed to diagnose selected grammatical points on the web which were implicitly taught by the visual, auditory learning style in forms of watching films, reading text and listening to podcasts. After receiving enough instruction, the students were given the immediate post-test (true/false or multiple-choice test) by the website to check their understanding of the selected grammatical points.
In the process of answering the test, the students who could answer the question, could go to the next level to answer another question and those who could not diagnose the grammatical points, enough hints in forms of feedback were given implicitly.
The obtained scores were of the mediated type, not the real scores which are given without any clues. In fact, in this method of teaching (web-based DA), the aspects of teaching and testing were integrated at the same time in the process of learning and led the learners to an autonomous level. The existence of visual support and movies were considered as hints that is the key feature of DA.
After three sessions for each grammar point, a quiz was administrated to the learners in the web-based DA group and during the test, they took the hints via computer in order to find the correct answer. The quizzes were in both productive and diagnostic format. It is worth noting that these classroom quizzes were scored and rated by the site, but the scores were not analyzed for the final data analysis section. During the assessment phase of DA, the learners received preplanned quizzes and tests which contained hints.
In the control group, the learners did not receive the assessment facet of DA. They were required to practice the same grammar points in visual, auditory learning style without quizzes and hints. There was no concurrent teaching and assessment while the student was studying because the detailed explanation of the grammar was provided online. The primary duties of the learners in the control group were to memorize and apply the grammar, while the primary role of the teacher was to explicitly teach the grammar online. The 12-session course of the treatment ended with the administration of the autonomy questionnaire and the grammar posttest. Independent samples t-tests were used to describe the findings of the data analysis in terms of grammar learning. Non-parametric Mann-Whitney tests were used to analyze these three tests since the ratios of skewness and kurtosis over their standard errors were larger than +/- 1.96 in order to publish the results of the autonomy pretest and posttest.
RESULTS
The first research question looked into whether online DA had any impact on the learners' grammar proficiency. The pretest results showed that there was no statistically significant difference between the high school students in the web-based DA group and the control group at the beginning of the study, demonstrating that the language learners in both groups had the same proficiency in grammar. The study groups were given the identical grammar test following the treatment. The results of the researcher-made post-test are shown in Table 1.
Table 1: Descriptive Statistics of Grammar Tests (Pretest and Posttest)
|
N |
Minimum |
Maximum |
Mean |
SD |
Pretest web based |
30 |
6.00 |
9.00 |
6.24 |
1.554 |
Posttest web based |
30 |
11.00 |
19.00 |
16.21 |
1.615 |
Pretest control |
30 |
5.00 |
8.00 |
5.98 |
1.543 |
Posttest control |
30 |
10.00 |
14.00 |
11.07 |
1.634 |
According to the descriptive results, there is no difference between the mean scores of the groups on the grammar pretest. A grammar posttest was administered following the treatment. The results showed that although the mean for the control group was 11.07, the mean for the students taking the grammar posttest in the web-based assessment group was 16.2. The table shows how obviously different the group means were. The parametric test assumption had to be tested since the differences between them needed to be statistically checked. One of the presumptions was that the data should be normally distributed. Table 2 shows the results of normal distribution of pretest and posttest in grammar.
Table 2: Kolmogorov-Smirnov Tests of Normality in Grammar
|
Kolmogorov-Smirnov |
Shapiro-Wilk |
||||
Statistic |
Df |
Sig. |
Statistic |
df |
Sig. |
|
Pretest Grammar |
0.147 |
30 |
0.200 |
0.746 |
30 |
0.548 |
Posttest Grammar |
0.142 |
30 |
0.200 |
0.744 |
30 |
0.546 |
The results of normality showed that the scores of both pretest, W (30) = .74, p > .05, and posttest of grammar, W (30) = .74, p > .05 were normally distributed. In this study, the pre-test of grammar was as covariate and the post-test was the dependent variable. To see whether there was a difference between the two groups, an ANCOVA was run since via ANCOVA, it is possible to eliminate the possible effect of covariate (i.e., the pre-test scores in this study) on the interpretation of the participants’ post-test scores. Furthermore, according to Dörnyei (2007), in quasi-experimental studies, the use of ANCOVA contributes to the reduction of the initial group differences. Based on the results, it was revealed that the assumptions of ANCOVA such as the assumption of homogeneity of variance was met. The results of ANCOVA are illustrated in Table 3.
Table 3: Analysis of Co-Variance in Grammar Test
|
Type III Sum of Squares |
df |
Mean Square |
F |
Sig. |
Partial Eta Squared |
Corrected Model |
270.933a |
2 |
135.621 |
711.234 |
.000 |
.944 |
Intercept |
109.947 |
1 |
109.795 |
613.044 |
.000 |
.912 |
Pretest |
77.332 |
1 |
77.311 |
437.611 |
.000 |
.904 |
Group |
199.146 |
1 |
199.220 |
1242.866 |
.000 |
.948 |
Error |
6.119 |
55 |
.179 |
|
|
|
Total |
3100.000 |
60 |
|
|
|
|
Corrected Total |
246.600 |
59 |
|
|
|
|
- R Squared = .933 (Adjusted R Squared = .924)
Table 3 shows that there is a significant difference between the two groups (F = 1242.866, df = 1, p < .001). It means that the learners in the treatment group performed better than the control group in grammar accuracy. Based on the results of ANCOVA, the web-based DA group outperformed the control or traditional grammar instruction group.
To answer the second research question, which aims to investigate the effect of web-based DA on EFL learners' autonomy both before and after treatment, a set of data analysis tests were used. Table 4 shows the normality test of the autonomy pretest and posttest.
Table 4: Kolmogorov-Smirnov Tests of Normality in Autonomy
|
Kolmogorov-Smirnov |
Shapiro-Wilk |
||||
Statistic |
Df |
Sig. |
Statistic |
df |
Sig. |
|
Pretest Autonomy |
0.123 |
30 |
0.200 |
0.043 |
30 |
0.025 |
Posttest Autonomy |
0.126 |
30 |
0.200 |
0.049 |
30 |
0.027 |
The results of normality showed that the scores of the pretest, W (30) = .04, p Ë‚ .05, and posttest of autonomy, W (30) = .04, p Ë‚ .05 were not normally distributed. Therefore, a Mann-Whitney test, as a non-parametric test was used. The median scores on the pretest of autonomy were compared between the experimental and control groups using a Mann-Whitney test because the assumption of normality was not maintained. According to the findings, the experimental group's median on the pretest for autonomy was 59.50, while the control group's median was 60 (Table 5).
Table 5: Mean Ranks; Pretest of Autonomy by Groups
Group |
N |
Mean Rank |
Sum of Ranks |
Median |
Experimental |
30 |
28.82 |
864.50 |
59.50 |
Control |
30 |
32.18 |
965.50 |
60.00 |
Total |
60 |
|
|
|
According to the Mann-Whitney test results given in Table 6, (Z = -.753, p >.05), there was no statistically significant difference between the median scores of the two research participants' groups on the pre-test of autonomy. This finding demonstrates that at the beginning of the study and before the treatment procedures, the members of the experimental group and the control group had the same level of autonomy.
Table 6: Mann-Whitney Test; Pretest of Autonomy by Groups
|
Score |
Mann-Whitney U |
399.500 |
Wilcoxon W |
864.500 |
Z |
-.753 |
Sig. (2-tailed) |
.451 |
A Mann-Whitney test was conducted to compare the two groups' post-test median scores on autonomy. The results of the autonomy post-test provided in Table 7 showed that the experimental group's language learners had higher levels of autonomy than those in the control group (M = 76.50 > 61.50).
Table 7: Mean Ranks; Posttest of Autonomy by Groups
Group |
N |
Mean Rank |
Sum of Ranks |
Median |
Experimental |
30 |
45.27 |
1358.00 |
76.50 |
Control |
30 |
15.73 |
472.00 |
61.50 |
Total |
60 |
|
|
|
The results of the Mann-Whitney tests (Z = -5.66, p < .05) (Table 8) indicated a rejection for the second null hypothesis because, in the autonomy post-test, the experimental group members gained better scores compared with the language learners within the control group.
Table 8: Mann-Whitney Test; Posttest of Autonomy by Groups
|
Score |
Mann-Whitney U |
7.00 |
Wilcoxon W |
472.00 |
Z |
-6.55 |
Sig. (2-tailed) |
.000 |
Qualitative Findings
The participant students uttered their viewpoints regarding the web-bases DA’s advantages and disadvantages to analyze the collected qualitative data, the phenomenological procedures recommended by Moustakas (1994) were employed in data analysis and reporting. First of all, the interview texts were read several times by the lead author in an attempt to make sense of the data and the interviews were transcribed word by word into written text to achieve familiarity with the data. To aim data familiarity, interview transcripts were read by the first and second authors independently and relevant or important statements were highlighted.
Since content analysis is a common method to analyze textual data (Berg, 2001), the interview data were analyzed using content and thematic analyses following Gao and Zhang’s (2020) steps of analyzing qualitative data. First, the data were read several times to ensure accuracy. Second, the open codes were generated. Third, the open codes were compared and grouped under axial codes. Fourth, the axial themes were classified under selective codes or dominant terms. Finally, the analyzed data were reported in detail. To ensure the confirmability of the analyses, sub-sample of the entire list of codes and themes were presented to a second coder to obtain inter-coder agreement coefficient. A high percentage (96%) was found, and further disagreements were negotiated.
The first question of the interview was to discover the learners’ points of view on the plus points of the web-based DA strategy. According to the interview results, most of the students in the treatment group preferred this type of instruction to the traditional classroom. They suggested that the strategy could be used in other English-related courses. The students expressed their points of view in the sentences. For instance, one of the students pointed out the role of the computer as a teacher in controlling the students’ stress as a result of assistance or mediation and stated that:
Extract 1: The role of computer that played the role of a teacher was noticeable among the students, and it helped everybody to adapt the stress and remove the anxiety.
The other plus point was its novelty:
Extract 2: It was definitely good because it was new experience in our city. Of course, we cannot limit internet to the city but I want to say in our country haven't seen so many sites like this web-site
Most of the students embraced the experience with open arms because of the innovation it presents and novelty it bears. Concerning the students' age group, utilizing modern technology for the purpose of teaching and learning will have pedagogical consequences. Another student reported the web-based DA’s good order and uttered it in the following fashion:
Extract 3: I can mention some of the advantages of the newly used mechanism for testing: it was good ordered. It provided me motivation to go forward
Another participant pointed out the factor of self-control as the other plus point:
Extract 4: One big advantage of the website was that I had control over my own performance. I was not under pressure to do anything. Everything was in my own control.
Moreover, the fun nature of the strategy and its active participation nature were the other merits.
Extract 5: The web-based DA that I did not experience so far was amazing and motivating since I received materials before the class and could study them.
Extract 6: Active participation and happiness in the class context increased my motivation and decreased my anxiety level.
The second question of the interview was on the demerits of the strategy under study. Most of the students believed that the merits of the strategy exceeded its downsides; however, they reported some of them such as unavailability of teacher that resulted in increased anxiety.
Extract 7: Although this strategy was full of fun, actually, the teacher was not always available. This fact made me stressed.
The ignorance of errors by the teacher (not the computer) and detailed instruction on limited grammatical roles were two key terms that the participants mentioned as the disadvantages of the strategy.
Extract 8: The teaching procedures were both new and different from what we had learned before, however I was baffled by the ignorance of errors by the teacher.
The third question of the interview aimed to discover the attitudes of the participants on the role of web-based DA on the students’ autonomy. Most of the students reported the attractive nature of the strategy that helped the learners overcome the difficulties of the grammar sub-skill and the repetition and high practice resulted in learners’ autonomy levels.
Extract 9: Most of the grammatical forms were practiced that this high practice resulted in grammar accuracy and high autonomy.
Extract 10: Computer-based feedback attracted my attention to the errors and resulted in high motivation to learn more and more about the target forms via the web and computer, this resulted in autonomy.
As it is axiomatic from the above extracts, the interviewees reported on the strategy's merits (fun nature, decrease of anxiety, independence, and autonomy) and demerits, such as being time-consuming, and the unavailability of the instructor as the important downsides. However, the course motivated them as it was attracting due to the high practice and mediated feedback.
The participants also pointed out the dynamic aspect of web-based instruction in which instruction and assessment are integrated in a way to access the untapped language potential of the learner. The hints bridge the “gaps” between the current language skills of the students and their desired language ability, so they can take complete control of their learning without the teacher help. This web-based platform gives them an opportunity to profit from online instruction and assessment to go back and watch, listen, read and test them over again. For further reference, the summary of the themes and sub-themes obtained from the 10 interviewees’ answers are presented in Table 9.
Table 9: Summary of themes and sub-themes obtained from the interviews
Theme |
Sub-Theme |
Frequency |
Advantages |
Interesting |
8 |
Decreasing stress/anxiety |
7 |
|
Motivating |
10 |
|
Help self-regulation/Autonomy |
6 |
|
Room for prior preparation |
4 |
|
Active participation |
4 |
|
Disadvantages |
Unavailability of teacher |
6 |
Lack of direct instruction |
7 |
|
Low support/feedback from teacher |
7 |
|
Time Consumption |
4 |
|
Role in Improving Autonomy |
High practice/repetition makes it possible |
8 |
Attracting the attention makes it possible |
7 |
|
Accessibility of the materials helps boosting it |
5 |
DISCUSSION
The purpose of the current research project was to develop a web-based DA for grammar instruction, which, at first, could serve the Iranian high school EFL students develop their understanding of grammatical features. It also used as a diagnosis and assessment tool. Second, it enhances the autonomy and self-reliance of the students, and, finally, it could check the students’ attitude toward web-based DA.
According to the findings, higher scores were obtained by the web-based DA group on the delayed posttest. Web-based DA was more effective than the N-DA as evidenced by the statistically significant level for the results of grammar accuracy. The ZPD-sensitive grammar hints and feedbacks provided by the web platform could move the students progressively from actual zone to learning zone. In other words, in supportive dynamic environment with the mediation of web, the students could smoothly move from unknown to known regarding grammar achievement. As a result of their participation in web-based scaffolding and mediation, the learners' mental grammar improved.
The ZPD represents a present-to-future approach to development (Valsiner, 2001) as it is indicative of what a learner will be able to do autonomously tomorrow based on what he or she is able to do today with assistance or mediation. In contrast, "static" evaluations follow a pattern from the past to the present, indicating current development rather than potential development. It will also be expected that the learner’s responsibility and autonomy increase because hints and prompts ranged from implicit to explicit, and the learners themselves were responsible for distinguishing the grammatically correct points. As a result, as for the effect of web-based DA on Iranian EFL learners' level of autonomy, the results revealed that the learners in the web-based treatment group outperformed the learners in the control group.
Additionally, interviews were held to examine the usefulness of the program to identify the beneficial effects of web-based DA education approaches in the enhancement of grammar accuracy. The majority of the students in the treatment group agreed that these teaching methods were efficient enough to meet their demands while also hastening and deepening their learning. They picked things up more quickly because they were more at ease and confident. Furthermore, their autonomy could increase because the web plays the role of a teacher; it gives the essential hints for reaching the answers. With positive views regarding autonomous learning, language learners who study online develop greater accountability, intrinsic motivation, and interpretive openness. Nonetheless, it should be pointed out that despite the pupils' positive feedback in the subsequent years, digital tools, such as web-based educational instruments can be effective with motivated teachers. These teachers should be in charge of creating suitable web-based programs and being concerned about students' progress.
These results are consistent with the results of previous studies that found that skills and sub-skills can be accomplished better in computer dynamic assessment environments, as they are much more useful in combination with non-dynamic enhanced learning environments (e.g., Ebadi & Saeedian, 2015, 2016, 2019; Estaji & Forough Ameri, 2020; Hidri & Fekri, 2020; Åžendurur & Yildirim, 2018; Vakili & Ebadi, 2022). The outcomes are consistent with the research conducted by Bhattacharya and Chauhan (2010), who used Web technology to increase the students' autonomy when learning English. The major distinguishing feature of web-based DA over traditional DA is that, it applies to a larger number of learners at the same time and the result of it, can be reported immediately for interpretation. Another feature of web-based DA is that it can make up the time-consuming nature of traditional DA in a fast way. Furthermore, the hints provided by DA can act as a kind of suitable compensation for the language learners' lack of background knowledge, which leads to make up the language learners' weak schema, and overcome their learning obstacles. Such hypotheses regarding the performance of the students with limited prior knowledge require more thorough empirical evidence. They showed more precise and active responses to hints and tips. They tried to improve their wrong answer in any item in the web-based DA environment. These suggestions act as teachers, advising and aiding learners with limited background knowledge to improve grammatical knowledge in instruction and assessment procedures. Hints allow the learners with a low-level schema to participate in self-directed and autonomous learning without explicitly supplying the correct solution, alternatively, forcing them to go through a step-by-step problem-solving process.
CONCLUSION
The main focus of this study was to help learners implicitly develop language abilities like grammar accuracy, to move them from their comfort zone, where they could not solve their grammar points by themselves, to the proximal development zone where they could deal with grammar challenges with scaffolding.
Some of the educational implications of the study are worth highlighting. The results of the current study have a number of implications for English learning and teaching procedures. Some implications of this study are relevant to EFL language instructors that, instead of utilizing traditional instructional methods and evaluation which are teaching at one time and evaluating at another time, they can integrate instruction and assessment in favor of learning at the same time. Teacher educators can make suggestions to teachers in the teacher training course, in implementing strategies of web-based DA within the classrooms, hoping to have more encouraged teachers who pay much more attention to this strategy in line with other training interventions. Understanding the interactive and corrective features of DA which combines instruction and assessment in new learning settings can be extremely valuable for teachers and learners. It would aid in promoting the students' self-control and making them more independent and in charge of their own education in self-study. If Web-based DA strategies are used at the appropriate level, they can help the learners recognize, engage, or develop effective ways of promoting their language abilities. Additionally, it is advised to employ the idea of "assessment as a teaching and learning technique" that was created in this study when creating more effective instructional activities for students.
Like all other studies, this study also had some shortcomings that require further investigation. The study involved only a convenient sample of 60 male Grade 11 EFL high school students and only the grammar of the 11th English book was studied. Other factors such as other levels, female students or other grammar points were not studied. Further study could, therefore, investigate the effects of DA for other levels and contexts using larger sample size and more generalizable sampling methods. Additionally, the effects of DA were not studied over time, so future researchers could look at the reliability of the outcomes across a longer time. Moreover, this study suffers from other limitations, such as using researcher-made grammar test. The future body of research can study web-based DA strategy with the other English skills and sub-skills, such as vocabulary and reading comprehension using more standardized tests.
Disclosure statement
No potential conflict of interest was reported by the authors.
ORCID
Abolfazl Fathi |
||
Morteza Aslrasouli |
||
Davud Kuhi |