Web Tracking and Students' Work Patterns in Online Language-learning Activities

Alysse Weinberg
Institut des langues secondes
University of Ottawa

Abstract:

In previous publications the author reported on the use of a website dedicated to French songs for an advanced French comprehension course at the university level. The present article analyzes web-tracking data captured from the website over two terms--fall 2003 and winter 2004. The variations in the amount of time students spent working on the online activities and their online behavior as they navigated through the Chansons de la francophonie web activities are presented. Research queries are made into the data to explore web behavior differences between day and evening students, male and female students, and especially diligent versus less diligent students. The website, the course, the students, the characteristics of the eChanson web-tracking system, and a few typical web navigation patterns for some students are also described. One result of this study shows that it is not necessarily the students with the lowest marks who spend the least amount of time on web activities (nor vice versa), nor do the most diligent workers on the web activities achieve the highest measured skill improvements.

KEYWORDS

Student Usage, Navigation, Website, Tracking, French, Gender Differences

INTRODUCTION

As computer-based learning activities become more integrated in the classroom, more tracking functions are becoming available to teachers. Tracking functions provide lists of web pages accessed by each student along with the date, time, and duration of the page visit. Often these tracking results are not looked at by the teachers because of the difficulty of extracting useful information from them. The accumulated amount of data can also become overwhelming (Renié, 2000).

However, it is possible to answer interesting research questions by attacking this bewildering tide of data. In this paper, we will examine web activity data records to investigate the following questions:

1. Do day and evening students work differently?

2. How much more time do hard-working students put into their web activities compared to less diligent students?1

3. Does the extra work done by the more diligent students translate into greater improvement between the pre- and posttests?

4. Are there differences in the web work patterns for male and female students?

5. What are typical web work patterns for more diligent and less diligent students?

0x01 graphic

31

This article begins with a description of the Les Chansons de la francophonie website. It then presents the context of the study, an advanced French comprehension course at the Second Language Institute of the University of Ottawa and the students using the website in the course over two terms: fall (September to December, 2003) and winter (January to April, 2004). The features and limitations of the eChanson web tracking system are explained. Finally the web-tracking data captured from the website is analyzed and results presented.

PREVIOUS STUDIES

As students work more and more with CALL activities many researchers have stressed the importance of observing their work patterns (Chapelle & Mizuno, 1989; Chapelle, 2001).

Research in the area of tracking students' navigation patterns and time spent on multimedia activities has been done by Canter, Rivers and Storrs (1985), Stanton and Baber (1992), Beasley and Vila (1992), Horney (1993), Orey and Nelson (1994), Recker (1994), Beasley and Waugh (1997). Different typologies were proposed to describe the navigation patterns used by students. Canter et al. established five global navigation strategies: scanning, browsing, searching, exploring, and wandering. Other researchers noted different navigation patterns such as linear versus nonlinear, top down, left to right, or number of accesses to a specific screen. The preferred navigational pattern depends on the multimedia environment used for the experiment. Much of the work on early tracking systems was done mainly on CD-ROM-based multimedia activities.

Duquette and Dionne (2000), Desmarais and Laurier (2000), and Renié (2000) described the tracking programs they used in their experiments based on the Vi-Conte laser disc and the Camille CD-ROM as well as their collection of huge amounts of data and the difficulties they encountered while analyzing it.

There has been less research done to track web-based activities which have less well defined boundaries compared with similar CD-ROM activities (i.e., students can get lost in cyberspace). Hwu (2003) studied learners' behaviors specifically in web-based activities. Students were to learn the use of two Spanish past tense forms with web activities installed on the WebCT platform. In this project, the students' knowledge gain correlated strongly with certain aspects of their work pattern: their use of explanation pages and the ratio of hits between video and explanation pages.

In a previous study, Weinberg (2005) compared two web-usage-tracking systems: the commercial product WebCT web-tracking system and a specially developed eChanson web-tracking system. She was able to determine that while students enjoyed using multimedia computer systems, they actually spent little time working on them. The students preferred the environment provided by the eChanson tracking system. They found it less obtrusive and more stable with a friendlier interface than WebCT. For the professor, it was easier to retrieve data from the eChanson tracking system than that of WebCT.

Many studies have investigated gender differences in relation to computer use. The present paper only looks at gender differences in browsing and working patterns. Beasley and Vila (1992) analyzed 100 students' working patterns with hypertext according to linearity and nonlinearity. Among their conclusions, they indicated that women had a more linear navigation pattern and therefore did not explore the content of the environment as much as men. However, Desmarais and Laurier (2000) reached a different conclusion. In their research with 28 subjects, they concluded that in the Vi-Conte environment men had the more linear approach and finished activities that they started while women tended to stop and restart the

32

activities later. They did not claim, however, that linearity is a characteristically male navigation pattern. Vila, Beccue, and Anandikar (2003) made an analysis of virtual reality navigation based on an experiment with 68 students. Gender influenced the tendency to turn left or right while navigating through a maze but did not influence the time spent on task or number of rooms visited.

THE CONTEXT OF THIS STUDY

Les Chansons de la francophonie Website

The website Les Chansons de la francophonie used French songs as stimulus material and was developed in HTML and JavaScript for the advanced French comprehension course at the Second Language Institute of the University of Ottawa. These interactive listening activities included true/false and multiple-choice questions, fill in the blank exercises, paragraph writing, and reading comprehension based on external web links related to the themes of the songs. The site presented 10 songs but only four were used during the course and thus for this experiment. The four songs were Venise by the group Duo Absinthe, Au bistrot d'en haut by Pierre et Vincent, La Manic sung by Bruno Pelletier and L'Acadie by Jean Rabouin. The first two songs came from France, and the second two songs were from Quebec.

The structure of the Chanson website followed the model proposed for listening comprehension by Mendelsohn (1994, 1995). For each song module there were three levels of activities: prelistening, listening, postlistening. The prelistening activities familiarized the students with the theme of the song and its singer. During the listening activities, students listened to the song and then worked on three or four different activities. The objective of the postlistening section was to reintegrate the material from the previous sections using a different skill, for example, preparing an oral presentation on a theme from the song or presenting a poem. In the pre-and postlistening sections students could navigate the web with external links. At all times students had access to an online dictionary.

Certain pages used a special server-side CGI file that automatically sent the web page results to the professor's email address when the students submitted their assignments. The email contained the answers or responses to all of the questions on the originating web page.

These activities were integrated into eChanson, the web tracking system that was specially developed for the Chanson website (see below and Weinberg, 2005).

The FLS 2522 Course

The data described in this study was collected from students' work in the fall 2003 and winter 2004 sessions of FLS 2522. FLS 2522 was the first of two advanced oral comprehension/expression courses offered by the Second Language Institute of the University of Ottawa. In this course students developed their listening and speaking skills and were systematically exposed to a variety of accents, registers, environments, text types, and delivery modes. The prerequisites for the course were that the students must either have passed the previous course or have taken the placement test of the Institute with a score of 60% or above. This course met twice a week for 1 hr and 30 min over a period of 13 weeks. One class per week was devoted to speaking and oral presentations while the other one was devoted to listening using authentic material coming from either videos or songs. The Chanson de la francophonie web activities were completed in weeks 2, 6, 9, and 11. The same professor taught four sections

33

of this course over two semesters: one during the day and one in the evening, both following the same outline. This course was a "hybrid" course in that some of the multimedia activities were done at home when students were working independently while others were done in the multimedia language laboratory with the language professor interacting with the students.

The Participants

Data from the fall 2003 and winter 2004 sessions are presented here. Coincidentally, both sessions had 31 students. In the four classes the majority of the students were women.

0x01 graphic

One student in the fall 2003 day section did no work and failed the course; consequently, this individual did not produce any tracking records. Another student in the winter 2004 day section came in towards the end of the course. Neither student is included in the table above, and both were dropped from the statistical analysis.

Most of the students in the fall 2003 session were in their 20s although there were 4 students over 30 years of age in the evening session. This was the first multimedia language course for all students. A questionnaire, to which 30 of the 31 students responded, showed that 3 did not have home computers and that only 20 had fast Internet connections. All the surveyed students were comfortable with computers. Eleven students had worked with French on the Internet before, and 19 had not.

For the winter 2004 session, there was also a total of 31 students in the two sections. One student was 39 years old, and the rest were between 18 to 25. All but one student had a home computer and 25 had fast internet connections. Two said they were not comfortable using computers. More than half of these students(57%) had already taken a previous language class with a multimedia component.

DATA COLLECTION AND INSTRUMENTS

Besides the web tracking there were other types of data collection used during the fall 2003 and winter 2004 sessions. These included language tests, surveys requesting user demographic information, and comments about the Chansons de la francophonie website.

At the beginning of the term, the students were given two tests to evaluate their linguistic proficiency. This proficiency test is used by the Second Language Institute of the

34

University of Ottawa and measures listening, reading, and general language proficiency at the advanced level. An analysis of variance was done using the Tukey-HSD. There was no significant difference found between the classes (p > .05), that is, all students on average were found to have a similar knowledge of French.

The students of the winter 2004 session took a dictation test as a pre- and posttest to investigate whether they had improved in their listening skills over the term. The dictation test used in other contexts has been found to be very useful in identifying advanced listeners; it has been used as a measure of advanced listening ability in many studies conducted at the University of Ottawa. It was first described by Hauptman, Wesche, and Ready (1988). The 112 word dictation test was read twice at a conversational rate and a third time in chunks. The sentences were long enough to challenge the examinee's short term memory. The dictation test was not marked for accuracy but rather for the use of words in correct order.

At the end of the term the students also completed a questionnaire requesting personal and demographic information and soliciting their feedback on the Chansons de la francophonie website and the web-tracking system.

The eChanson Web-tracking System

The eChanson web-tracking system was developed by the e-Learning Centre at the University of Ottawa. Using ASP files, the system tracks who was using a web page and records in an Access database the times the page was opened and closed. The Chansons de la francophonie website with the tracking system was accessible from anywhere on the Internet, and all students had to register with the eChanson tracking system before doing the Chanson activities.

Students logged into the system and were presented with the main menu of the web-based activities. The length of time students worked on any page was calculated automatically from the opening and closing time of each page. For every internal link that students clicked on, the system created a new record with the following data: student number, directory or song name, page name, start date, start time, end date, end time, and the time spent browsing. External links, links on the eChanson song pages to web pages external to the system, were also recorded. However, for technical reasons, the system could only capture the number of times that the students visited the external pages, but not the duration of the visits to those pages. Consequently, no analysis was done on external page visits.

Web-tracking Data Description

After the final grades had been submitted at the end of the course, the following table data was downloaded from a special professor-reserved site and used to create similar tables in Access.

1. Page_Hits: with directory, page_name, student_number, start_date, start_time, end_date, end_time, seconds_duration;

2. External_Page_Hits:with directory, page_name, student_number, start_date, start_time, number of visits, link_name, link_URL;

3. Pages: with directory, page_name, URL;

4. Students: with Student_number, course_code, last_name, first_name; and

5. Courses: with course_code, name, semester, year.

35

Pages were sorted into two types, working activity pages and navigational index pages. Analysis focused on the working activity pages. The navigational index and menu pages were not included for the most part. Significant activity pages were identified as those requiring substantive work and included all the homework pages and all the listening activities in the lab.

The data from all the sessions were cleaned and compiled into two large tables: 03_04_Page_Hits and 03_04_Students. Most of the data analysis was done from the 03_04_Page_Hits table which held the following columns: Page_directory, Page_name, Student_number, Start_date, Start_time, Number_seconds, Session_code (2003-Fall or 2004-Winter), Day_Evening (Day or Evening), Page_type (Activity or Index), End_date and End_time. This table had 6,305 data records: 3,101 from the fall 2003 and 3,204 from winter 2004 sessions. The page hits were divided into 1,889 activity page hits, 4,333 index page hits, and 83 hits on unassigned (and not counted) pages.

Limitation of Tracking: Data Cleaning and Validation

The large amount of data collected had to be manipulated and analyzed in sophisticated ways that required the help of a statistician and a data analysis expert. The data were uploaded to an Access database and filtered, sorted, and analyzed at this location. Some results were exported to Excel for further graphical display and presentation.

The fact that a web page was open did not mean the student was necessarily working on it. It was possible that a student just opened the page and walked away from it. The time recorded by the tracking system may not accurately reflect the time that the students were actually engaged in active work on the page. The records with low duration times tended to reflect more accurately that the student entered the page, considered it, and then chose to leave it. With page visits of longer duration times, there was greater uncertainty about the level of engagement and whether or not the student was working on the activity the whole time. In some cases the students had to go and print out a song, and this would confuse the time tracking. Hwu (2003) observed students not working on the page while the timer was ticking and also the contrary, students working while the timer was not ticking.

Extensive data cleaning was necessary which required a certain amount of judgment. The goal of the data cleaning was to remove any spurious data records that would adversely affect the statistics we were trying to employ. Records having extremely short or unrealistically long durations were deleted as were duplicate data lines. For example, one student seemed to spend over 5 hours on a certain page, and this record was deleted. There were 64 cases in which a student had two pages open at the same time, but only one of these two pages was used for the analysis. In an earlier analysis Weinberg (2003) found four instances of duplicate pages in 240 data lines. There were seven cases in which the long duration activity page seemed to be duplicated and to have the same start and end times. (The developer of the system left his test and development data in the same database as that of the production data.)

There were data records, mostly of index pages, having durations of zero seconds or one second. Also, some of the index pages had times associated with them of an hour or an hour and a half. Index page records with durations of zero, one second or over two minutes were deleted and not included in the calculations.

A researcher must examine tracking data very carefully for overlapping times and other data problems before analyzing the results. Automated routines to check for duration

36

overlaps and sorting can assist this process. Generally, despite the need for the careful cleaning, the quality of the data at the end was very good, and we ended with 6,305 records from the four classes giving page hit and duration information.

TRACKING RESULTS AND INTERPRETATION

This section presents the analysis of the variations between the amount of time students spent working online, correlations between web work and final grade, navigation patterns between index pages and content pages, differences in work habits by gender, and descriptions of some individual students.

Time Spent on Online Activities in 2003 and 2004

This section examines the varying amounts of time different students worked on their web activities.

Average number of hours between different classes

The graph below shows the average number of hours each student spent working on activity pages during the whole course. Time spent on index pages is excluded.

0x01 graphic

More time was spent doing online activities during 2004 than in 2003. On average, the 31 students of the fall 2003 sessions each spent about 3.1 hours doing activities, while

37

on average the 31 students of the winter 2004 sessions each spent 6.0 hours doing the same activities. The explanation for this is that at the end of the fall 2003 session the professor discovered that many students were not working very hard online. As a result, the professor decided to assign marks to some of the activities and required them to be submitted as online homework for the winter 2004 session. The amount of time spent on the web pages almost doubled.

Do day and evening students work differently? In 2003, the day students spent slightly more time online than the evening students that had the four mature students. (Mature here means over 30 years of age.) In 2004, the evening students, with only one mature student, spent more time working online. It is sometimes thought, and night school teachers occasionally say, that evening students work better than day time students. However, the results of the analysis of our data are ambiguous in this regard and do not allow us to generalize whether or not evening students work longer or harder than day students. Cunningham (2000) reached a similar conclusion, there was no difference in the class performance between day and evening students, even though they had different pressures and time constraints.

Variations in total time spent working on web activities

The chart in Figure 2 contains a data point for each student in the two groups. The data records are sorted between the least diligent students who spent the least amount of time working online and the most diligent students who spent the greatest amount of time working online.

0x01 graphic

Again, the line shows that the fall 2003 students, with fewer compulsory assignments to hand in did less work online than their winter 2004 counterparts who had more compulsory assignments.

38

How much more time do more diligent students put into their web activities than less diligent students? The figures in Table 2 show the amount of time the more diligent and less diligent students spent online.

0x01 graphic

The unadjusted statistics include all the students. The adjusted statistics do not include the top two best performing nor bottom two worst performing students. The purpose of providing the adjusted statistics is to provide a less biased estimate since the extreme cases may lead to a less representative estimate of the mean values. In this case, only the calculation of the ratio changes substantially in the two sets of calculations. The ratio of top-to-bottom students is a soft number and can swing wildly depending on how little work the student at the bottom does. Nevertheless, using the adjusted statistics of Table 2, and leaving out the best and worst performing students over the two samples, the pattern of use shows that the more diligent students did three or four times as much work as the less diligent students.

In summary, the longer working students spent more time on each page and did more pages. The shorter working students spent less time on the significant pages and failed to do some assigned pages.

It was calculated that the average web activity over the term for the top half of the students in the winter 2004 session (sorted by final grade mark) was 5 hr 43 min. The average web activity for students in the bottom half was 4 hr 55 min, only 48 min less. Similarly the difference for the fall 2003 session was only 18 minutes, the upper half of the students averaging 3 hr 48 min of web work and the lower half of the students averaging 3 hr 30 min.

Relationship Between Web Work and Final Grade

Figures 3 and 4 display the students' final course grade and the time they spent on the web activities, excluding the time they spent on the index pages, for the fall 2003 session and the winter 2004 session, respectively. The final course grade was a composite of different marks: pop quizzes (10%), speaking (30%), homework (10%), class participation (10%), midterm (20%), and final exam (20%). The results of Pearson's correlation analysis showed no significant relationships (p = .671)

In Figures 3 and 4, the line through the middle of the chart represents the students' final course grades, highest grade on the left-hand side and lowest grade on the right-hand side. The jagged line represents the time the students spent working on their web activities.

39

0x01 graphic

In the fall 2003 session it was the students receiving average grades who were spending the most time on their web activities.

0x01 graphic

In winter 2004, there was a tendency for the students who received higher grades to spend more time on their web activities although the correlation between web time and final grade for the whole group was weak.

40

Winter 2004 session: Correlation between gain in listening skills and work on web activities

Does the extra work done by the more diligent students translate into greater improvement between the pre- and posttest? As mentioned above, both the day and evening students in the winter 2004 session took a dictation pre- and posttest to measure their improvement in listening skills over the term. Students' scores on the pre- and posttest were submitted to a t test. The mean of the pretest was 21.9 (out of 28 possible points), and the mean of the posttest was 23.3. The difference between was significant (p = .000), showing the students in both groups improved their listening skills over the term.

In the context of overall significant improvement in listening skills, was there a relationship between the amount of time the students spent on the web activities and their gain scores? Figure 5 shows the amount of time the students spent on the web activities and their gain scores, multiplied by 100. The line at the bottom of the chart represents the students' gain scores. The jagged line at the top represents the total number of minutes the students spent working on the web activity pages during the term, again excluding the time they spent on the index pages.

0x01 graphic

Pearson's correlation analysis actually showed a significant negative correlation between working on web activities and increased listening skills (p = 0.042). Some of the least diligent students showed the greatest increase. Also, some students did worse on the posttest than on the pretest; their gain scores are shown as negative. These backsliders were usually the middle-range students. However a couple of the students with little or no improvement were among the best and brightest students in the courses. They had nearly perfect scores on the pretest and so had very little margin for improvement.

A question may be raised, however, whether or not the listening component in the web activities or the reading/writing involved in the activities was the primary component. Perhaps the work of navigating through web pages, reading, and filling in blanks focuses more on reading and writing and relegates listening to secondary status.

41

Index Page Navigation

The median for time spent on index pages was 7 sec, and the mean was 40 sec. There was a large difference between the median and mean because there were many index pages left open for 200 sec or more, although more than half were open for 7 sec or less. There were 4,333 index page hits generated compared with 1,889 activity page hits, that is, it took on average 2.3 index pages for students to get to an interesting activity page. Using the median value of 7 seconds to evaluate an index page and 2.3 index pages to an activity page, the average time to navigate to an activity page was about 16 sec.

However, part of the problem of defining a precise navigational time to an activity page may be that the index pages are locations where the students rested or stopped between activity pages. While three fourths of all the index durations were 16 sec or less, one student left an index page open for 1 hour 9 min, while others left it open for an hour, less than an hour, and so on. It is difficult to determine when index page time was being used to evaluate navigational paths and when students had finished an activity and switched their focus away from the computer. It does highlight, nevertheless, the use of the index page to pause and rest between activity pages.

Gender and Online Behavior

This section examines differences in online work patterns of men and women. Calculations were made to see whether women or men worked longer at their web activities and whether one gender browsed around the lesson space more than the other gender. Table 3 shows the average number of minutes that the men and the women in both sessions spent on the web activities.

0x01 graphic

The data show that for the fall 2003 session, the 21 female students on average worked on their activity pages for 11% more time than did the 8 male students. The data from three students, two male and one female, were dropped from the fall 2003 session analysis because two did just minimal work, 13 minutes or so, another did no web work at all. In the winter 2004 session, the 25 female students on the average worked for 64% more time than did the 6 male students. These two samples, although small, seemed to indicate that women spent more time than men working on their web activities.

While the figures in Table 3 indicate women spent more time working on the activities than men, one calculation about whether or not women browsed around more than men gave ambiguous results. The analysis of the data to answer this question was based on the assumption that any activity page opened and then closed in less than 90 sec represented a phase of browsing rather than sustained work. Activity pages generally required a minimum of 3-4 min to listen to the song, to read, and to make a serious attempt at answering the

42

questions. All 1,889 hits for the activity pages, again excluding index pages, were divided as either being open for less than 90 sec or open for 90 sec or more. These records were then divided once more by gender. Counts and ratios were made for the four groups. The results are presented in Table 4.

0x01 graphic

The figures in Table 4 show no clear gender distinction. In the fall 2003 session the men tended to browse more, while in the winter 2004 session the women tended to browse more. The winter 2004 ratio contradicts the finding of Beasley and Vila (1992) that women have a more linear navigation pattern and therefore do not explore the content of the environment as much as men.

In summary, the analysis presented here seems to indicate that women spent more time than men working on their web activities but that there was no clear gender distinction for web browsing behavior.

Analysis of Some Individual Students

What are typical web work patterns for more diligent and less diligent students?

Navigational spreadsheets for individual students were produced listing each web page the students visited. These spreadsheets showed what time the students entered the page and how many seconds they spent working on it. Spreadsheet lines were highlighted for those pages on which students spent more than 2 min. Harder working students usually had approximately four pages of data navigation listings compared with only one page of data listings for the less diligent students.

This section briefly describes the web behavior of two diligent students and one less diligent student in order to better understand their online working patterns and work habits. The more diligent (MD) students both had listening final marks above 90%. MD1 had a final course grade of around 75%, and MD2 had a final course grade of around 85%. Both MD1 and MD2 were good, hard-working students beyond their web work, and they were observed to participate actively in class. The less diligent (LD) student (LD1) had a listening final mark of around 70% and a final course grade of about 60%.

MD1 visited 131 web pages giving about four sheets of web access data. Her data listings showed her working late in the evening and around noon on her French web activities. MD1 opened the "Index pages" very often, although she only spent a very short time looking at them. She moved quickly to the content pages. She spent a significant amount of time, sometimes up to 50 min, working on the content pages. She followed a linear navigational route, going simply from the main index to the song index to the activity and then back to the song index. Occasionally she would look at the same song activity again and then go back

43

to the index for a new activity. MD1 seemed to like to click on her web browser page refresh button. Once she made half a dozen calls to the same page, generating a half a dozen data records, within a minute.

MD2 preferred to work during the day. Over the whole term she never worked past 9:00 p.m. in the evening. During one 1-hour lab period, she worked consistently starting at 6:17 p.m. and finishing at 6:52 p.m. After the lab period, she kept working, from outside the lab, at the assigned activities for three fourths of an hour more, starting at 7:59 p.m. and finishing at 8:43 p.m. During the whole term she was observed to work systematically and conscientiously through her activities. Her answers to assigned homework questions were always correct. She visited 151 pages total. She navigated quickly through the many index layers to get to her work pages: 17 sec, 12 sec, and so on. She tended to explore all the pages available from the index page. However she only stopped to work significantly at the assigned pages and did not do more than what was requested. She would work for 53, 41, or 47 min on content pages. This was noticeable throughout all her work in the course.

LD1 was a less diligent student. During the whole term he navigated his way into only 36 tracked web pages of which just 3 were done outside the lab. LD1 only worked on 4 pages for more than 10 min, whereas other students, on average, worked on 10 pages for 10 min or longer. He skipped many assigned pages and did no web work at all between March 9 and April 4 inclusive. During one 90-min lab session he started working at 4:33 p.m. (the class had started at 4:00 p.m.) and he finished at 4:56 p.m., so he was not working on the web for half an hour at the beginning and half an hour at the end of the class. During that time he worked consistently on one page for 19 min and spent the rest of his time navigating between the different index pages.

This pattern of avoiding class work shown by the web-tracking system is similar to the attitude this student showed in class. He was consistently late or left before the end of class. He missed over 20% of the classes. He did not participate much in class although he was quite fluent in French.

In general, the same behaviors that students exhibited in class carried over into their web work. The good students applied themselves both in class and on their web assignments. Other students who were not so dedicated to their class work got distracted when working online.

CONCLUSION

There are inconsistencies between the two sessions. In the fall 2003 session it was the students working longest at the online listening activities who received only average grades and who showed only moderate improvement in their listening skills. The high-mark students spent less time on the web activities. In the winter 2004 session the opposite occurred with the high-mark students spending more time on their web activities, although the correlation between the amount of time they spent on the web activities and their improvement in listening skills was weak. Perhaps students with lower listening skills feel a greater need to work with the activities.

Our study showed that there was a carry over of normal classroom behavior to the online work. Students who were observed to work well in the classroom usually worked long and diligently on their activity pages. Students who seemed less motivated in the classroom were distracted at their web work and worked only half heartedly at it.

44

Our study also showed no clear gender distinction for site-exploring behavior as measured by the ratio of pages worked on for 90 sec or more compared to pages worked on less than 90 sec. This lack of gender distinction is contrary to the work of Beasley and Vila (1992) that indicated that women had a more linear navigation pattern and Desmarais and Laurier (2000) who concluded that men had the more linear approach. It should be kept in mind that the male samples in the present study were quite small and may not be representative.

The use of web-tracking data needs to be approached cautiously. A researcher cannot do global statistical analysis without first carefully searching the data for irregularities and doing required data cleaning. This writer found that to be true both for the current project and for her previous research using WebCT data. A data specialist may be needed to sort through the large amount of data collected.

The important uncertainty regarding the use of the data is that the length of time that a student has an online page open, as provided by simple automated tracking, does not necessarily represent how carefully that student is actually working with the activity. A more precise measurement of the degree to which the student is focused on the work will require assigned homework submission, screen captures, key stroke counting, direct observation, videos, interviews, and so forth. Automated web tracking will always be hampered by these limitations, although it does provide some insight into student behavior online.

NOTE

1 In this study, a hard-working, or diligent, student is one who records more minutes of time working on an activity web page, as opposed to a less diligent classmate who spends fewer minutes engaged in such an activity. Our definition does not include any sense of special care or perseverance other than what can be inferred from the fact that the web page is open.

REFERENCES

Beasley, R. E., & Vila, J. A. (1992). The identification of navigation patterns in a multimedia environment: A case study. Journal of Educational Multimedia and Hypermedia, 1 (2), 209-222.

Beasley, R. E., & Waugh, M. I. (1997). Predominant initial and review patterns of navigation in fully constrained hypermedia hierarchy: An empirical study. Journal of Educational Multimedia and Hypermedia, 6 (2), 155-172.

Canter, D., Rivers, R., & Storrs, G. (1985). Characterizing user navigation through complex data structures. Behaviour and Information Technology, 4 (2), 93-102.

Chapelle, C. (2001). Computer applications in second language acquisition. Foundations for teaching, testing and research. Cambridge: Cambridge University Press.

Chapelle, C., & Mizuno, S. (1989). Student's strategies with learner-controlled CALL. CALICO Journal, 7 (2), 25-47.

Cunningham, R. L. (2000, Winter). An empirical investigation of the performance of evening and day students. Journal of the Academy of Business Education. Retrieved May 18, 2005, from http://www.abe.villanova.edu/proc2000/n030.pdf

45

Desmarais, L., & Laurier, M. (2000). L'analyse des schémas de navigation en ELAO. In L. Duquette & M. Laurier (Eds.), Apprendre une langue dans un environnement multimédia (pp. 281-301). Outremont, Québec: Les Éditions logiques.

Duquette, L., & Dionne, J. P. (2000). La résolution de problème dans les exercices lacunaires en L2 et l'environnement multimédia. In L. Duquette & M. Laurier (Eds.), Apprendre une langue dans un environnement multimédia (pp. 179-210). Outremont, Québec: Les Éditions logiques.

Gay, G., & Mazur, J. (1993). The utility of computer tracking tools for user-centered design. Educational Technology, 34 (3), 45-59.

Hauptman, P. C., Wesche, M. B., & Ready, D. (1988). Second language acquisition through matter learning: A follow-up study at the University of Ottawa. Language Learning, 38 (3), 443-475.

Horney, M. D. (1993). Case studies of navigational patterns in constructive hypertext. Computers and Education, 20 (1), 67-82.

Hwu, F. (2003). Learners' behaviors in computer-based input activities elicited through tracking technologies. CALL Journal, 16 (1), 5 -29.

Mendelsohn, D. (1995). Applying learning strategies in a second/foreign language listening comprehension lesson. In D. Mendelsohn & J. Rubin (Eds.), A guide for the teaching of second language listening (pp. 132-150). San Diego: Dominie Press.

Mendelsohn, D. (1994). Learning to listen. A strategy-based approach for the second-language learner. San Diego: Dominie Press.

Narcy, J. P. (1991). Comment mieux apprendre l'anglais? Paris: Les Éditions d'Organisation.

Rézeau, J. (1999). Profils d'apprentissage et représentations dans l'apprentissage des langues en environnement multimédia. Alsic, 2 (1), 27-49.

Orey, M.A., & Nelson, W. A. (1994). Visualizing techniques for examining learner interactions with hypermedia environments. (ERIC Document Reproduction Services No. ED373747)

Recker, M. M. (1994). A methodology for analysing students' interactions within educational hypertext. (ERIC Document Reproduction Services No. ED388288)

Renié, D. (2000). Apport d'une trace informatique dans l'analyse du processus d'apprentissage d'une langue seconde ou étrangère. In L. Duquette & M. Laurier (Eds.), Apprendre une langue dans un environnement multimédia (pp. 281-299). Outremont, Québec: Les Éditions logiques.

Stanton, N., & Baber, C. (1992). An investigation of styles and strategies in self-directed learning. Journal of Educational Multimedia and Hypermedia, 1, 147-167.

Vila, J. A., Beccue, B., & Anandikar, S. (2003). The gender factor in virtual reality navigation and wayfinding. In R. H. Sprague (Ed.), Proceedings of the 36th Hawaii International Conference on Systems Sciences (HICSS) IEEE (p. 101). Los Alamitos, CA: IEEE.

Weinberg, A. (2005). Les chansons de la francophonie website and its two web-usage-tracking systems in an advanced listening comprehension course. CALICO Journal, 22 (2), 251-268.

Weinberg, A. (2002). Virtual misadventures: Technical problems and student satisfaction when implementing multimedia in an advanced French listening comprehension course. CALICO Journal, 19 (2), 331-357.

Weinberg, A., & Knoerr, H. (2003). Learning French pronunciation: Audiocassette or multimedia? CALICO Journal, 20 (2), 315-336.

46

AUTHOR'S BIODATA

Alysse Weinberg has been teaching French as a second language at the Second Language Institute of the University of Ottawa in Canada for many years. She is a co-author of two French textbooks, À bon port and Points de rencontre. With Hélène Knoerr, she co-authored the Complete idiot's guide to learning French as a second language, two CDs of multimedia activities for listening and reading comprehension--Compagnon de voyage--and one on French pronunciation--Compagnon de parole. She regularly publishes on language learning.

AUTHOR'S ADDRESS

Alysse Weinberg

Institut des langues secondes

600 Ave. King Edward

Ottawa, Ontario, Canada

Phone: 613 562 5800, ext. 3465

Fax: 613 562 5126

Email: weinberg@uottawa.ca

47