Introduction

are important not just for the evaluation of existing software, but for the design of software specific to the special needs of this population as well.¹

To examine these two questions it makes sense to begin with a list of features that we know are important to young, typically-developing (TD) software users and then ask whether such a list also reflects the preferences of users with ASD. The beginnings of such a list can be found in early empirical work by Malone on the design of motivating instructional environments.² Surveying 65 children ranging from kindergarten through eighth grade, he had them rate the subset of 25 computer games with which they were familiar on a three point scale (don't like, like, like a lot). Each game was then analyzed for the presence or absence of 10 features, based on the characteristics of computer games at the time and theories of motivation by Banet ³ and others. The features Malone used were:

1. Existence of an explicit goal
2. Computer keeps score
3. Audio effects
4. Randomness involved in game
5. Speed of answer counts
6. Visual effects
7. Competition
8. Variable difficulty level
9. Cooperation
10. Fantasy

There are three advantages to basing our own survey on an updated version of Malone's original list. First, of course, is the existence of prior results that show high positive correlation in typically-developing children. The second advantage is that the list itself is based on the application of a number of cognitive and educational theories of intrinsic and extrinsic motivation to the problem of learning in general and software design in particular.

The third and most compelling advantage of starting with Malone's list, however, is that most of the features on it constitute areas of experience in which we might a priori expect to find differences between typically-developing children and those with autism. Such expectations would be based on differences between TD and ASD children that are part of the defining characteristics of the syndrome. For example, we might expect that visual effects would be more important to children with ASD since they tend to rely on visual reasoning and visual organization of information. Similarly, we might expect the existence of randomness and variable difficulty levels to be negatively correlated with use for children with ASD who generally seek predictability in their environments. Speed might also be a negative factor for those children whose symptoms include motor planning impairments. Further, it is assumed that most children with ASD do not have the kind of ability to reason about an opponent's strategy that makes competition challenging and fun, that they prefer solitary to cooperative play, and that they are incapable of or are highly restricted in their fantasy. These characteristics of the syndrome, then, predict group differences for the features competition, cooperation, and fantasy. Expectations with respect to explicit goals, scoring, and audio effects are more equivocal. It is possible to imagine that conventional goals and scoring imply competition and would, therefore, be unimportant; but it is also possible to argue that by representing the goal and scoring visually such features would become both salient and motivating. Audio effects such as music might be an important enhancement whereas some sound effects might be particularly aversive for those children with auditory hypersensitivities.

Of course, a plausible story for why there might be group differences does not guarantee that such differences will outweigh individual variability. However, the fact that we have such a priori expectations means that by comparing the preference for these features in software activities with the preference for the same features in non-software activities, we may be able to find evidence pertaining to a third question:

Such a question has important implications for the special education community, as it begins to address the issue of what role computers might have in levelling the educational playing field.

We have stated the three questions central to our survey's design as questions rather than hypotheses to underscore the exploratory nature of this work. Because the survey was exploratory, we offered a large number of features as potential discriminators between the two groups of children, including but going beyond Malone's original list. Although this has implications for achieving statistical significance given a moderate group size (as discussed below), it seemed the appropriate course of action for a preliminary study.

Description of the Survey

Question 4 collected information about the child as a computer user, including who was responsible for purchasing software, what criteria were used, age at which independent interaction was mastered, use of an accompanying manual for the software, and choice of hardware platform. This question was intended as a gross measure of the comparability of the two populations as software users. Significant differences along these dimensions could introduce a potential bias in the result of the feature analysis. For example, if the TD children chose their own software primarily for entertainment purposes and the parents of the ASD children chose primarily therapeutically-oriented software, then the a priori distribution of some features within the software could be different, and thus, the perceived relative importance of the features could be different as well. Question 5 was intended as a refinement to this gross measure and asked the respondent to list the child's three most and three least favorite software titles.

Primary Measures of Interest

The scale is unusually worded because some children with ASD have a limited emotional range or limited ability to show emotion, especially positive emotion, in conventional ways. A scale with endpoints labelled Strongly dislikes/Strongly likes thus seemed inappropriate. A scale with the endpoints Absence very important/Presence very important seemed problematic as well; how do you judge whether the absence of something is important? The rather unbalanced verbiage shown above was an attempt to try to solve this problem. However, a scale labelled Avoids activities with this feature/Prefers activities with this feature might have been a better, and more conventional, choice ⁴.

The main difference between questions 6a and 6c was the set of activities to which the features were to be related. Question 6a asked participants to rate the following 23 features in the context of the child's computer activities:

• Clear goal (e.g. get 10 points, answer all questions)
• Visible reward (e.g. points going up, stickers awarded)
• Sound effects
• Music
• Randomness (e.g. different puzzle each time)
• Speed of response counts
• Visual effects (e.g. animations)
• Competition
• Multiple difficulty levels
• Involves fantasy or role-playing
• Use of animated character guides
• Use of on-line help
• No reading required
• Text in addition to auditory info
• Rich visual detail
• Relevant theme (e.g. trains, maps)
• Particular subject matter (e.g. math, science, stories, painting)
• Open-ended/child-controlled activity (e.g. painting, exploration)
• Program-guided activity (e.g. math drill, Carmen Sandiego)
• Fact-based activity (e.g. Jeopardy style game, math problems)
• A variety of activities on the disk
• Ability to use alone
• Ability to use with another person

The list of 23 features given above both subsumes and extends Malone's. In five instances the description of one of Malone's features was changed. Existence of an explicit goal and Computer keeps score were changed to Clear goal and Visible reward in an effort to more clearly distinguish the existence of the goal per se from the use of a visible cue to signal progress toward the goal. Variable difficulty level was changed to Multiple difficulty levels to better match the language used in marketing software. Cooperation was changed to Ability to use with another person to contrast this feature more clearly with the added feature Ability to use alone (one may argue that this change alters Malone's intended meaning but it is, for our purposes, the more pertinent choice). Finally, Fantasy was generalized to Involves fantasy or role-playing.

Features were added for one of three reasons. In two cases, we simply teased apart one feature of Malone's into multiple related features. Malone's list includes Audio effects whereas ours distinguishes Sound effects from Music. Similarly Visual effects is split into Visual effects such as animation versus Rich visual detail. In three cases we added features Malone would not have seen in 1980 but that seem characteristic of early learning software today (Use of animated character guides, On-line help, Variety of activities).

Although Malone's list contains many features that we might expect to be differentially preferred by children with ASD, it was never Malone's intention to derive his list with an eye toward the special needs of this population. Thus, in the remaining eight cases additional features were added because they reflect important characteristics of current software specifically with respect to the information processing needs of children with autism. The features No reading required and Text in addition to auditory information were included because many children with ASD have difficulty processing auditory information. Software without text generally relies more heavily on auditory presentation of information, whereas early learning software that includes text tends to present information redundantly to the visual and auditory channels. The features Relevant theme and Particular subject matter were added in response to the restricted interests and fixation on specific topics that are one of the diagnostically-relevant characteristics of the syndrome. The inclusion of a feature to detect a preference for Fact-based activity was motivated by the same literature that argues for a limited ability in fantasy and pretend play. The remaining features--- Ability to use alone, and Open-ended/child-controlled activity versus Program-guided activity ---were motivated by the defining characteristic of autism, a preference for solitary play and internal locus of control.

Data Collection Method: Subjects and Procedure

Survey distribution was performed in two stages for each of the two populations, the ASD group and the TD group. In the first stage, an invitation to participate in the survey was posted to appropriate listservs. Listserv readers who responded to the post were then sent surveys via email. Surveys were, for the most part, completed on-line and returned via email. In a few instances, differences in email programs made this impossible and surveys were returned in hard-copy form via the postal service.

The invitation for the ASD group was posted to the lists autism@maelstrom.stjohns.edu and aspberger@maelstrom.stjohns.edu at the end of April, 1997. The request for participation was later forwarded by one respondent to a group of parents interested in hyperlexia (a characteristic that is sometimes found among high-functioning children with ASD). The post invited response from individuals willing to fill out a short electronic questionnaire who had a child (or children) who used commercial, off-the-shelf software in a home, therapy, and/or school setting and who had been diagnosed with any of the autism spectrum disorders (autism, pervasive developmental disorder, Asperger's syndrome, etc.).

Although it was easy to identify a likely source of potential participants for the ASD group, finding a comparable on-line source for the control group of typically-developing children was more difficult. Since participants from the autism community were self-selecting on the basis of an interest in children's use of educational software, we targeted lists for the TD group that appealed specifically to individuals interested in this topic as well: chi-kids@acm.org, netnews.misc.kids.computer, netnews.alt.comp.shareware.for-kids, netnews.alt.education.home-school, netnews.misc.kids, and crc@classroom.net. Most of the responses we received for the ASD group in the first week were for pre-teen children who were using software in the home. Thus, to maximize the percentage of usable data from the TD group, the post to the TD lists was revised slightly from the original to request individuals willing to fill out a short electronic questionnaire who had a pre-teen child (or children) who used commercial, off-the-shelf software in the home.

In collecting the data, no socio-economic information was requested of participants. Socio-economic equivalence of the two groups seemed likely by virtue of choosing Internet listserv readers who had personal computers in the home. While it is a weakness of this exploratory study that such an equivalence was not established empirically, we had to weigh the usefulness of such a comparison against the possibility of reduced participation due to concerns about providing this kind of information, especially over the Internet in electronic form.

Results

Response Rate

We received 77 requests for questionnaires from individuals associated with children in the TD group. Of those, 45 individuals (58%) returned one or more surveys: three individuals returned surveys for two children, and two individuals returned surveys for three children. Of the base number of 52 surveys, five were excluded on the basis of inadequate information (two individual children and one of the sets of three children from a single household) and three were excluded on the basis of disability (one case of aphasia and two cases of speech problems that required some support within a mainstream classroom environment), leaving 44 responses for the TD group. Of these 44 responses, 40(91%) were provided by the child's parent or guardian, three were provided by a family member other than the parent, and one was provided by the child's teacher.

Population Characteristics

Disability is the only significant difference between the groups for these characteristics. Overall the ASD group may have come from the less-impaired portion of the spectrum: only 9% (5 children) were non-verbal, compared with reported rates of 50-75% in the literature. The distinction made here between higher and lower functioning children with ASD was predicated on a combination of educational situation and communication level. A lower-functioning child was defined to be one who required a special education classroom or whose communicative level was well below norm for chronological age (including, for example, a five year old child who was verbal but reported to be at the two- to three-word stage). All children considered to be higher-functioning were home-schooled, partially or fully mainstreamed and communicated verbally in full sentences.

The difference in male/female ratio between the groups reached only marginal significance. Since autism differentially affects males at a ratio of about 3.5:1, in any sample of this population we would expect the distribution of males and females to be similar to the one found here. All other things being equal, we would expect the split for the typically-developing group to be 50/50, resulting in a significant difference between groups for this characteristic. The male bias in our data for the TD group may reflect gender-related preferences in young children for computer activities, or may have some other cause. However, the disparity between the groups on this measure seems inevitable given the skew in the male/female distribution for the ASD population. We address the potential implications of this disparity below.

As shown, there was no significant difference in the number of readers in the two groups (Chi-square = 1.35). The distribution of readers among the higher and lower-functioning subgroups was fairly even, with 16/38 (42%) readers categorized as lower-functioning and 22/38 (58%) as higher-functioning.

In addition to being generally comparable, the two groups appear to be drawn from essentially the same population of computer users, at least for the gross measures we considered:

On average, children in both groups mastered basic interaction via mouse or keyboard around their fourth birthday. The youngest age of mastery in both groups was 1.5 years, and the standard deviation for mastery in both groups was 1.8 years.

As noted in the table, the difference between groups in the use of Macintosh versus PC platform is marginally significant, a fact for which we have no obvious (or plausible) explanation. To understand whether the difference might have had an impact on which software the children used (and, thus, on which features they disliked or found important), we can compare the software titles given in response to question 5 of the surveys. Analysis of the data from question 5 shows usage in both groups is dominated by popular titles that are available on both PC and Macintosh machines.

In almost all cases the adult responding to the questionnaire was solely or partially responsible for the purchase of software. Overwhelmingly, the primary criterion for purchase was educational. Only 20% of the TD group chose entertainment as the main reason for choosing a software title. For the ASD group, 11% gave entertainment as the main reason and 13% chose software primarily for therapeutic use.

Feature Analysis

Recall that the large number of features to be considered in looking for differences between the ASD and TD groups is a function of the exploratory nature of this survey. Unfortunately, a side effect of the feature set size is the relative reduction in power of the statistical tests for the given population sizes and, therefore, the corresponding adjustment of the significance level to p <= .002 ⁶. However, since the study is exploratory, we will want to consider in our discussion values that might well have reached significance in a survey with a more focussed feature set. Consequently, we include in our reporting values for which p < .01 as well.

The difference between diagnostic groups for the three computer-only features (Sound effects, Animal characters, and On-line help) were evaluated by t-tests with none reaching significance (p = .62, p = .03, and p = .67, respectively). The remaining 20 features were evaluated using a repeated measures analyses of variance with diagnosis as a between-subject factor and environment (computer vs. non-computer) as a within-subject factor. The next table shows those features having significant differences between and/or within groups. None of the interaction terms (diagnosis x context) was significant.

The column labelled ASD vs TD examines significant differences between the groups in their attitude toward features independent of whether those features are found in a computer or non-computer environment. In other words, preference for each of Randomness, Competition, Text and audio, Visual detail, and Use with others was significantly different as a function of disability with the computer environment neither contributing to nor detracting from the effect. Examination of the means, collapsed across contexts, shows that Randomness, Competition and Use with others are differentially preferred by typically-developing children, while Text and audio and Visual detail are differentially preferred by children with ASD. Further, Visual effects is a feature of interest that children with ASD may prefer more than their typically-developing counterparts. Similarly, Multiple levels of difficulty may appeal more to typically-developing children in all contexts.

The column labelled Computer vs Non-Computer examines significant differences between contexts independent of disability. In this case, means reveal preference for each of Randomness, Visual effects, Music , and Child-controlled activities were all greater in the computer context, regardless of whether the child was autistic. Interestingly, Program-guided activities also showed a marginal context effect independent of disability.

Discussion

Expected and Found

In contrast, Randomness, Competition, and Use with others are differentially preferred by typically-developing children (recall that randomness was one of the critical features for engagement found in Malone's original study). As a negative factor for children with ASD, Randomness and Competition may be particularly problematic because they feature prominently in current software design. Within-group measures show that children with ASD are significantly more willing to accept randomness in a computer context than in a non-computer activity; however, in either context the average attitude toward randomness is still on the negative side of the scale.

Competition shows no such ameliorating effect by context and is similarly negatively viewed by children with ASD. Further, the case for a between-groups difference for Competition may be even stronger than our data suggests. Given the amount of data collected, it was inappropriate to try to include gender as a potentional factor in our analysis. The literature on gender differences in computer use, however, indicates that a competitive feature in software is differentially preferred by males. Our data shows that the ASD group disliked competition despite being predominantly male (81%) while the more balanced TD group (66% male) rated competition favorably overall (that is, despite a higher percentage of females than the ASD group had). Thus, if we consider the scores for Competition in light of the gender ratios of our two groups, it seems likely that the difference in preference, at least among males, is even larger.

Use with others, showed a small positive preference by the TD group but was viewed as essentially unimportant by the ASD group (the mean value collapsed across contexts corresponds to the doesn't care choice on our scale). This is the least problematic of the discriminating features, however, since most software that can be used with others can also be used effectively alone. Moreover, using the same logic about gender ratios applied above, it may be the case that the difference detected for Use with others is artificially inflated. If, as the literature suggests, it is the female portion of the TD group that contributes the positive scores for this feature, then the imbalance in gender percentages between the groups may be the source of the effect. To see if Use with others (and Competition) show gender-related differences for the ASD group in this age range would require collecting significantly more data.

The seventh feature to show an effect for diagnosis was Multiple levels of difficulty. The essence of the difference seems to be a relative lack of importance for the ASD group compared to a definate preference for the TD group. We conjectured in our initial discussion that the characteristically rigid play of children with ASD might translate into a tendency to try to use software in the same way time after time. However, it also seems possible that children with ASD simply do not notice the change in difficulty level or do not derive satisfaction from attaining higher levels. An additional survey question needs to be designed to tease apart these possibilities.

Expected but Not Found

Although we made a number of a priori conjectures about which features might show group differences, our third question---Are children with ASD more like their TD peers when on a computer?---was strictly exploratory⁷. The interaction terms of the repeated-measures ANOVA (diagnosis x context) address this issue. Although none of the interaction terms reached significance, two seem to be worth further attention: Visual effects (p=.03) and Competition (p=.04). In the case of Visual effects there seems to be the potential for an interaction such that children with ASD can be said to be even more sensitive to visuals on the computer than they are to visuals in non-computer activity when compared to typically-developing children. Given the independent differences both between-groups and within-group and the high means in all conditions, it seems possible that the interaction was obscured by a ceiling effect. With respect to Competition, the potential direction of the interaction corresponds to an even higher aversion to competition on computers. As described above, such an interaction may have been obscured by gender percentages in the two groups.

Conclusions

We found significant differences between the groups with respect to five common features of software. The use of randomness in presenting information, the use of competition as a motivating factor, and the ability to use a program cooperatively with others were all more important to typically-developing children, while the complementary use of both text and audio information and richness of visual detail were differentially important to children with ASD. The data was suggestive of two additional effects: preference for visual effects such as animation in the ASD group and preference for multiple levels of difficulty in an activity in the TD group.

Contrary to expectations, the children with ASD did not have a stronger preference for thematic, subject-restricted or factual material, nor were they significantly more averse to fantasy or role-playing activities. Most surprising was a strong attraction in the ASD group to educational software that involved speeded response.

The similarities between the groups were just as interesting as the differences. Both sets of children responded strongly to the use of music and sound effects in their software and preferred child-controlled to program-controlled activity. The children in this age group seemed to feel somewhat positively toward the use of animated character guides and neutrally with regard to on-line help facilities. Although the TD group had a significantly higher preference for using software with others, both sets of children enjoyed the computer primarily as a solitary pursuit. Scores also showed no distinction between the groups with respect to the importance of a goal or visual reward in computer play.

The role of computers in our educational system is of increasing economic and philosophical concern. Real inequities in access to the technology have been recognized for many years and many populations, including children with developmental delays. But the question of physical access, while fundamental, may be ultimately less important than the question of intellectual access. A computer in the classroom is only as useful as the appropriateness of the software it runs. The pilot study reported here is a preliminary step in understanding how the information needs of one population may differ from the norm. In addition to being an interesting comparison in its own right, two purposes are served by studying this question. With such information:

1. it may be possible to articulate a set of rational design principles for educational software for this population, at least with respect to the differences revealed. As an example, the information from this study has already been incorporated into the design of a software environment for language intervention for children with ASD.
2. it may be possible to create a metric for evaluating off-the-shelf educational software with the special needs of children with ASD in mind. Such a metric could be used both as an organizational tool for a database of commercial software and as an objective set of criteria for recommending such software to therapists, educators, and parents.

A larger-scale survey, focussing more narrowly on the features suggested by our preliminary data, would undoubtedly contribute to these goals.

Endnotes

² Malone, Thomas W., Toward a Theory of Intrinsically Motivating Instruction, Cognitive Science, Volume 4, 1981. Note that while there are many current published ratings of early learning programs (and some of these are based on appraisal by young users), none includes a feature-based analysis that correlates presence or absence of a set of features with eventual approval or disapproval. back

³ Banet, B. Computers and early learning: A new direction for High/Scope Foundation. Calculators/Computers, Volume 3, 1979. back

⁴ Of the 103 respondents to the questionnaire, only one commented on the unusual nature of the rating scale. In general, participants did not seem to have trouble interpreting the scale, using the full scale and occasionally introducing non-integer values (e.g., 3.5). back

⁵ In retrospect, sound effects probably should not have been omitted, being an increasing component of non-computer electronic games). back

⁶ The adjusted p-value is found by dividing the usual significance level (.05) by the total number of tests to be performed on the data (23). back

⁷ For the sixteen features we included in the list because of clinical differences between children with ASD and their TD counterparts, it might reasonably be expected that we would have found significant between-group differences in the non-computer condition. However, by using the repeated-measures ANOVA, significance in the diagnosis x non-computer condition is defined relative to differences in the diagnosis x computer condition (i.e. the two tests are not computed independently). Without significantly more data (or a reduced feature set) we can only look to the means for justification of our initial reasoning. The means table shows that in all instances but one (Factual material) the difference in preferences between the diagnostic groups for the feature in the non-computer condition is in the expected direction. For example, Fantasy shows means of 3.21 (ASD) versus 3.79 (TD) while Subject shows means of 4.09 (ASD) versus 3.82.back