Our site, "RWX Models for Active Worlds," serves as a guide to understanding and working with 3-D avatars in the rwx format. Since the primary purpose of this site is to provide step-by-step tutorials for manipulating avatars, our usability tests were designed to determine the tutorials' overall effectiveness and ease of use. The data gathered during these tests were used to assess the current prototype, and suggest further design changes. The test method was as follows:

User Description

The subjects consisted of two users, both of whom are graduate students at Indiana University. One individual had intermediate familiarity with 3-D environments (Active Worlds), while the other had no previous 3-D environment experience at all. We intentionally chose users of different skill levels in order to ensure our tutorial was useful for both novice and intermediate 3-D environment users.

Testing Materials

Because the tutorials require the specific tool, RenderWare’s Modeler program, users were tested on the computer where Modeler is installed.

Procedure

The testing procedure was pretty straightforward. Users were directed towards our site, and also shown the location of the Modeler program. They were then asked to change the colors of an avatar after following Tutorial 1. They were also asked to change the textures of an avatar after following Tutorial 2.

Throughout the procedure, users were asked to ‘think-aloud’ their thought processes concerning each step of the tutorial. In addition, they were encouraged to express any questions or opinions they had while manipulating the avatars.

Data Collection

Data were collected from three main sources. First, we examined the users’ verbal ‘think-aloud’ comments. Secondly, we administered a short post-test questionnaire that allowed the users to rate the overall effectiveness and satisfaction with the site. The questionnaire consisted of both open, and scaled question types. Thirdly, we interviewed the users regarding any other difficulties or their feelings.

Results

Results of our data analysis indicated the following:

• In general, preliminary test results were encouraging. The users were able to complete both tutorials without any serious difficulty, and felt the tutorials moved at an adequate pace. Also, both participants used the initial exercises as a starting point for additional avatar manipulation.

• The main usability issues that arose were related to file transfer and Modeler operations. One user discovered that texture files needed to be in bitmap format in order to display correctly in Modeler's material painter function. Also, one user had difficulty replacing an original avatar texture with another, and she felt the instructions for texture loading needed to be more explicitly stated. Other lesser problems included a general difficulty distinguishing which color shade to manipulate, and the tendency of Modeler to occasionally generate errors and lock-up. Since Modeler is still in beta stage, we hope that further versions will be more stable.

• In terms of numbers, the post-test questionnaire indicated an overall satisfaction with the process of following the tutorials (4.5 on a 5 point scale), and ease of use (4.5). In addition, both users remarked that they would like to learn more about avatar design in general.

Redesign Implementations

After review of the usability data, we implemented the following design changes: