This is an evaluation of two established multicast shared workspace tools together with comments on three newly-released tools. The established tools, wb and nte, have been evaluated both as support for distributed team meetings and in a specialised teaching and learning application where the users were less technically knowledgable. Comments on the three new tools, wbd, mDesk and TeleDraw are based on the documentation provided by the developers and on test sessions carried out at UCL and the University of Exeter.

The main aim of the evaluation is to assess usability but additional information about the tools’ functionality is given to provide a frame of reference.

Wb

Wb is the shared whiteboard tool developed by Van Jacobson and Steven McCanne, both of Lawrence Berkeley Laboratory, University of California, Berkeley, CA., to provide a shared display function for multimedia conferencing over the Mbone. Versions are available for UNIX platforms (including Sun, Dec Alpha, SGI) and for PCs running Linux. At present there is no Windows or Macintosh version.

While wb can be used on its own as a collaborative tool, it is more frequently used in conjunction with video and audio tools in a full videoconference. For meetings, it provides a normal shared whiteboard on which participants may write, draw and type with all contributions visible to all participants. In a seminar, its capacity to import PostScript pages allows it to act as an overhead projector. In this mode there is a support tool from the same authors called wb import.

As with the other Mbone tools, wb can be started through sdr or using a command line invocation, with options including: username; conference name; encryption key; ttl; orientation and maximum size of importable postscript file.

Table 1 provides a summary of the tool’s features. All functions are visible and accessible from the main window, a factor which probably makes it easier to learn how to use the tool.

A toolbar on the right, vertical edge of the drawing area offers tools for: freehand drawing, straight line, arrow, rectangle, ellipse, and for erasing, moving and copying objects, choosing font , colour and brush size. On this bar there is also an Undo button, a Print button and clickable to alter the orientation of the page.

The exit is clearly marked, with the Quit button also visible on this toolbar.

Buttons on the bar along the lower, horizontal edge allow the user to create new pages, navigate through the pages and import files.

The default behaviour of wb is to enforce a page change on all participants if one participant changes page. It is, however, possible for an individual to override this and "lock" a page (s)he is working on. This allows considerable flexibility of operation. Limited resizing of the window is allowed. Such resizing is not enforced on other participants but objects on the page will be redrawn and scaled to ensure that the user retains the same view, proportionally resized.

Input is via mouse and keyboard. Although all participants in the lesson can use wb simultaneously, it is not possible to modify what another person has written. It is also not possible to see other participants' cursors.

Wb has been used for some years to support meetings of distributed research teams and to show slides during seminars. In both roles it has been found to perform well. In these situations the user groups have almost always been confident computer users with some knowledge of networking.

The aim of this evaluation was to assess the usability of the tool for the average, non-technical user. Some information was gathered from observation of the tool in use during one-to-one consultation meetings between a research student and a statistician. The main basis of the evaluation was, however, a ten-week language tutoring field trial.

The users in the field trial were language teachers and students (mainly of Arts and Humanities) in the Higher Education sector. The age range spanned approximately 40 years. None of them had a background in computing, but all except one had used word processing applications. A 10-15 minute initial training session was given prior to the first use of wb in a language tutorial.

Over a period of ten weeks, four small groups of students took part in weekly language tutorials. This amounted to almost 60 hours’ use of the tool. There were 12 participants, 10 of whom took part in the full course of 8 or 9 two-hour tutorials. The other two users participated in 2 or 3 sessions only. In all sessions, wb was used in conjunction with rat and vic, with rat being used in hands-free mode, to allow particpants to talk and write simultaneously.

Prior to starting the tutorials, teachers had identified activities which must be supported in a conferencing system for language teaching. These were conversation, grammar, reading, writing, filling in gaps, vocabulary, pronunciation. In addition, observers noted that the users needed to carry out the following shared workspace operations in order to complete these tasks:

• Import text files.
• Import image files.
• Read text from the screen; read aloud and silently, skim and scan text on screen.
• Refer to the text during discussion (find the appropriate point in the text and make others aware of it).
• Mark words and phrases, make annotations.
• Write – usually short sentences and phrases or individual words.
• Move objects, including text.
• Draw, mainly rough sketches
• Make corrections (moving or deleting objects, altering spelling).

In three of the four weekly sessions the language required accents on imported and keyboard-input text.

Evaluation took place mainly through observation. HCI specialists sat in the same room as teachers and students and filled in an observation sheet during each session. Language teachers acting as expert observers made their own notes, watching unobtrusively from remote workstations. Data was also obtained through administering questionnaires and rating scales, conducting interviews with participants, and recording group discussions and workshop sessions.

Wb was used in every session and for many different tasks. Table 2 shows how often the main functions were used.

Basic typing tool	100
Eraser tool*	90
Change page	79
Select new colour	69
Basic tool also used for drawing	66
Create new page	55
Import postscript	45
Import plain text	6.9
Move text	0

Five functions (type, draw, erase, select colour and change page) were used in virtually every session. Users also frequently selected a new page and imported postscript files. Other tools and facilities were used quite rarely and in one group discussion it was suggested these were not needed or that they might be hidden behind drop-down menus. This may relate to the particular task for which the whiteboard was being used during this trial. It might also reflect the amount of training users were given; perhaps a refresher course part way through the trials would have drawn their attention to the potential of some less-used features.

All the activities named by teachers were able to take place, and the users managed to carry out the operations listed. They reported that they found the whiteboard easy to use and that it made the sessions very interactive and enjoyable. They felt that the short training session they were given had been adequate. There were some problems, however, and these are listed in the following section.

There were some difficulties with page-changing, arising because it is easy to scroll through several pages when one wishes only to move to the next one. In this particular application there was some reluctance to select a new page (users preferred to delete text and reuse the same page). The figure showing the page number is so small that users were often unaware of it and this made navigation more difficult.

There was occasional confusion over the three possible ways to erase entries in wb (select eraser, backspace or use right mouse button). Users also expected to be able to use the right mouse button to erase any object on which they clicked and were disconcerted by the fact that what was erased was the last object they had entered.

Users familiar with a word processor experienced some adjustment difficulties, since wb has no cursor, no automatic carriage return and one cannot select or edit single elements of a block of text. Those who had previously used a drawing package also had to adjust to the fact that graphical objects cannot be selected and modified once they have been created.

Some found the mouse too crude as a drawing tool. This was particularly so in the statistics consultation session, where the statistician wished to draw numerous small diagrams to illustrate points. In this case, the task was severely constrained by the shortcomings of the tool.

Importing a file into wb is a simple operation. It was made more intimidating for users by the fact that the tool was running on a UNIX platform, so that they had to locate files within the UNIX directory system. Making such files easy to locate (putting them in an immediate subdirectory of the one from which the tool is run, for example) would improve the usability of the tool. In a later trial, in which this was done, the tutor was able to use this feature with confidence.

Users commented that they found it tiring to read long passages of text on the screen - although good quality, appropriately-sized print helped. Problems could also arise through different reading speeds, when one student wanted to change page before others had finished reading the current one.

The Undo button appears only to undo certain actions. The Print facility is not very flexible. Some configuring is needed to make it work at all. It also seems possible to print only the whole of the whiteboard contents whereas, for this application, participants often wanted to select one or two pages. Although it is possible to retain a copy of the whiteboard contents via the Print option, users felt that a separate Save function would have been useful, particularly if it operated on selected pages.

Occasionally, activities were constrained by the fact that objects are owned by their creator and cannot be modified by others; useful teaching and learning activities could be devised if it were possible to override this safety mechanism.

It was also unfortunate for the language teaching trial that there was no facility to type accents on text. This could be solved by using target language keyboards, or by reprogramming keyboards to generate the desired characters' special key combinations.

This is a general-purpose shared workspace tool and it was not designed specifically to support applications such as the one in these trials. In fact, it does support such applications remarkably well. It is stable and robust to packet loss, to the extent of being usable as means of communication when audio is not intelligible - though eventually subject itself to packet loss in extreme conditions. Users found it enjoyable to use and easy to learn. If it could be used with a graphics tablet, or similar input device, it would be suitable for an even wider range of activities.

It is possible to see numerous new functions which might be useful but perhaps it would be a shame to lose the simplicity and learnability by adding a large amount of extra functionality. Instead, it might be better to concentrate only on improving a few of the functions already in place.

Nte is a shared text editor designed for use on the Mbone. It was written by Mark Handley, at UCL. It is worth mentioning a few points made in the accompanying documentation. Nte is not intended to be a whiteboard but a text editor; the absence of graphical display features is not a shortcoming of the tool. Nor is it a shared word processor. The tool was designed to be used in conjunction with other communications channels. Human protocols which make use of these must be developed in order to get the best use from nte.

Versions are available for UNIX (Sun, SGI, HP) and PC (Linux, NT, Windows95). Like wb, it can be invoked via sdr or through the command line. Its command line options are also similar to those offerred by wb.

The main features are mentioned in this section, but Table 1 gives a more complete list and allows comparison with wb. Although the tool does run under Windows, some functions are not accessible because there are only two mouse buttons.

Menu names are consistent with those used in many commercial applications and the options offered under the File and Edit menus are what a user would expect, with Quit accessible from the file menu, for example. The names of the other menus (Colour and Font) show what choices they are likely to offer. The only possible exception is the Show menu, which gives the user some freedom to select what is visible on the interface.

File: The choices here assume the user knows the difference between plain and structured text. Plain text files can be loaded and the current document can be saved as a plain text file. There is a size limit for imported files; if they are longer they are truncated. Options to load and save structured text files (ie retaining the information available when they were created) are also offered. These are not fully stable and seem only to work on very small files.

Edit: The user is expected to know what constitutes a ‘block’ of text. Delete block deletes the currently-selected block; it disappears slowly, so that participants can see the change happening. Lower block behaves like send to back in graphics packages, and allows the user to access blocks that were partly hidden. It acts only locally. Undo does not work in the present version.

Colours: The current default colour and the colour of the block currently being edited can be set. Colours are intended primarily to distinguish between different people editing the same document. One cannot set parts of a block to different colours - only the whole block. The colours have been chosen to be fairly dark, to make text readable.

Fonts: The current default font style and the style of the block currently being edited can be set. Options supported are normal, italic, bold and bold-italic in a courier fixed size font. This is to enable documents to be saved as plain text, without the problems that would result from variable width fonts. As with the colours, the different styles are intended mainly to aid disambiguation of different users’ text. Again, changes operate only on whole text blocks.

Show: This allows parts of the application interface to be hidden to save screen space. When the user opts to show the list of participants, the Read Only, Lock and Encrypt buttons also become visible and the writing area is reduced.

Unless the creator locks a block of text, anyone in the session can edit that text or delete it. Many people can edit the same block of text simultaneously, but if more than one person tries to edit the same line at one time, a conflict will occur, which results in only one of the changes being preserved.

An important feature is that nte always tries to tell the user who did what. Unlike wb, nte offers a pointer which is visible to all participants and uses labelled icons to identify the source of actions, as they take place. However, the documentation points out that network failure can cause delay. When this happens, nte may do something unexpected, although it will reach a consistent result. Provided that communication is possible via another channel, this should not pose a significant problem.

As with wb, the evaluation is based on the use of the tool both to support research team meetings and also for a specialised teaching and learning activity, the type of small group tutorials already described.

The tool was used over a period of more than a year, in regular conferences between members of a project team distributed throughout the world. The main use of nte was to import the agenda and to make notes of decisions and obligations agreed during the meeting. Notes were saved by individuals and were also used as the basis of the minutes. Since discussion of an item might continue over several meetings, saved text was often modified between meetings, and then re-imported for further consideration. Nte was used in every session.

In the case of the language tutorials, there was a single course, offered to two small groups for 22 weeks. The users had the option of using nte in every session but did not always do so. Data was collected in the same way as for the evaluation of wb but over far fewer sessions. Because of the reduced number of sessions, the participants did not have time to become expert users of the tool. It would be fair to say that they used it for long enough to encounter usability problems but not for long enough to solve them or to explore ways of using the tool to support this particular application. These users were all previously familiar with wb. Training was given prior to use of the tool, in the form of written instructions.

In all the above-mentioned cases nte was used in conjunction with audio and video channels (using rat and vic). In the meetings, rat was used in push-to-talk mode. The users in the language tutorials used it hands free, so that they could use the mouse to operate nte, whilst they were talking.

Most of the language learning activities previously identified by teachers were still required.. These were identified as: conversation, grammar, reading , writing, filling in gaps, vocabulary, pronunciation. The nature of this particular course meant the operations that the users were required to perform in the shared workspace were slightly different from those identified in the earlier trials. They were:

• Import longer text files (this was an advanced course, based on reading)
• Read text from the screen, aloud and silently, also skim/scan text on screen, to locate a word or sentence.
• Refer to the text during discussion (finding the appropriate point and making others aware of it)
• Mark words and phrases that are not understood
• Record vocabulary and other notes.
• Write, both single words and phrases and slightly longer passages (two or three sentences)
• Create a table with two or three columns
• Fill in the table, with two students and possibly the tutor working simultaneously.
• Move text
• Make corrections (correcting spelling or vocabulary by deleting and adding).

This particular class did not use imported graphics, even when they were using wb as the shared workspace. This seems to have been dictated by the nature of the course they were taking.

The language here was French. Both imported text and the participants' entries required accents.

Nte generally performed well as a support for meetings. It is particularly robust to packet loss and where teams are distributed across the world it can be important to have a viable means of communication when the network is very congested. In this situation, nte appeared to be usable even when wb was subject to long delays and loss of text

Text files imported were usually quite short, so the size limit did not cause problems. Colours provided a useful means to identify participants’ contributions and the facility to save proved extremely useful.

Some improvements were requested, however. It appeared to be advisable to edit the file before saving, adding information about who made contributions, since once the file is saved this information is lost. When saving text, nte also saves large amounts of white space, if this has been created, which can be a minor irritation. The saving and re-importing of structured text only seems to work on very small files, and is not stable.

Nte is a general-purpose shared text editor. Whilst it was used successfully in project meetings, its suitability for a specific teaching and learning application such as this one has not previously been evaluated. Having become comfortable with wb, the users, despite encouragement, were for some time reluctant to use this new tool. When they did use it, they managed to carry out the operations they needed but not with ease. The main user problems concerned:

• The size of file that could be imported: This was a course based on reading quite long texts. Because the file was often truncated, considerably more work was needed prior to the lesson, to ensure that the text was complete. This was compounded by the fact that whenever text is imported it appears at the top of the page. A second, quite laborious operation is needed, to drag it down to the desired position.
• The identification of participants: The labels on icons are taken from the participant’s login id or from a command line option. In a case such as this, where a group of students have a common login id, all their icons have the same label. Unlike most of the other Mbone tools, nte does not allow the user to change name during a session. In these tutorials, therefore, colour remained the only identifier.
• Font size: All the users found the font size too small for reading comfortably. One tutor did see some advantage in the small font size, saying that it allowed a large amount of text to be visible on screen at one time, perhaps making it easier for students to scan through it.
• Lack of accents on keyboard-entered text: This is the same problem that users found with wb – but the graphical capabilities of a whiteboard at least allow them to draw accents in afterwards. Text imported, with accents already included, was displayed satisfactorily.
• The difficulty of establishing common reference: The interface is very flexible. It is possible, via the Show menu, to show the participant list. Doing so will obscure an area of screen into which another participant may well be typing text. The screen area is scrollable but users can all scroll independently. This means that when the document covers the equivalent of several pages, it is possible for participants all to be looking at different areas. This caused considerable frustration and confusion.
• Difficulty marking individual words: They can not be highlighted or coloured differently and, unless the original is double spaced, underlining is also not possible.
• The fact that text cannot be restructured into different sets of blocks: Blocks are not sufficiently fine-grained to support all the activities that are needed in language teaching and, once created, cannot be subdivided. The most obvious example concerns an imported file, which becomes a single text block.

A number of positive points were also made. Users particularly liked the pointer and the labelled icons. They were also very ingenious in finding ways around their problems with the tool. The tutor, for example, inserted his own markers into the text to provide signposts. They also used non-alphabetic characters imaginatively to help highlight words. Having to take these measures did impose an extra burden on the users, however, which ought not to be necessary.

Nte is already a satisfactory support for meetings, when used by those with some experience of networked applications. To make it usable by less expert users, for tutorials such as those described, the following modifications should be considered.

• Allow users to override the size limit on imported files.
• Provide a single larger font, as an alternative. Perhaps double the size would be appropriate.

• Make it possible to alter the user’s name after start-up.
• Allow blocks to be re-defined after text has been entered.
• Add some form of signposting to help navigation through long documents. The small document map provided does help to some extent but the user may not have this showing on the interface. What is needed is reference points, perhaps page numbers.

• Take some measures to enforce a common view on all participants. This might involve causing all participants to scroll when one does. It might simply mean making it harder to change the interface once the tool is running.

Other shared workspaces include TeleDraw, Wbd and Mdesk. They are all in various stages of completion and none appears in a final, stable, released version. None of these tools has been formally evaluated. Wbd and mDesk have, however, been tried out by several users. A short description of each of the tools is followed by comments based on this preliminary testing.

TeleDraw is a platform-independent shared whiteboard from Stuttgart University Computer Center. It is an object-oriented line-drawing program which can be used either as stand-alone or for Mbone conferences. At the time of writing, only a cut down version of TeleDraw, called TeleCanvas, has been released.

TeleDraw supports a number of graphics primitives including free-hand curves, straight lines, open and closed polygons, elliptical arcs, rectangles, ellipses, elliptical chords, pies and text, with visual attributes (line colour and style, fill colour, font name, style and size). Existing graphics objects may be scaled, moved, rotated, grouped or rearranged on the screen. Bitmaps may be included but only moved and scaled rather than arbitrarily modified. Text objects remain editable - and the individual nodes of an open or closed polygon may be moved or deleted separately (although the extension of an existing polygon is not yet supported).

A TeleDraw document may consist of several "pages", which can all be modified. Each page can be write-protected and hidden individually, and the order of pages within a document can be changed at any time.

The complete user interface consists of various components interacting with each other. The main component contains the actual drawing and provides the TeleDraw functionality. The other components (function, tool and attribute bars) may be omitted or modified - thus allowing for customised versions of the tool.

A view of the interface is provided on the TeleDraw webpage. At first sight it looks overcrowded and cluttered, with menus and/or tool bars on three sides of the screen. However, the facility to customise the interface and only display tool bars that the user needs, should solve this problem of overcrowding. TeleDraw seems tool seems to support much of the extra the functionality required by some current users of wb and nte, and we look forward to the release of TeleDraw.

mDesk is a tool developed at Luleå University in Sweden. It is described as a "multicasted co-operation environment".

The designers have tried to keep the interface simple by allowing the user to choose which parts of the application to open.

The small main window contains four buttons, corresponding to the following four main functions of mDesk:

• A whiteboard
• A vote facility which enables participants to cast votes on a given subject.
• A basic chat facility.
• A window showing the participants currently using mDesk on the same multicast addresses.

Clicking on any of these buttons will open the appropriate window. Clicking the button in the main window again, will close the window. This simplicity is appealing. Quit buttons on the individual elements would probably be an improvement, however.

The whiteboard tool itself also has a simple main window but here the approach is less successful. The main window contains three buttons only. To use the tool, however, it is more or less essential to use these buttons to open the palette, toolbox and page-navigation windows shown below. These, in turn can lead to further windows and dialog boxes, so that soon the screen may look very cluttered. Using this whiteboard with multicast audio and video tools would be likely to compound any existing window-management problems.

The icons used in the application could be clearer. The ones on the main window and in the whiteboard are not instantly recognisable and the teapot icon in one of the whiteboard toolbars was a mystery for some time. (It brings up a dialog box for importing image files, when selected with the right mouse button).

A number of features offered by mDesk would make it popular with users. These are outlined below.

• The whiteboard supports a number of graphics file formats, including gif and jpeg, making it more flexible than wb and more easily compatible with a range of other applications. At present, importing an image takes an unpredictable length of time, sometimes more than five minutes. The length of time appears to increase as more objects are created in the whiteboard, although this observation is based on limited experience.
• Text and graphical objects can be selected and modified after creation, another feature which users have requested. It can, however, take some time to update every participant’s screen when modifications are made, which would limit its usefulness in interactive sessions.
• A pointer can be selected and this is visible to all participants. At present it is always coloured red and therefore individual pointers are not distinguished, but confusion might be avoided by additional voice communication.

Like wb, mDesk has a concept of pages and these are easy to create and select. It appears that users are not forced by the tool to look at the same page, although it could be that this simply happens very slowly. Another nice feature is that the user can open a "mini-view" of a page, which is like a small document map. This could be useful, although the fact that this can be done for every page might increase screen clutter.

From the short tests that were run, it would appear that the tool still crashes quite often but if it is made more stable it will offer a lot.

The last shared workspace tool is wbd from LUTCHI at Loughborough University. An alpha version has just been released. Wbd was written to allow better integration with the LUTCHI ssd session directory / simplified conferencing environment, which is attempt to provide an integrated interface to a set of multicast tools.

Wbd is compatible with wb. This means that no one is forced to use it if they have access to wb but that it would be useful to those operating in a PC environment in which wb does not work.

As the interface shows, it is modelled on wb, but there are minor differences. The activity window offers fewer options. It is intended to have an extra feature, represented by the Grab button, but this is not yet implemented. It seems not to have the following wb features:

• There are no page numbers.
• The page does not automatically change to the page where someone is typing or drawing so it is very difficult to be certain that everybody is looking at the same page.
• There is no obvious way of deleting. Right clicking will delete, as like in wb, except that it seems to delete things in random order.
• The control window does not appear automatically when starting up wbd, but can be opened by <shift><left mouse click>, which is not an obvious key combination.
• It does not seem to offer encryption.

In the tests that were run, input from wbd was displayed in wb and vice versa. To a user familiar with wb the tool is easy to learn, although there are some small differences in ways to carry out operations (moving text can only be done via the Move button, not with the middle mouse button, for example.).

The tool is still not sufficiently stable or predictable to allow a full evaluation.