3DM: A Three Dimensional Modeler Using a Head-Mounted Display

Jeff Butterworth, Andrew Davidson, Stephen Hench and T. Marc Olano

Comments:

Frank’s Blog

Paul’s Blog

Summary:

This article describes an intuitive 3D modeler called 3DM which uses a head mounted display to facilitate the understanding of spacial relationships in a 3D environment. In spite of an increase in the demand for 3D models most modelers are hard to use. A large source of ambiguity with traditional modelers is caused by the use of a 2D screen to represent 6D objects, and a 2D mouse to manipulate them. 3DM sets out to overcome some of the obstacles by placing the user in a virtual 3D environment with a 6D pointing device.

Inspiration for the project was drawn from easy to use programs such as MacDraw which possessed a simplicity and ease of use, requiring no long learning curve. New techniques described, are applicable to many other programs. A 6D 2 button mouse and a simple user interface is employed with a tool palate. Its position can be fixed in space or can follow the current view. The cursor representation changes to reflect the selected tool. Exploration of the model environment can be performed by either walking for proximal excursions, or flying for more global ones. Zooming in and out and a home view button are supported. The screen image is rendered at between 15 to 30 fps.

Surfaces are created in place, and can be done in several ways. An interesting extrusion tool enables manipulation in a dynamic manner according to 6D mouse manipulation. Standard editing tools such as cut, copy, paste, undo, are present. A novel hierarchy feature enables instances of an object to be created so that changes to the parent are passed on the children.

The results emphasize the package’s efficiency, ease of use, good representation of spatial relationships between objects, and advantageous use of the 3D environment. The lack of constraints resulting in difficulties in aligning objects accurately is mentioned and a zoomable grid offered as a solution. The goals of the project have clearly been met as users freely experimented with different shapes and editing features.

Discussion:

The notion to develop an easily accessible 3D modeler using VR in the early 90’s was very forward thinking. In many regards this same goal in a simplified non VR version was taken up by sketchup as it appeared in its original form in 2000.

Although the resulting images look a little primitive from today’s perspective, for the time it is very impressive. It is a shame that a little more detail was not presented concerning the integration of the various components.

An interesting compliment to the HoloSketch paper.

TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning

Jeff Lieberman and Cynthia Breazeal

Comments:

Drew’s Blog

Manoj’s Blog

Summary:

The article presents the use of a wearable vibrotactile feedback suit to improve human motor learning. It is proposed that this can be used as an extension of the human teacher whose movements can be analyzed and used as a model to provide real time corrective vibrotactile feedback to the student. This is seen as a more direct and precise real time method of communicating motor movement. The need to map visual cues from the teacher onto oneself is avoided and unlike verbal feedback the vibrotactile system has a very short response latency which is critical in learning. Additionally, the system can be used for rehabilitation, teaching the blind, or as a full time motor skills teacher running in the background.

The use of a virtual reality teacher has been shown in some complex learning tasks to be more robust against distraction than its human counterpart. At the core of the proposed setup is a Vicon tracking system, which enables both teacher and student movements to be analyzed. Calibration for individual conformational differences is performed before testing is conducted. However, this rather bulky and costly system is intended to demonstrate proof of concept and not as a final production implementation. The vibrotactile actuators are of an electromagnetic type which enable control of both frequency and amplitude of stimulation with no spin up latency.

For the purpose of this study joint angles are used to measure position. To induce corrective movement, vibrotactile actuators on the side of the limb that it needs to move away from are triggered. Rotational movements are induced by using a saltation effect.

For the experimental evaluation, 40 subjects were divided into two groups. One group was given visual and vibrotactile cues were as the other was given visual cues alone. The testing consisted of three parts:

The first part consisted of presentations of still images which the subjects were asked to imitate. Each of these was presented for 5 seconds with vibrotactile feedback (depending on the group) where the subjects were asked to assume the positions as quickly as possible.

The second part consisted of a series of videos demonstrating movements which the subjects were again asked to imitate. I am assuming again that vibrotactile feedback (depending on the group) was active immediately for each video.

The final part was a questionnaire.

The vibrotactile group commented that more concentration was required but with time performance would improve. Analysis of their movements revealed that there was a significant frame by frame improvement over their non vibrotactile counterparts. Vibrotactile communication of hinge movements were much clearer than rotations. The vibrotactile group showed 27% improvements on hinge movements and little or no improvement on rotations. Repeated trials showed 7% acquisition improvements on hinge movements as subjects became accustomed to the system.

Discussion:

This is a very novel concept. In particular the use of a more direct line of communication without the use of language for motor movement learning is interesting. In many regards, this is a much more naturalistic paradigm as we are predisposed to process touch while executing motor movements for mid course corrections when dealing with collisions etc. Real time stimulation as a mode of communication during a movement would be far simpler than using language. Ultimately, this should lead to faster acquisition of movements as there should be less repetition.

The article mentions several possible improvements such as the placement of the tracking marks, stimulators and in particular the rotational feedback.

From a learning perspective, a behavioral comparison of movement acquisition between the two test groups would be interesting.

HoloSketch: A Virtual Reality Sketching/ Animation Tool

MICHAEL F. DEERING

Comments:

Drew’s Blog

Josh’s Blog

Summary:

HoloSketch, which is described in the paper, was to create as an easy to use virtual reality 3D geometry creation and manipulation tool using head tracking, stereo shutter glasses and wand manipulation. Up to this point virtual reality technology has not been used for mass market applications. A possible reason for this is attributed to the resolution limitations of head mounted displays.

Unlike other systems, HoloSketch 3D objects are constructed in a 3D environment using head tracking. The objects are fixed not only in 3D space, but also in scale, and are not subject to distortion or “swimming” traditionally caused by to changes in viewing position. This independence of viewing position makes this system unique, in fact a ruler can even be held up to the objects in virtual space.

The presentation of tool pallets is arranged to overcome the traditional 2D problems of superimposition by remaining in 3D and dissolving between the objects and the radially arranged pallet. Sub menus are also supported. Selection is made by poking and impaling menu items with the wand. Traditional functions such as the creation of object primitives, grouping objects and attributes are supported. The editing features take advantage of a naturalistic grasping action on the wand combined with key strokes for moving operations. In addition there is a 10X reduction in wand sensitivity to reduce jitter. More advanced features such as rotation of objects about an axis can be achieved without numeric entry, scaling, flight path, environment, as well as looped movies to name a few. The standardization of units enables screen size independence.

The system was tested as whole by a graphic artist over the period of a month and determined to be easy to use after an initial adaptation period. In particular it seemed difficult to let go of traditional 2D habits of not moving ones head in order to take full advantage of the 3D virtual reality nature of the system.

Discussion:

The article presents a very powerful and well executed 3D drawing environment. In particular the use of head tracking to enable 3D objects to remain fixed as the viewer examines them from different angles is quite unique and of tremendous value. The head tracking system combined with dynamic calibration for inter ocular separation working seamlessly in the background to achieve this is impressive. However as is mentioned in the article, due to differences in the fundamental architecture of traditional applications it would make compatibility with HoloSketch technology difficult. At the time of publication, in view of the features offered by HoloSketch this may not have been a disadvantage. The evaluation of the system certainly needs more than one individual test, and more information on the implementation of the wand would have been most welcome.

Since the publication of this article, the applications of this system outside 3D drawing have range from remote control robot surgery to industrial, hostile environment and military applications.

Wearable EOG Goggles: Eye-Based Interaction in Everyday Environments

Comments:

Paul’s Blog
Kevin’s Blog


Summary:

This paper presents an eye tracker which uses Electrooculography (EOG) for eye motion sensing and context recognition. The use of eye movement gestures has advantages of privacy, and is a natural indicator of attention and intention. Furthermore it does not alter the temporal sequence actions by requiring a dwell time to make selections. The EOG system has advantages over the video based eye trackers which are bulky, obtrusive and not geared for online embedded processing which is a critical necessity for eye movement based communication.

EOG was shown to be adequate to read eye movements robustly in everyday situations with a mobile setup. This was first tested with a commercially available system, after which a purpose designed one was constructed. Unlike previous studies which focussed on using eye movements for object selection, the present one was concerned with fast interaction. EOG recorded Saccades, fixations and blinks have been reliably used for robot control from fixed positions.

The EOG goggles designed and used in this project are a miniaturized highly wearable and self contained stand alone set up with embedded recording. In order to compensate for physical activity an ambient light meter and an accelerometer were incorporated into the frame. EOG measures the electric potential field surrounding the eye. Any movements of the globe, which itself is a dipole, result in a change in the surrounding potential field which is translated into a directional movement signal by the system.

Blinks are detected and removed in this particular application but can be used for selection etc. Consecutive Saccades in the horizontal and vertical planes are recorded. A Continuous Wavelet Transform - Saccade Detection algorithm is used for Saccade detection. Detected Saccades directions are mapped into “L”, “R”, “U” and “D”. Diagonals are determined from simultaneous combination of these and mapped into “1”, “3”, “7” and “9”. Recognition of several consecutive gestures is done through string matching.

The results for performing a test structured in the form of a game of increasing difficulty showed that 30% of the subjects had difficulty maintaining concentration, but overall the system outperformed speech and hand gestures.


Discussion:

The EOG system presented as a more compact and portable alternative to more traditional eye tracking devices has tremendous utility. Additionally, the use of the naturalistic modality of eye gestures is interesting.
However, without extensive practice the performance of unnatural eye gestures must be very distracting as it is using a modality which has a very high attentional priority in a very non natural manner. Repeatedly performed forced Saccades, independent of objects in the visual field would be potentially disturbing.
The use of this system in a mobile setting makes no mention of OKN or how it would be filtered out. There is some mention of an accelerometer but no indication as to how it is implemented. Are eye gestures only possible while stationary? Unlike video based eye trackers where fixation positions, or dwells are recorded in addition to Saccades no mention is made of how spontaneous Saccades would be separated from gestures.
The exact nature of the test is not identified, and the basis for concluding the superior performance of eye gestures in comparison to speech and hand gestures needs explanation.
This system would be better suited to non dynamic environments where spontaneous eye movements and the demands on the operators attention can be predictably controlled.

Vogel, et al. Distant Freehand Pointing and Clicking on Very Large, High Resolution Displays.

Comments:

Manoj’s Blog

Drew's Blog

Summary:

This paper presents the need to manipulate objects presented on large high resolution displays from both close up and a far. The pointing device must be able to transition smoothly through this range to be effective. Under these requirements traditional devices are not adequate. A more naturalistic approach of using hand gestures is presented. The vocabulary of hand gesture is described along with some of the challenges in their implementation. For example the calibration of the air tap posed some difficulties as there is no lower limit to the movement. In contrast the thumb trigger although providing tactile feedback proved uncomfortable and tiring. Both visual and auditory feedback signals were presented to aid the user. Additionally a graded ambiguous posture visualization was created which greatly facilitated learning. Three types of pointing were explored. Ray casting using the index finger and a dynamic recursive lowpass filter. Relative pointing and clutching using a dynamic lowpass filter to eliminate jitter in combination with relative measurements taken from a starting position. A simple and effective use of cursor orientation was used to provide feedback when clutching. Finally RayToRelative pointing with simultaneous recalibration.
The goals of the study were to compare the task completion time, error rate, and recalibration characteristics. The user was required to hold their hand in a predetermined position for two seconds before the cursor and first target would appear. The user was required to select it before the next trial. RayCasting was found to be faster where clutching would have been required or when selecting large targets but its high rate of error prevented it from being practical. No significant difference was found between Relative and RayToRelative techniques.


Discussion:

A very interesting paper presenting a minimalistic solution to maximize the potential of very large high resolution display usage.
The work is clearly presented and thoroughly executed and tested.
A point which is covered in the article itself is the elimination of hand markers, to make the execution solely in the hand of the user.

Hansen, et al. Noise Tolerant Selection by Gaze-controlled Pan and Zoom in 3D.

Comments:

Manoj’s Blog

Drew’s Blog

Summary:

StarGazer, a 3D pan/zoom interface for presenting noisy data is described. A two tier concentric arrangement of letters is used as a keyboard with which the system is tested. A rectangular area in the center of the display is used for zooming in on the display space. During use, a visual saccad results in a radial translation of the target area to the center of the screen where it is magnified. The greater space between objects at the center of the display increases accuracy and enables additional contextual information to be presented. Panning enables adjacent areas to be explored while maintaining the same level of magnification. Visual and auditory feedback are provided. Unlike other eye tracking preparations, StarGazer uses the gaze vector to determine where in 3D space the eye is looking.
Testing consisted of six preparations, three screen sizes with or without noise. Each of the 48 subjects were assigned one preparation. They were asked to type their name on a standard keyboard and the given an explanation of the StarGazer system. They were then given a two minute familiarization period with the system and finally they were asked to enter their first and last name as quickly and accurately as possible.
Two further tests were performed with 3 subjects to explore the effect of latency, and with 7 subjects to explore the effect of zoom.
Results are questionable as the testing procedures do not have consistent group sizes or adequate controls.


Discussion:

The StarGazer interface is a creative solution for overcoming the limitations of affordable eye tracking technology and providing a screen size independent solution.

However there are several points which are not satisfactory:
No description of the type of noise introduced is given.
No gaze direction vector is supplied by the eye tracking hardware, yet the use of this by StarGazer was identified as a novel attribute.
During testing, there is no control for individual differences as subjects were not required to perform all tests.
Furthermore, subsequent tests varying latency and zoom were conducted with disproportionately smaller subjects some having prior experience.
WPM comparison with another system showed StarGazer to be half as fast in its original form.
Robustness from noise attributed to low error rate could be due to other factors, i.e. concentration due to task difficulty or unfamiliarity etc.

A more comprehensive testing procedure with proper controls would give a more representative evaluation which could be compared to other similar systems.

My email address is mrussell (at) tamu (dot) edu

I am a first year masters student in computer science.

I am originally from London in the UK but have been here for many years.

I am interested in taking this class because sight and touch are critically related to my research interests. They are amongst the most dominant sensory input modalities for awareness. The combination of theses two modalities or the ability to abstract information between them is of particular interest.

I am not certain exactly what I will be doing in the next 10 years. Whether in industry, business or academia. I am certain my research interests will have a dominant role.

I am not sure what the next biggest technological advancement in computer science will be.

Eye tracking and wearable hepatic sensory and tactile devices. Both for Human machine interaction and behavioral measurement.

I can think of two individuals in particular:
Nicolo Paganini for his daring and technical brilliance, to hear and see him play with my own ears and eyes.
Leonardo da Vinci whose imagination transcended the technology and materials of his time.

One of my favorite films is KOLYA a Czech film directed by Yan Sverak. A wonderful cross-section of life.

My research interests are in psychophysics, visual pattern and motion perception, visually guided motor action, spatial awareness, selective attention, abstraction, learning and memory.