This paper presents Essential Reality works on 3Dimensional Human Computer Interface for changing 2 Dimensional visual to 3 Dimensional visual. Using data glove it's possible to put it on like a normal glove and it then acts as an input device that senses finger movements and hand position (3coordinates) and orientation (3 coordinates) in real time. They even experiment with providing tactile feedback in the glove so that it would feel different to bend your fingers depending on whether you were "grasping" a data object with the glove. To provide three Dimensional Interactive Graphical User Interface for Windows Operating System. This type of interface is very natural and provides more room for placing windows and other graphical objects on the computer screen. The virtual reality data glove is used to provide interactivity for handling and manipulating various icons and graphical objects on the computer screen.The current Human Computer Interface for influencing icons, folders, graphical objects and tasks in Windows is 2 Dimensional. The mouse is the critical interface that forces 2 Dimensional Graphical User Interface as the icons and windows are placed 2d Surface. The limitation of surface do not allow large no of windows and icons to be positioned on the screen. If more no of windows are forcibly open some of them get overlie. However in 3 Dimensional spaces we get one more dimension and it's better to work with 3 Dimensional instead of 2 Dimensional. We can operate on windows and icons in 3 Dimensional visual. Number of windows and icons can be situated top, down, left, right, and even in back surface. As the windows are pushed back their appearance will turn in to small due to perceptive vision. From user side it looks like a tunnel in which icons and windows glide in the space as the windows are dragged to the for front they turn in to bigger and they pushed back they turn in to smaller. Windows can also be slanted to reduce their size from the user point of view. This will facilitate wrapping of more of icons and windows on computer screen using P5 Data Glove. To provide three Dimensional Interactive Graphical User Interface for Windows Operating System. This type of interface is very natural and provides more room for placing windows and other graphical objects on the computer screen. The virtual reality data glove is used to provide interactivity for handling and manipulating various icons and graphical objects on the computer screen.
Keywords:
3D user interface, 3D displays, data glove, gesture recognition, perspective projection, GUI, HCI, WIMP
Introduction
The graphical user interfaces (GUIs) that are known as WIMP (which stands for Windows, Icons, Menus and Pointing) have been dominant for more than two decades. In a time when computer tools and applications became used by millions of people, software engineers realized that the interface was one of the main factors that determined the success of an application. WIMP user interfaces provided a "de facto"standard that, thanks to the existent consistencies in the look and feel of application interfaces, gave the user ease of learning and ease of use. As a result, user interface design was introduced in the life cycle of software development, and many methodologies have been proposed which support the development of user interfaces [1].
Interface technology advances towards a fourth generation of user interfaces, a "post-WIMP" generation which is not only based on 2D widgets such as menus, forms or toolbars, but also on 3D user interfaces that enable a more natural and intuitive style of interaction Human-computer interaction currently faces the challenge of taking advantage of new technologies, which efficiently use the capabilities of the computing systems of today and more effectively match human capabilities and perception. We investigate how 3D GUI can be useful in software applications of today and identify general areas where 3D technologies can be more useful than other technologies [2]. This paper consists of a survey of 3D standards, GUIs and in particular 3D GUIs. There is no doubt that 3D is going to be the way of the future when it comes to software. The revolutionary 3D Wonder 3D Desktop improves and makes it easier to use your Windows PC. Its 3D GUI replaces the decade old 2D desktop with a more productive and enjoyable 3D interface. Lets you organize your tasks and icons in previously impossible way. Animated real-time .3D Worlds create an overwhelming entertaining and fun to use desktop.
The existing 3 D environment in the world of proprietary software
In the last few years some interesting software application has appeared that have brought 3 dimensionality to the world of computer graphical interface. In the literature the term "3D interface" is used to describe a wide variety of interfaces for displaying and interacting with 3D objects. True 3D interfaces i.e. interfaces with all its components in a 3D environment have not yet had any major impact outside the laboratory. The interfaces nowadays referred to as 3D GUIs are almost exclusively "hybrids" between 2D and 3D interfaces. In this thesis we are introducing complete 3D GUI having 3D Windows, Icons, Menus, and Pointer (WIMP) and more on those 3D characters.
Apple was the first mainstream vendor to choose three-dimensionality when it released MAC OS X in 2001.This operating system interface introduced some functions able to take advantages of the 3d graphics cards installed in Macintosh computers. There are semi transparency effects able to permit a better management of overlapped windows, something that recalls the 'glass metaphor 'used 6 years later in Microsoft Window Vista. [3]
Without doubt the "glass" is one of the main 'new' features of Aero the 3D GUI present in the Windows Vista pack. Aero's main effects are, beyond the semi-transparencies, , a 3 dimensional windows managing system called Windows Flip 3D. With this feature a new concept of Spatial organization and management based on a perspective effect that allows better management of the desktop space in the Windows graphical interface has been introduced. This makes possible more natural navigation. This new function requires a powerful graphics card creating technologies limitation because not all existing computer have similar capabilities, this trend, with the pretext of 3d GUIs, will surely continue. [3]
Now talking about open source we must begin with the Sun looking Glass project, supported by the Sun Microsystems software house. In this system, windows start in 2d normal mode but can be manipulated as 3d objects that can be set any angle or turned completely around the user but still in appearance of 3D not the real one. [4]
In the Linux/Unix world we focus in particular on the Compiz project, the first software to popularize the 3D interfaces on this operating system. The effects of virtual desktop and cubes are effects that make possible fast access to all six segments of off-screen space. Instead, with the concept of the virtual desktop and its 3D visualization, the off-screen space is now shown. But the limitation is still the 2D characters and 2D Input Device. [5]
3D Top uses the icons that are present on your normal desktop and represents them in 3 dimensions instead. This approach enables you to fly around your desktop, change the shape of the icons and arrange or drag them in 3D also. In addition you can create colored spotlights, paintings (bitmaps), background and floor textures, clocks, flags (shortcuts) and whatsoever the future may bring. But It is Hybrid version of 3D GUI not the real one. [6]
The Cubic Eye is the revolutionary 3D Application Platform transforming the way people view and interact with their data. The only 3D platform based on cubes, Cubic Eye is the evolution of the traditional 2D user interface into a real-time 3D environment, providing a richer, more productive, more enjoyable experience. But this application is virtual vision not the real vision.[7]
Currently available application will provide the features of rotation, scaling, tilting, Selecting, moving, windows, and icons. But all this functionality is done using mouse and keyboard. This way of handling of 3D objects is some what difficult and not that much efficient in working.
One more cons with current application available in market are that it does not provide any facility for adjusting the fonts according to the rotation of 3D objects. So its reliability is not that much powerful,
Visualization
In a 3D GUI the visual elements are genuinely three-dimensional: they are situated in xyz space, are defined in terms of 3D coordinates, need not be flat and may contain spatial regions (volumes). At this stage we plan on changing the structure of WIMP and characters, which are essentially 2D constructs, which gives real of 3D presentation and provides more advantageous.
To implement this 3D WIMP components our prototype use perspective projection. Here to display real 3D interface to 2D display device conversion is the necessity. Formulas for this are
xd1 = ((xc * z1) - (zc * x1)) / (z1 - zc)
yd1 = ((yc * z1) - (zc * y1)) / (z1 - zc)
xd2 = ((xc * z2) - (zc * x2)) / (z2 - zc)
yd2 = ((yc * z2) - (zc * y2)) / (z2 - zc)
(xc ,yc, zc ) is the point from where user look inside the object
(x1,y1,z1) and (x2,y2,z2) two original end points of segment
(xd1,yd1) and (yd2,zd2) after perspective projection two end points
There are 5 main thrusts to our prototyping work.
3.1 The desktop manager
The desktop manager metaphor we have used for the 3D space in which we arrange windows is that of a tunnel. The user is positioned in the middle of the tunnel looking toward the other end. The tunnel, and windows in it, is displayed with a perspective projection. The tunnel shaped desktop provides 5 surfaces to manipulate with multiple windows.
Figure 1.3D Tunnel Shaped Desktop
3.2 Window Manager
Windows may be positioned at arbitrary depths in the tunnel. The normal orientation is orthogonal to the longitudinal axis of the tunnel, or more simply, ``front-on''. As a user pushes a window further into the tunnel its size is diminished in the normal inverse size to distance relationship of perspective projection. In addition to the front-on window display mode there is a ``hanging'' mode where the windows are hung on the left or right walls of the tunnel, as shown in figure 1. Hanging all the windows allows the user to quickly gain an overall idea of where the windows are in the tunnel.
Figure 2. Multiple windows placed in 5 surface of desktop
The first is an overview area to the right of the tunnel which may be seen in figures. The overview area provides a plan or "top view of the tunnel" and is provided in addition to the main ``down the tunnel'' view. Windows may be selected and then moved up and down in the overview area, corresponding to back and forward in the tunnel. The last major component of our window manager is a console, positioned at the bottom of the screen as shown in figures Controls are provided on the console for changing global settings affecting windows. There is a button to hang all the windows at once, rather than hanging individual windows.
3.3 Character Manager
To tilt and rotate the multiple windows with normal 2D characters cause the problem of sharp visibility as 2D character are not compactable with 3d window manipulation. In our prototype we have introduced 3d characters which are compactable with 3d perspective projection of individual window as shown in figure.
Figure 3.XY Rotation of Window & 3D Characters.
Figure 4.YZ Rotation of Window & 3D Characters
3.4 3D cursor : Virtual Hand
The fourth thrust of our 3D GUI prototyping work is exploring the design and usability of a 3D cursor controlled by a 3D input device. One of our 3D cursors is shown in figure where it is being used to select a 3Dwindow, Icon etc.
Figure 5.3D Cursor: Virtual Hand
Most interactive 3D graphics applications employ a 2D cursor and 2D pointing device for interaction. Given that the user is manipulating 3D objects or worlds in these applications, this gives rise to a fundamental mismatch in dimensionality. In the context of a 3D application the cursor ``floats'' over the top of the objects rather than being part of the scene. We believe the use of a 3D cursor, introduced into the scene as an object in its own right, controlled by a 3D (six-degree-of-freedom input) device is a better approach for interactive 3D applications, including a 3D user interface, than the traditional approach. Recently, work in the area of 3D interaction has intensified. We believe this to be a significant area for research in its own right, with the potential for many important practical outcomes.
3.5 Icon, Device and Taskbar, Menu Manager
Icon manager will active the window by just selecting or clicking on particular icon. Taskbar manager will generate icons on the taskbar for active window.
3D Start menu is implemented with our own 3Dcharacters to apply realistic view.
This four feature are visualized in the Figure 6 ..
Figure 6. 3D start menu and sub menu
Device manager will start timer and captures the 8 values (ø1 to ø5 and x, y, and z) from p5 Data glove. It will maintain circular queue and generate average value, global variables are used for storing purpose.
Figure 7 Actual Working and Interfacing
INTERACTING WITH Interface
The most natural way for humans to manipulate their surroundings, including the windows e.g. on their desktop, is of course by using their hands. Hands are used to grab and move objects or manipulate them in other ways. Hands can be used to communicate with others and state intentions by making Postures or gestures. the most promising approach to minimize the cognitive load required for learning and using a user interface in a virtual environment is to employ a gesture recognition engine that lets the user interact with the application in a natural way by just utilizing his hands in ways he is already used to.[7]
4.1 Applied Hardware:
The glove hardware we used to realize our gesture recognition engine is a P5 Glove from Essential reality, shown in Figure. The P5 is a consumer data glove originally designed as a Game controller. It features five bend sensors to track the position of the wearer's fingers as well as an infrared-based optical tracking system, allowing computation of the glove's position and Orientation without the need for additional hardware. The P5 consists of a stationary base station housing the infrared receptors enabling the spatial tracking. The attainment of position and Orientation data is achieved with the help of reflectors mounted on prominent positions on the glove housing dependent on how many of these reflectors are visible for the base station and on which positions the visible reflectors are registered, the glove's driver is able to calculate the orientation and position of the glove.
Figure 6. Essential Reality P5 Data Glove
During our work with the P5, we learned that the calculated values for the flexion of the fingers were quite accurate, while the spatial tracking data was, as expected, much less reliable. The estimated position information was fairly dependable, whereas the values for yaw, pitch and roll of the glove were, dependent on lighting conditions, very unstable, with sudden jumps in the calculated data. Because of this, additional adequate filtering mechanisms had to be applied to ascertain sufficiently reliable values. Of special attention is the very low price of the P5. It costs about 50 €, by comparison to about 4000 € for a professional data glove, which of course provides much more accurate data but on the other side doesn't come with integrated and transportable position tracking. Indeed, the low price was one reason we chose the P5 for our gesture recognition, because it shows that serviceable interaction hardware for virtual environments can be realized at a cost that makes it an option for the normal consumer market. The other reason for our choice was to show that our recognition engine is powerful and flexible enough to enable reliable gesture recognition even when used with inexpensive gamer hardware.
4.2 Recognition Process
Our recognition engine consists of two components: the data acquisition and the gesture manager. The data acquisition runs as a separate thread and is constantly checking the received data from the glove for possible matches from the gesture manager. As mentioned before, position and especially orientation data received from the P5 can be very noisy, so they have to be appropriately filtered and smoothed out to enable a sufficiently reliable matching to the known postures.
First, the tracking data is piped through a deadband filter to reduce the chance of jumping error values in the tracked data. Alterations in the position or orientation data that exceed a given deadband limit are discarded as improbable and replaced with their previous values to eliminate changes in position and orientation that can only be considered as erroneous calculation of the glove's position. The resulting data is then straightened out by a dynamically adjusting average filter. Depending on the variations of the acquired data, the size of the averaging values is altered within a defined range. If the data is fluctuating in a small region, the size of the filter is increased to compensate jittering data. If the values show larger changes, the filter size is reduced to reduce latency in the consequential position and orientation.
Figure 8.P5 Gesture Trainer
The resulting data is reasonably correct enough to provide a good basis for the matching process of the gesture manager. To lower the possibility of misrecognition, a posture is only accredited as recognized when held for an adjustable minimum time span. During our tests it showed that values between 300 and 800 milliseconds are suitable to allow a reliable recognition without forcing the user to hold the posture for too long.[8]
System Evaluation
FEATURES
COMPIZ LINUX
AERO MICROSOFT VISTA
MAC OS X APPLE
GLASS PROJECT SUN MICRO
CUBIC EYE
MAW3
OUR SYSTEM
TUNNEL SHAPED DESKTOP
√
Ã-
Ã-
Ã-
√
√
√
ROTATION AND TILTING OF WINDOWS
√
√
√
√
√
√
√
3D CHARACTERS
Ã-
Ã-
Ã-
Ã-
Ã-
Ã-
√
3D INPUT DEVICE
Ã-
Ã-
Ã-
Ã-
Ã-
√
√
REALISTIC PERSPECTIVE PROJECTION OF 3D WIMP
Ã-
Ã-
Ã-
Ã-
Ã-
Ã-
√
Conclusions
In this paper, we suggested the 3D Desktop System which can help users of personal computers to work under more comfortable environments. Our system provides users with powerful 3D desktop which is handled by 3D input device .Our system is efficient and flexible in handling the numbers of task spaces and provides more smooth graphics by specially generated 3D characters and 3D Cursor.
