Over the past decade, the research in techniques that combine virtual and real world has grown rapidly. Different realities (Virtual, Mixed or Augmented) have been applied in design sector in order to merge physical space with virtual world. These realities can be helpful to designers when presenting their ideas and concepts in less abstract and more realistic way. There is a variety of explanations and definitions of realities. The one I chose is focused on the difference in perception and interaction as important features of design. The need for a classification approach suitability issues for the adequate adaptation of different realities and technologies. It can also help to maximize benefits of reality technologies for certain design activities and help designers in comparing different technologies.
Figure 1. Order of reality concepts ranging
Source: http://www.sd.polyu.edu.hk/
As we can see from Figure 1, reality concepts range from Real Reality through Mixed Reality to the Virtuality. The definitions of particular realities are sometimes very difficult to identify. Some reality can cover a part of another one. The range in Figure 1 adjusts design comprehension and collaboration for specific activities along a design process.
Virtual Reality
The visualization technology creating a total virtual environment is called the Virtual Reality (VR). This computer-simulated environment is able to simulate places in the real or physical world. During the recent decades, architecture visionaries adopted ideas of virtual reality when they wanted to present their design concepts. Computing technologies have been developing and supporting more and more advanced graphic capabilities (including CAD software, graphics hardware acceleration or head mounted displays).
The architecture designs and urban plans in larger context used the virtual reality concept in 2001 for the first time. Since then, the progress has rapidly accelerated and working in VE became a part of architects' profession.
Virtualized Reality
Virtualized reality world enables users to see a virtual reconstruction of a real-world event. It is a simulation of a place in the real world. This technology places a set of cameras at an event and allows the user to fly around to watch the event from different positions. Take a football match as an example. The use of virtualized reality would enable us to watch a match from the point of view of the quarterback. Since 1993, the technology that could enable us to see these views of the world has been an area of ongoing research.
Virtualized Reality can bring great benefits to architects and landscape designers, especially in a design review of a final proposed model. The medium could enable the designer to appear in the proposed space and perceive the perspective, the lighting, etc.
Augmented Virtuality
Merging a real object into a virtual environment is provided by technology called Augmented Virtuality. Multi-modal, layered, physical elements are dynamically integrated into the virtual environment and can interact with the virtual world in real time. There are various technologies that use the Augmented Virtuality technology, such as streaming video or 3D digitalization of real objects.
Design sector gives space for more innovations in Augmented Virtuality but despite its potential capability it has not received particular attention compared to Augmented Reality or VR.
Mediated Reality
Computer-Mediated Reality refers to augment or removed information from perceived reality. The perception is provided by some kind of electronic device, for instance a wearable computer, EyeTap or a smart phone. The device works as a visual filter between information from real world and information received by a viewer.
Mediated Reality has been used for architectural applications to remove or mask visual data. The technology is also operated with interactive computer interfaces.
Augmented Reality
The term Augmented Reality (AR) refers to an enhanced view of a physical real environment merged with augmented layer of virtual computer-generated elements. Artificial information can be saved and restored as a layer on top of the real world view. AR is a sub-mode of Mixed Reality (Fig.2) adding the information layer proceed in real time.
The AR technology has potential to bring many benefits in the design sector, especially in architecture and urban design. Head-mounted displays, virtual retinal displays, sensors and actuators are most commonly used for visualization purposes.
Figure 2. Order of reality concepts ranging from Reality (left) to Virtuality (right)
Source: http://www.sd.polyu.edu.hk/
Amplified Reality
To amplify reality means to blend actual surroundings with computer generated images. A real object directs the flow of information and unlike with the AR system, the user is in control of the information. Amplified Reality enhances the perceivable features of a real object in the real world while AR is focused on how the user perceives reality.
Figure 3. Classification according to correlation between perception and action and level of interaction
Source: http://www.sd.polyu.edu.hk/
As indicated in Figure 3, Mediated, Augmented and Amplified Realities have a similar level of perception and action and the closest position to the real reality from listed realities. Similar level of perception can be identified in their technology definitions; the virtual element is adjusted with the real object. High correlation between action and perception makes those three realities beneficial for planning activities.
Apparently, Amplified Reality has the highest interaction with real objects. The second highest interaction belongs to Augmented Reality. For better understanding, I use an example to explain the difference between these realities. Imagine that you are going to colour walls in your room with a new colour. One possibility is to re-paint the walls. The other one is wearing a pair of eyeglasses with glasses coloured with your preferred colour that would change your perception of the walls. Wearing coloured eyeglasses represents adding a layer of information to the real environment. Thus, changing the perceived information without an interaction creates AR. Re-painting the walls corresponds to amplified reality because of an interaction with the real world. To sum up, an Amplified Reality system changes the attributes of the physical object while in Augmented Reality, the impression of the perceiver is altered. Mediated Reality has a lower interaction with the real world environment. It is an environment in which the real objects of Mediated Reality are altered or removed by the imposed virtual objects.
Augmented Reality in relation with Architecture
A growing research in computing and new technologies brings us new opportunities and development in many sectors, from medicine to art. The design sector is looking toward various IT technologies that enable improvements in current state-of-the-art architectural visualization, design and construction process, etc. Firstly, the design concept is created in designer's fantasy, followed by developing the concept during the design process and finally bringing it into the physical world. In the 20th century, the development of virtual environment brought new opportunities in the design process to architects and designers. In the past, the only way that the design process worked, was that architects made technical drawings and built models. Thanks to Computer Aided Design (CAD) packages we are able to make our abstract visions more realistic by rendered visualisations. The process of working in CAD consists of creating 3D graphical model and rendering it on a graphics workstation. Development of Virtual Reality enables more and more ambitious visualisations. In VR, it is possible to simulate a walk through the 3D object. This facility offers a simulation of the viewing of the new design in virtual environment to the customer and designer. However, could we show the new design to the client in a real environment? There is one possible solution, using Augmented Reality where a user would view the virtual information simultaneously with the physical world.
Augmented Reality has an important contribution in design process. The final architectural project or urban plan heavily depends on the effectiveness of collaboration between team members. AR tools provide better understanding of the design concept and support effective collaboration, while visualisation aid facilitates it just to certain content.
Collaboration within research
Augmented Reality has been developing since the second half of the 20th century. Researchers and scientists came up with a variety of applications in many different sectors. AR tools usable in design and architecture are still in progress. In the past, the researchers worked primarily on scientific visualisation in AR environment. The collaboration with designers is essential. The collaboration started just recently and there are some usable tools which need to be improved in order to be applied. Architectural studios and universities already participate in this collaboration and provide many ideas and concepts.
Collaboration in design process
The quality of design, particularly large scale design and sophisticated projects, often depends on successful collaboration of team members. A design meeting involves a team of participants while the actual design is still made by individuals who are using CAD software and PCs. When special problems are discussed and solutions suggested, it is up to the individual to perform and apply those changes in the design. Collaboration is then limited just to early design sketches. AR technology supports the collaboration in common meetings and allows manipulation witch spatial problems. Interaction mechanisms allow all the users to be involved in the work of individuals.
Applications
Many of the applications can be useful also in the design sector. This paper presents AR tools in different phases of design. First two examples, Sketchand+ and BrenchWorks, are prototypes that apply AR system into the design sector. The other two examples, AR CAD and ARTHUR, are relatively well developed by researchers as well as architects and intended to be used in architecture and urban planning.
Sketchand+
Sketchand+ is a collaborative AR application aided 3D sketching tool developed by Hartmut Seichter, an architect and software engineer. Seichter's goal was to integrate the new AR technology to create and edit sketches, the early phase of design. The system was developed in 2003 and it was the first prototype of using AR in architectural practice.
The devices for using sketchand+ system are an augmented digitizer-tablet and strokes for creating 3D sketches. The software handles audio and text messages. It is controlled from within the Mixed Reality environment using tangible interaction techniques. Sketchand+ works with AR Toolkit and an early version of the libTAP VR/AR framework.
In the following pictures, you can see a user with AR glasses creating a 3D object and working with it. Firstly, he creates a model by using a sketchpad and stroke and then he observes the model in the real physical model.
Figure 4. Sketchand+
Source: http://technotecture.com/
BenchWorks
The application was developed by the same architect and engineer, Hartmut Seichter, one year after the Sketchand+. It represents the next generation of Sketchand+ and facilitates devices and techniques usable for urban design proposals. BenchWorks uses AR Toolkit System in combination with optical magnetic tracking. The system can be used by two people simultaneously observing virtual and real objects. This early prototype provides the precision of a magnetic tracking system and free manipulating of tangible interfaces.
Working with BenchWorks is illustrated on the images bellow.
Figure 4. Benchworks
Source: http://technotecture.com/
CASE STUDY 1
Augmented Round Table for Architecture and Urban Planning (ARTHUR)
ARTHUR was released in 2004 as another AR interface for roundtable design meetings, supporting mainly urban and architectural proposals for multiple users. The goal was to develop interactive collaboration based on AR technology and integrate CAD systems into AR environment. An augmented round table provides the view of physical working environment and virtual components where real objects are used as tangible interfaces to create a virtual environment. Collaboration is supported by verbal and non-verbal communication by the table. The system provides optical see-through AR displays. For using the application, you need AR glasses, the ARTHUR round table, twin cameras fitted in the glasses and computer vision techniques. ARTHUR is based on many particular components, for instance MORGAN AR framework, computer vision based mechanisms or AR displays.
Figure 1. The basic components of the AR framework
Source: http://www.jvrb.org/articles/34/
Figure 1. See-through head mounted display
The AR glasses and the real world placeholders provide all users around the table equal access and contact to the design user interface. The user is able to observe the scenario in 3D, using head mounted displays. It is possible to render the scenario monoscopic or stereoscopic. 3D visualisation augments see-through video and the view from head mounted camera is layered by the virtual scene elements.
Computer vision is used as the basic user input. Figure 1. Computer vision systems
There is a major difference between input devices. Natural input is an input such as hand gestures or fingertip moving, and marker based device, for instance a placeholder object and wands. CV system controls the marked based devices. There are two types (Fig.), one is placeholder object and the other one is wand-like pointer. Placeholder objects are moved in the table plane. They can be manipulated by a user. Pointers have 3 buttons to pick the operation with a pointer.
Source: http://www.jvrb.org/articles/34/
Conclusion and Future work
