Augmented Reality

Introduction

Augmented reality (AR) is a field of computer research which deals with the combination of realworld

and computer-generated data (virtual reality), where computer graphics objects are blended into

real footage in real time (Ashley, 2007). Our company's product, Home Design Augmented Reality (HD

AR) focuses on enhancing options for interior design professional and real estate agencies. Our product

allows an opportunity for real estate agencies and interior design professionals to provide a service to

their clients that will not only give their clients a realistic view of what they are purchasing before their

purchase, but also will differentiate these businesses, giving them a competitive edge that will disrupt

their competitors while enhancing their services to the consumer. Home Design Augmented Reality will

combine real and virtual reality, and be interactive and take place in real time, while being registered in

three dimensions.

Technology

Description of Augmented Reality

Augmented Reality (AR) involves changing a person's perception by introducing virtual objects

into a person's real world view, thus providing a "mixed reality" environment, in which real and virtual

objects coexist.

In effect, AR is "the less obtrusive cousin of virtual reality" (Schmalstieg et al, 2002) because

everything is not rendered virtually, in fact relying on the real world view to determine what virtual objects

to introduce, and where to place it, relative to a person's view. A graphical representation of the virtuality

continuum, as drawn by Anders, shows the relationship between virtual reality, here called virtual

environment, and AR:

Figure 1: Virtual Continuum (Anders, 2008)

4

H D A R

Components of Augmented Reality

The Computing Aspect

The background of Augmented Reality came from the term "computer-augmented reality"

(Wellner et al, 1993), which was initially described as the "opposite of virtual reality". Eventually it was

called AR, dropping the word "computer" altogether, which just happens to be the most important

component of AR. The virtual objects are computer-generated data, and as such are introduced by

superimposing them onto the aforementioned real world view. These computer generated virtual objects

can simply be text data, a 2D image, or a rendered 3D image, and is limited only by the collective

capability of the AR's computer system. The ever increasing power of today's computer systems with

high-end multicore CPUs, capable chipsets, appropriately-sized RAM, high RPM hard drives, and video

cards with fast GPUs that are able to send out the necessary video output is more than capable of

handling the hardware requirements of an AR solution. The other side of the coin is that software

development for AR development has been, for the most part, specialized. However with the emergence

of the web 2.0 era, software can now be easily obtained for Augmented Reality, like the open-source

osgART 1.0 SDK (HITLab NZ, 2006), the bourgeouning mobile app market, with Google Android

development in particular, or a custom solution built by inhouse or outsourced software developers, which

we will leverage.

The Display Method

As for the display, we mentioned that one must see two views superimposed together: the real

world, and the virtual world. The view can be simply seen through an appropriate video display device,

may it be a head mounted display (HMD) or an LCD. Generally there are two category of displays:

optical see-through, in which translucent glasses are utilized, and video see-through, where the output is

a combined video feed of live video from the person's perspective, with augmented reality elements

already superimposed (Azuma, 1997). An example of this is utilized by the AR Android application

Wikitude. Utilizing the Android powered G1 mobile phone, which sports a 3.2 megapixel camera, it

enables the picture preview function of the phone as the "live video" from whence an AR virtual object, in

this case Wikipedia-derived text/picture information, is superimposed. A video can be found at

http://www.youtube.com/watch?v=8EA8xlicmT8. In the aforementioned video, one can see the AR

application utilizing the 2 megapixel video camera to shoot live video, then virtual elements are

superimposed on top of it, which is then presented on the LCD display. It is the Kappa Group's

observation that almost all AR applications now uses video-see through, and as such we will be utilizing

said display category in conjunction with a portable LCD screen.

The Tracking System

For AR to successfully place virtual objects in the right place, the technology must know where to

relatively superimpose objects. The virtual objects should be aligned properly with the real objects in the

real world (Bajura & Neumann, 1995) for successful operation. As such, the common method for tracking

is to use fiduciary markers. Fiduciary markers are used as points of reference by the AR technology, so

that proper placement of virtual objects can be achieved (Bajura & Neumann, 1995). These markers are

applied manually to real objects, which allow the AR technology to register successfully.

5

H D A R

An example of AR utilizing fiduciary markers is CNN's Holographic Interview System technology

that they debuted on the 2008 Election Night coverage. Utilizing 35 video cameras in a 220° ring

configuration, backed by a green screen, video is shot, angles are synchronized, data is crunched, and a

finished video stream is then shown (Chen, 2008). A CNN explanation and interview video can be found

at http://edition.cnn.com/video/#/video/tech/2008/11/06/welch.faux.hologram.explainer.cnn . The fiduciary

marker is the red circle on the floor in front of CNN host Wolf Blitzer, from which the AR solution

superimposed the computer generated video image of CNN correspondent Jessica Yellin on live video.

However, since fiduciary markers had to be placed beforehand in the real world, some AR developers are

turning to non-fiduciary based solutions. This solution is attractive because it allows mobile point of view

(POV) AR solutions that does not lend well to preplaced markers. As such, some AR applications are

increasingly incorporating new technologies such as GPS coordinates, markerless video tracking, and

accelerometer and compass for image registration to resolve tracking issues. The aforementioned

Wikitude AR application utilizes such a marker-less tracking solution.

Our Innovative Product: Home Design AR

Technical Overview

We are therefore proud to offer our Home Design Augmented Reality product, HD AR. HD AR is

a product solution that allows 2D and 3D visualization of appliances and furniture in an otherwise empty

room or space in a house or building. Running on today's powerful laptops coupled with a 2 megapixel

webcam, HD AR uses the video see-through display method and generates an augmented reality view.

Tracking is done by using markers from our Metaio-based Marker Generator Program. Finally, whether

by using pre-loaded content or downloading from a repository by connecting on today's broadbandspeed

wireless connection, HD AR provides a plethora of options in populating the area on which content

is being generated for. Let us delve into the specifics of our product.

Development

In this globalized-driven industry, it is not uncommon to go outside traditional software

development practices. As such we have outsourced the bulk of creating HD AR software to Hexaware

Technlogies (www.hexaware.com), an India based software solution provider. We are using a Software

Development Kit (SDK) from Metaio GmbH (www.metaio.com), a global company based in Germany that

is one of the leaders in providing commercial augmented reality software development tools (Ashley,

2008) in the world. With their Unifeye Ultimate Edition v3.0 SDK, our Hexaware software programmers

takes advantage of a framework that has been created already, leaving them to concentrate on the end

product instead. Although the bulk of our programmers are in India, we have a core group of US-based

lead programmers that decide on the general direction of our product and act as relationship managers

for the Hexaware team. The HD AR US Software Team works in close conjunction with our

management, marketing, and customer support team to ensure a common direction in development of

HD AR.

The HD AR Software

The HD AR product centers around an innovative, Microsoft C# software application, based on

Metaio Software's Unifeye Technology Platform (Metaio, 2007) for the Microsoft .NET framework, and

customized by our software programmers as a packaged modular software for the home design industry.

6

H D A R

The software comprises the HD AR Content Creator module, which handles furniture and appliance

virtual objects creation, the HD AR Marker Creator module, for creating printable fiduciary markers to be

placed onsite, and lastly, the HD AR Viewer module, the program that handles HD AR's display

processes. The HD AR Viewer was extensively tested successfully on a variety of low and high-end

hardware running Microsoft Windows XP and higher, although we recommend a minimum system

requirement that is delineated further down.

Furthermore, we host HD AR Virt, an online repository website from which designers using the

HD AR Content Creator can submit and upload their objects for storage, sharing, and selling, among

other things, and from which the HD AR Viewer can download content to be displayed.

We utilize Amazon's cloud services for several functions, including creating server instances on

Amazon's EC2 for running HD AR Virt's front end, PHP forum, and beta development of our HD AR

Browser PlugIn, which we'll discuss more later. We also use Amazon's S3 for storing HD AR Virt VRML

models, and Amazon's SimpleDB to hold our datastore.

Hardware Considerations

As mentioned earlier, today's computers are getting more powerful based on Moore's Law, but

the caveat of this is battery life on laptops and portable devices has been quite lacking (Moore, 2007). As

such we decided that wearable computing utilizing AR is not the best way to innovate. Instead we are

going to leverage current desktop-replacement performance laptops that are available Off-The-Shelf

(O.T.S.) to easily run our HD AR solution.

The minimum system requirements are as follows:

• 2Ghz CPU

• 2GB RAM

• graphics card with a discrete GPU

• 200MB free space

• USB port

• WiFi connectivity (802.11a/b/g/n), when available

• Broadband card (EDGE or HSPDA capable)

Video Acquisition

Using today's high quality webcams, we are able to get quality live-source data on which to provide not

only the "reality" part of the HD AR output, but also to work with our synchronization and tracking system,

to allow proper placing of virtual objects. The minimum webcam requirements are as follows:

• 2 Megapixel sensor

• 24 bit true color depth

• 640x480 pixel resolution

• 30 frames per second

• autofocus capable

7

H D A R

Marker Creation

When creating fiduciary markers, HD AR's Marker Creator module utilizes a regular inkjet or laser printer to print out a 6 by 6 inch bi-tonal (black and white) square pattern, which is then sampled as a 36

bit digital code (Metaio, 2007). A bi-tonal border is used to estimate proper 3D posing of the virtual

objects based on the marker. All encoding and decoding is handled by HD AR. An example of a fiducial

marker printout:

Figure 2: A 6 inch by 6 inch fiduciary marker printed out by HD AR's Marker Creation module (Metaio, 2007)

The marker is to be placed on the floor where virtual objects are to be rendered.

Typical System

As of now we are using as our testbeds HP 8710p performance laptops with a 2.10GHZ T8100 Intel Core

2 Duo processor with 3MB L2 cache, 4GB of DDR2 RAM, and an Nvidia Quadro NVS 512MB graphics

card, running a Windows XP Professional OS. We will be happy to have HP VARs (Value Added

Resellers), as well as other manufacturers, work with customers in ensuring they have the necessary

hardware to run HD AR.

If the laptop is equipped with a swivel type webcam that allows one to rotate it towards the open area,

and it meets the minimum requirements, that makes it much easier for the customer. However we find

that our current testbed works as well with a Logitech QuickCamPro notebook webcam. Again we are

more than capable of having VARs assist customers in provisioning equipment.

8

H D A R

Business Plan

When it comes time to buying a house it is important that people are able to find the perfect

house. While going through the long home buying process it can be very difficult to imagine what the

house will look like fully furnished. However, with the help of augmented reality real estate agencies are

able to show people’s furniture in each house they are looking at. There may be some houses that may

allows a home owner to have their dream kitchen or theater room, while other houses many not be large

enough for all of the furniture people may have. When looking at the back yard someone may wonder if a

children’s playground, pool, and/or shed will fit in the back yard, with augmented reality this is a

possibility. Real estate agencies with augmented reality will have the ability to show their customers

exactly what the house they are viewing will look with all of their furniture and future plans.

The process of real estate agencies implementing augmented reality into their daily practices will

be our company’s main focus. In order to convince real estate agencies of how augmented reality will be

used to help them sell homes it will be necessary to demonstrate the abilities of augmented reality to the

executives of the real estate agency. The entire augmented reality system will consist of a computer and

software. The computer will come preloaded with software that will allow the real estate agents to

customize each house to people’s specifications. When the house has been customized for the home

buyer the real estate agent will be able to show the home buyers what their house will look like with all of

the furniture and future plans in the house on the computer. This entire process will be demonstrated to

the executives so they are able to see how simple the process will be for their real estate agents to use.

Convincing the executives of real estate agencies they need to spend large amounts of money on

the augmented reality system may be difficult. However, by showing them the power of the augmented

reality system and the ease of use with our software they will realize how this will give them an advantage

over other real estate agencies. Being the only real estate agency that has augmented reality systems will

attract many more customers, because they will want to be able to see what their furniture and future

plans of the house will look like before buying the house.

Focusing on selling augmented reality to the executives of real estate agencies will be more

successful than selling it to just one home buyer due to its cost. Each home buyer would have to pay a

very high price to get the augmented reality system, and have to spend a lot of time learning how to use

it. However, we will train real estate agents in this software so they will be able customize houses to the

home buyers preferences. Also, since the real estate agency will be using the augmented reality system

for several customers its cost will be distributed across all of the customers, making it affordable for them.

Having augmented reality in home buying will help make the home buying process easier; because of this

real estate agencies that have augmented reality systems will attract many more customers than the

agencies that do not have the system.

Another area of focus will be on selling our Augmented Reality technology to interior design

professionals. The equipment will be the same as the real estate agencies (a laptop and software).

Interior design professionals will be able to use this technology to show the changes they are planning on

making to their customers. The software will be able to show changes to a customer’s home such as wall

color, furniture, blinds, curtains, paintings, carpeting, etc. Just as the real estate agents the interior design

professionals will be trained in how to use the program so they are able to meet all of their customer’s

demands, and be able to show them the changes to make sure they approve of them before they spend

the money on the changes.

9

H D A R

Having a dual approach to selling our Augmented Reality technology we will be able to maximize

our profit and growth rate of our company. We are able to sell to both of these markets since the software

and training will be the same. The only difference is the interior design professionals will focus more on all

the details inside the house, while the real estate agents will focus more on the larger items in the house.

The interior design professionals and the real estate agencies that have our Augmented Reality system in

place will have a distinct advantage over their competition. Those who do not have our Augmented

Reality system will fall behind those who invest in our system, which is why we will be encouraging all

interior design professionals, and real estate agencies to purchase our Augmented Reality system.

Conclusion

Our company will effectively use Augmented Reality in home design , incorporating the use of

Video, computer laptops and fiduciary markers. Our company stands apart from other AR companies

due to research we have done into using desktop laptops to run our HD AR solution. With the usage of

video acquisition and synchronization, our system will provide all that a home designer and inter

decorator needs to help clients visualize their project. Augmented Reality is the wave of the future. No

more need to have to pay draftsmen to draw tedious designs and construct models that take time to

produce. We seek to minimize the amount of time it take to create home designs, simplify the home

buying process due to its ability to attract more customer to RE agencies. A smart, effective, sleek

product that will triple the ROI, an attractive element for investors. Take a look at HD AR and make it a

part of your experience in home design.

References

Anders, P. (2008) Designing mixed reality: perception, projects and practice. Technoetic Arts: A Journal

of Speculative Research, 6(1), 19-29.

Ashley, S. (2008) Annotating the Real World. Scientific American, 299(4).

Azuma, R. (1997) A Survey of Augmented Reality. Presence: Teleoperators & Virtual Environments, 6(4),

355-385.

Bajura, M., & Neumann, U. (1995). Closed-loop tracking for augmented-reality systems. IEEE Computer

Graphics & Applications, 15(5), 52-60.

Chen, J. (2008) How the CNN Holographic Interview System Works. gizmodo.com/5076663.

Fuchs, P. et. al. (2002) Assistance for Telepresence by Stereovision-Based AR and Interactivity in 3D

space. Presence: Teleoperators & Virtual Environments, 11(5), 525-535.

HITLab NZ (2006) OSGART. AR Tool Kit for OpenSceneGraph. http://artoolworks.com/community/osgart.

Malone, M. (2007) Moore's Second Law. Wired Magazine. 12(4), 2.

Metaio (2007) Developing Augmented Reality Applications with the Unifeye SDK whitepaper,

http://www.metaio.com/products/sdk/.

Schmalstieg, D. et. al. (2002) The Studierstube augmented reality project. Presence: Teleoperators &

Virtual Environments, 11(1), 33-54.

Wellner, P. et al. (1993) Back to the Real World. Communications of the ACM, 36(7), 24-26.