Augmented Reality

Multi Targets in Qualcomm Augmented Reality SDK

I’ve been working on the next step of our project this week, trying out multi targets in QCAR and toying around with ideas on how to implement interactions in such systems.

Multi Targets is a way of tracking physical boxes with square or rectangle shapes by registering a trackable image for each side of the box, and letting the system know what the dimension of the box is. This allows you to create occlusion effects (virtual items appearing to be hidden behind the real physical item), an effect that is hard to achieve with normal Augmented Reality.

To set up a Multi Target in Qualcomms AR SDK here is what you do:

First, find a suitable box that has a lot of features on the cover, to ensure that it will track well. In my case this cereal box made a good candidate.

In the best of cases you would have the package textures in digital format, but in most cases when your not working directly with the manufacturer you’ll find yourself having to scan the box yourself. If you have a professional scanner this is no big deal, but in my case I had to make due with the camera of my iPhone. If you do the same, make sure to take the picture in good and even lighting conditions to avoid problems when analyzing the image for tracking.

After scanning all the sides, create a new Multi Target trackable on the Qualcomm developer portal, and enter the dimensions of your box. The important thing here is to get the ratio right, the actual values you use correspond to how you 3D content will be scaled in relation to the trackable.

Once the trackable has been created, you upload the pictures for each side of the box and have them analyzed. The front side of my cereal box gave a four-star rating which will track well in most conditions.

Once all this is done, you Multi Target trackable is ready for download and use in your AR projects. In my next post I will go into detail on how you use it in Unity and add 3D content to augment your target.

December 28, 2011  1 Comment

Getting Started with Mobile Augmented Reality

Augmented Reality campaigns have become more and more common, and many designers and developers have started to look at it as a method to present content in new and unique ways. It has however also gained a stamp of being “gimmicky” and not bringing any real value to a product, something the AR community has been struggling to overcome in the last year. By bringing affordable and integrated solutions to the market for developers and designers, companies such as Metaio, Qualcomm and others are hoping to bring new life and new innovations to the field. And with new advances such as gravity aware AR and live occlusion, they are.

So what if you want to jump onboard and develop the next generation of cool AR apps the go beyond simple gimmicks and changes the way we use mobile devices forever? Well, in this guide I wont give you a golden ticket to solve all that, but I hope to help you get a running start on developing your first mobile AR app, and giving you some pointers on how to make it great.

So lets get started!

The Platforms

During the last year a whole new world has opened up for AR developers with multiple platforms to choose from, all offering different advantages depending on your needs and price point.
Some of the biggest ones:

String is focused on bringing powerful AR graphics to iOS, with a goal of empowering creative agencies with AR tools. It’s also expensive, with a $499 minimum price point for their cheapest commercial license.

Qualcomm is a new player in the AR field, but has decided to release their developing tools for free commercial use for both Android and iOS which is really cool. They are a growing platform with new developments in the works.

Metaio has been around for some time, one of the early innovators in the AR field. Their mobile SDK has some unique technical features that are really cool, such as gravity-aware AR. They just recently introduced a free version of their SDK, but it comes with a incredibly ugly Metaio watermark for all your apps. If you want the real deal, you have to fork out at least 4.950€ (ouch!)

Total Immersion has also been around for quite some time, and has powered some of the biggest AR marketing campaigns out there. Not really for the indie developer though, they focus more on big brand agencies.

So which one should you choose? Well, it all depends on your budget and technical requirements. I personally recommend going with Qualcomm, simply because it is completely free for commercial use (Metaio just released it’s environment for free use, but unless you pay up you are stuck with a horrible Metaio watermark in all your apps).
To get started with Qualcomm’s AR environment, simply sign up on their development portal here:
https://developer.qualcomm.com/develop/mobile-technologies/augmented-reality

Qualcomm also offers a great integration with Unity3D.
What is Unity3D you say? Well, it’s the greatest tool in your AR toolbox, and I’ll tell you why.

Unity3D

Unity3D is a cross-platform game development toolkit that allows you to design and build 3D gaming experiences on a single code-base and deploy it across almost every thinkable platform. But you are not building a game you say? Not to worry, Unity is an amazing tool for all your AR developing needs. All the major AR platforms, including Qualcomm, has extensions to Unity that lets you easily create, place and align objects in relation to your AR markers without writing a single line of code. However if you are so inclined, you can use code to easily bring your projects to life using animations, interactions or basically anything you can think of.

To get started with Unity, you can download a free version from their website. Sadly, to be able to deploy these projects to your iPhone or Android device, you need to buy the respective version of Unity. This might sting a little, but I really recommend it since Unity will make your developing process so much faster, saving you tons of time. And time, as we all know, is money.
If you just want to try it out, they have a great 30 day trial of the pro version, which allows you to deploy apps for mobile devices. So try it out first and see if you like it.

Connecting Unity with Qualcomm’s AR environment is a breeze. You simply download the Unity extension on the Qualcomm developer portal and import the downloaded packages right into Unity (follow the guide Qualcomm gives you to make sure you dont miss any steps). Try out the included projects that Qualcomm has supplied, and you will have your first AR app up and running before you can say “gimmick”.

Making people go “WOW!”

So you got a teapot displaying on a ugly marker. Thats great. Cool. But it also kinda sucks.
One of the key challenges of Augmented Reality is making things look believable and real. To make a really immersive AR experience, you want augmentations that make people look beyond the screen, thinking the augmentations have materialized out of thin air and are there for real.

There are some key concepts to make things look great. One of them is lighting and shadow. Another is imperfection (real objects are never perfect). I wont go into details here, instead I will point you to an incredibly useful post over at PixelPlacement.
He offers a ton of great advice on pushing your AR implementations to the next level, and even supplies some pre-built packages that you can drop right into Unity and use in your projects.

Another thing to consider is how people will interact with your AR experience. Sure it’s easy just to overlay some digital content on a poster and call it an innovative marketing campaign, but to really create something that makes a difference you can engage people in much deeper ways. Consider how and where your app will be used, and implement interactions accordingly. Will people interact with your AR content through the screen? Or will they interact by “touching” the virtual objects in the physical world? (Qualcomm has built in support for this with “virtual buttons”, though it doesn’t always work as great as you imagine) All these decisions need to be based on the context of how your app is being used, and the only way to figure it out is to try and try again. Put your concepts into the hands of people and let them try it. Observe how they, in the way that is most natural to them, try to interact with your augmentations. Adjust your designs from their feedback.

Also consider using the device itself as a way of interacting with the AR environment. The distance that people are observing your AR environment at, for example, gives you an idea of what they are trying to accomplish. Are they looking up close to see the details? Or looking from afar to take in the whole environment? The great thing is that you have the power to measure this through the device, and act accordingly by changing content depending on the context that it is being used in.

These are just some ideas to get you thinking. I’m sure you will come up with tons of exciting new ways to engage people in your apps.

Keep Going

So you have your first AR app up and running, and you have tons of cool concepts in your head about where you want to take it. Where to go from here? Well, get building. It wont happen unless you make it happen. Start simple, and expand on your projects with cool new features. Dont be afraid to experiment. Augmented Reality is still in its cradle, so try to think outside the box and bring new ideas to the table.

Another thing to keep in mind is that the AR business is constantly growing and evolving, and it helps a lot to keep up on the latest changes and releases. Create a dedicated Augmented Reality feed and follow great blogs such as AugmentedBlog, join some AR companies on Twitter (such as @String). It helps immensely to keep up to date and follow the latest changes in the industry.

I hope this guide has been helpful in getting you up and running, and I look forward to seeing your cool AR projects in the future. If you like, you can also follow me on twitter (@freeortman) for random musings about AR and other things.

December 15, 2011  1 Comment

KinectFusion Could Change Augmented Reality Forever

This is one of the most impressive things I’ve seen in a long time. Microsoft Research has presented their new Kinect research, named KinectFusion, and it shows huge potential in the field of Augmented Reality. Imagine having this kind of technology embedded in a pair of AR goggles for example.

August 31, 2011  Leave a comment

Future Visions

Look, I love future-concept videos as much as any other tech geek… but for the love of god, if you are going to create a believable concept for 2015, stop trying to make every interface look straight out of Star Wars, complete with Space Lazers sound effects. And another thing, what 10 year old kid in 2015 has any reference to a floppy disk (look at 3:10) ? Hell I barely remember them anymore.
I know the video is from 2007 any I should probably give them a break, but…no.

If you want to sell me on AR, instead show me how beautiful and seamless it could truly be:

Right now I am in the middle of the second week of my internship, and most of my time is still spent diving deep into reports, blogs, videos and whatever I can dig up on Augmented Reality. I am finding a lot of interesting stuff, and one source that has been very helpful is the Layar blog, especially the “ARspiration category”.

http://www.layar.com/blog/categories/augmented-reality-inspiration/

August 24, 2011  1 Comment

No More Gimmicks

When researching the commercial work that has been done in the field of AR, you will generally come across the same things a million times. It seems every company and their grandmothers has held a campaign with QR tags that you can hold up to the webcam or smartphone and get some gimmicky 3D model doing some animation. Especially car companies seems to love this type of campaigns.

Thats why it’s very fun to come across AR solutions in marketing that fill a real use. One of those that I have found is TryLive Eyewear created by the well known AR company Total Immersion. It lets you try on glasses using your webcam, and I was really surprised with how well it works. You can try it out yourself at www.zoff.co.jp/mirror, but the page is all in Japanese.

Here is a demo video showing how it works:

And this is how sexy I look in glasses:

I have been working on documents for the university today again, writing oppositions on a old thesis to practice my academic reading and writing. Not the most exciting stuff, but it needs to be done so I can get started with the fun stuff and really focus on becoming a true AR ninja. Spending late nights in the office working these days, since my accommodation is far away and consists of a inflatable mattress in a otherwise empty room. I am only spending six weeks in Stockholm to start off the thesis project, then me and my friend that I am doing the internship with are moving to Singapore to continue work there.

August 18, 2011  Leave a comment