top of page
  • Writer's pictureDarien Smith

AR Project - Furniture Mapping App Visualisation

Updated: Apr 15, 2023


Background


AR to me has always felt like a bit of a gimmick, offering an experience that is fun for about 5 minutes, then forgotten about. For this reason, I have never considered using AR for creative output until this point. It is only when I then started coming up with ideas for my own AR experiences that I started to see its potential for storytelling and visualisation.



Testing

In class, I was given the opportunity to use Adobe Aero, a simple-to-use program specifically designed for creating AR experiences. With a short timeframe of only 30 minutes, I was tasked with using Adobe Aero to create a short narrative experience. This task gave me the opportunity to play with all of Aero’s tools and serve as a means to get to grips with constructing AR scenes.


What I created for this task was a scene where a robot walks to an orbiting moon, the robot does a dance, and an asteroid then flies in, collides with the moon, and blows it up. This was not a very riveting narrative experience or a technically impressive one, but it taught me how to use Adobe Aero and how easy constructing AR experiences can be in general.



Planning


When first coming up with ideas for an AI project, I had an ambitious idea of creating a real estate-centred phone application where the user could use AR to visualise and map furniture within an empty room. The point of this application provide visualisation to potential home buyers so they can better conceive what an unfurnished room may look like after furnishing.


I quickly realised however that this idea would likely be too technically advanced and time-consuming for the scope of this project. Though I did not move forward with creating this furniture application, from this idea I came up with something more appropriate for my project while still retaining the concept of the original idea.


My new idea was to use a business card as a base and to use AR technology to display a live preview demonstrating how the aforementioned furniture application would work. This is to help anyone that is interested in using the app better understand how it works visually, while also providing a call to action as a means to help secure potential clients.


This is the idea I had in mind for how I wanted the experience to go:


1. The user would scan a business card with their phone to initiate the AR experience.

2. An empty room emerges on top of the business card.

3. A character would enter the room

From this point on, I wanted to add an element of interactivity to the experience. I felt this would make the experience more memorable and fun for the user. This is how I planned to achieve this:


4. A pop-up would spawn prompting the user to tap 1 of 3 buttons.

5. Each of the buttons would spawn in 1 of 3 premade furniture arraignments.


At this point, I felt it fitting to use the character as a native element. I wanted this character to react to the different furniture arrangements, much like a user furniture mapping app would, positive, negative, and indifferent. This is to demonstrate the emotional impact such an app could provide. As previously stated, the point of this AR experience is to make real estate agents want to buy or learn more about the mapping application so it is important to show a visualisation of their clients enjoying it, even if it is simulated.


This is a storyboard to outline how I wanted the experience to look and function.







Creating the AR experience

Choosing my tools


When starting the creation of this AR experience, I had to decide what software I would use. I played with Adobe Aero and created a basic 30-second experience as a means to get to grips with the software and its tools. Though Aero was seemed simple to use, and already had many premade asset installed, I had some big problems with it. The main problem I had pertained to the I characters and their animations. I felt the default directable characters were very limited in what they could do. Even something as simple as having the character turn around on the spot seemed to be impossible with the default animation sets.


I also was not a fan of how the animation system worked as a whole. There was no way to play back my later animations without first having to watch in real-time all the preceding animations first. This meant creating longer projects with many animations would become very time-consuming.


Due to these problems, I searched for a different platform in which to create my AR experience. This is where I found Blippar. Blippar is web-based AR builder which I was hoping would be better than Aero for my purpose, however, Blippar also posed some big issues. The first being the textures. As you can see from the image below, I was able to apply a texture to a plane I created in Blippar, this play was to act as the floor of the scene, however, the full texture was not being used and there was no way to adjust its scaling.


Original floor texture:


Imported floor texture:


I also found the animation features difficult to use. Though Blippar has a timeline for its animations, the very thing I wished Adobe Aero had, the actual usability of the animation system was strange and inconsistent at times. I found it frustration to use and not very intuitive. This ultimately turned me off using the program all together.


I decided to give Aero another chance before looking into other AR creation programs.


After consulting with my lecturer, I was informed about Miximo, an Adobe website which allows users to share and use a wide range of open-source animations. I felt this would be the perfect solution to solve the low-character animation issue I had with Aero. Though the other problems I had with aero still prevailed, I felt this was mitigated by the otherwise very intuitive and easy-to-use interface.


Importing a Character


When first conceptualising this project, I felt the most difficult part would be creating and animating the character, however, through fairly new online tools, this process was actually quite straightforward, at first.


Using a website called Ready Player Me, I was able to quickly and easily create a 3D character.


The character creation tools were very easy to use, and are similar to the character creation tools that many video games possess.


After creating the character, I was able to export it as a 3D asset, specifically a .GLB file.


From here I attempted to upload this GLB file straight into Mixamo, that way I could apply animations to my 3D model. This is where I ran into a problem.


It seemed that Mixamo does not recognise GLB files. To solve this issue, I turned to google.


I followed this online tutorial explaining how to import my .GLB file into the blender, a 3D modelling, rendering, and animation software. From Blender I could then export the file back out as an .FBX file, a file type that is supported by Mixamo.



Again, I ran into problems. When trying to export my model out of Blender, only one of the model’s textures was exported with it. In my case, it was the shirt texture.


After searching across forums ( https://blenderartists.org/t/fbx-export-has-no-material/1305970/15, https://www.reallusion.com/FeedBackTracker/Issue/Texture-mismatch-when-exporting-to-OBJ-or-FBX) and youtube videos explaining the specific export setting required, and possible troubleshooting solutions, I came up with nothing. I even went as far as using a different computer with a new instance of Blender to see if this solved the problem; It did not.


Finally, I found a video that explained the process of combining all the model’s textures into one, I believe this would have solved the problem, however, this appeared to be a very time-consuming and tedious process. Considering I could simply use one of the pre-made characters within Miximo and achieve the same effect, I felt this lengthy process was not worth doing.



Importing Animations


As previously stated, I decided to use one of the characters on Miximo for my project. Along with the character, I needed some animations to tell the story I had in mind. Through Miximo this was a very easy process as everything was already compatible with the character I had chosen. Miximo also presented a large gallery of animations to choose from meaning I could find what I needed for the vast majority of scenarios. If however I was unable to find the animations I needed, I did come upo with a back up plan. I briefly considered using an AI based motion tracking software such as Deep Motion. This a machine learning based sftware that generated 3D animation from video.



Ultimately I decided against using this program for two reasons, the first being that I felt these using such a tool would lead to more incompatibility issues that would be time-consuming to resolve outweighing the low scope of this project. The main reason for not moving forward with Deep Motion is the animations on Miximo happen to cover all the animations I need for this particular project. Deep Motion, however, is definitely something to keep in mind for future 3D projects of mine.


From this point on, I got to work compiling all the animations I would need for my scene. To help me out with this, as well as set out my exact animation path for the story, I created a document listing all the animations I needed and in what order they should occur:


Referring back to this document, through Miximo I found and downloaded all the animations I needed.


Within Miximo there are some simple customization options that allowed me to make adjustments to the animations before downloading them. This helped with matching the animations with one another, as well as adjusting the intensity, speed, and length of the movements.


The controls, though basic, helped me to better tell my story by adjusting the charicters level of expression. Furthermore, these customisation options helped me better fit the animations to the character model, as sometimes the arm distances were slightly off.


Environment


Regarding the scenes enviroment, I used Aero’s built in ‘interior design’ models, combined with 3D assets Free3d.com. Specifically, I used this sofa model for one of the room arrangements https://free3d.com/3d-model/couch-sofa-742041.html



Animating the scene


After compiling my furniture assets, it was just a case of arranging them into different orientations and saving them into a folder within Aeros ‘Scene’ section. I was then able to animate these folders (which I named room sets) all at once. This saved me a lot of time and kept things organised.


Animating the character was a fairly simple process. Once all the character animations were imported into aero, they would display in the ‘animation set’ drop-down section of the ‘play animation’ dialogue, just like using the directable characters built into Aero, only with more control.


There was however a lot of trial and error with the animations, for example, sometimes the animations did flow well with each other, so I had to move them around or replace them entirely. A big issue I had was orienting the character when I wanted him to turn around. I wanted to use an animation to make the character turn around, as oppose to a janky looking rotate or spin action. The issue with using an animation to make the charicter turn around is that only made it look that way, in actuality the models axis stayed stationary relative to the environment. The issue that arose from this made itself known when I applied another animation after I made him turn around. When attempting this, the character would instantly turn back and face the original position he was facing before the turn around animation. The only fix I had for this was to rotate the character manually, the problem with this is it looked weird and unrealistic. The best solution I could come up with was to try and hide the rotation within other animations as an attempt to make it look as natural as possible.


The fact that something as simple as turning a character around was still very difficult to pull of in a natural way really highlights the limitation of Aero. If I were to do another AR experience of this scope or higher, I would do more research into other software that could do a better job, or maybe even use Blender. For this project though, Aero just about got me by.


This is my animation set-up for the finished project.


I used many ‘hide’ and ‘show’ actions to change between the different furniture sets. I also frequently used tap triggers to allow the user to interact with the story and change the furniture themselves. Though linear, I believe the interaction makes this experience more fun and memorable. Adding this kind of Interactivity into this business preview also better demonstrates how the furniture mapping works as this too would be interactable in a similar manner.


QR code to the finished project

If there are any problems viewing the AR experience, here is a video instead.:




Final thoughts


A few things changed from my initial plan for this project, the main one being that instead of allowing the user to switch between the 3 different room arrangements as they saw fit, I instead let them view the room setups one at a time. I made this decision so I could guide the user through a slightly better narrative experience that they otherwise couldn’t have had.


Overall, I believe as a means of advertising, AR could be a powerful tool, It succeeds in explaining visually what a business is all about, while also making for a memorable and unique experience.


From creating this project, I not only learned about how to use tools to create AR experiences, but I also learned about the potential of AR. I now see It is not just a form of gimmicky, short-lived entertainment, but it can also make for an effective advertising, and previsualization tool.

The biggest issue I see with AR as a whole is its lack of convenience. Having to pull out your phone and potentially download an app which may or may not work (like Adobe Aero for Android) just to view the experience I feel takes too much time and effort, especially for use in advertising, a world where you need to grab people attention immediately and put your product in front of prospective clients with the least amount of resistance possible. I believe AR advertising would be significantly more powerful if it could be viewed without the need of a phone at all. If AR glasses are something that exists in the near future and are worn by a large percentage of the population, I feel AR advertising would be a much bigger part of our lives. No pulling out phones or special apps required, all it would take is a look and your prospective client would initiate your AR advertisement; because of this, believe AR could be the next big advertising medium.

11 views0 comments

Recent Posts

See All

Comments


bottom of page