00:00:00
00:00:01
#11 - Pioneering VR & AR Technologies Since 2014
16 Plays3 months ago
In this episode, we dive deep into the virtual and augmented reality world with a seasoned expert who has been pioneering in the field since 2014. Our guest has worn many hats—developer, entrepreneur, CTO—and has been instrumental in shaping the immersive technology landscape.
Starting as the co-founder of Immotionar, he developed ImmotionRoom, a groundbreaking full-body VR system using Kinects and VR headsets. It was a remarkable feat in 2014 when VR was still in its infancy. He then co-founded New Technology Walkers, a consultancy group specializing in cutting-edge technologies. He later became the CTO of VRROOM, where he spearheaded the development of an innovative platform for immersive shows. Notably, VRROOM achieved the #1 spot on the Pico Store during its alpha launch and facilitated record-breaking events like the Jean-Michel Jarre concert with 75 million views in 24 hours.
Our guest has also contributed to numerous AR/VR projects across various sectors and provided strategic consultancies to companies venturing into XR and the metaverse. His blog, "The Ghost Howls," is a celebrated resource in the VR community, earning accolades from HTC and Onalytica as a top VR influencer. He has shared his insights at prestigious events such as WCVRI, View Conference, SXSW, and AWE, and has been featured in major publications like Forbes and Huffington Post.
Join us as we explore his journey, insights, and the future of immersive technologies. Whether you're a VR enthusiast, a tech professional, or simply curious about the metaverse, this episode promises to be a fascinating and informative experience.
Transcript
00:00:00
Speaker
So let's go. Hello, Antony. Thank you for joining us today for this 11th episode of the Numero Mesh Podcast. um So the previous one for those that haven't listened was about an introduction, about Metaverse, about virtual reality, augmented reality. So if you haven't listened it yet, I highly recommend to listen to it. Hello, Anthony. How are you doing? Hello. for hosting me here. I love everyone. I listened to his podcast. I'm fine today. There is the sun, so I can be pretty happy. Thanks for asking. The weather is nice. So maybe can I ask you to present yourself a bit, please?
00:00:48
Speaker
Yeah, sure. I don't want to make people fall asleep during this podcast, so I'll try to be pretty fast in my introduction. I'm a i'm a VR guy, a VR guy since 2014. I remember um I tried for the first time an Oculus Rift Dedicator that was of a colleague of mine. and and I fall in love with this technology and since that moment I always try to work in the VR VR sector into various startups always being either the dev or the tech lead or the CTO or anyway the the tech guy working on goggles and in 2016 I also started my blog about the VR and VR and I'm pretty active on social media especially it's written and linked in so I like to
00:01:42
Speaker
speak about these technologies and speak with the community. So there are a lot of amazing people that are in our space or i interact with them daily. And that's pretty cool. So this is basically me 10 years developing in XR and also writing a lot of words about that. That's good. So I would like to understand a bit more. You didn't start by AR or VR originally when you started when you finished your studies or you finished your yeah universities or you don't know what what what did you study? Can you maybe share a bit more about so the the experience that leads you to develop? Did you went to the university? What did you start by virtual reality or did you start by another technology?
00:02:31
Speaker
Well, actually, when I was at the university, now you'll you'll make me feel a bit old, but when I was at university, the VR was mostly a technology for the movies, because we were in the various phases of, let's say, winter of VR, so it was before Palmyra Lucky launched the Oculus Rift. So we studied a bit something, oh, there is this thing called VR, there are sunglasses, but very, very fast. were mostly focused on 3D graphics, like the one of video games. And I was also studying computer vision, through which I got in contact about the studies they were doing about ah putting virtual objects in camera streams, something that I learned being called augmented reality then. And um so basically, I was the passion for multimedia and technology. That was what I was doing.
00:03:25
Speaker
Then in 2014, I got contacted by a previous classmate of mine in all years after the degree. And actually, we both liked ah multimedia technology. And he wanted to create a startup. And he was fascinated like me by the the commercials of the Google Glass that at that time were promising have everything about that device. And we wanted to make a start about AR with Google Glasses. We bought the Google Glasses for $1,500 just to discover that they were basically horrible and unusable for everything. So we just wasted a lot of money.
00:04:10
Speaker
And while I was crying in the corner saying, okay, all our ideas have to be put in a trash, the the my buddy, which is called Johnny Rosagalina, was like, oh, but I have another device. It can be about plan B. It's called the Illustrif Dikichu. I had no idea what the hell it was. So I said, yeah, why not? He brought it to the office. um He made me try it. It was a demo called Tuscany, where you could see yourself in some Tuscany villa, in an island, et cetera. For today's standard, it was a super happy demo. But the time you know I put the headset on, I was like, wow, that's magic. I'm in another place. That's magical teleporting. I love this technology.
00:05:00
Speaker
That's the moment where really my mind clicked. I would say also my stomach clicked because after five minutes I wanted to vomit and to remove the headset. That literally was ah one of the defining moments of my life and that's where we actually then started our first startup together, which was called Emotional because we were amazed by the technology, but at that time there were no controllers, no avatar, nothing. So we tried to create a system to give a full body to the user. And that's how I really started entering professionally into VR. Okay. So when you say no controller, what do you mean exactly? Maybe for the people that that don't know, what does it mean?
00:05:45
Speaker
Oh, it means that basically you put a headset on your head or these big glasses. The headset was connected to the computer. You couldn't use it like the Quest ah today or alone. You had to attach it to a computer and you controlled VR with the mouse and the keyboard, which today's looks like a joke, but that was how VR was at that time. You looked the Below you, there was no avatar. To move around, use the keyboard. And there is nothing you could have on your hand like to shoot or to interact, et cetera. Mouse and keyboard for everything. So that's why we thought we should do something about that. OK. And so to them, I'm familiar with some VR and AR systems. But what what are the most what are the best um controllers that exist?
00:06:40
Speaker
Well, I think most of them are the same after Oculus standardized the input. I would say that, anyway, there are some interesting take, like the Valve Index Controllers are amazing because they can sense all the fingers, which is something most controllers cannot do. So that's their, let's say, superpower, for which they are pretty cool. And also, you can open your hand with the Valve Index Controller throwing object because they clench on your palm, so you don't have to grab the controller, just make it, let's say, clench your palm so you can open the hand to throw objects, you can sense all the fingers, there are particular devices.
00:07:24
Speaker
But there are also other controllers which have some interesting superpowers, like the Quest Pro controllers have cameras on board. That's pretty crazy. They have three cameras, if I remember well, per controller, and the controller performs the tracking itself. So it's a it's pretty cool. You can put it everywhere and keep striking. and But actually, if I had to choose my favorite controllers are the ones of the Oculus Rift, the CB1, which is a 2016-year device.
00:08:02
Speaker
but very incredibly comfortable, balanced and resistant. You could smash it into a wall and probably the wall was going to break, but not the controller. there are ii as a going to a Okay, so just to give a bit more detail, maybe you can complete and and you tell me if if it's correct or no. So to explain a bit more about the the headsets and the the controllers, the it's a kind of accessory because in the VR and AR world for persons that have tested some technologies or for people that haven't tested, there are two main parts, let's say of two main pieces of the physical technologies that is one, glasses,
00:08:52
Speaker
so what we call headset that is ah compatible with oculus, meta, quest 2 and i think so also other supernatural if i'm not making mistake etc and accessories for these headsets and glasses that are controllers so it's not a controller like the PlayStation but it's a bit similar in the concept because it's something that ah I can have in my hand or you can have in your hand and on this controller I can pay perform some actions and you can pay perform some actions so I guess there are some buttons um generally it's three four I think in the case of the
00:09:35
Speaker
The meta one, it's maybe four that they are visible, maybe five with a small ah um trigger similar to a gun, let's say. And there is a kind of a ring. and um So it allows to perform some actions, I guess, related to the applications. Tell me if I have made some mistake or if you want to complete maybe Anthony? No, no, I think I'll explain it well. OK. cool. So yeah, I didn't also say, but I did not ask you about your studies to make you feel old, but just to understand. And so when you launched this first startup,
00:10:18
Speaker
and It's about ah focusing on um full body ah system related to VR e etc. Maybe can you develop a bit more about the technology? I have seen that it was Unity 3D, what are the technologies? What are you concretely doing related to the human body? What I have seen also that you have won a Faber Day prize in January 16. Can you explain us a bit more the different ah awards that you get, the different main projects and and and cool stuff that you want to share?
00:11:00
Speaker
Well, I think that apart from the awards, I think that what was cool was really the project. It was but the you know ambitious so and maybe too early in time and we needed also a bit more experience but basically it's crazy. you know you are ah You try a device, you see, okay, it doesn't have full body, and say, okay, I try to solve the problem myself. And that's something that, you know, was naive and cool at the same time. So, but how cool do we, just true guys in Turkey and Italy, trying to add full body to VR, so make the user see an avatar of all the parts of the body? Well,
00:11:46
Speaker
It's simple because there is a technology that is made to track your body and that's called the Microsoft Kinect. My partner had Kinect in the office and so we tried to mix the data of the Kinect which was tracking the whole body of the user with the data of the headset which will make you see yourself in a fantastic reality. and making a lot of networking stuff and the data fusion stuff, so I'm not going to bore everyone with these details. Basically, we are mixing the data of multiple keynets, fusing it together to give you a good approximation of the pose of the data of the user. And this way, you could look behind you and see your full avatar, see your hands, see your feet,
00:12:42
Speaker
because not you're in the sense of your avatar, science avatar seat, but they were put in the same way of your body. And then you can even walk in place to walk in of, since there was no controllers, how could you walk in VR? And the the the thing is, we were you walk in the same spot, in place, and we moved you in virtual reality. This way, since you are making a moving moment, with your body, the brain was also tricked into the fact that it was moving. So it um it gave less that called motion sickness effect. You know, sometimes people try VR and they can get noses. With that movement, that the nose effect was very reduced.
00:13:34
Speaker
And so people were able, let's say, to try some kind of futuristic VR 10 years ago. And it was cool that this system also worked with a Gear VR. So the first, let's say, cool headset that worked with your mobile phone put inside. And so usually without wearing any sensor and any controller, you could use an untitled headset and walk around the space and interact with objects in 2015, I guess, for the Gear VR. So this is what we were trying to do. Of course, ah it was a prototype. We were looking for money you know to make a system that was more stable.
00:14:21
Speaker
But I still remember when we went to exhibitions and people are putting a headset, putting the hand in front of their face, looking the hands in VR, like, wow, they were not expecting that. So it was pretty, pretty, you know, futuristic stuff. OK, there is a blog post that is still online that I have read that is that has been published in 2017. I'm not going to to read everything here but I will link it in the description in the description on YouTube and on the blog post because each episode has a blog post that can be found on numeromesh.com
00:15:05
Speaker
What it says is to give some technical um idea for people that are more technical than me is that um there is two lines that say, we have a braided in Motion Room runtime to version 0.4.2. With this update, you will get a free tracker server that will let you use in Motion Room runtime with VR reach to easily use Kinect as positional tracker for your cardboard and get a C sharp client to make any kind of dot .NET applications to communicate with our tracking service. So there are a lot of information that can be found.
00:15:48
Speaker
And you are also thanking people, so BEPS Engineering, a great IoT company that gave you the possibility to follow your dreams. Max Ariani, who believed in your dream like no one other, didn't help you in all possible ways. Andrea Basso, who has been an important mentor for you. When available, Sasha and Jonathan, for being your supporters and giving you strengths, you will love them forever. So it's pretty um good to see this um blog. I think that it was one of your first entrepreneurial project. Diving more into the people that helped you in the technology for developers.
00:16:30
Speaker
Can you maybe give a little bit more details about how it has evolved ah since ah today? what Because you have a mentioned Kinect, so just I will finish to to explain a bit more what is Kinect. If I remember properly, it was a kind of... ah motion sensing device for the Xbox. Personally, I used it for the Xbox and Xbox One, if I'm not mistaken. So it was um a device that was containing RGB cameras. So i'm but I'm not going to explain what is it because I think that I'm not qualified here. And infrared projectors and the textures that map depth through either structured light or time or flight calculations, which can be
00:17:17
Speaker
in turn be used to perform different um actions and real-time gesture through reco recognition, body skeletal detection, and other capabilities. So it also contains microphones that can be used for speech recognition and voice control. So it was the really, really first step before. There was something for PlayStation, I don't remember, but um basically what it allowed, if I remember, it was to play bullying instead of using the mo the controller of the Wii for the people that play the Wii, the Nintendo. Basically, you could dance with um Microsoft Kinect. I will put also a link about Microsoft Kinect. It has been released in 2010, if I'm not mistaken. um so Maybe some people that are listening to us have used this. Maybe some people know it was connected by USB. and Personally, I was using it in
00:18:14
Speaker
to play some games, but it was pretty, yeah let's say, basic. And I'm not sure that a lot of people were playing, but it has been a big marketing um thing for Xbox at that time. And it it seems I didn't know that people like you were building on this ah technology. And it's ah it's super interesting to to know. And ah after i I remember that also it have been ah it have been connected, integrated with Microsoft Azure. So if we have some people from of listening here,
00:18:52
Speaker
and That's everything I can say. i i I would like you to explain maybe a bit more for technical people um what exactly was the challenges and what you decided to move on because as as far as I understood, you said in one of your personal ah blog posts, the ghost holes, I will put also the the link ah because I think this blog is also still um available, um what you that you were moving on and that you would continue to develop, but develop different stuff well I will cut this straight to the point the technical problems we had were mostly that
00:19:34
Speaker
We were using ConnectV2 and we started using ConnectV2 the moment that ah basically Microsoft discontinued it. So we were left with lots of bugs that we could not solve ah because the runtime of Connect, like ah the hands were trembling and we could not make them stop because of course we it's not an open source runtime, so we cannot modify it. And so our system had lots of problems, but there were no alternatives to Kinect. And so that was a big problem. We could maybe try to develop some track of ourselves with some AI tech, but we needed money. We needed funding that never arrived. So the moment why we went on is because without money, we couldn't go on with that adventure. It was undoable. And that's a pity, but
00:20:28
Speaker
and Let's say that's part of startup life, so projects, 90% of startups fail in the first three years. So that was, let's say, a good adventure. We learned a lot. we I learned a lot, so personally, both on the technical side, but also on the entrepreneurship side, because I started, that it was just a pure tech guy. I had no idea. about others being an a entrepreneur, doing marketing, raising money or whatever. I, thanks to Emotioner, I did some experience and also attending the European Innovation Academy that was like an accelerator that came to my city in 2017, I would say. ah We learned a lot about what really means being a true entrepreneur. So that was like
00:21:20
Speaker
An interesting adventure and the funny thing is that a lot of years later Microsoft released a new version of Kinect called the Kinect for Azure, which was much better for tracking. So maybe we if we if we had in 2014 the Kinect for Azure, we could have had maybe a better story because the tracking is much better with the new version. But the funny thing is that even this new model has been discontinued, so Microsoft keeps the ah launching and killing the Kinect continuously. But at least this time, they left the runtime to be used used with other hardware to be accepted or on the back, hardware if someone on the tech side wants to investigate still full-body Tracambia, that's how do they can get. and so the The issues were both technical and financial. and you know we As we went on,
00:22:18
Speaker
And we parted way with ah the co-founder, Johnny, and we went to do different things after that. Okay, that's pretty ah clear. And as you were writing, I think it was one of the hardest decision you have taken. And that you were sad about it. So that that can be totally understandable, especially for people that listen to us that are that are entrepreneurs. So then you decided to move on. So yeah I guess you continue to to work in this sector. What what is the what is the next step then?
00:22:58
Speaker
Well, the next step is you mentioned the guy before reading my article, which is Massimilian Oriani. He was a guy that saw our technology and fall in love with it. And here, he's always been a gamer. And he we decided to let to do something together. So we started consulting people about 3D VR, VR gaming, this kind of stuff. And luckily, that was also the period 2017 of the ah winter of VR, ah the winter of modern VR. ah So after the hype of the meta at that time, Facebook acquiring Oculus.
00:23:41
Speaker
And the promise of consumer VR, then consumer VR is launched and it's too expensive. And at a certain point, people realize these headsets are not selling enough. So there is a lot of disappointment about VR. VR is dead, blah, blah, blah. So it was not the best moment when VR, but we we kept doing our work, doing consultancies, sometimes not strictly about VR, but about ah gaming, about ah unity development, about things like that. And then at a certain point, we ah basically also with the collaboration of N-Elephants, we decided to investigate one thing. Is it possible to use the Vive Focus Plus, which is was a VR headset, the Vive Focus and then Vive Focus Plus?
00:24:34
Speaker
to do some kind of augmented reality. The white focus was an interesting device, and it had two cameras in front of the device itself. These two cameras had, you could in sdk gas the the images that these cameras were saying. So applying some mathematics, we were able to Make the user see through the headset. So what's now is called the past or mixed reality or whatever We created it hiking a Sample of the wire focus plus so we unlocked mixed reality in 2019 putting also ah How we did it as open source some months later. So have everyone could do that and And basically, three years before Meta released, the lead developers to that on the quest, we were doing something similar, a bit more rough, of course, on the back focus. And so then HTC was quite impressed by what we did. And so we partnered with them, and with their support, we created a fitness game in mixed reality.
00:25:47
Speaker
called Hit Motion Reloaded, which was one of the 20 launch titles of the Bifocus Plus, the only one in mixed reality, because no one was developing a mixed set reality on a via headset at that time. And it was practical because the idea was that you could do fitness, you could play like a boxing training, but disguised as a game in your home. But since there was mixed reality, you could see your surroundings. So you could see what you had around you. This is why you didn't risk punching walls or other people, et cetera, while you were training on a headset.
00:26:24
Speaker
So that was another cool thing that we were able to do. We also was school going to Shenzhen to launch it at the VIVE ecosystem conference. And the height that the game was put on the VIVE port store. And years later, ah we also ported it to Quest as soon as Meta-enabled pastel for developers. So we were able to There is still a demo of the Quest version on the App Lab if someone wants to look for Hit Motion Reloaded. And it was a pretty good game. I think it's still ah more than four stars of review on App Lab. So that's ah that's been another cool adventure. we We kept doing consultancies and other stuff like that. But let's say the highlight of our agency has been Hit Motion for sure.
00:27:17
Speaker
Okay. um So this is for the the entrepreneurial side. side what What kind of big changes do you see um as ah as a consultant that you were not seeing as an entrepreneur? Because I guess, I don't know how how you feel about that, but that when you develop a project for yourself, And when you develop a project for someone else, let's say, or for another company, I guess that's the that's pretty different because you are in different positions. Maybe the technologies are different. So what differences did you see between building your own um technologies and building your own companies?
00:28:08
Speaker
And being a consultant, what were the advantages and the disadvantages that you see and maybe the technologies that learn, the different products that you use, etc. Well, um let's say that when you do something of your own, you of course are more motivated because usually you're doing something out of your own passion, so you really want to build it. So you are motivated to work at night, you know, to deliver something cool and that's ah that's very good. ah So you you are putting yourself, let's say, in the product.
00:28:46
Speaker
And that's the cool side when you do something for yourself. The good side though of making consultancies is instead that you work with a lot of different technologies, because one day comes someone that wants to use HoloLens, another day comes someone that wants to use the Vive, and then there is 3D AR tracking, or maybe just Unity 2D, but you still learn something about that. When you work out of consultancies, you get in touch with many people, all of them, asking different stuff. So um that's ah how you also learn many more things in a way. So you don't stay with what is called, let's say, ah vertical knowledge. You become more horizontal. You get to know more that.
00:29:32
Speaker
So that's a good side of doing consultancies. When you do product, and I saw that also with my latest experience, it happened in Rome, you're trying to focus on the technologies of your product. So this is what I see as main differences. And the other one is that, of course, ah depending on when you are gone, when you do consultancies, you or usually try to work on the latest technologies. Not always, but hopefully yes. While when you have your product, it depends because ah you have update cycles. So maybe you stay for two years with the older version of Unity and you never see the new one. So ah you lose some of the latest up updates and then you have to catch up later by yourself. So it's a pass and cons of both things, but I enjoy it and I still enjoy doing both.
00:30:28
Speaker
Okay, so for the people like me that doesn't have um much and technological or developer knowledge, um Unity, I think you mentioned Unity, Unity 3D. And if not, the I think people will come across this this term in in if they look after information about virtual reality and augmented reality. So it's a popular cross-platform game engine used for developing interactive experiences, so including virtual reality and augmented reality applications.
00:31:06
Speaker
And Unity 3D refers to the software itself, which provides a range of tools and features for creating 2D, 3D, VR, and also AR content. And C-sharp um stands for... um ah It's a programming language. that is often used within the Unity 3D environment. And I guess that a lot of games are developed with with this technology. I cannot mention any of these, but I guess there are a lot of popular games that are built in this um language and within this environment. And when someone mentions Unity 3D or C Sharp,
00:31:55
Speaker
they are referring to the combination of the Unity 3D engine and the C-sharp programming language language for creating interactive content. And C-sharp, it's a powerful and widely used language in game development that is flexible and performant and easy to use inside this Unity ecosystem. So there are developers like you, I guess, something that use C-sharp to write scripts.
00:32:26
Speaker
and code that control the behavior of game objects, manage interactions, implement game mechanics, and more within the Unity 3D projects. I have read this term also on your scaredgos.com blog, that the website that I will link in the in the description. um Maybe another question that I have for you because there are a lot of terms that and people know when we speak about web, when we speak about apps, people know more or less the difference between the website and what we can call a web app. Let's say if we can speak like this.
00:33:06
Speaker
and mobile app and because it's pretty, let's say, easy to to know, do you have the the app on your phone or not? Do you use the computer? So I think people understand more or less this multi-media and technological terms. but maybe can you explain your definitions or what you think about the difference between VR and AR and also computer vision because for me it's not that clear so I guess that there are other people that may need this clarification please. Well it's a basic but at the same time complicated question because
00:33:46
Speaker
There are, let's say, some mismatch on the definition, for example, of the term ah mixed the reality. So you may hear some differences here and there when people explain what's VR, what's AR, and what's MR. So I will stay with what for me is, let's say, one of the correct standards that is the, what is called the reality virtuality continuum by Milgram and Kishino, which is basically these people that try to categorize the different realities. And they say they see realities as a spectrum. So there is the real world on one side of the spectrum, which we all know.
00:34:35
Speaker
There is ah the virtual reality on the other side. Virtual reality is I am in a totally computer-generated world, which has nothing in the relation with the real world I'm in. I can be in my office, I put a VR headset, and I see myself on Mars. So these are the two extremes. And then there are ah in the middle, according to this definition, there is the mixed reality. I mean, whatever you mix between the real world and the virtual world, when you create a mix of the two, you are inside mixed reality. And so to make an example, what are the types of mixed reality? The most common one people refer to is augmented reality. Augmented reality is I see the real world
00:35:26
Speaker
but I see on it some terms that are some elements which are virtual. So for instance, classical example is Pokemon Go. um I looked with my phone, I see the straight line in front of me, but I also see a virtual Pokemon on it. And since I'm mixing a real environment with a virtual element in it, I can say that's augmented reality. The reality has been augmented with a virtual element. There are other forms of mixed reality according to this definition. One is called documented virtual reality, but now I don't want to complicate too much the situation.
00:36:06
Speaker
I think that this is, let's say, a pretty easy and clear definition of the different realities, and that's why it's the one I use myself. But there is some everyone is inventing terms for marketing reasons, so I saw use the extended reality, extreme reality, extra reality. Now there is special computing by Apple. But at the end of the day, ah I think that the the basic terms are the one I said. and Regarding computer vision, computer vision is that arm of computer science that basically gives vision to computers. So basically it's analysis of images and videos so that a computer can understand what's in the image of videos.
00:37:01
Speaker
the the computer can understand what he sees in a way. So let's make an example of computer vision. You for sure know all the filters that now people use on mobile, buy Snapchat, et cetera, to put glasses on your face, put some dog ears on your face, or do some kind of face effect. How does the system work? There is ah your front face camera of the phone that grabs images, then makes a video, basically. And there is a computer vision system that, from a meme image, which understands where is the face. And where is the face, then you can add some augmented reality elements on it. So it's the it's a system that can analyze images and detect what's in the image. Basically, that we can say it's in layman terms, what's computer vision.
00:37:53
Speaker
okay um that's pretty clear i had also some notes about the different frameworks and And so sorry for some of the audience that are not technical, but it's the the goal of the this podcast to give um competitive they have and a competitive advantage for people that are looking to educate themselves. So after we will, um and let's say, go back to more um common question about the the journey of Antonio, et cetera. But I wanted to mention some of the most common frameworks and platforms used for
00:38:32
Speaker
AR and VR development that includes Unity 3D, so we discussed this. In ruling Jain that is similar to Unity 3D and it's another powerful game in Jain often used for AR and VR development that offers high fidelity graphics and advanced physics simulation and robust toolset for creating immersive experiences. There is also ARKit for iOS an AR core for Android that are platforms provided by Apple and Google respectively for creating augmented reality applications on mobile devices that they offer APIs and tools for developers to integrate AR capabilities into their iOS and Android apps. There is also Viforia that is an AR development platform that provides tools for creating marker-based
00:39:29
Speaker
and markerless AR experiences. It supports different devices and platforms, includ including iOS, Android, and Unity. And there is OpenXR that I think, if I'm not mistaken, Charlerie has spoken about the in the in one of the podcasts in French. and So for the people that don't speak French or didn't listen to the podcast, it's an open standard dart for VR and AR development. aimed at providing a common API for interacting with different AR and VR hardware on platforms. So it allows developers to write code that can run across multiple devices and platforms with minimal modifications. So these are just a few examples of the many frameworks and platform available for AR and VR development. So the choice, of course, depends on the project requirements, the target platforms, the developer preferences, and the the resources available.
00:40:25
Speaker
So this is just to give a bit more of content side context. I don't know if you want to precise something or mention something Antonio about that um this technical, let's say, yeah part. Not exactly. Okay. um So yeah, so coming back to you and your journey, so we were speaking about consulting. Are you still consulting? Don't know what what happened next? what the What are the projects that you have worked for? And and yeah, what did you what did you do since you started the consulting? Have you launched um other projects? I know that the answer is yes. um But can you tell tell us more about what happened after?
00:41:11
Speaker
Well, basically with with Max, so we the new technology forecasting is still up, so we still consult here and there. Let's say that we both are um mostly focused on other activities, but we still consult when needed, so that's still something that stays up because People, companies are always in need of someone with experience that can help them in integrating AR and VR. This is what we can do. So ah that thing is still alive. But we both, let's say, are dedicated our time to other things on in different, let's say, companies.
00:41:53
Speaker
So, I'm, for instance, after, let's say, I started dedicating more and more time to a company called Vroom since the twenty the end of 2019. Basically, I joined this startup that was about making apps or something like that for our festivals and cultural institutions. The moment that the CEO, Louis Kachutola, decided that he wanted to explore the future of events. So he said, okay, Venice in festival school, but what if we'll do that in VR, for instance? So he wanted to explore this and was looking for someone to join this new venture of his company. And I was like, okay, maybe I can work with you.
00:42:47
Speaker
and We started experimenting a bit on how to create events in VR. We he got some connections, we made some tests, but then the pivotal moment arrived with the COVID. The moment that the COVID arrived, everyone wanted a virtual event. And that's ah how the company, let's say, took off. Because we started in 2019, and then in 2020, the horrible thing happened. And yeah, so basically we we found ourselves making in the three, four years that come later concerts in virtual reality for crazy institutions, I mean crazy important institutions like the Venice Film Festival.
00:43:36
Speaker
or South by Southwest which in the United States is a super important tech event or for important artists like Jean Michel Jarre which in France is super famous and it was famous also worldwide ah because ah he's, let's say, one of the founders of modern electronic music. ah And we did a few concerts with him. We won some awards here and there. It's been pretty crazy.
00:44:08
Speaker
ah in twenty At the end of 2020 we made a concert with Ja, with which was inside a virtual Notre Dame. He got millions of worldwide views, was broadcasted on the French TV, so we had really Very interesting adventure. And then in 2022, sure ah we decided to start to build our own platform because we did our concerts mostly in VRChat. And we decided to want to build a platform to make VR plurality concerts. And that was another big adventure because making a platform is super complicated.
00:44:53
Speaker
People complain when there are bugs on VRChat or Engage or Roblox to talk about the platforms that are known to the the mainstream, but making this kind of systems is super, super, super complicated. So we did these this platform with a few concepts. ah in it with some artists like Manny White, like Max Sans, et cetera. And it was another, let's say, ah potential success. With the Alpha launch, we even arrived at the first spot as the most downloaded free app on the Picostore. And with that, ah that thing now has been put on hold a few weeks ago.
00:45:44
Speaker
Again, because of lack of fonts, but maybe we can really live again or things like that, but it was anyway um very, very interesting. It's an experience and a very cool product. Yeah, I have checked on YouTube. So Versailles 400 was a mixed reality concert live from the Hall of Mirrors on December 25th, 2023.
00:46:16
Speaker
yep to celebrate the 400th anniversary of the Chateau de Versailles. And Lynx Mixed Reality is redefining the boundaries of mixed reality. So the concept we have spoken just before with the Lynx R1 headset. Maybe can you explain what is this engineered in France, Lynx R1 headset, because it seems that um it was allowing the artist to share this experience with an audience. But exactly how how this happened, how this concept happened exactly? Well, the link set is a fantastic, an even very interesting project because basically there is a guy called Stan LaRock which decided to try to make a headset in France, which is his country, and basically challenging big giants like ah Meta, but with a team of like 20 people.
00:47:14
Speaker
So, and he's super smart, I know him very well. And he did a successful Kickstarter and he did this mixed reality headset with a first version with the the the mission of trying to propose, you know, something alternative to Meta and to other big giants. And the the headset is kind of a dev kit for now. And it's mixed reality. So when Jean-Michel Jarre was wearing it, he could see the audience around him. He could see his instruments, but he could play his music and basically broadcast it to our ah to our system. So the idea was that Jean-Michel Jarre could play his music in the real world and in the virtual world at the same time. And with a mixed reality headset, that became possible. So that's how it worked.
00:48:10
Speaker
yeah i'm watching them i'm watching the website that is links dash air dot com so l why and y ynx-air dot.com um and there are accessories as ah controllers and the travel case and the everything is sold out um and and you can subscribe for updates so it's a project that personally I didn't know um it's pretty impressive and the the there was as you say the kickstarter
00:48:48
Speaker
a project by Stan Narok that I also didn't know that 1216 backers pledged 725,281 euros to help bring this project to life I think it was in and end of December 2023 as well um um so yeah all the links of the this info will be of course in the blog post related to this um to this episode. So live concert, ah big milestone for cultural applications and big, I think, technology statement, of course. um I had other question that came to my mind. um I guess that's one of the coolest projects you have worked on. um
00:49:42
Speaker
But what, and in all the projects you have worked on, What are the biggest challenges you have faced? You mentioned the Kinect challenges. um So what ah what were the biggest challenges and how did you ah ah overcome them? Because you also mentioned that it's complicated. But I guess to realize an event like this, as you said, it's not easy and there are challenges and we cannot just you cannot just say, okay, there are challenges, i I stop, you need to find solutions. So what were the challenges and what kind of solutions did you you find?
00:50:21
Speaker
Well, it depends always on the project. I think that one of the main challenges is that still at NWA VR is a new field. And this means that there are no clear standards. A lot of things are experimental. You have to keep yourself updated. So it's a matter of surfing on a new technology and surfing on the problems that may come with it. and finding solutions via Discord server, so Google search or asking to other people. It's it's a matter sometimes times of just ah trying to solve a lot of problems that come down the line. One common problem that everyone has in the VR field now is money, and that's a common problem because now we are not in the, let's say,
00:51:18
Speaker
most hype moment of XR. So it's how to find investments and fund the projects. And without money, of course, no startup can survive. So every time that the that I worked anyway on an XR project, ah budgeting was very, very important. And on the user side, of course, there is always the the problem of the user interface. I mean, how do you create an experience in VR that is engaging, that doesn't give motion sickness, it is easy to understand, and blah, and blah, and blah, and blah. And the other thing is how do you cope with the fact that there are not many people with a VR headset? So for instance, when we did the concert, how do you
00:52:08
Speaker
make a concert that is not just watched by 1,000 people, but that you have many more. And for instance, what we did in all our most important performances was streaming a video about it on the most important platforms like YouTube, Facebook, Twitch, et cetera, so to increase the reach of the work we did. So these are the some of the main problems that I found in my last years in this tech space. Yeah, it's pretty interesting. I think you mentioned two points that I wanted to mention. So um the first one is, how do you stay updated with the latest trends and technologies? Because it's um pretty hard to find information.
00:53:00
Speaker
I think it's a pretty new industry as well. So as you said, following news, ah you mentioned also some conferences and events. I think it's it's a good practice. Joining online communities, forums, as you said, it can be Reddit, it can be Discord, it can be Slack channels. listening to this podcast, of course, yes um using new technologies, continue to learn, reading papers, joining developers communities. There are some, I guess, so if you have some links, Antonia, you can also share them. And follow also leaders and influencers like you on on social media.
00:53:41
Speaker
um The second point that I wanted to mention and it's, it's it's I asked these two questions also to Sean and I said, but why it is so hard to get profitable for AR and VR startup and why is it hard to get funded? And what I researched and what I found, and I guess you would tell me if you are agreeing or not, is that there are different reasons. The first one is that the development costs are pretty high because these technologies and these applications require specialized keys and technologies that can be costly.
00:54:19
Speaker
And it creates, of course, good experiences, but it involves investment in hardware, in software, in people, and and and people are are expensive. One other thing is that it's pretty long to develop I think compared to traditional software applications and and it can extend the time to market and increase the development cost. I think also there is a new maturity on the market because it shows promise but it's still
00:54:53
Speaker
Relatively recent and nascent comparing to other industries, the market size and the user adoption rates and may not yet justify the investments because we don't see people using these technologies. And I think it's hard for startups to demonstrate immediate um profitability. I don't have found one yet. And there are hardware constraints. There are, as I said, user acquisition, retention and monetization because If it's not possible to find revenues and to make revenues via advertising or via subscription fees, um these experiences are are pretty expensive. um And regarding the funding,
00:55:36
Speaker
I perceive that there is a high risk that investors are seeing and and and this can be justified by all the points that I have mentioned before. It's hard also also to find ah metrics because um to convince an investors and of course the startup needs to show ah growth and but potential shell and viability and also the familiarity,
00:56:08
Speaker
and the the the fact that to secure funding from ted traditional source sources it's hard because people are not just familiar with the potential and the opportunity and also the time horizon I guess that if someone invest I don't know because I haven't spoken to investors yet but I guess that the time frame is pretty long. An investor cannot seek for short-term returns. So that maybe also can explain um why it's so hard to to to get profitable and and to get funding. Do you see other reasons or do you want to share any other advices or or experiences about that?
00:56:50
Speaker
Well, I think, more or less, to summarize everything, I think that the the the important thing to understand is that investors are not, let's say, no profit. They give the money to people. They want returns. And VR is not in its ah best shape. So it's proven every time that it needs more time. VR, and it's, I remember when I spoke with a senior vice presidents, those Sony pictures, is that people who want to say in VR should commit to 10 years because it's going to be a long ride, a fun ride, but a very long ride. And so investors want their money back usually in three, five years and knowing that VR can take a very long time is not good for that. The market is more. It's hard to make money because the market is more.
00:57:46
Speaker
compared to mobile, to PC, or whatever other technology. Plus, doesn't help, of course, a journalist that the first problem start to get panic about the real ecosystem doesn't working, et cetera, et cetera. So that's the reason. But it's also true that we are actually in a moment where let's say, Apple entered the field, Google and Samsung are entering in a few months. So actually, I think we are in a moment where XR will raise again, but will be a slow raise from the ashes. So actually, I think we will see more investments in the coming months about XR, but still, it's a hard field to be in. And there are some success stories, which is good.
00:58:40
Speaker
But I can tell you even many more failure stories because it's really hard to make a decent money about it because it's not just about making some money. To survive a company, you have to make hundreds of thousands of euros, millions of euros sometimes. So it's um it's a complicated field to be in. but the In the long run we will succeed, but our investors will invest and when we are closer to that moment. That's for sure. um Speaking about the time horizon and the development process, can you please walk us through the typical development process for creating an ER VR experience from start to finish?
00:59:28
Speaker
Well, it's a not that different from other projects like 3D games or such. Usually you start from a purpose like what are you building? Why are you building it? I don't know. We want to do a concept for the end of the year. And then you try to refine this idea. So, for instance, where it became in 2020, we want to do a concept with Jean-Michel Jarre and Otto Dam to promote the the city of Paris and to find shotguns, et cetera. Then you basically um start ah creating a team about that because you maybe have some internal resources, but you may need some external people to help you, maybe with audio, and maybe with tracking, et cetera. Once the idea is refined, it should be
01:00:24
Speaker
um It should become super detailed. So it should come from an idea to exact specifications. Yes, you want to do a concert with Ja, but how many songs ah ah with the word graphics style, et cetera, et cetera. And usually this is something that the different heads of the departments, the tech guy, the art director, the sound engineer, et cetera, work together. So they start defining every detail of the application and then also to delve into the technical design. So not only ah what is the
01:01:06
Speaker
ah you know, what is the style of the graphics, but also with what program you make the graphics, what shaders are you going to use, and what game engine. So Unity Unreal, as you mentioned before, et cetera, et cetera. So as a developer, for instance, at this stage, I make ah the technical architecture. So which technical part, talk with other technical part to reach the result. After that you refine the needed team hiring people that you may need. You start working and while you start working you should coordinate and with the other people of the team check with the project manager if the development is in time, if you are late refine the things that are being laid.
01:01:57
Speaker
Usually, this is the moment where there are problems and you need to find solutions to problems. You have to change some of the originalal idea some part of the original idea because maybe you discover it's not doable, et cetera, et cetera. And then after the development is finished, you have the final application or final let's say experience anyway. you need to test it so you need some people that try it a lot of times discover all the bugs all the problems that are there you solve them and then after all this big mess you go live so you release your product or you launch your experience and that's the moment where what happens happens so you do your VR event and
01:02:46
Speaker
Maybe you there is some bugs in it still that you didn't find are interesting. The show must go on. Usually it's normal and you you just have to smile and let things happen. And after that, you you keep refining it and improving it and things like that. So what's something that is very important for me is always the the science stage. Everything should be thought carefully before you actually start working on it. Otherwise, it's a recipe for a disaster.
01:03:18
Speaker
Okay. um
01:03:23
Speaker
So this was answering one of my questions about the the development, let's say, cycle or the the typical development process. Another question that I have is, what do you see as the most exciting opportunities for AR or VR in the near future?
01:03:50
Speaker
I don't know, otherwise I would be working on it. Let's say there are many opportunities, if I have to say something, for instance, one of the fields that in my opinion is going to be interesting is the mix of technologies, especially mixed reality, augmented reality and artificial intelligence. Because there is a lot of hype about AI, I think, in part justified, in part not. But it's a very powerful tech. And I think it's very cool if mixed with mixed reality. So I think the opportunity is trying to use these technologies together and try to create something that can give value to the user.
01:04:35
Speaker
Okay. And regarding the different industries, people often speak about gaming, of course, often speak about healthcare, often speak about education. Concretely, how do you think these technologies will impact these industries? Like what will we see into gaming changes changing? What will we see in education that will change in healthcare, et cetera? Do you have already some some project or some some things that you see happening Well, I think that, you know, we already seen the transition in some fields like, ah you know, for instance, industrial prototyping, already VR is widely used, in remote assistance, AR is widely used. ah In education, not that much. I guess it's also, there are some practical problems about it, but it will be.
01:05:32
Speaker
Training painting in VR is super powerful. Gaming, we saw already some amazing games in VR, but of course the technology should be more widespread to affect more the whole gaming industry, which is enormous. So I think we are already seeing this transition, and I think it's a matter of value except for poses like In training, it makes people learn some stuff in a more efficient way with more attention of information and also makes people do some safety training, which can be very dangerous totally safely with just a simulation.
01:06:14
Speaker
With prototyping, people can prototype in VR instead of making and prototypes with chalk or other, let's say, maph physical materials, which are very, very expensive. So I think what we'll see is the improved use of this tech. Probably the field where I think that we can do more and we will do more is education. i I think that, the you know, having a virtual tutor powered by AEI for students is the the end goal we should all have because everyone of us has a different way of learning stuff and having a customized MR tutor will be great for that.
01:07:04
Speaker
Yeah, that's true. um it It brings me to a question that um I don't know if there is a specific answer, but i I was asking this question a lot for the blockchain technology because initially I come from blockchain. And now I'm diving into a multi-duality, virtual reality and metaverse and artificial intelligence and so on. And I always try to answer a question that is, do thispro when when I was speaking to someone that have that that have an idea or a company that is looking for raising funds or or different persons that are, do you really need blockchain? And I'm asking this to myself most of the time because i I'm not asking this to the people directly.
01:07:51
Speaker
And I would like to ask you the same question. ken Can you, but of course it can be a podcast by itself, so I don't want to spend, because we we already have spoken a lot, but how can someone decide, okay, my project my project needs AR or VR, or maybe for doing what I want to do, maybe I don't need this technology. Well, ah that's a good question and also you know very complicated to answer. But in general, I'm the first one that when people ask me a consultancy, I try to understand if they really need the RMBR.
01:08:29
Speaker
That's the first thing. Sometimes people just heard about the technology and want to use it because of the hype or whatever. But that's not the right approach. The right approach is user technology if it is needed. I think that ah depends on the use case. VR for instance is very useful if you need to available to users in some other place doing something. i I'm not good at explaining it with a few words, so I'll make a few examples. Maybe it's better. You need to do some industrial training, which can be dangerous. Of course, you use VR. ah You can do a lot of simulation training totally safely. It's fantastic. You want people to meet from all the parts of the world and discuss a common prototype.
01:09:18
Speaker
Of course, you use VR because you want to people to meet and ah remotely, but they want to interact with 3D elements, so it's fundamental ah that they discuss a prototype in VR. You want people to meet and have a chat. Do you need VR? ah No, you can use Skype, you can use Zoom, it's much more efficient. I open a link on my PC, I'm in Zoom, I see people in the face, it's much better. because there is no 3D needed value.
01:09:51
Speaker
I see some people making some things or I frame ah an image on, I don't know, a cereal box on with my phone and I see a video in AR. Is it useful? No, it's a video. You can open it on your screen in 2D. Do you you frame a cereal box and you have a little game about scoring goal in AR. Is it useful? Yeah, it can be a marketing initiative in AR. I would say whenever you need something that surely needs to be 3D for a good reason, then it pays special 3D interactions and broad interactions. Usually said it's a good candidate for AR and VR.
01:10:44
Speaker
ah When ah you just need some info or something like that, maybe it's not needed. There are some exceptions to this rule, but let's say more or less this is some example to guide ah people in understanding when it can be useful and when not. Cool. um Then last two questions and then I propose you to conclude. the um The one that I want to start is, can you please share any tips or advice for aspiring AR, VR developers who are just starting out?
01:11:24
Speaker
Well, I think that my I will be quick on this. I think that many people start ah wondering what's the right way. I didn't study RMBR at the university. My way of learning was by doing. So, my suggestion is that the you start learning, let's say, um on a book, on a course at university, on an online course, etc. The field you want to learn. So, if you want to be a developer, I don't know, you learn a Unity our development on Coursera or Udemy or whatever.
01:12:08
Speaker
And then you start doing something and learn by doing because it's a technology that you still have to do a lot of practice to to be learned. So that's my suggestion. Okay. And sorry, I have two at the end to conclude. One that is, have you encountered any memorable or unexpected experiences while working on a project? um What do you mean? Something that was unexpected or something memorable, a bit like fun, or I don't know, something that you you say, hey this this was an experience that was unexpected.
01:12:50
Speaker
Well, I'm thinking and, you know, maybe it will come to my mind later. I can only say that something that wasn't expected in my project sometimes were the success and maybe some projects at or like when we did a concert with Jarrah, you know, really in the day after that was 75 million views on the social media. That was kind of crazy because you're like, ah Did I really take part in such a successful thing? you know That's that's ah the the most the biggest surprises I had, were mostly of that type, let's say. ah And sometimes also on the negative side, you launch something, it breaks.
01:13:37
Speaker
It's also the negative side of unexpected things. Of course, okay, that's cool. That's what I wanted to know. And then the real last one. um How do you envision the future of AR and VR technology? How do you see it involving in the coming years if we have to speak in in five or ten years again? Well, um and in 10, 15 years, what I imagine is that most people will have an AI headset most of the time, which will enhance their life. So we will live in a constant mixed reality, hopefully hopefully, which will be a beneficial one, with the AI being supportive for our activities.
01:14:28
Speaker
So, for instance, just to make an example that Mark Zuckerberg did, ah why should I have a physical TV and put it in the on the walls and then it breaks and then it's a mess and paid 120 euros when I could have for one euro a TV app or my headset, put it everywhere, make it follow me, e etc. The good thing of these EXAR technologies is that the virtual elements can do what they want. They don't have to obey the law of the physics, so I can have a virtual TV everywhere.
01:15:02
Speaker
So I imagine having assistants like now we have Siri or now Chachibiti et cetera that help us ah constantly maybe with the some human appearance. So we can have some virtual assistant helping us everywhere. um And then I guess we will see because ah it's always hard to predict what is going to happen in the far future because there are a lot of ripple effects. so Remember that to close, why it's so hard to predict, you know, just to make an example, if you ask people 50 years ago how they imagine today, there have been a lot of good predictions about some science ah fiction, authors, et cetera. Some people predict in flying cars or TV, radio. People predict a lot of things, but very few predict that like internet, which
01:15:57
Speaker
ah arrived as an invention to connect people and then basically disrupted totally our lives. So it's hard to imagine how some technologists that maybe today are still in embryonic phase will change everything. One of them is AI. You know, people say what AI is going to do in five to 10 years. And I don't know, ah for sure is going to influence a lot of things it was already doing. So 10 years from now, what AI can do for us in mixed reality, I think no one knows, not even Sam Altman. So it's going to be an exciting ride. And I think what we should do is adapt. ah We should adapt fast to whatever technology is going to adapt.
01:16:44
Speaker
Okay, that's pretty good. So thank you. In conclusion, I think that I can say that you have shared an incredible journey. m You have adapted in a world that needs constant um adaptation and in a world that is constantly ah evolving. And I can hear that you maintain um a positive spirit and that you continue to leverage your skills in in this ah ecosystem. So I'm glad to have this to have this discussion um today. So you um I guess that you are still, um and i and I could hear that you are still passionate about this ah
01:17:32
Speaker
technology, yeah you have a blog that I will share where you are sharing insights and reflections and the so I think one of the lessons also is that we need to be creative and resilient, have also a sense of community if we want to move this industry forward. And the dedication is making the difference. Dedication, I guess, is always making a positive impact yeah professionally, personally, and it inspires us and it reflects the basic spirit of innovation that defines the this world of immersive technology. So thank you so much, Anthony, for having um joined me today. And yeah, I'll let you ah conclude and see you around.
01:18:24
Speaker
Well, it's it's been my pleasure. Let's see ourselves in VR then. OK, ciao. yeah