Pedro Gil Farias & Hugo Pilate were one of the 7 selected artistic project for the first #RiT artistic residency, back in May 2023, in V2_ (Rotterdam). In this framework, Dark Euphoria asked them about their relationship to XR, their artistic path and favorite tools. With this 2 out of 7 special interview, discover Pedro Gil Farias & Hugo Pilate unique look on eXtended Realities…!

text

Dark Euphoria | Interviewer: Céline Delatte – Happy to meet you two and discover more about your specific approach to XR; which seems very collective and participative…  But, before we talk about that, could you each tell us a bit about yourself?

Pedro Gil Farias: I’m originally from Portugal, now based in Rotterdam. My background is in industrial design. Then moved into this idea of participation and futuring, got interested in focusing on the public space and on matters of commoning. For me also this idea of using DIY technologies and approaches that are approachable, like the accessibility of tools to come together as a group.

Hugo Pilate: Actually we have pretty similar backgrounds. I also come from industrial design and workshop facilitation. Getting mostly for corporate clients, to come together and prototype their ideas out of cardboard or styrofoam, could be a bridge, a product, or a neighborhood, to better understand what each person was thinking or align their vision. Then, during the pandemic, I started replacing cardboard with digital tools. So turning to platforms like Miro, Minecraft, or Fortnite Creative, to see what I knew of collaborative activities could translate to these sandbox environments.

Pedro G. F.: You want the origin story of how we met? Hugo was in India working at Quicksand Design Studio and they did a project on the future of humanitarian aid that I saw Anivash Kumar (Quicksand co-founder) give a talk on at Primer2020. At the moment I was researching the intersection of speculative and participatory design and decided to reach out to Avinash…

Hugo P.: …And he put us in touch since I had told him I was planning on moving to the Netherlands. Then we collaborated on a couple of projects for Mozilla Festival and for Extinction Rebellion. We made online gathering spaces that were often paired with weird micro-workshops. And through that we started collaborating more and more. 

C.D. – What was your first contact with XR? How did you actually meet XR?

P. G. F. – I started to do traditional printmaking and for me the combination of analog and digital was really interesting… Now I’m screen printing, but before I wasn’t.
I was doing lino cutting and was thinking how I could bring a digital layer to the print, which is a more straightforward use case for XR. That was where I started tweaking and experimenting with it.

H. P. – My point of entry was facilitating prototyping workshops at National Institute of Design (NID) with Salil Parekh where over the course of a week, we’d get students who didn’t really work with digital tools to tell a story through a space that they made. They’d storyboard it, then build it in cardboard, then film it, and then have to recreate that in a game engine. So it was more about the digitization of experiences and digital storytelling. Trying to extend what we could do with both analog and digital mediums. That was kind of the starting point. 

H. P. – We made a website for this project “whatamess.city” where we documented all our influences as well as the tools that we used. 

Pedro G.F.: But in terms of XR tools specifically, we use a lot of photogrammetry and 3D scanning in general. We used Polycam. It’s an app that’s downloadable on any phone. And for newer phones, especially iPhones or iPads – that we used in the workshops – it makes use of its LiDAR scanner. It was incredible with the participants, even yesterday, someone was asking like: “but how much time does it take to do the scan? Like four hours, six hours?”. Actually, you do the scan and with the LiDAR, in 20 to 30 seconds you get the scan made. And for me, it’s the confrontation of how accessible it is. Especially with the 3D scanning. And that was also part of our project: what’s the use case of it for a non-expert, for a normal consumer or for someone that just wants to go around the city and scan things…  


H. P. – I think the other thing that’s really nice about these tools is that even though a lot of the 3D scanning and the AR we’ve used has been around for quite a while, the way it’s getting now integrated into smartphones and tablets that you might buy just to watch videos or play chess is actually really incredible… When we did these workshops, participants came out with very practical questions on how to start by themselves, whether or not they identified as creative individuals. It’s not like “Oh, you had this expensive equipment that I could never get”. Some even downloaded the app before the workshop ended. So that really changes your relationship to XR work, to realize you could try it on your own device.

C.D. – Do you have any tips to find resources on XR?

P. G. F. – With this kind of stuff, I always get surprised by how much there is and then it’s like a pick and choose… You see a tutorial on SparkAR, on augmented reality… and you just mash it up together. What I like is people that bring this technology through a different angle or through their own topic, especially urban space… Like Tobias Revell, a designer from the UK that makes blog posts that sometimes have nothing to do with 3D or AR but then he always tries to make a 3D snapshot, 3D rendering and include it in the beginning of the text. I find it interesting how these tools can integrate other topics. 

H. P. – For me it’s mostly through artists like Matthew Plummer Fernandez or Theo Triantafyllidis, I gain a lot from following their work. For the photogrammetry, one thing that tipped us off was also a Corridor Crew video, they are VFX YouTubers based in LA. They had made a video playing with game engines and 3D scanning, there’s a chance we fell for the product placement… 

P. G. F. – Another is Oddviz,  an artist collective with an architecture background. They go to cities, they scan a lot of things and then they make these archives of things they scan, but in a really geometric patterned way, resulting in amazing panels of it. They were the first ones, for us, coming from design, to really try to go in between these artistic, but still research, ways of using XR. 

Hugo P.: It does feel like “XR” is almost too specific of a term. When you think of how porous the lines between the industries using these technologies have become: film, theme parks, new media and gaming industries Like the ILM StageCraft screens used to shoot the Mandalorian, the fact that game engines were used to create the backdrop elements to me is really crazy. So we really need to think about XR beyond the headset and the smartphone. (Speaking to Pedro:) Preparing for the residency, you were looking at the XR works prior to smartphones, and that was really helpful. A tipping point when you brought this up to say: “if we are going to use a touchscreen, how do we make sure that we don’t take it for granted?”. At some point we even tried to do everything on the tablet: the 3D modeling, the scanning, the AR. We ended up wanting to have a little more freedom so we didn’t restrict ourselves to that, but I think that was a big point that we wanted to try and make. 

“We really need to think about XR beyond the headset and the smartphone.” Hugo Pilate

Pedro G. F.: That’s again, coming back to the research. Coming across a screen, in parallel with screen printing, the DIY aspect of it… It’s amazing that people can just start doing XR like this. Even though I don’t use the face filters of Facebook and Snapchat, the fact that people are doing these things with no prior experience of 3D and they’re making custom filters, it’s almost punk. Except that’s Meta / Facebook. But there’s something there that’s also DIY.

H. P. – I mean my younger brother is studying business administration but he has taught himself 3D earlier than I did as an aspiring industrial designer, because I was waiting for my teacher to tell me how to do it. And he’s just casually 3D scanning things on holiday and putting it in Unreal once he gets back home. But even at Epic Games, they have a very clear picture of this, they call it the gamer to creators spectrum. They’re super aware of it. So all their product suite is mapped on it. XR is too often seen as visual augmentation with different degrees of opacity, that’s very limited. That’s why we love Leon van Oldenborgh‘s work, for example, that uses locative media arts and GPS because that’s also part of XR.

C.D. – It extends the meaning of extended realities…
Would you have an advice for artists or creative people who want to begin with XR?

H. P. – We started mostly with this project. 

P. G. F. – We don’t recognize ourselves as XR artists…(…) We were researching papers from the 2000s about virtual and extended reality. In SMSLINGSHOT project by The Constitute, before using smartphones, they were making an XR Nokia catapult. They would write a message on a Nokia phone and they would slingshot it into a LED public display. And for us, the one sentence that struck first,  was this idea put forward by Eriksson and colleagues that these innovations on XR focus too much on introverted devices. You put on your VR, you’re introverted, even though you can put the screen so other people can see what you see. Even with Pokemon Go, what you do is also seen by others. It’s like: this cool combination of real world and digital world; how is it going to merge? I was really skeptical and I still am. It’s like AR and VR is limited to the sense that you put the digital layer and that’s it, but there’s no connection of one with the other.

H. P. – That’s also why we chose to have one device, one tablet, that was then shared by all the members of the group. Each time we ran a workshop, we limited participants to five captures in order to give the process a sense of preciousness through scarcity. And the group of five to eight people had to deliberate on what to scan. Having one device, also brought up the question of isolation often linked to digital experiences. The idea that the PlayStation was meant to be played with a bunch of people side by side, and then it became alone in my room. Even if now you are likely connecting from your room to massive online communities. But the side by side physicality is still a lot more rare. So that’s one aspect we’re trying to bring back. 

P. G. F. – Depends on how people learn and use stuff. You can have a project and you jump into a software that you don’t know, but at least you know more or less what you’re going for in an experiment. Other people also learn it without any project in mind. 

“Most of the innovations on XR focus too much on introverted devices.“ Pedro Gil Farias

H. P. – I think the main point is to avoid doing it just for the sake of using XR. Having a story, having something to tell is crucial. It’s really crazy the amount of documentation that is online, especially with YouTube,  even for TouchDesigner, on SparkAR, on Aero; there’s this incredible amount of documentation. 

P. G. F. – Coming back to the iPad and the PolyCam again, you can go to Unity and if you never code, if you never use a game engine, you can easily get scared away. And that’s the first problem I had with the Lino cutting and AR. I went into Unity and I got frustrated. I managed to get something out of it, but that’s why maybe I never touched it again… (…) With Aero, it’s all so natural the way that they managed to do it. In the workshop we managed to scan stuff and make a prototype in augmented reality in only two hours. Which is crazy just because PolyCam and the Aero exist. Otherwise it wouldn’t have been possible.

H. P. – And the AR took 10 minutes. It was two hours because we were walking and talking. You really have to break the bubble of the XR. For me, the path-in was creative coding and open source software development, much more than any sort of VR-specific or metaverse hype. We used Mozilla hubs for the Haul Earth Ledger project, that’s what got us to start experimenting with XR. As XR artists we have a lot to gain from looking into all the work that’s gone into making this space what it is before it was called that.

P. G. F. – And for me it’s the analog. If I can find ways to combine screen printing, which is the most analog stuff that you can get with the digital thing, it’s perfect. 

H. P. – The other type of creative coding and open-source-related space we’ve started exploring with this project is game modding. And how opening up your game engine allows people who are gamers to then intervene on your creation by adding their mods to their games. Like the elephant in the room I think is that we’re not modding our hardware compared to some others. We chose the consumer product as a way to say: “you can do it from your sofa”. And that was important and valuable. But then I think the flip side is that we’re not right now in a capacity to hack our tablet for instance. If you’re gonna say analog, then it seems like that’s the reality.