Mars College

Little Martians — from ceramics to 3d animations
29 Oct 2021 back to blog

Hello World!

As I discussed in my last post, I started creating the Little Martians series on the first year of our project, using local clay. After discovering what happens if one heats up untested clay to high temperatures (the sculptures melt in a very interesting way!) I decided to try high end clay for a while, so I could learn much more about all sorts of high temperature glazes. One of the greatest things about a project in which I can easily produce a lot of small sculptures — thanks to the silicon molds! — is how much experimentation with painting on ceramics I can do. I’ve been also learning about how to make the clay look cracked (it depends on the way I insert it into the mold), how to press the flexible molds to get different features out of each martian (they can have wide faces, slim, deformed etc), and more, there are so many variables to play with, making every martian unique.

Little Martians - Process from Vanessa Rosa on Vimeo.

Painting on ceramics means a lot of chemistry. There are underglazes, oxides, glazes, stains, slips, overglazes, one can add pieces of glass, metal, and so many other things. A ceramics master should understand each one of the ingredients of this mix, to know what they do and how they interact. The good news is, if one is not concerned about achieving a regular uniform outcome, one can combine it all and learn in practice what happens — and study the chemistry little by little while having fun with the sculptures! Working in a collective ceramic studio is also extremely helpful to learn faster as one can watch other artists on a regular basis. I had the enormous pleasure of being a member at Sculpture Space NYC for a couple of months, where I could play with all their custom glazes. There’s where I met the marvelous ceramic artist Leilah Babirye and enjoyed exchanging techniques with her. I’m grateful for how she taught me to fearlessly apply as many glazes as I felt like it. Several Little Martians sculptures and a StyleGAN2 animation made out of them. Plus some photos with 3d photography AI models. Music by George Gershwin, Rhapsody in Blue.

After I created a bunch of Martians, I started to animate them. First, I’d take simple photographs with my phone and edit them on Adobe Lightroom, then send it to Runway and apply the First Order Motion model, an AI model that is capable of tracking a face on an image and applying the movement tracked out of another face from a custom video — including your personal webcam. There are several Colab notebooks with this model as well. One can also just apply the Lip Sync model to make the images talk. The outputs of the First Order Motion model usually have low resolution and it might be worth it to apply a super resolution model on top, though I found the results still unsatisfying. Another option is to use specialized apps for this: MugLife is by far the one with best output quality and people can improve the face mesh over the photo, creating a much better tracking. Tokkingheads is another great app, as it gives the option of high resolution outputs and, like Runway, one can use custom video. The downside is that one must pay more to have access to these apps, while Colab can be free and Runway has some free features.

But great as they are, these AI models are still very limited. So I decided to learn photogrammetry, which turns out to be easy and quite a lot of fun. I researched the best recent softwares and chose Regard3D, because it’s free, open source and has high quality. It’s a software made for researchers, which is why it presents up to date algorithms, it’s just as good as many of the paid ones, it just has a very simple looking, uninviting, user interface. One needs to follow some tutorials, like this one, watch some videos if you want to know more, and you’re good to go! In my experience, it does make a big difference to use a good camera, and it was better not to edit the photos at all (at least I thought it was big trouble to give anything other than the original files to Regard3d as it could not recognize automatically the camera I used if I edited the photos or exported images out of a video). Lighting is also a key issue and it’s tricky to photograph glossy ceramics — my models came out with these white circles on their texture. Therefore I still need to improve on more diffuse lighting and apparently adding a powder, spraying, doing something to reduce glossiness is helpful. When taking the photos, one should aim for good focus, a turntable is helpful to get all angles, and it’s key to take photos moving around the object gradually as images need to overlap a lot to give the software accurate information. There are many great tutorials on photogrammetry tips! I’ve been taking around 50 to 90 pics to each of my sculptures, and my computer is not the strongest, so it’s been taking me over an 1h to create each 3d model on Regard3d.

Martian living in the simulation — inside Blender 3D!

I export my beautiful models as .obj and then I can start playing with Blender. I’m very new to Blender and it often gives me headaches. I’m constantly googling every little thing as it’s not intuitive at all to someone new to 3d. Happily, after following the classic Blender Guru Donut Tutorial series, I now have the basic skills to move around — again, stoping all the time to check more specific tutorials. Knowing what one can do with these softwares is key, so one can know what questions to ask google, what techniques can be applied at all. I’ve been following a lot of amazing artists on Twitter and Instagram to discover what this field can do: Marjan Moghaddam, Nude Robot, LeMuet, Yann Le Gall, just to mention a few. There are artists doing fascinating work in terms of 3D and AI, such as the legendary Alex Mordvintsev and Ajay Jain (on the combination of CLIP+NeRF) — soon we will have another boom in 3d possibilities! Plugins are also really helpful to skip a lot of phases: there are plugin for improving the model’s topology (the polygons that make the 3d models), plugins for animating and connecting with Motion Capture and AR, plugins for improving lighting, etc etc. Some of these plugins are free, some have free trials, and some have to be paid for. Sometimes, another open source software can be used in place of a plugin, and one can switch from Meshlab to Blender, and to whatever other software. Otherwise, I still need to learn what Houdini Fx, Cinema 4d, and all these other softwares can bring to my models.

Anyway, it’s a LOT of learning, but very rewarding — at least when I don’t get frustrated with troubleshooting, spending an hour to discover something quite stupid I had no clue (the way one saves the files also matters! I learned with some pain to save each blender project and .obj file on a specific folder). It’s fascinating that all these techniques are becoming accessible to simple humans who have a laptop, internet connection and can speak English. Most of the learning material is in English, so there’s still quite a language barrier as multi language educational content is far far behind. Happily AI can help with automated translation.

Now that I have these basic skills, I can start the more audacious chapter in the Little Martians project! Storytelling!!! The Little Martians will present an archeological/speculative research series about humanoid history! With many conflicting ideas on what happen to humanity. But that’s a whole other discussion, a conversation for my next blog post ;)

posted by: Vanessa