Digital Terraforming
Learn how a real–world object was converted into a 3D digital object by exploring multiple solutions. 3D objects provide learners with a more complete connection to the course content.
The project
In some circumstances, students can’t be at a physical location. These circumstances might include barriers to travel, weather, illness, laziness, or a pandemic that heavily restricts our ability to be in close quarters. When teaching many subjects, theory will only take you so far. It’s sometimes necessary to introduce props that will aid in student understanding and retention. That said, even under optimal circumstances props can be limited in their quantity and thus, having one prop for a class of 50 students creates challenges in itself. Plus, what if that prop breaks? And how do you assess a whole class on their knowledge of that prop? Here’s how I solved these challenges.
The challenge
An instructor has physical models of geological formations, as shown in the picture below. Students need to study them and are tested on them. Each physical model has several small numbers on it, which are associated with an information sheet. Obviously, being able to see these numbers is a big plus. In addition, there is only one model located on campus — so even If students were on campus together, access to this model would be limited as students would have to study it in groups or sign the model out individually. Therefore, having a digital version available to study at a moment’s notice would be ideal.
First I‘ll show you the end result, and then take you on a journey through the mountaintops of trial and the valleys of error that led me to my solution.
We initially considered using video or still images to display models. However, because there is so much information on each model (and that information is very fine) the decision was made to try and digitize these physical models and have them represented as virtual 3D models. This way, students can manipulate the models in much the same way that they would in reality.
So, how do you recreate a physical object as a digital one, you ask? Good question! Before I attempted any solutions, I had a few hurdles to overcome. First, the physical model’s surface is reflective, which makes lighting and capturing more difficult. It’s also hollow, making it more difficult to prop up for capture. Additionally, the models are produced by a company and, thus, copyright must be adhered to. What’s more, some modelling software is very specialized and costly with a high learning curve. Lastly, I want the end user to have as close to a one-on-one experience as possible, meaning that the end user can manipulate the digital object as easily as they would a physical one.
After overcoming these initial hurdles, here are some of the solutions I tried.
Solution one: Photogrammetry
A simplified explanation of this process is as follows:
- Take between 30 and hundreds of photos of an object in a cylindrical pattern, capturing every angle of the object with roughly 80% overlap between pictures.
- Run those photos through specialized software, which then generates a 3D model along with a texture.
There are many ways to do this and the process can get quite technical. Suffice to say, I tried this method but because the models I was trying to recreate were reflective, the results were terrible. The only way to get a quality model and textures using this method would involve spray painting the object a matte colour (usually black) and then going through the photogrammetry workflow. Because we only have one of each of these models, this was not a feasible option and the time it would take would diminish any returns gained. In the end, this solution would take 6 hours per model and produce unusable results
Photogrammetry — REJECTED
Solution two: Handheld 3D scanner
We have a device that attaches to an iPad and acts as a lidar scanner, which allows me to quickly scan an object into 3D. It works great, and one of the big benefits of using this solution is that it’s not as affected by reflective surfaces.
Another reason for using this solution is that it produces a textured model in minutes, drastically reducing production time. The main downside to this method is that model fidelity is generally fairly low. Although I had high hopes for this method, the results were irregular and poor. This method worked better than photogrammetry, but because of the size of the physical model, the poor model reproduction accuracy, and lower resolution of the textures, this solution ultimately did not work. Note that there are other hardware solutions in this category, but as they would cost between $5000 and $100,000 to use, they were not realistic options.
Handheld 3D scanner: REJECTED finished product scan
Solution three: Using apps and phone cameras
There are many apps that use the camera on your smartphone to create 3D models. I tried a few and although none of them produced viable results, at least it was a quick and easy test. Pictured here is an app called Q-clone. Although it had one of the best results, it also shows how the photogrammetry capturing process is done. Unfortunately, because Q-clone uses a printed sheet of paper and because of the size of the models I am developing, I wasn’t able to test it on the actual geographic model. Regardless, textures were muddied and blurred and geometry was imprecise.
Using apps and phone camera: REJECTED
Solution four: Photoshop with specialized plugins
This was a long shot, but it was very promising. I gained access to a Photoshop plugin that takes real–world locations and, by using a heightmap in the app, recreates those locations as 3D models using the map’s texture information. While it yielded great results, the resolution was fairly low and it was very difficult to find real–world locations with all the required geological features. I also needed to include cross sections of the earth to provide more information, which would require high amounts of instructor interaction and thus cost more. This did, however, give me the idea for my last attempt at creating heightmaps.
Photoshop with plugin: REJECTED
Successful solution: Handmade heightmaps and multiple software solution
Using the specialized plugin for Photoshop gave me the idea of using my own heightmaps to generate a 3D model and then adjust as needed from there. This process, although initially difficult to map out, is a fairly simple. The most time-consuming aspect is creating the heightmap itself. For this process I use an illustration iPad app called Procreate. Working from the highest to the lowest elevation, I create selections and fill them in with gradations of grey, starting from white at the highest points to more dark grey on the lowest elevations.
Once I have a greyscale heightmap, I take that into ZBrush (a 3D sculpting software) as an alpha and generate a 3D model based on it. This results in a fantastic model that is slightly exaggerated and needs minimal editing. Now I have the geometry completed and need only the texture applied.
In ZBrush I create UV’s and reduce the amount of polygons needed so the new 3D object will run more smoothly online. I then separate out each side of the model and create UV’S for each.
I export the entire model from here and apply photos of the model to each respective side. Once I have done this, I open up the model in ZBrush again and apply those textures
At this point, with a little bit of tweaking, I’m done!
The last step of this process is to import this model with its textures into an online tool called Sketchfab, which is how all the interactive models within this article are being displayed. I adjust how the model is displayed in Sketchfab, add annotations and sounds, save it, and copy the link to send it out. This model can then be embedded into websites or even included in an LMS system as a page or exam
