Its crazy to think that in around 20 years, humans may become a multi-planetary species. If you think about it, humans evolved from hydrogen atoms a few billion years ago, where eventually, stars and supernovas created heavier elements. These heavier elements reacted with hydrogen to form organic molecules and a few billions years later, humans began to walk the Earth. A few million years after that, humans may be able to bring themselves to another planet. 🤯
Humans have been able to achieve unimaginable things such as bring a man to the moon, create a robot that can clean your floors and create chocolate chip cookies 🍪, despite all of this, we are quite fragile. To survive, humans require a certain temperature, a certain amount of food and a certain amount of water. We also can’t be exposed to harsh environments where there is too much radiation or not enough oxygen. All of these conditions are not met by any discovered planet besides Earth. This is not good news if we want to colonize Mars. This is why we send robots to explore Mars before we do. Spirit, Opportunity and Curiosity have been exploring the Martian surface for around two decades now so humans can better prepare ourselves for colonization on this planet.
Roombas and Mars Rovers
A couple years ago, I came home from school, took off my shoes and just around the corner I saw this circular flat object that was moving toward me. As a 10-year-old I thought to myself “What is this wizardry?!”. I later learned that it was a Roomba.
At the time I was fascinated with how this circular object worked. How was it able to not fall off the stairs? How was it able to make a 90 degree turn when it bumped into something? It took me a good seven years before I understood the technology behind it.
The sensors and algorithms a Roomba uses are quite similar to that of a Mars rover.
Rover Hardware
To understand how software uses sensory data on Mars rovers, let’s introduce some of the hardware that is carried these rovers. Some of the hardware listed below is not related to the software that is going to be discussed but were brought onboard the Perseverance rover and are used for experimental purposes.
- RAD750 — radiation-hardened computer (200 MHz, 2 GB flash memory, 256 MB RAM, 256 kilobytes of electrically erasable programmable read-only memory).
- Six stereo hazard cameras (“Hazcams”). Four on the front and two on the rear end of the body. These cameras are used to detect hazards around the rover such as rocks, sand dunes and trenches. Front “Hazcams” are also used to guide engineers to move the robotic arm when collecting samples. The rover frequently stops to take stereo images of its surroundings for autonomous navigation.
- 2 colour stereo navigation cameras (“Navcams”). This helps engineers navigate where the rover is going. Before the rover “drives blind” or sets on a trajectory without performing any checks on its surroundings, Navcams” ensure the trajectory path is safe.
- “CacheCam” is a single camera that records the process of the rover taking samples. This way, scientists have record of the entire sampling process to account for any sources of error.
- MastCam-Z takes colour images, video and 3-D stereo images. These images are run through an algorithm to help interpret how far away certain objects or terrains are from the rover. In other words, it interprets pictures similar to how humans interpret pictures and this helps the rover navigate.
- SuperCam helps analyze rock samples. It fires a laser at rock targets that are beyond the robotic arm’s reach and vaporizes the rock to reveal its elemental composition.
- Planetary Instrument for X-ray Lithochemistry (“PIXL”) uses X-ray fluorescence to identify chemical elements in very small areas (i.e. grain of salt).
- Scanning Habitable Environments with Raman & Luminescence for Organics & Chemicals (“SHERLOC”) is a combination of tools that specify in the search of organic material. It is composed of spectrometers, a laser and a macro camera.
- WATSON camera captures larger, context images for the detailed information that SHERLOC collects. Since this camera is mounted on the arm, it can be used to observe other parts of the rover
- Mars Oxygen In-Situ Resource Utilization Experiment (“MOXIE”) is an instrument that turns carbon dioxide in Mars’ atmosphere into oxygen
- Mars Environmental Dynamics Analyzer (“MEDA”) will provide measurements for temperature, wind speed and direction, pressure, relative humidity, dust size and shape
- Radar Imager of Mars’s Subsurface Experiment (“RIMFAX”) analyzes geologic features under the ground. It does this using radar waves to probe the ground.
Rover Software
A key motivation for developing autonomous navigation in rovers is due to communication latency and bandwidth limitations. It takes approximately 20 minutes for a command to reach Mars from Earth. With computer-vision, rovers are able to carry out their tasks much faster and make their own decisions when faced with an emergency.
Stereo vision
Stereo vision is a technique used in computer vision that originated from how human eyes perceive the world. The formal definition of stereo vision is “the ability to derive information about how far away objects are based solely on the relative positions of the object in the two eyes”-source.
Our eyes perceive images of the world at different angles. You can do a simple experiment to try it for yourself: place a finger in front of you and close one eye. Then open the eye and close the other. The images you perceive of your finger should look a little different. This ability is especially crucial in recognizing the world because our brain pieces the two images together to perceive the world in 3-D. This is exactly the role of stereo vision in rovers and Roombas. 3-D depth perception in these machines function based on the fact that different lenses perceive the same image at different angles.
Stereo vision usually uses two cameras. When one camera takes a picture, the computer cannot recognize how far away objects are. In order for the computer to recognize this, a second camera needs to take multiple images. When the second camera takes an image that is similar to the image taken by the first camera, the computer is able to perform triangulation calculations and figure out the distance of certain objects.
Stereo vision helps the rover recognize how far away certain objects are, hence, it is crucial for the rover to be able to navigate its way around Mars.
Visual Simultaneous Localization and Mapping (“VSLAM”)
Seventy years ago, people relied on handy maps to navigate themselves to their destination. That was the case until about twenty years ago because the Global Positioning System (“GPS”) became widely accessible. No more wandering around complicated neighborhoods for hours.
As you may know, GPSs are super handy but unfortunately, they are not available on Mars so rovers will have to rely on another method to determine their position. This is where VSLAM comes in.
You might recognize VSLAM through your Roomba. The vacuum has a camera embedded into it which takes pictures of its environment. The picture is then fed through an algorithm which is able to detect distinctive patterns in the pixels and recognize shapes (i.e. corners; outlines). The algorithm will then use odometry, an algorithm, to determine its position. Mars rovers detect its position very similarly to your Roomba.
You might be wondering how a computer understands an image. Pictures are very intuitive to humans but how does a computer that runs on 1s and 0s understand a picture? Well, an image is actually composed of pixels which have colours that correspond to specific numbers. When certain numbers are recognized in certain patterns, the algorithm is able to recognize shapes and objects.
Perseverance Rover and Ingenuity
I am SO excited for the discoveries that are about to occur on the Perseverance mission. I’m most excited for the helicopter that is attached to the bottom of the rover and released when the rover lands on Mars, Ingenuity — a Mars helicopter.
Ingenuity is an autonomous helicopter that will fly over Mars. This experiment will help scientists better understand how to design future helicopters for the Martian atmosphere.
Due to the 20-minute time lag between Earth and Mars, Ingenuity will be autonomous. It is able to fly autonomously because it contains accelerometers, gyroscopes, altimeters and inclinometers. The data from these sensors are entered into an algorithm which will make desired corrections to the helicopter.
TL:DR
- The Perseverance rover is filled with many science experiments and instruments: RAD750, “hazcams”, “navcams”, “CacheCam”, MastCam-Z, SuperCam, PIXL, SHERLOC, WATSON, MOXIE, MEDA and RIMFAX.
- The Perseverance rover is an autonomous rover that uses stereo vision and VSLAM to navigate on Mars.
- Ingenuity is an autonomous helicopter that will use multiple sensors to navigate itself on Mars.
