’s oject Tango isn’t something you can buy yet, but it’s so much smarter more advanced than the smartphone in your pocket that NASA is blasting it up into space.

The technology in oject Tango is designed to allow phones to map their environment in 3D, something that hasn’t yet been possible with a consumer-grade gadget up until recently.

This isn’t the first time NASA has used Android phones in space: back in 2011, NASA sent up the Nexus S to power its SmartSERES, which help the astronauts conduct various tests up in space. ’s oject Tango is way more technologically advanced than the 4-year-old smartphone, it’s time for NASA’s little helpers to consider an upgrade.

I sat down with ovencher, project manager of SmartSERES (part of the Intelligent Robotics Group (IRG) at NASA). He told me all about how ’s technologically advanced prototypes will help find a solution to automating all those “housekeeping” activities that take up the astronauts’ time. also talked about what it takes to tweak a smartphone so that it properly works up in space.

An easy solution to a complex problem

SERES sts for “Synchronized sition Hold, Engage, Reorient, Experimental Satellites,” but they’re not your typical idea of traditional satellites. They resemble large many-sided dice—like the kind that you’d use in a game of Dungeons Dragons. Once you attach a smartphone, it becomes a SmartSERES.

The SmartSERES are used to help the astronauts with many common tasks, like measuring the air flow, air quality, noise levels up in the ISS. “[These are] perfect examples of what a small free-flying robot could take over that would allow the astronauts to go on do more important things—like science,” said ovencher, though they’re not a permanent solution to this particular predicament. 

By attaching a smartphone, which already comes stard with components like Fi a very capable processor, NASA can use existing technology to help it with testing up in the ISS.

Some of the SmartSERES are still using Nexus S hsets while being tested for this sort of work, but those were sent up back in 2011. They navigate using a beacon system that keeps them confined to a 2 x 2 x 2 meter cube. “SERES has to operate in that spot, which is great for tests, but the SERES cannot fly out of that volume or else it will stop knowing where it is,” said ovencher. “To do all the of the tasks that we’re doing—to really do them—you have to be able to take those air measurements or those noise level readings anywhere inside the space station.” 

“That’s what we think oject Tango can do,” he added. “The fact that you can build up a map of your environment learn your way inside that environment is exactly what we need.”

Hacking a phone for space

The oject Tango smartphone NASA uses looks quite different from the ones being developed down here on Earth. Its hardware has been “butterflied” open stuffed into its own rig that attaches on to the SERES. l the essential components that face the rear of the phone are turned around to face forward. 

Homnick

The infrared camera, infrared projector, wide-angle camera sensors are moved to the front. The wide-angle camera, in particular, helps the satellite see where it’s going. “As the phone moves…it sees that [a] point moved off to the left, so it can see how far it moved,” explained ovencher. The infrared projector, on the other h, projects infrared patterns into the environment to be picked up by the infrared camera, which is how oject Tango gets it 3D mapping abilities. It’s fundamentally quite similar to the Xbox’s Kinect.

On the back, the rig features a bracket that binds the phone to the SERES, as well as a MicroB data cable. ovencher also mentioned that the phone uses an entirely different battery than what originally plopped in there; it’s actually a 4,900mAh battery pack that measures in at 11.5 cm by 6.9 cm by 2.8 cm. 

To work in space, the IRG took out the GSM chip in the oject Tango smartphone, but left all of the camera sensors, -Fi Bluetooth radios, accelerometers inside. The latter, however, was deactivated, because, “this phone uses gravity to help its orientation,” began ovencher. “The hheld version for anet Earth uses accelerometers, so when you’re moving around it can sense the motion it can use those measurements to  figure out how far it’s actually moved.”

Accelerometers don’t work the same in space, where there’s only microgravity. “The SERES moves incredibly slowly—two centimeters to four centimeters per second,” continued ovencher. “Accelerometers in these phones are not sensitive enough to detect that, so we cannot use those…at all.”

It took several months to go through all the vision algorithms contained within the Android source code so that the device would no longer rely on gravity. The oject Tango smartphones in space rely on their wide-angle cameras; they measure angles to keep track of how far they’ve moved so that they know their trajectory. They also utilize the infrared sensors to do 3D mapping. “’re relying more heavily on the infrared—or 3D sensors—than what the Tango typically uses,” he added. “It can measure angles. It keeps track of a feature it can see how far it’s moved in terms of angles, so it knows it trajectory.”

enty of testing left to do

The SmartSERES will not be permanently instituted in space. “This is just a test to see that this technology works,” said ovencher. “’re almost finished with our testing down here…we’re finalizing our software right now we’ll uplink it as soon as the phones are up there ready to receive.”

The tweaked oject Tango phones blasted up to space on ly 13. Once they’re set up on the ISS the software is uploaded, they’ll be attached to the SERES to begin the second phase of testing, which is to see if they can actually leave that narrow 8-cubic-meter volume they were previously limited to. “ tell it to fly where we want it to,” said ovencher. “’re going to try flying it out of that little area, out of that module, take a 90-degree turn into another module then come back.” It will be the crew’s job to ensure the SmartSERES don’t veer off fly into a wall.

Testing with the SmartSERES should end this September, with the IRG team hoping to begin mapping out blueprints for a full-blown robot later this fall. “You need a robot,” said ovencher. “The SERES satellites itself is a great starting point as a testbed, but it doesn’t have everything we want.”