From their offices on the second floor of the Museum Parke Building in downtown Los Alamos, a team of scientists led by Descartes Labs co-founders Mark Johnson and Steven P. Brumby are on a mission to better understand planet Earth by analyzing massive amounts of imagery.

Brumby is the company’s Chief Technology Officer and a theoretical physicist who led a machine-learning team at Los Alamos National Laboratory incubating the technology for seven years with $15 million of research money.

Descartes Labs raised $3.3 million of its own funding and spun off as a private company last August. Descartes Labs is focused on taking satellite imagery, turning it into something coherent and analyzing it for useful data.

“We’re teaching computers to see,” Brumby said. “We’re beginning this journey by understanding satellite imagery to enable real-time global awareness — whether it’s food production, energy infrastructure, the growth of cities, or our impact on the environment. Not only will we identify what’s in a single image, but we’ll also be looking at how our Earth changes over time. This deep understanding of our planet will allow us to be better stewards of this world and give people worldwide the knowledge to make intelligent decisions about the future.”

Johnson is from the San Francisco Bay area. He is the CEO of Descartes Labs. He explains that the team’s underlying technology is based on breakthroughs developed while his scientists were working at LANL.

“Steven (Brumby) worked on genetic algorithms for 10 years at Los Alamos and then began working on neuroscience models of vision and deep learning algorithms seven years ago,” Johnson said. “National labs in the US often work on blue sky research – think the Internet or the Manhattan Project – and apply this research to real world problems for the government, giving the technology a solid practical foundation.”

Descartes Labs’ specialty is in taking deep learning and other advanced machine learning algorithms, which are typically applied to categorizing still images found on the Internet, and developing them so they can be applied to vast amounts of data. There are three major components to this technology:

• Data pipeline – Descartes Labs is building a pipeline that can handle the massive amounts of visual data coming from all sensors on the planet: satellites, UAVs, cameras, cell phones, etc.
• Image understanding – Descartes Labs uses advances in deep learning and artificial intelligence to teach computers how to identify important information buried in visual data, from everyday photos through to exotic scientific datasets.
• Pattern recognition – Once we’ve identified things in single scenes, we look for patterns over time: what’s changed, what’s changing, what looks like something from a year past, etc.

Johnson has worked on startups like Kosmix (acquired by Walmart), Powerset (a foundation of Microsoft’s Bing) and Zite (which was bought by CNN). After the Zite deal, Johnson moved to New Mexico to start Descartes Labs.

At LANL, Brumby co-invented GENIE — image-analysis software that was capable of identifying elements such as water and beaches in satellite photos. Later, he developed a neuroscience-based deep-learning algorithm with applications in such areas as social media and surveillance video.

“We’ve now made a breakthrough in deep learning, which uses our knowledge of the human brain to make computers as intelligent as human beings,” Johnson said. “We realized that this project had a commercial opportunity,” Brumby said. “So that’s why we networked ourselves to Mark.”

Descartes is in the process of developing what amounts to a 400-frame video of the Earth since 1980, with each frame containing 4.5 billion pixels of data.

Johnson holds an undergraduate degree in philosophy and computer science from Stanford University. Brumby holds a Ph.D., in Theoretical Particle Physics and a B.Sc. with honors in Physics from the University of Melbourne in Australia.

Brumby will present a talk about the Descartes Labs story at the Entrepreneur and Start-up Mixer at 5:30 p.m. Wednesday, July 8 at Diamond Mix, 1900 Diamond Dr., in Los Alamos.