MATTEO LUCCIO / CONTRIBUTOR / PALE BLUE DOT LLC / WWW.PALEBLUEDOT.COM

FIGURE 1. Hawaii in true colour, courtesy of Astro Digital

The shift in the Earth observation(EO) market from selling pixels to selling finished intelligence products has sharply accelerated in the past couple of years, due to two factors. First, satellite imagery has become commoditized, partly due to the launch of dozens of small satellites. Second, advances in artificial intelligence, cloud computing, and cloud storage have greatly expanded the number of people who can access sophisticated analyses of this imagery or run the analyses themselves.

For this second installment in this new series on geospatial analytics, I discussed these developments with:

Descartes Labs

ORIGINS AND COLLABORATIONS

FRITZ SCHLERETH DESCARTES LABS

Descartes Labs was founded by a group of former Los Alamos National Lab scientists who had been working for a long time with very large datasets, including geospatial datasets and satellite imagery. While these datasets were becoming more and more prevalent, Schlereth explains, these scientists were often hamstrung by the technology to which they had access. “They were not able to quickly scale up computer resources to deal with these large datasets and develop machine learning algorithms that would automatically cleanse and prepare the datasets for scientific analysis,” he recalls. “They were also not able to make these data available to others for large-scale analysis.”

So, they left the lab, started a company, and set about building that capability. Schlereth is responsible for bringing the company’s technology to the market, including its platform and datasets, as well as its relationships with customers and the services it provides to them.

From its inception, Descartes has partnered with teams at Los Alamos National Lab that use its platform. It has also been working with researchers at universities and plans to expand its network in that area.

DATA AND ALGORITHMS

Descartes’ imagery archive consists mostly of a collection of public datasets. “We work a lot with Landsat, MODIS, Sentinel-1, Sentinel-2, and Sentinel-3,” says Schlereth. “We are getting Sentinel-5 online as well. We have Sentinel 4 on our roadmap along with dozens of other pubic and commercial datasets.” In addition to geospatial datasets, the company also collects weather data.

FIGURE 2. Wind turbines found visually in North America using computer vision with Descartes Labs GeoVisual Search

Besides housing that imagery, the company prepares it for scientific-grade analysis by converting it to surface reflectance and by co-registering datasets and rationalizing them with respect to each other. “We work hard to make sure that, within a single analysis, we are enabling the scientists to pull from all the different data that are available,” Schlereth explains.

The company has also introduced its own derived datasets, as well as pre-packaged algorithms that give scientists shortcuts, such as water maps. “We make it easier for users to focus on a deep learning application, such as object classification,” Schlereth says. “We already have some of our own, pre-trained classifiers that they can use for that.”

PLATFORM AND SOLUTIONS

Descartes has created a computing environment that is easy to use and accessible to scientists, most of whom are not experts in cloud engineering, though they understand the cloud’s importance to their work. It has built a technology platform and put public APIs in front of its data archive and in front of many of its tools. Some of its customers, in business and academia, use its platform similarly to how they use Google Earth Engine (GEE), for example, to do their own analysis and solve their problems.

Meanwhile, its internal science team provides services for customers that lack their own capability to use its platform. “Essentially,” Schlereth explains, ìour science team uses our platform in the same way that an external customer would to solve a problem in, say, agriculture or energy. We have both a platform business and a solution business.”

In the geospatial world, Schlereth points out, the term ‘platform’ is used to describe many types of offerings. “GEE and DigitalGlobeís GBDX are examples of platforms that are similar on the surface to what we’ve built. However, it is important to emphasize that we’ve focused on machine learning as a primary purpose of our platform, and that informs the architecture and design decisions that we’ve made.” So, while acknowledging similarities between GBDX and Descartes’ technology, he sees DigitalGlobe first and foremost as a partner. “Certainly, their high-resolution dataset is an incredible resource. However, I also see opportunities for joint customers where we can provide advanced analytics in domains such as agriculture, where we may combine different data sources from different public and commercial sources.”

To that end, Descartes creates a synthetic layer from the electro-optical data that it pulls, and processes it for surface reflectance. It also creates its own cloud and cloud-shadow layer and co-registers its datasets so that features on the ground are mapped consistently across all of them. “The end goal of all of that is to make it very easy to the researcher to incorporate different datasets for the same problem,” Schlereth says.

FIGURE 3. Synthetic Aperture Radar (SAR) over North America from ESA Sentinel-1, courtesy Descartes Labs

PROCESSING

Many of Descartes’ projects rely on either traditional remote sensing-based approaches, such as developing a yield model for crops by measuring the spectral signature of each pixel and then using linear regression; or on deep learning, for example, maping the power grid and quantify wind or solar power production. “Some buyers essentially buy access to our platform and do their own work, while others buy a ready-to-use solution. Perhaps the most powerful option is for the customer to buy both a solution and license platform access. For example, a customer can buy a map of power infrastructure in SE Asia and use it in a model they build for themselves using our platform.”

Images collected on different days by different sensors are often hard to compare, due to slight differences in atmospheric conditions and in the behavior of the sensors. To ensure its data users that they can compare a pixel collected on one day by one instrument to a pixel collected on a different day by another instrument, Descartes cleans and calibrates optical data and provides a top-of-the-atmosphere product for all of its datasets, as well as a surface reflectance product. It also strips out clouds and cloud shadows, because many instruments do not already mask them. “To make it easy to work with the data, we wanted to provide our own, consistent cloud mask across our imagery collections,” says Schlereth.

Descartes also works quite a bit with synthetic aperture radar (SAR) data and makes it available to users of its platform. “We’ve made it easy to combine SAR with optical data so that the user can stack different raster layers for use in the same analytic or machine learning process,” Schlereth says.

“Essentially, our science team uses our platform in the same way that an external customer would to solve a problem in agriculture or energy. We have both a platform business and a solution business. In the geospatial world, the term ‘platform’ is used to describe many types of offerings.”

–FRITZ SCHLERETH

USE CASES

Many researchers use multiple satellite imagery and geospatial analysis platforms. Descartes’ users tend to be scientists, some working in commercial fields, such as agriculture and energy, and others at research institutions. “Our use cases tend to be heavy on the machine learning side,” says Schlereth. His company provides both solutions for specific industries and business needs—such as agriculture models, land cover maps, and production facilities monitoring—as well as tools that allow research teams to develop those analytics themselves. “Between those two extremes, we’ve also built a series of more generalized components, such as water maps and forestry layers. We also do a lot of object detection.”

Descartes has prioritized agriculture, energy, and industrial production for steel and other kinds of industrial products, and is building several layers that relate directly to those applications. “Our goal is to produce a broad and deep collection of layers that are useful for geospatial analysis, such as maps of where water is, or more abstract ways of processing,” says Schlereth, “such as, different NDVI products that are already processed and ready to be used, primarily for agricultural analysis.” The company’s users can work via an API with both processed imagery and derivative datasets, such as measures of air quality or crop maps. “They can pull these derivative datasets into their own systems or develop more sophisticated models using our derivative datasets as an input.”
Besides agriculture, energy, and industrial production, Descartes also has projects in other areas. “A lot of this boils down to understanding regional or global supply and demand around a commodity,” Schlereth explains. “In agriculture, the best example is understanding grain supply and what fields are likely to produce, or how animal populations and feed demands are changing over time, or how a transportation network moves that commodity.”

With respect to energy, he points out, the company focuses largely on production, demand, and infrastructure. “Often the question is ‘What is power demand going to be because of some weather condition?’ We can also look at the change that alternative energy generating sources have introduced into the system, such as the growth of solar and wind power.”

Astro Digital

FIGURE 4. River delta in Madagascar in false colour, courtesy of Astro Digital

ORIGINS AND COLLABORATIONS

BRONWYN AGRIOS
ASTRO DIGITAL

The backgrounds of Astro Digital’s founders include mechanical engineering, mapping, business, and physics: Chris Biddy, the company’s CEO, has been building cubesats his whole career; Agrios worked for years at Esri and also at Planet; and Mikhail Kokorich, the company’s business manager, is an entrepreneur with a physics background. “We know that designing a satellite system for the sake of designing a satellite system is not going to get us into a good place with customers,” says Agrios. “It is about identifying the changes in the market, building software, and designing satellites to answer those questions that are in the market.”

In developing its analytic capabilities, Astro Digital collaborates closely with the National Snow and Ice Data Center and the Laboratory for Atmospheric and Space Physics, both at the University of Colorado at Boulder. These centers’ skillsets range from building satellite calibration systems and operating satellites to atmospheric science and remote sensing.

FIGURE 5. Astro Digital imagery allows for the creation of vegetation indices (NVDI -Normalized Difference Vegetation Index)

THE BET

Treating smallsats like large ones misses the point and erodes their value, Agrios argues. Instead, they should be purpose-built and integrated into a system that can ingest the pixels they collect, analyze them using computer vision and machine learning, identify changes and trends, and answer scientific and business questions. This is facilitated by the coverage and refresh rate that swarms of smallsats enable for a fraction of the cost of legacy satellites. “We are producing reliable coverage and high-quality imagery,” she says.

Astro Digital began by focusing on the information needs of commercial agriculture, including commodities trading and supply chain management. “Four years ago,” Agrios recalls, “we made the bet that companies would be more open to accepting information based on computer algorithms and that cloud computing would be increasingly low cost.” So, the company decided to build a system with a 22-m resolution to capture “deep stacks of pixels that can be ingested into a machine learning system.” The focus on computer vision guides the company’s choices, from its sensors’ resolution and spectrum to the filters used on its spacecraft to how it downlinks the data. “It is not the resolution or the frequency that set us apart. It is the fact that this is a system-based approach to build for a next-generation remote-sensing system that relies on machine intelligence.”

THE BUSINESS MODEL

“Nobody is buying imagery anymore,” Agrios points out. “Nobody will be buying imagery anymore. They will be buying a subscription to an API and have imagery show up right in their application. This is a disruption in the business model that is being fueled by small satellites.” Sophisticated customers, who have a significant investment in a remote sensing pipeline, access images from the API, then process it and analyze it themselves. “Market growth happens when customers access information about their areas of interest where processing and information is handled within the Astro Digital environment,” she explains.

Astro Digital’s customers subscribe to specialized products—for agriculture, disaster management, etc.—that the company keeps refining over time with feedback from them, but does not customize for each customer. “Our goal is to provide a suite of products and not be a services company,” says Agrios.

FIGURE 6.

FIGURE 6-7. Making maps and results screens from Astro Digital

THE CONSTELLATION

Astro Digital plans for its Landmapper constellation to be fully operational by mid-2019. It includes both the Landmapper-BC, for broad coverage—a moderate-resolution imaging system that will provide daily global coverage—and the Landmapper-HD, for high definition. The company launched the first Landmapper-BC satellite in mid-January and will begin launching the Landmapper-HD in the third quarter of 2018. It will take the constellation 18 months to be complete and operational, from the launch of the first satellite. This includes both launching all satellites and commissioning them (the time it takes to bring them to operational status).

Creating a high-quality, well-calibrated set of pixels to ingest into the analysis enables Astro Digital to identify features, calculate trends, and detect meaningful change, Agrios explains. Depending on the problem, the analysis can range from regression analysis to examine the relationship between variables to training a neural net to learn from a series of observations.

“They will be buying a subscription to an API and have imagery show up right in their application. This is a disruption in the business model that is being fueled by small satellites.”

– BRONWYN AGRIOS

A commodities trader, for example, may need to assess the yield of a crop, to know how much of it is going to be harvested. “What we can do is create something like a crop classification signal, which can tell you, on a global scale, whether a pixel is corn,” says Agrios. “We can tell that to you very early in the planting cycle because we have not only a recurrent neural network that is trained to carry out that very specific task, but also a large archive that goes into defining that. We are looking for specific signals that get used to help inform somebody else’s answer.”

The company has researched how to use neural nets and multi-source moderate-resolution imaging to do early classification of crop types and is constantly trying to improve the quality of its input images. This includes using machine learning to detect cloudy pixels. Building an autonomous analytical system to do this, Agrios points out, requires the image quality to be as close to perfect as possible.

FIGURE 8. Agriculture Dashboard of Astro Digital

CONCLUSIONS

While Astro Digital has its own satellites and Descartes does not, both Descartes and Astro Digital go beyond collecting EO data to prepping it for ingestion into and analysis by a system. Both companies provide their data and analyses in a wide range of formats, depending on each customer’s specific needs and levels of technical sophistication. They are two of a growing number of companies that are taking full advantage of advances in the collection and machine-processing of EO data to address complex business and scientific questions.

FIGURE 9. Agriculture in Italy shown through the near infrared spectrum, courtesy Astro Digital