Taylor Geospatial Engine (TGE) exists to accelerate the development and commercialization of geospatial innovation emerging from academic research. Through its Innovation Bridge Program, TGE connects academia and industry to turn research-stage concepts into technology that’s quickly usable by many organizations. In 2024, TGE supported the development of Fields of The World (FTW)—the largest benchmark dataset for extracting field boundaries from satellite imagery. Now, FTW is in Phase 2 which expands the work and focuses on producing usable field boundary data in collaboration with partners in sustainable agriculture, supply chain, and food security. As part of our FTW journey, we’re profiling key contributors who are making this vision a reality.
When Caleb Robinson joined Microsoft’s AI for Good Lab five years ago, after completing his PhD at Georgia Tech, he brought with him a passion for computational sustainability and a focus on using satellite imagery to tackle real-world problems. Today, as a Principal Research Scientist at Microsoft, he co-leads the Geospatial ML research group and is the lead researcher on the Global Renewables Watch, rapid damage assessment, and global building density estimation teams.
From Academic Research to Global Impact
Robinson’s journey into field boundary detection wasn’t a direct path. His background in semantic segmentation and land cover mapping from satellite imagery provided the technical foundation, but it was the broader mission that drew him in.
“The Microsoft AI for Good Lab is a completely philanthropic effort,” Robinson explains. “We partner with external organizations who have a problem and data, and if we can bring AI to help tackle that problem with them and create an impact, we work on those types of problems. Field boundaries fit that description really well because you can do so many things on top of field boundaries for different agricultural uses once you have them.”
Now, through partnerships with organizations like the UN-FAO (Food and Agriculture Organization of the United Nations), Robinson and his team are working to create field boundary maps that could help countries navigate complex regulations like the EU’s deforestation rules (EUDR).
The Technical Challenge: More Than Meets the Eye
For those outside the field, the concept might sound straightforward: use AI to identify farm boundaries from satellite images. But Robinson is quick to point out the complexity lurking beneath that simplicity.
“There’s a whole laundry list of things that go into a good dataset. From making sure projections are right—the Earth is a sphere and we’re looking at tiny flat sections of it—to ensuring there are no clouds in the imagery, to making sure field boundary data in polygon format aligns correctly with the imagery. We spent the time upfront to do that, and now it’s the fun part: playing with it.”
That upfront investment was substantial. Robinson estimates a large block of the effort for the first Fields of The World paper went into creating the dataset itself, with a smaller portion dedicated to modeling. But that foundation has proven its worth. Now that the team has shifted focus entirely to model development, they’re seeing significant gains.
“One thing that has surprised me is the delta from one model version to the next, and the improvement you can get over that first version,” Robinson notes. “I thought the out-of-the-box techniques would do better than they have. The details of what to put in the dataset matter, and the details on the modeling matter as well.”
FTW includes approximately 1.6 million parcel boundaries and over 70,000 samples. Each sample includes instance and semantic segmentation masks paired with multi-date, multi-spectral Sentinel-2 satellite images, enabling detailed temporal and spectral analysis.
Why Not Just Walk the Fields?
When explaining the project to non-technical audiences, Robinson emphasizes the scalability challenge. Ground truth data—having someone physically walk around a field with a GPS-enabled phone—remains the gold standard for accuracy. But it’s simply not practical at scale.
“You need field boundary delineation data for a lot of food security applications, and that’s often costly to get manually,” he explains. “It takes a long time, or you need a partnership with a company that collects data from tractors with GPS, which is not a scalable solution for monitoring agricultural outputs at country scales. What we’re doing is taking satellite imagery and using AI models to produce estimates of that same data for downstream applications.”
The field boundaries themselves are just the beginning. They serve as containers for countless other agricultural insights: crop type identification, yield estimation, irrigation assessments, and disaster response applications like identifying flooded fields.
The Power of Collaboration
One aspect of the project that clearly energizes Robinson is the team itself. When asked what excites him most about being involved, he pivots quickly from the technical challenges to the human element.
“The team is incredible. I don’t think Taylor Geospatial Engine (TGE) could have put together a better group of people. Everyone is really easy to work with, super sharp, and passionate about what they do. It’s a pleasure working with the whole team.”
This isn’t just professional courtesy—it reflects a fundamental principle Robinson has learned through his work: never try to solve applied real-world problems without domain expertise. “You will mess everything up, and it won’t be useful because no one will even know about it,” he says bluntly. “That partnership with domain experts makes a project successful from square one. You have to spend time with them to build up the language—what do all the AI and machine learning terms mean, and what do domain expert terms mean?”
This collaboration extends beyond just avoiding technical pitfalls. Robinson points to recent discoveries that might surprise those new to remote sensing: “If you’re trying to estimate the area of fields, having your model predict which pixel is a field and summing the total area is guaranteed to be the wrong answer. You have to take a statistical approach. That’s not obvious to someone without a remote sensing background, and I only learned this in the past year.”
Scaling Toward Something Real
The project is reaching an inflection point that Robinson finds particularly exciting. Recent work by one of the FTW contributors demonstrated the system’s potential by running the current best model over the entire United States in minutes rather than hours or days.
“I really want to get to the point where we can press a few buttons and see what the output looks like over very large scales,” Robinson says. “People understand maps, and once you have a map of the model output, it suddenly becomes real, and more people get excited about it.”
Open by Design
The project’s open-source nature isn’t just a nice feature—it’s central to its mission. Unlike many corporate research efforts that develop privately before releasing results, FTW and Microsoft’s AI for Good operates in the open.
“We’re working in public,” Robinson emphasizes. “If someone’s passionate about field boundaries or applied projects that could have an impact in the world, they can hop right in and ask questions on FTW GitHub Repo or commit code. We definitely encourage that.”
“You have data and a really capable set of collaborators that’s a mixture of technical experts and domain experts, plus the infrastructure to get those people in the same place working together. I think you can do remarkable things with that combination.”
For potential contributors, Robinson sees the project occupying a sweet spot: “It’s not so large that you couldn’t find something interesting to contribute within a semester-long class project if you were a motivated undergrad. But it’s also not so small that it’s fragmented. New contributors can onboard pretty quickly and do something useful.” Find out how to get involved and contribute to FTW.
Field boundary outputs from running a model trained with the FTW benchmark training data.
Looking Forward
As Robinson balances this work alongside other projects at Microsoft (and debugs the occasional sleepless night with his new baby), he sees FTW as more than just a research exercise. The techniques and community being built have staying power.
“As I work on projects like these, my toolbox of techniques always expands,” he reflects. “There are things I’ve learned while working on Fields of The World that I’m already applying in other projects. And there’s the community that TGE is building—being part of that is valuable long term.”
The twice-yearly in-person gatherings that TGE orchestrates play a crucial role. “That time together is critical,” Robinson notes. “A lot of the momentum for the first paper came out of those in-person meetings.”
With the foundation laid and models advancing rapidly, Robinson and the FTW team are moving toward something tangible: global field boundary maps that could transform how we monitor and manage agricultural systems worldwide. For a project rooted in computational sustainability, the path forward is both technically demanding and profoundly practical—exactly the kind of challenge that drives Robinson’s work.
_____________________________________
Principal Research Scientist, Microsoft AI for Good Research Lab