→ What Makes a Flood Model Useful? Fathom's Global Flood Risk Data

PLAY

Show notes

Nissim Lebovits, a technical fellow at Radiant Earth, guest hosts this episode with Oliver Wing, Chief Science and Product Officer at Fathom, a flood risk modeling company spun out of the University of Bristol in 2013 and acquired by Swiss Re in 2023. Fathom publishes its methods in peer-reviewed journals and relies on partners to distribute its data, which leaves a team of two dozen scientists free to focus on building the data itself.

Ollie walks through the product stack, from 30-meter flood depth maps to risk scores to a full catastrophe model, and explains why elevation data matters most: run a flood model over a surface model like the Copernicus DEM and every city becomes a hill that water can’t reach. He describes a customer base split roughly into thirds across insurance, banking and capital markets, and corporates, governments, and NGOs, and how Fathom serves that range by deliberately under- and over-engineering its products and letting distribution partners fill the gaps. The two discuss where machine learning helps and where physics still wins, why trust in the team behind a model is what customers are really buying, and how a reinsurer’s view of risk differs from a city planner’s.

The second half turns to uncertainty and openness. Ollie argues that multi-model ensembles share the same blind spots and should not be read as probability distributions, that the public release of property-level climate scores showed the cost of giving people models without context, and that most of the remaining problems sit between the model and the person using it rather than inside the model. He closes with Fathom’s distribution strategy: cloud-optimized GeoTIFFs on S3, an API, research access through Zenodo and Google Earth Engine, and a free UK flood model on the Oasis Loss Modelling Framework aimed at breaking vendor lock-in in catastrophe modeling.

Links and Resources

• Fathom – global flood risk data and catastrophe models
• Oasis Loss Modelling Framework – open, vendor-agnostic framework for catastrophe models, where Fathom’s UK flood model is now free
• SWOT – the NASA and CNES Surface Water and Ocean Topography mission Ollie expects to improve river channel representation
• Copernicus DEM – a global surface model that needs correction before flood simulation
• Nate Silver, The Signal and the Noise (2012) – the book Nissim cites on how commercial weather forecasts skew government probabilities
• Nadia Eghbal, Working in Public (2020) – the book Nissim cites on the economics of open source maintenance

Key takeaways

1. Elevation data comes first – Freely available elevation data captures rooftops and tree canopy. Without a corrected terrain model, cities look like hills and flood risk looks flatteringly low.
2. Customers buy the team – A flood model bakes in millions of judgment calls on gray areas. Peer review, documentation, and honesty about limitations are how a vendor earns the trust to make those calls.
3. An ensemble is not a probability distribution – Models built on the same data and assumptions fail together. Exploring uncertainty inside one transparent model can be worth more than licensing five that agree.
4. Tailor uncertainty to the user – A reinsurer’s science team wants full disclosure. A homeowner reading a climate score needs the minimum information required to make a sound decision.
5. Interoperability is the bottleneck – Customers struggle most with fitting data into their existing systems, not with model accuracy. Open standards like Oasis and cloud-optimized formats on object storage do more for adoption than another increase in resolution.

Transcript

(this is an auto-generated transcript and may contain errors)

Nissim Lebovits: Okay, I think we’re live. I think we’re streaming on YouTube. I can’t see YouTube, but I I I trust that there will be a chat or something so we can we can get started. Okay, Ollie, thanks for being here. And thanks for everybody who’s listening to this. You’re listening to Great Data Products, which is a podcast about the craft and ergonomics of data. My name is Nissim Lebovits. I’m a technical fellow at Radiant Earth, which sponsors this podcast. And this month I’m filling in for Jed. I’m guest hosting. And today I’m gonna be talking to Ollie Wing.

Ollie is the chief science officer at Fathom, which is a global flood modeling company based out of Bristol in the UK. They do really cool work. Ollie, why don’t you tell me a little bit about that work and what you do?

Oliver Wing: Sure. Thanks for having me. Really delighted to be here. So I’m the chief scientist at Fathom, and we are a flood risk modeling company based in Bristol in the UK. But as of, well, three years ago now, we were acquired by global reinsurance giant Swiss Re, which has changed some things but not changed many others. And we’ll provide interesting fodder for our discussion later on about all the different use cases of the sorts of tools that we built. But as a kind of potted history, we spawned out of University of Bristol. The university is probably 200 meters from where I’m sitting right now. And that was where raster based inundation modeling kind of was invented, kind of began by my PhD supervisor, Paul Bates. Up to that point, it was very much like, you know, survey a transect along a floodplain and then do a 1D model and interpolate between it. But with the advent of remote sensing data and the sort of 80s and 90s that then built up to, well, how do we make use of all of this cool data to actually do two dimensional flood modeling? So that’s what was created there. And then the journey of this being wielded by maybe those that are more aligned with the tradition of being a geographer rather than engineer really sought to figure out how to scale up these approaches to providing answers at scale rather than necessarily the most accurate answer locally, if that makes sense. So to try and really solve the information gap problem that geographers love to solve. And so some PhD students at the time were doing that. And then they thought, hey, let’s commercialize this. Let’s turn it into a business. Why are we spending a third of our time teaching undergraduates, a third of our time begging the government for some money, and a third of our time actually doing the work when we could spend a hell of a lot more time doing the research and get interested commercial end users to fund it for us?

So that’s kind of been the model. We’ve been very lucky enough from its inception in 2013 not to need to raise a great deal of money in organically, so to speak. So we’ve always kind of wielded the levers of power ourselves as scientists. And that’s very much then trained the direction that the business has taken. And like I say, we were then acquired in 2023 and still very much do the same sort of work very much operate like a research group at a university. And if you kind of took fathom science team that’s 20 or 30 strong, depending on who you class as a scientist versus an engineer versus developer or so on, you know, the research output probably puts us in the top five of universities worldwide in terms of flood modeling research. So our niche has always been to be the folks who kind of defining the state of the art as a private company rather than consuming the state of the art from universities, which in many ways you see an interesting and maybe not quite so positive parallel with the advent of AI, how the traditional engine rooms of scientific inquiry and research and so on was the university sector and I think should be for the most part. And how that is now much more in private hands. But anyway, that’s the history of Fathom.

And our job really is to arm businesses and communities and governments or whoever has an interest in understanding flood risk with the best possible data. And I use best as like our synonym for great with a capital G that you folks use at Radiant. And we can unpack all the meanings of that over the next hour or so.

Nissim Lebovits: That sounds that sounds like a good idea. That was not what I had planned, but yeah. So I it occurs to me, I’ve never actually asked you what it means to be chief science officer at Fathom. We’ve you know, in conversations that you and I have had before, we’ve talked a lot about what Fathom publishes, but I don’t actually understand what that means. And I’m curious about how you think of it as, you know, having just talked about the relationship between Fathom’s academic background and and the private sector, what does it mean to translate science? For industry.

Oliver Wing: Well, interestingly, my full title is Chief Science and Product Officer, which has not always been, but I have inherited the kind of product, the product domain within the company as well, which I think is quite telling in terms of where our focus is in terms of what it is that we do and what our proposition is. Because really, we are a data product company. We really do invest in kind of building the raw IP and creating relatively standard-ish geospatial outputs as a result. And we spend much less time and have much less expertise in packaging that up in a really nice way, making it incredibly user-friendly and a glossy user interface, and so on. The engine room at Fathom is the scientific inquiry. And we generally rely on partners to distribute it in a way that end users are likely to consume it. So whether that’s climate risk platforms or whether that’s just geospatial platforms like Esri or so on, we’re really keen to not have to invest in all of that ourselves and focus purely on what our value proposition is, which is building the data in the first place. So as chief scientist, I kind of get with the benefit of the many smarter people in my team, decide on where it is we should focus our efforts. How lucky am I to have two dozen people who are like professorial grade at an institution, unencumbered by all the administrative and teaching duties of universities. And I just get to sort of direct the strategy on what it is that we build and how we build it and who we build it for. So in many ways, that kind of the science and product divide doesn’t really exist so much anymore. We’re trying to figure out how we rapidly operationalize cutting edge research. And so it all starts really with the customer and what they want and what they’re willing to pay and how they consume it. And we work our way back from there and figure out what’s the delta? What have we not already got that they want? And I do feel that we’re at quite an interesting inflection point, really, in terms of getting to the kind of philosophical core of when do you stop, what’s good enough, what actually are the biggest priorities when it comes to further refining and developing models. You kind of see an interesting parallel in the climate world, where if you’re given 10 times more compute power as a climate scientist, what would you do with it? And you have people who are like, well, I’d increase the grid resolution of the models so I can run these models more accurately.

You know, of others that say, no, no, they’re fine as they are. I would just run them more times to explore more of the uncertainties and more of the scenarios. And then you’ve probably got a third group that’s like, whoa, slow down. We don’t need to throw any more money in computers at this. Can we at least define what it is we’re trying to achieve? And I sort of feel like we’re in a similar space in flood modeling and risk modeling in general. You know, now that we’ve got this mass proliferation of data providers all providing some sort of insight and some sort of niche, know, well, what does good enough look like? And I think that that really is what a great data product should aspire to be able to articulate.

Nissim Lebovits: Right. So maybe it’s helpful to talk a little bit about what you guys provide now as a kind of context. Because I think it’s it’s pretty remarkable. Like Fathom is putting out global, is it 90 or 30 meter resolution now? And it’s it’s hazard, but then you also have a bunch of other layers on top of that. And this is stuff that was kind of unthinkable 10 years ago, maybe even five years ago, the idea of having this at a global scale. So maybe could you share a little bit about like what you’re publishing now and then

Oliver Wing: 13.

Nissim Lebovits: Maybe some of things that people are asking for and and how you’re thinking about those asks.

Oliver Wing: So our product stack is sort of an infinite continuum of different bits and bobs of data. And in some cases, for customers who want to build their own models and just take, you know, pick and mix bits of inputs from different places, in some places, they want to get rainfall data from somewhere else or they want to get elevation data from somewhere else, which is a really good way if you have the resources to build a model. Because if you’re building it, you’re understanding it, you’re understanding its sensitivities, its uncertainties, its limitations. And that is the best way to use it in its most effective way. So in some ways, we have loads and loads of products. And in some ways, we just have one, which is a global catastrophe model, which essentially, a cat model is sort of an insurance tool, but it’s not strictly just for insurance, which basically you use to work out what the flood risk to a single location or a portfolio of locations might be and get that as an answer in terms of monetary loss per event. And the event component is pretty important because you need to know what could be hit simultaneously. That’s really what creates the tail events. If you think of Hurricane Katrina, if all those buildings flooded in isolation at random across 10,000 years, wouldn’t really matter very much, very manageable. It’s the fact that it all goes at once that creates the catastrophe. So that’s really what catastrophe modeling all is. And if we work our way back from that, the real kind of pedigree of Fathom is building the flood maps that underpin it that predict the depth per 30 meter grid cell for a whole range of frequencies ranging from really high frequency flood events to really low frequency flood events and doing that in the most rigorous and transparent and reviewed way that we possibly can do. And then one more step from that is the elevation data. The elevation data is the most important part of a flood model. And so because most freely available elevation data sets, in fact all freely available elevation data sets, really only represent the Earth’s surface, i.e. Roofs of buildings, canopies of trees, all sorts of artifacts within them, they’re not presented in such a way that a global flood modeler can actually run simulations over them. If we run a simulation across the Copernicus DEM, for instance, All of the cities would be big domed hills because the buildings would be picked up by those. And hey, presto, they wouldn’t be at risk because water’s not going to flow up that hill. You get a very flattering view of what global flood risk would be, but it’d be a dreadful one. So we have to correct or augment those data sets to ensure that they represent terrain that we can then simulate over. So we do lots of things, but also one thing, depending on your point of view.

Nissim Lebovits: Right. This so this reminds me of the the the sort of bigger question of not just what you’re building but who you’re building it for. You’ve kind of alluded to to reinsurance, which Swiss re who owns Fathom is a reinsurance firm. Are are reinsurance, I guess, firms? Are they called firms? Are are are they your primary customer? I mean, what is what is the breakdown of who is buying these products look like? And how does that maybe shape what you provide?

Oliver Wing: Yeah, we have a huge range of customers kind of spanning anyone who you think might be interested in flood risk, which makes my role when I’ve got my product hat on pretty difficult and certainly on the commercial strategy side quite difficult as to where you actually then focus your resources. Initially, we were set up, assuming that we were set up for the insurance sector. That’s where this kind of began in 2013. And we thought these global tools would be really helping insurers close the protection gap and start writing business in places that they don’t normally write. Thailand 2011 was an enormous flood event that I think to date is still the largest insured flood loss, where insurers just didn’t have models and didn’t know that they were accumulating so much risk behind flimsy defenses on these enormous Southeast Asian floodplains.

In some ways, we were sort of set up in response to that to show that, hey, you need more than just a Europe and an American flood model. If you’re going to write business here, you need to understand the risk of it. And so some of our early clients were insurers and brokers who were interested in that sort of question about how to avoid the mistakes of Thailand 2011. And also the World Bank and the UN and development agencies who, like us, thought the real value in the data was in Well, there’s no model for Africa or South America or Southeast Asia. And so how can we help some of these emerging economies avoid the mistakes in the West, so to speak, that we’ve made in terms of where we’ve developed and how non-risk averse we might have been? But really, probably from about, I don’t know, 2020 or so, the demand for this sort of information just went through the roof probably in response to well-intended but less well-crafted climate disclosure regulation, where large businesses and banks and many entities really had to start understanding what climate risk was. And so that led to demand for credible, trusted, transparent information that we were well positioned to then service. So we’re now in this position today where, you know, Probably it’s a relatively even split between a third of our revenue comes from the insurance sector, which would include reinsurance and brokers and so on. Another third from the kind of banking and capital market space. Asset managers, investors, banks, and so on. And then probably a remaining third miscellaneous bucket, would be a combination of corporates, governments, NGOs, engineering consultancies, that sort of thing. So it’s very broad, very broad. And again, sort of fits with our model of not needing to create a tailored solution for every individual customer. Know, we’re sort of get out of jail free card is we just create the data, we build data products, and we sell it to people who are capable of consuming those data products. And if you aren’t capable of consuming them, you get them via one of our distribution partners. So it sort of allows the scientific pedigree of the team to focus really on the scientific problems rather than the kind of data and accessibility and interoperability problems which frankly are the biggest issues when it comes to how do I make good use of data.

Nissim Lebovits: I’ll ask the question that I know Jed wants me to ask right now, which is I don’t know if you’ve seen his sweet spot graph, but Jed has a a sweet spot graph that he likes to show. And it’s you know, it’s it’s a parabola, more or less. And the middle of the parabola is sort of the ideal level of processing, where if the data are not processed enough, nobody can really use them because it’s a real pain. But if they’re over processed for a specific use case, it’s sort of limits who can use them and it it is a really poor ROI. So I’m wondering how you guys have landed on on what you feel like is a sweet spot for the level of processing of this data product and how you provide it to people and how that is influenced by who is using it.

Oliver Wing: Well, my cop-out answer is we have multiple sweet spots. We try and cater to all needs in that regard. So we’ll have pretty raw data for those that are capable of, you know, that have geospatial staff and can understand this stuff and can implement it within their own systems, you know, major companies and particularly major insurance companies absolutely have the right level of expertise to be able to consume complex, messy data. That’s their day job. So that’s no problem. And then there’s the middle ground, maybe the peak of Jed’s parabola, which is a lightly processed piece of data, which will maybe look something like a score. So you compress these multitudes of information into a single metric so that people who don’t want to be overwhelmed with terabytes and terabytes of grids that give you flood depths, which to many people would just be too esoteric to make much use of, that actually just sort of gives you, how risky is this place and how risky is it relative to another place? Just give me that information. And so we have that in our of risk score stack, which kind of integrates all that information into something relatively simple and consumable. And then we have the arguably over-engineered element of it, where we have a cat model that is kind of an end-to-end piece of software built in an open source framework, which we can talk about a bit later as well, where it’s maybe a little over engineered towards the insurance sector when it doesn’t really need to be. These are the ultimate risk management tools, but because they’re built by, within, and for, and use the language of, insurers other entities can become a little hesitant about whether they’re suitable for them. To consume it, you have to upload your data in an insurance-specific format and put all of your deductibles and policy limits and the financial structures of the policies that you’re writing to be able to get any output from the model. And it’s like, what if I’m just a non-insurer and just want to know the risk to this location? Well, yeah, you can absolutely do that, but it wouldn’t be immediately obvious that you could.

So in that regard, yeah, my answer is that we under and over engineer quite deliberately and try and fill in the gaps with distribution partners, ultimately. The clunky analogy really is to try and be the intel inside to climate risk information. We’re hyper-Uber specialized in flood risk. And you’ll find a bank who’s like, well, Cool, but I need to know about earthquake and wildfire and tornado and hurricane. It’s like, we’re not going to be one of these sort of venture capital backed companies that suddenly within six months has a US wildfire model. That’s just not our expertise. It’s not possible to do that credibly. And so we’re quite happy partnering with others to be the flood in that equation. So the line between competitor and collaborator becomes pretty blurry in this space, which I think it’s pretty cool.

Nissim Lebovits: One of the things that first drew me to Fathom’s work is the level of technical sophistication. I think this kind of speaks to what you were saying about the, you six-month timeline for a wildfire model. You guys are really serious about this. But you’re also doing some pretty bleeding edge stuff with like, you know, the the SWOT satellite and, you know, deep learner physics informed deep learning models. How do you guys balance, you know, when you’re thinking about building this stuff between the the level of technical sophistication, especially with the hype around AI right now, and also a commitment, not well, not just a commitment to, but a responsibility economically to a kind of conservatism given that like the the insurance industry is quite literally risk averse. You know, you have people who are like really need you to be reliable, but you also have this sort of cutting edge push. So how do you how do you balance that?

Oliver Wing: It’s a difficult balance and I rely on having a brilliant team. Mean, the fortunate thing about Fathom’s vision, mission and value proposition is, you know, I could just play fantasy flood modeller and pick all the people that I would want all over the world to work on these problems because it’s a great place to work and we do really cool stuff and it has very few of the drawbacks of other places where you could do similar amount of work.

So it means that we can kind of bring that kind of go fast, take risks, break things, cut corners where they’re worth cutting, but do the hard graft where it really needs doing. Because we kind of have the experience of where to go looking. Where are the aspects of the models where they’re pretty, there’s no right answer, so you just need to make a quick one and move on to something more material. So it kind of comes with experience and it comes with Again, understanding the customer and what they’re looking for. And so we never work on any particular problem that we don’t think somebody is willing to pay some money for its solution, or that we don’t think is a material limitation in the model, and that we don’t think will create a strategic differentiation from the competition so that we’re always staying ahead. Mean, one of the big risks to our business model given that we publish everything that we do, is that you can just replicate exactly how our model is built. And the reason why that doesn’t kill us is because there’s a bit of a lag between the writing, the reviewing, and the publication of the paper, and the actual bleeding edge of the science. So we kind of owe it to ourselves to constantly stay ahead of the own curve that we’re creating, because otherwise we play catch up and we lose our differentiation. In that way, we kind of We kind of set a pretty quick pace, but we go slow where we think we need to go slow. And so AI is a very interesting one, because you lots of interest and very little deep expertise on this topic. And you have lots of people who actively want to consume a model because it’s AI-based and lots of people who actively don’t want to consume a model because it’s AI-based.

And nobody can really articulate why or what the benefits might be. That’s not true. It’s not nobody. It’s difficult from an end user perspective to be able to differentiate the wheat from the chaff, the good from the bad. As always, you land on a sort of middle ground of a hybrid approach. Physics inform machine learning or what have you. I always get slapped on the wrist by our machine learning engineers for saying.

We’ve built this model that’s learned physics. They’re like, it hasn’t learned physics. It’s just a data-driven model that produces the answers that you’ve trained it to produce. But the reality is we’re creating really neat tools to plug the deficiencies of more traditional techniques. But it’s by no means an end-to-end solution. Shouldn’t be deluded into thinking that you’ve got a ChatGPT interface where you say, what is my flood risk? And it produces an answer. That’s absolutely not what this stuff looks like.

It’s about using physics-based models and the well-understood equations where that is computationally tractable to do so and where a data-driven solution just cannot get out of sample in a reliable enough way, and using machine learning techniques for exactly the problems for which it’s designed for, which is of wrangling with messy data, finding patterns in it that you would otherwise not spot, and generally tidying it up.

So really we use machine learning as an input to physical models, or we use machine learning to kind of sharpen up the core scale detail within a physical model, rather than this sort of completely unsupervised generative approach. And it kind of gets me on to the main point of what I think our differential is and what a differential should be in this space is trust, really. You’ve got to be able to trust the model.

That might come from being convinced by me waffling away on a podcast, or it might be from very detailed documentation or peer-reviewed articles or whatever. But ultimately, as an end user, yes, you’re kind of buying the quantitative data and the model, and you’ve got a grid with some numbers in. But I really think that you’re buying the team that built it. You’re buying the expertise and the value judgments that they baked into it millions of gray areas that somebody somewhere had to make black and white because that’s what data has to be. And in turn, you need to have faith that they’ve made a reasonable choice. And you may well outsource that because you think, these folks are the experts in this. And that creates a really interesting dynamic between model vendor and model user and what it takes to be competitive in this space because It isn’t clear-cut what makes a great model. You can validate these things within sample to death, but you’re using them out of sample, so how would you possibly know? Ultimately, it comes down to how you conduct yourselves as an organization and how you support customers in its use and maybe where you push back on misuse and maybe… Toning down some of the hyperbole around how awesome and accurate and amazing all this stuff is and actually saying, well, I don’t think you should be using a model to do that. This is beyond the state of the science. For example, others might take a different point of view. They might just want to go as high resolution as possible and talk about how incredibly accurate these unvalidatable predictions might then be as a result. So it’s a really interesting interface because maybe different to some of the other data products that you guys talk about in your day-to-day, which are much more kind of observationally grounded. And there sort of is, to at least a higher degree, some degree of truth that could be sought through observing. In some ways, everything’s a model. You’ve still got to process the satellite image and so on and so forth. But this is like truly generative modeling where you’re taking a bunch of observationally-based inputs and then applying them with some equations to make a prediction that’s very difficult to verify. And so all of the usual rules don’t really apply there. Well, surely if it’s higher resolution, it’s therefore better. Or surely if this valuation metric reduces my confidence bounds quantitatively by this much, it’s therefore better. In model world, that’s That doesn’t really apply.

Nissim Lebovits: Yeah, I find flood models really fascinating in this sense. I I’m sure this will be gratifying for you to hear, but as we’ve talked over the last couple of years about flood modeling, I’ve really come to think of it as as one of the most epistemologically interesting kinds of climate data out there. Cause you know, I I sort of think of the other end of the spectrum being building footprints data, which I work with a lot, which are very easy to explain to people because you show them the data and then you show them the satellite base map and they’re like, yeah, there’s the building, it’s right there.

And you can walk to it. And it’s really a binary question of like, is the building there or not? And maybe the exact boundaries are not perfect, but it’s pretty easy to wrap your head around the concept of of what a building footprint refers to. Whereas like a flood map, the you know, it’s sort of like this platonic ideal of a flood that is ne you’re never gonna get that exact footprint. It’s contingent on so many different factors, like you know.

Not just like the volume of rainfall, but when the rain fell and if there was rain before that, or if there wasn’t rain before that, or if the sewers are backed up. And so it can be really hard to communicate this. I have found in my in my work with municipal governments, for example, that it can be really hard for people to wrap their head around even just the idea of a hazard layer and a return period. And then you get into concepts like like expected annual loss know and these risk curves and stuff and it like it’s just it’s so hard to to to get through to people. And i I think it’s also because different entities are thinking about risk in different ways. Like I tend to think of reinsurance firms as as almost like poker players, right? Where they’re less concerned about the individual hand and more about the long term of their their earnings over time. Where like you can lose a hand here or there But if you play the averages and you distribute your assets or your investments well across a wide range of places, then what matters is the average. And so they that’s they’re really concerned with distributions because they can distribute assets and stuff. Whereas a a city government really doesn’t have that luxury because you’ve only got one city. And if you get one out of, you know, out of sample event, it’s kind of screwed. So I think this question of of communicating the data is is really important and in particular in terms of of uncertainty, which is like even more of a of a of a difficult concept to explain to people. So how do you guys think about I n not even marketing per se, but scientific communication. I guess marketing is a part of it in terms of trying to take these incredibly complicated topics and distill them in meaningful ways for your end users or people who might be interested in the data.

Oliver Wing: Yeah, mean, uncertainty is this labyrinthine topic. And I think it has sort of different definitions and meanings depending on who you are and what matters. It’s quite a personal thing, uncertainty. And so if we’re trying to see the wood for the trees in this conversation, really, uncertainty is the difference between what we need to know and what is available to make a decision on I think we can break it down into epistemic and aleatoric uncertainties. But honestly, I don’t think that’s particularly meaningful in a sort of data use perspective. And I’ve sort of these days take the view that most of the uncertainties really kind of live in this liminal space between the model and the user. It’s less related to, well you know, the resolution of this is coarser or the vertical accuracy on the elevation data is, is worse here or you’ve used a method for extrapolation that’s, that’s potentially less reliable. But, but, but the main issues really arise when, a human confuses the model for reality that they don’t realize that it’s, of course they realize it is, but they assume that it’s a perfect representation of the real world in all of its messiness. So yes, There’s quantitative heft and skill to creating these tools. But as we talked about, there’s this kind of subjective expert judgment-based underbelly. And so we get onto all of those aspects of trust and so on that you’re talking about as to why that’s then so important when consuming a model. So how do you then go about communicating it? Well, you need to do so with some degree of humility, but not undersell so much. You need to be clear on on what the use cases are, as you say, a reinsurer is doing very different things with these sorts of tools than others. They have the expertise to know what it is that’s important for their decision making. These are institutions that are typically sort of holed up in the Alps somewhere and have been around for centuries. They’re sort of the the quiet backbone to the global economy, actually. I mean, if they failed, the whole system fails. And so they really, really, really need to understand extremes. They insure the insurers. Potentially after them, there’s governments. That’s about it. And so it means that because it’s almost so existential that they understand this problem, they don’t really need to get engaged in, I don’t know, the beauty parade that can often pervade discussions of model quality and what makes data good and so on, that they know that they don’t need to get it right for every location. They just need to really understand how bad it could possibly be and make sure they’ve got enough money stashed away to be able to pay out on that when it happens, as well as remain competitive in a competitive market. And that’s very different, as you say, to the city planner who does need location-level skill you know, ultimately…

When you kind of boil it down to our constituent components, I think I just have to rely on my previous answer of trust. Who do you trust to provide this information? And are they being honest about model limitations, or are they just putting a glossy marketing spin on it? Are they people that just look like they’re here to make a quick buck, or do you have an organization with a roster of mission-driven global experts building this stuff. We’ve taken an answer and an approach to that, and others have taken different ones. And ultimately, we’ll see who wins.

Nissim Lebovits: That’s a that’s an excellent plug. That was well done. Yeah, I’m I’m fascinated by this the concept of reinsurance, which I think you actually are this is silly, but probably the person who introduced me to. I had no idea that there were insurers for insurers and how central this is to like the global economy. It’s like really weird that this is sort of there in the background. But it it makes a lot of sense in many ways w when you kind of think about what’s going on And one of the things that most fascinates me about it is, you know, we have this fetish. I come from a very technocratic background, right? I’m a city planner by training, and city planning comes from this idea of like, if we just have the right data, we can sort of top down organize stuff. And and so there’s this this this fetish of the idea that like we’re gonna plan everything with data, which in politics often doesn’t work for a couple of reasons. One, you actually ideally in a democratic system don’t have that much top-down control, right? Like we don’t want to be Robert Moses anymore, and that’s probably a good thing. And along with that, you have very different political timelines because you have re-elections. And so every four years you kind of have to worry about you know, convincing people to vote for you again, which can promote short-term thinking. So it’s it’s very local. It is you have fairly limited power and you have this very short timeline that you’re making decisions on, which means that Long-term investments like flood infrastructure for your city tend to get deferred because nobody wants to be the person who, you know, raises taxes by three percent to fund it. Whereas reinsurance, it seems like, is maybe of all the industries that are out there, kind of the one that’s most attuned to to risk and climate risk and decision making. I mean, I I know after the California wildfires, for instance, there was kind of this big panic of like, what happens if people stop getting home insurance because all of a sudden California is a no go zone for insurers. I don’t know if I have a a particular question there. I’m just really like I’m fascinated by this concept.

Oliver Wing: It’s a really interesting dynamic, isn’t it? Because in some ways, we see governments suppressing the very rational market forces that would discourage development and occupation of risky areas. And part of me, as a person, is hesitant to think that the market has the solutions and that private companies are going to solve that because ultimately, yes, they provide signals as to where you should build and where you shouldn’t and who should get covered and who shouldn’t. But they’re ultimately there to make money as all businesses are. They might have a mission and a vision and so on. And there might be a sort of bond of trust that they have and what their role is in the global economy and society and so on. But ultimately, they’re not a charity. They’re not there to pay out they need to be collecting more in insurance premiums than they’re paying out in claims. Otherwise, they won’t exist. So I’m a little torn in many ways. I do think that if governments can take some of that long-term thinking and start to apply it in a consistent and coherent way, and that mostly comes through the form of what capitalists generally don’t seem to like, which is regulation, and actually having coherent regulation and enforcing it. That’s sort of the answer. In the meantime, yes. I think it is positive that as the financial sector in general hungers for higher quality data on flood risk or climate risk or have a broadly want to talk, that they are going to start making, you know, beyond just tick box disclosure exercises, they are actually going to start informing business decisions with these model data sets. And they’re going to start understanding that they can get a competitive advantage by doing so. And that’s going to mean that some places will have very, very high cost of getting insurance cover. That will mean that getting a mortgage will come with a massive interest rate because you’ll be a massive credit risk. It will mean that investors will be less interested in this area because the valuation of this asset or this portfolio of assets is lower, and so on and so forth. And so you’ll start to see this rational shift towards not developing and not continuing to occupy the most risky locations on the earth. But it isn’t a given that the private sector will ensure that that’s done in an orderly and equitable way. That’s ultimately what governments have to be able to do.

So part of me is content with the idea that, well, at least global capital potentially has a good solution to this. It might not be the best one, either.

Nissim Lebovits: Yeah, I I worry about the consequences of a massive overnight devaluation of everyone’s homes, especially in the US, where like we’ve, you know, for the last basically since World War Two, promoted the idea that your primary asset is your house. And so if suddenly, you know, millions of people’s homes are worth nothing, that is gonna have pretty pretty substantial economic consequences.

Oliver Wing: Yeah, and it builds on…

It builds on this discussion I have a lot in the insurance industry in particular. It’s like, what is it that we’re trying to quantify? It we’re actually trying to quantify the true risk, or are we trying to provide a quantification of the perception of risk, or justify our own perception of risk? Because those are two very different things. You see, particularly in model world, a lot of highly uncertain problems having very, very similar answers which from a science perspective is completely incoherent. You can look at submissions to the regulator for the Florida Insurance Commission for hurricane wind losses. And they’ve got a very, very stringent regulator there. So to be able to use a model to set insurance rates for Florida, you have to pass this commission. And it’s hard to know if this kind of model convergence is a consequence of unintended regulation or market forces or whatever. But for all of the models that have succeeded in this process, for the probable maximum loss for hurricane wind in Florida, which an event beyond our imagination, in essence, they all produce answers roughly at the round 90 to 100 billion mark. And I think this is an unknowable thing. This is unquantifiable. We don’t really even know the fundamental physics behind why a big hurricane or a small hurricane might form. And yet, multiple quasi-independent views of this unknowable thing are pretty similar. And to me, when you have a critically, epistemically uncertain problem, you want as many independent views as to what the answer could be as possible from a diverse group of people with different backgrounds and different experiences. And so, There’s many avenues you can go down with that train of thought. But really, my point is, well, are we actually trying to understand the risk, or are we just trying to create a common currency for it so that we can trade it effectively? And I think that potentially goes with something even as simple and fundamental from a global perspective as American home values and how important that is to people. There was a a massive unregulated experiment with the disclosure of public climate scores on real estate websites at the start of this decade. And probably at the time, lots of people, even well-meaning people, thought, wow, this is really cool. How amazing that the public can get information from these models on what their flood risk or their fire risk might be. And yet, I think this is an example of the unintended consequences of open or democratized or whatever buzzword we want to use, models where you provided the model but without context and without understanding the end user and their capacity to understand its limitations. And sure, in aggregate, it’s probably a force for good that people can have a sense on whether their home is more risky than another home. But what do you say to the person in, you know, wherever they might be in America who’s not been able to sell their home or has had to take a haircut on the price of it because of what might have just been a model bug ultimately. And that is sort of unforgivable, I feel. And it’s a consequence of lack of regulation and lack of the public sector taking a lead on what is ultimately a problem that that is in the public interest and it’s been left to private companies to fill in who, you know, can can do a good job of the modeling but ultimately they’re there to make money. That’s their incentive. And I have very mixed feelings on, you know, the the overuse of model data in that regard, because it’s kind of good to raise awareness of risk but done in an incoherent way, less so.

Nissim Lebovits: Well, yeah, and you’re making me think also of of FEMA’s own models. I mean, the the whole institution of FEMA is so kind of shot through with perverse incentives, just in terms of like the it’s propping up entire communities around the US coast that probably shouldn’t be living where they live. And that has to do with, you know, one after another political institution asking for carve-outs and handouts, often bipartisanly and you you’ll if you look at the maps, you’ll often see these places where people have clearly filed to have the flood zone modified for regulatory purposes to exempt their, you know, whatever construction project they have. And so you’ve got these people building in the floodplain that like know that they’re building in the floodplain, and FEMA can’t really do a lot about it. I remember in in Philadelphia, maybe maybe a decade ago, there’s this The the Schuylkill River runs through West Philadelphia and past this neighborhood called Manayunk. And there’s this this island in the middle of the Schuylkill there that people have been trying to build on for decades. And finally, about 10 years ago, they convinced the city council member to let them build apartments on it. And you know, it’s clearly in the FEMA floodplain. He got a carve out for the city. Predictably, it floods in like 2021, I think, 2020, during this big hurricane that hit Philly.

And they had to send the fire department to rescue everybody. And it was like, you know, such a clear-cut case of we should have known better. And then flash forward to this year, and he’s carving out another exemption for another part of the same island to build more housing on. And it’s sort of it’s like it’s really staggering that we have all these data. It’s not like there’s a lack of information. We have all the data we could possibly want to make a decision there, and and yet we still don’t. And it sort of makes you think like, well.

You know, if if data doesn’t do it, if you have the data and it’s just like about these kind of bad actors, basically, not not saying the councilman specifically is necessarily a bad actor, but there are clearly people here with short-termist, perverse incentives making obviously bad decisions in spite of having the data. So it sort of makes you wonder like, well, okay, well if we had the best data in the world, you know, how much of a difference would it make if this was this is the kind of decision making that we’re engaged in?

Oliver Wing: Well, think it, frustratingly for a podcast all about data, it’s sort of the data is not really what we should be talking about. There’s no problem there. Often the conclusion after hours or days or years of discussion on this topic is that what we have is absolutely fine. It’s more than enough. We can invest another a few million dollars in a more refined model in a certain place and make it a bit more accurate by some metric or another. Or we could instead try and link up the modeler and the end user in a more coherent way and figure out what it is that they’re trying to achieve and what good enough looks like and potentially discover that we already have it. And instead, the main thing is how to use it safely as this sort of shared endeavor with shared accountabilities for misuse. And so the problems here, I think, are down to weak regulation and weak enforcement of them, which you see all over the world, really. Mean, in the UK, there’s this pressure to build more homes to address the housing crisis. And often, the thing that’s claimed to be in the way of that is too much red tape. We’ve to cut cut red tape so that we can build more. And that often means making sure you can build more easily in floodplains. So housing developers can build these things in 18 months, two years, and run the gauntlet in that development period and hope they don’t get flooded. And then once they sell all their houses to individuals, the risk is fully passed on to homeowners. And then, of ultimately, the state will keep bailing them out, which is kind of necessary, but also perverse at the same time. And that’s, you know, we sort of talk about how private solutions might resolve that to some extent in terms of, you know, well, an insurer might might not cover the cost of ensuring that project, or an investor might not finance it, or it would be so unattractive to a homeowner because their interest rate would be so much higher or their insurance premium would be so high. But yeah, mean, ultimately, it’s about connecting the end user in these cases. And I do think the government has a role to play.

Of course, it has a role to play in doing that in a more coherent way, but it’s very hard for them to reconcile the kind of supply side and the demand side of the equation. And you end up with the sort of, you know, the countless perverse examples of where people are incentivized to continue to live in risky areas. And I don’t know what the political solution is to it. And that’s why, you know, the experiments of publicly disclosing what the climate risk of a given location is, I think, has its roots in being quite well-intentioned. But how do you expect a random homeowner to interpret the uncertainty in their climate score to be able to think, I know it says six out of 10, but the elevation data has plus or minus 50 centimeters of error, so I could be a two out of 10, or I could be a 10 out of, like, No normal person is capable of doing that. And so you need to be incredibly careful when revealing that kind of information to people. And that whole experiment, I think, has really changed how I would ideally think about communicating models to a variety of end users. And I used to think, tell everybody everything. Fully disclose everything.

Total documentation, I’ve written thousands and thousands of words on model limitations and what to do about it. And I sort of think, actually, for the vast majority of people, they should go nowhere near that. You need the minimum amount of information required to make a decision. And in some cases, that might be, sure, there’s loads of uncertainties, but you don’t have the capacity to take a decision amongst that You’ve got to understand that we can never be 100% certain, but this is the best course of action for the decision that you’re taking. And that in many ways, not so much of a tangent, but a sort of extension of the thought. But the uncertainty is kind of part of the equation anyway. Mean, risk and uncertainty can be pretty synonymous, depending on your point of view and what your body of work is and so on The uncertainty should just be a part of that. You should think, yes, this is risky, but also it’s uncertain, so I need to add some loading on the decision in some way. All of these things are possible to do. But yes, the first step of having political misincentives is a very tricky one to begin with. Because you then not only are you not moving forwards with the problem of overexposure and risky areas, you’re kind of just, at the minute, we’re focusing on how do we stop going backwards rather than moving forwards. You’ve got this huge population of people living where nobody should be living. And they get flooded, and they get a load of taxpayer money, and they rebuild in the same place. And you just think, this is nuts. And so there are undoubtedly people far more expert than me in in these sorts of things will know of more sensible and equitable and affordable solutions to that to ensure that those that are in harm’s way have the means to get themselves out of harm’s way and stay out of harm’s way. And those that can afford to move or can afford to deal with the risk and be at liberty to do so, then don’t get a huge bailout every time the inevitable happens. There are solutions there. But like I say, this is This is far from what makes great data. This is how to use great data.

Nissim Lebovits: Yeah, it is there’s there’s only so much you can do. I mean you can you can lead a horse to water, but you can’t make him drink, right? And and I think it also, you know, going back to what I said about sort of technocratic fantasy of how people use data, when you look at how it gets used in the wild, especially in the public sector, data is often a very small fraction of what One ends up informing a decision. So all of our sort of fears about the need to caveat very heavily and provide 16 different return periods and climate scenarios and stuff, especially like if you’re thinking about local government, they’re gonna take one of those and apply it. And that’s gonna be like, you know, 20% at best of what factors into their decision making. When you also think about, you know, local political interests and maybe pressure from the real estate sector, and just like the you know, I I’ve I’ve been in Plenty municipalities where there’s like one GIS analyst whose job it is to be, you know, the person maintaining the cadastral record and finding all the data sets. And also they’re like the hydrologist because nobody’s got a hydrology degree. So you’re just kind of like doing the best you can with what you’ve got. Which on the one hand is is an argument for caveating stuff carefully for somebody like that, but also for just providing data in kind of a a a simple way, especially because it often gets used more as a communication tool than necessarily a decision making tool. You know what I mean? Like it’s it’s it’s not necessarily that they’re going to look at every single bit of the data set. They’re often going to show like a map and that’s going to be an impactful map. And people are going to be like, yeah, we probably shouldn’t build houses there. Which maybe is is more important than an expected annual loss number if you’re in a local government context, for instance.

Oliver Wing: Mm-hmm. Mm-hmm. Yeah. Mean, ultimately, the idea of a model making a decision is a terrifying concept, really. We should reclaim our agency. They are only there to support decisions. And all of the messy gray area, swampy bit that sits in between the model and reality is up to a human to interpret and act upon. And that only comes with providing the quantitative and the qualitative, making sure that you are communicating uncertainty but tailoring it to the user in a way that is likely to mean that they make a better decision as a result. Mean, one kind of argument in the insurance sector really is in underwriting where the incentive of an underwriter is just to write more and more insurance business. They want to collect as many as much premium as they can. It’s somebody else’s job to make sure that they don’t write so much that if there’s a big event, they go bust. Somebody else will deal with that. So they are just, and I’m over-characterizing them here, I’m being very unfair and underwriters. But if I was to present the full range of possible answers to what is the flood risk at this location I want to write a policy for and, they then realize that in a flood model, the true answer is somewhere between 0 and 100% of loss. Well, what’s their incentive? They’re going to assume that the risk is really low and write a really competitive price and then do that for so many locations that they might clear the market out in that year and wipe the competitors off the face of the Earth because they’re just underpricing everyone. And then when something happens, they go bust and the financial system with it. It’s sort of back to that initial point about providing just enough information so that it can be safely used. Don’t assume that your end user is capable of really understanding the intricacies of when is an answer decision relevant and when do I need more information to be able to do that.

Nissim Lebovits: So I’m I’m realizing we have chat that I should be monitoring, which is exciting. So we have two questions in there. Well, one’s one’s one’s not a question. One is some some pointed feedback from my friend Max. Max Lenormand says he strongly disagrees with I believe was my last point about not communicating uncertainty. We’ve done it for weather forecasts on I think all phones. It’s not simple, but he says he trusts people. He he trusts and believes that we should bring people along.

So I guess I I guess that does raise the question of how, you know, how you communicate uncertainty as part of a data product in a way that is to your point the minimum necessary for people to make an effective decision. Because I’ve also like, I mean, I think we see this a lot in climate, for instance, where sort of bad actors or people acting in in bad faith will use communicated uncertainty as a way to dismiss a problem say, well, climate models, you know, aren’t certain. There there’s there’s uncertainty in there. And uncertainty often becomes a kind of a a kind of cognitive bias toward n inaction. Which I so so there’s a a real important sense of like both communicating uncertainty well, but not communicating uncertainty in a way that gives ammunition to people who are trying to discredit climate information.

Oliver Wing: Yeah. Yeah. And I don’t think either of us has said not to communicate uncertainty. It’s about tailoring it more to the end user and what they’re capable of understanding. And I think weather forecasting has been quite successful with that. Mean, as a non-meteorologist, sometimes, if I, you know, it says 50% chance of rain, I just think, for God’s sake, why can’t you just tell me if it’s going to rain or not?

You know, people are capable of consuming that and making their own decision, but sort of the impact of the decision is, well, you know, I might get wet because I haven’t brought an umbrella with me because it was a coin flip as to whether it was going to rain or not and I didn’t make the right choice. And of course, there are more severe scenarios in terms of misforecasting weather. But anyway, the point is that these are verifiable forecasts where you have tons of data to be able to validate whether the model is actually performing well. They’re fundamentally not very often going out of sample. And so you have so much data to test the reliability of the forecast. You just don’t have that in the sorts of models that we’re talking about. If we’re talking about what does a flood model actually do, well, arguably it’s providing an incredibly uncertain forecast of next year’s flood events all the range from no floods happening to massive floods happening. So putting a percentage on that in a weather-style app is sort of meaningless. And you can’t validate that it’s even got skill in. You would expect that at least if you said it’s a 10% chance of this loss, in the long run, that loss should happen 10% of the time. You can’t test models in that way. So I think it’s a long-winded way of saying it’s just very important to communicate depending on who it is that you’re talking to. If I’m talking to a big insurance company with a team of PhD level scientists on staff that are looking to integrate a new flood model into their decision-making framework, it’s absolutely like full disclosure of everything. And they’ll already know it anyway because they’re skilled end users in this space, and they understand the value of the data, and they understand its limitations.

There are certainly cases where total disclosure just creates more confusion than value. Absolutely.

Nissim Lebovits: So Nate Silver actually makes an interesting point about weather forecasts and the signal and the noise, which is at this point, I think 15 years old or more. But he talks about how commercial weather forecast providers are mostly just repackaging government weather forecasts. And they tend to be less accurate because they have an incentive to be more conservative in their forecasts because they have to worry about losing users because they’re tied to things like ad revenue and stuff. And so if you’re getting your weather from the US government, I think it’s NOAA who tends to produce it, they will tell you there’s a 50% chance of rain because there is genuinely a 50% chance of rain. But if you’re getting it from, you know, whoever the weather provider is is repackaging that and and basically selling it, they’ll tell you 60% chance of rain because a false negative is worse than a false positive.

They would rather you take your umbrella and not get rained on than the opposite, which is an interesting, you know, little i it it’s it’s an interesting bit about communicating uncertainty because they have an incentive to communicate uncertainty inaccurately so that you don’t get mad at them, which, you know, in theory, at least NOAA does not, which I find I find fascinating. I don’t know if that’s changed since that book was written, but I thought that was really interesting.

Oliver Wing: Yeah, that is really interesting. Didn’t know that. I can sort of, I mean, I kind of think these private weather forecasting companies are total charlatans really, because they just repackage publicly funded data. But that point aside, I don’t think it’s entirely unreasonable that they practice that in that way. Because, know, NOAA is providing, as far as I understand it, the sort of direct output of an ensemble of weather forecasts. And in 50% of them, It rained, and in 50% of them, it didn’t. And therefore, this is the probability of rain. Whereas what the kind of repackaged data is providing is a sort of more human interpretation of that. It’s like, I don’t want you to just think it’s a coin flip that it’s going to rain. Look, it’s a pretty good chance it’s going to rain. Why don’t you just take the raincoat with you? It’s really not the end of the world. It’s not going to cost you very much. And so in some ways, I don’t hate that they’re just different products for different people.

Nissim Lebovits: Well, yeah. And I mean it’s it’s what I would say in terms of like designing floodplain regulations at the municipal level. Right. Like, you know, w you can’t make you can’t make parcel level decisions about where a flood zone should be based on ninety meter resolution data, but you can just tack on a little bit more. Just use the ninety meter, tack on a little bit more and be like, don’t build near this. Just, you know, for for be cautious.

Oliver Wing: Yeah, like what really is the cost of, okay, yes, my 100 year flood zone that follows this perfect line in model land, you know, why wouldn’t I just maybe be a little bit more risk averse and just extend it a little bit more? And this does sort of happen in management practice when it comes to freeboard where you’ll add a nominal amount of elevation onto a building to ensure that it sits above this level. But I sort of feel like what’s the risk of being more conservative with that. But that’s why ultimately the use of these tools requires you to be human, requires you to have a risk appetite and a strategy for sating that appetite. Because there’s opportunity in a competitive market to take a more optimistic view of the risk and make more money and higher risk of losing out. And in other spaces, particularly when we’re talking about, you know, government regulation, I really don’t think there’s very much incentive at all to take the more optimistic view of a risk. Why would you need to do that unless we’re truly talking about crippling scarcity of land or that if I just built this 100 meters over there, it would have so much more amenity value than if I had to build it a little further up the hill. I just don’t really believe that those are actually material and that you can’t take a more risk averse approach to planning. But obviously that’s just one small aspect of what these sorts of great data products could be used for.

Nissim Lebovits: I I have it in my notes to talk to you about why multi model views are not a great way of accounting for uncertainty. Let’s unpack that, ‘cause you mentioned them a moment ago and it it kind of reminded me that I wanted to talk about that.

Oliver Wing: Yes, well, they are a bit like democracy in many ways. It’s the best we have, but fundamentally flawed or whatever the Churchill quote is that I absolutely butchered. Absolutely taking multimodal views is kind of a prerequisite, but it’s not sufficient to understanding uncertainty. You could say the same thing about flood modeling. You could say it about climate modeling. You could say it about probably all sorts of models in that It’s only really if you have truly independent views of a particularly uncertain problem that having multiple, yeah, that would actually be beneficial in really bounding what the uncertainty is. And too often, we kind of mistake lots of models producing an answer giving us, would give us a lot more confidence than if one model gave an answer. And I think all things equal, that’s probably probably true, maybe, depending on how the others were built. But you see it in the IPCC, you’ve got all these loads of lines on your kind of iconic graph of how much the world will warm as a function of global emissions. And you view all these lines and you think, huh, so one of those will be true. Because look how many lines there are, so many different independent estimates. Good. But I mean, that’s completely false interpretation because these models all have the same fundamental strengths and all the same fundamental weaknesses. And so if one is wrong, they’re all likely to be wrong. And until you have a method for accounting for that, you haven’t got a probability distribution. You’ve just got a distribution. And so I think that that is potentially what’s missing from the kind of drive towards you need multiple models. My example of the Florida Commission.

Multiple models will not help you in that situation because they all kind of say the same thing anyway. And so you’ve spent five times as much money on securing the license for that model, validating it, implementing it within your systems. That would cost you tens of millions of dollars a year to go through that process, only to discover that they’re all calibrated to the same set of unrepresentative data anyway. So what is the benefit of multiple models in that scenario?

And so with limited resources, taking one model and exploring the uncertainties within it, and that’s something, not really here to like plug our stuff, but that’s something that you can do within our tool. Yes, it’s only one model and it’s created by a single team with its own biases and its own specific sets of expertise, but you can tweak the parameters within it and see how the outcome will shift as a result.

You can input more specialized or more generalized, whatever you want to do, more generalized data when it comes to vulnerability modeling. Maybe you have a better idea than I do about what data center vulnerability might look like and input that into the software and see how the results change. That’s kind of what the point of model transparency is all about, that you change a parameter and you figure out how much it changes the result. So doing that in a structured and coherent way would be both easier and more valuable than securing multiple licenses to different models, I think, if you’ve got limited resources. Or alternatively, go through that same exercise with a variety of models. But just to say, without nuance, if I bought every model that was available to me, that would lead to a better decision than if I just used one. I think.

I don’t really believe that to be true. But look, there’s plenty of examples where a careful assimilation of multiple models that diverge in areas where there’s multiple reasonable solutions to a given problem and being able to appraise those differences and blend the results together can absolutely lead to better decision outcomes. But yes, all I’m really saying is that don’t mistake the ensemble for a probability distribution because they’re not.

Nissim Lebovits: So we have a question in the chat about last mile delivery. I wanna hold on to that until we talk about the sort of open source open data plan that you guys are sort of cooking up at Fathom. Before we get to that, I wanted to ask about El Niño. Just because this is, you know, from what I’m hearing, this is like pretty historically unprecedented, very much like a an an out-of-sample kind of event. Are you guys thinking about it? How are you making sense of the consequences? How does it fit with the modeling that you guys have done?

Oliver Wing: Cool, yes, very topical. I’m probably going to commit climate heresy here and maybe downplay it more than, well not really downplay it, but in some ways I’m kind of tired of the topic because this is, it’s interesting because it is a naturally occurring phenomena that creates a lot of local warming and a small amount of global warming layered on top of already unprecedented global warming, which is the real issue here. The El Niño thing, it depends on where you live as to whether this is a problem for you. There’s spikes in risk in California and Southern USA for flooding and drought across Southeast Asia and difficulties with food supply in other parts of the world. Absolutely this is part of a normal cycle of that happening. And, you know, we have seen impacts of it, but they’re sort of, you know, no excuse for not being prepared for that, really. Mean, El Niño and La Niña is just a natural oscillation and we know largely what the impacts play out like. Now, that’s not to say that there aren’t any problems here, but what I’m trying to really get at is that this isn’t some like, massive amplification of global risk. This is just a redistribution of where weather happens. And if you’re unlucky, bad weather will occur more in a certain place. And if you’re lucky, bad weather will occur less in a certain place, which is kind of the same thing every year anyway. And I do wonder if just psychologically, because it’s creating warming, that that is sort of why it’s being covered so much. I mean, would we be talking about a super La Niña, which actually cools temperatures slightly. But from a flood risk perspective, it’s way worse than El Niño. Way worse. El Niño causes most of the rain that would cause flooding in Southeast Asia to just flood the Pacific quite harmlessly. So what an amazing benefit of the Super El Niño. Super El Niño in the North Atlantic, it’s been a historically quiet hurricane season. What a celebration we should be having about that Now, I say that sort of tongue in cheek because it’s no joke that there are places where risk is increasing. But I remain somewhat perplexed at the sort of sweeping generalizations associated with increasing risk, and particularly since we’re both in Europe, where the impacts, apart from food supply chains, which may or may not transpire to be real, are essentially unknown. Know, have British newspapers writing about, super El Niño means it’s going to be stormy this winter. I mean.

What possible evidence do you have for that? So it’s a bit of a frustrating topic for me. So I find it very strange, the sort of obsession with it. And I don’t want to downplay the realities of the risk that it brings locally in certain places. But I’m also a little weary of the hype as well.

Nissim Lebovits: I guess it it’s in some ways reflective more of our own cognitive biases than of the reality of data, which you sort of can only understand taking a global perspective, right? It’s it’s how people are are are prone to like blaming general stuff on climate change. Like any kind of weird fluctuation in weather is always attributed to climate change, which in some ways undermines the the seriousness of the stuff that really is attributable to climate change when just like every weird quirk of the weather is attributed to it. But it’s also the distinction between local and global understandings of risk, because you can have in the aggregate lower risk as a result of El Niño. But if you live in a place where it’s flooded more this year and you’ve mentally made that connection, you have a very different perception of how it impacts risk.

Sorry, just checking to make sure I’m not missing anything. Siddharth, we are going to get to your question in a sec. I wanted to first ask Ollie about Fathom’s open data, which is I guess kind of the the the the culmination of this conversation. So you guys are you guys are thinking about open data. We’ve we’ve talked a lot about, we’ve talked at this point now a lot about you know your strategies for communication and stuff. You guys are thinking about an open source publishing strategy. You mentioned the I think it was the the Oasis flood modeling framework that you alluded to. So yeah, tell us about that.

Oliver Wing: Yeah. Huge topic. So interesting to kind of follow this on with a lot of my experience related to the public disclosure of climate risks and the dangers of doing so. And in tandem talking about how can we make data much more accessible to people who need it, and maybe even free in some cases where that’s warranted.

Because we currently don’t have particular intention of releasing our flood risk data for homeowners to consume or anything like that. But we do want organizations and businesses to, or maybe not even businesses, but just to be able to access our data much more easily. Mean, it’s both astounding and depressing. And to me, as a scientist, but less kind of on the… Data side and more on the science side if such a distinction really exists. Almost quite boring that the biggest problems that we find our customers face is nothing to do with the quality of the data and nothing to do about whether it validates well against X, Y or Z, but is just purely on the question of interoperability and how can I consume this data within my systems and how can I make use of it and how does it…

How does it meld with all the legacy systems that I use and all the incumbent vendors of information that I currently license? That is the number one topic. And so from a science strategy perspective, that’s annoying and difficult because that isn’t really what our thing is. We build the data, throw it over the fence. That’s a bit of a generalization. But ultimately, we just want you to use the data that we have and off you go. Whereas now, because we just hear it so consistently that, you know, well, I want to use your model, but I don’t have my data formatted in a way that’s going to be able to talk to it in any coherent way. Yeah, it becomes a struggle. So there are sort of multiple prongs to this. The first general one is that actually all of our data that we publish in peer-reviewed by design and definition is available for non-commercial use, because it has to be, because we can’t just write a paper on something and then not allow you to actually verify whether we’re talking rubbish or not. So that’s always been the case. And we do struggle to make that as frictionless as we would like when it comes to accessing the data, not only because we’re a regulated business and can’t just openly distribute data because If somebody in a sanctioned country then accesses that data, that’s a liability for us. So there’s all sorts of things that make this quite difficult. And so we then have to of gate the access, and contracts have to be signed even when it’s free, which is frustrating because I don’t really care about the legal stuff. Least somebody here does. Otherwise, we would have been sued by now, I’m sure. So that becomes tricky. How do we distribute data in a way that means we don’t? Have that there’s no legal liability for doing so, and also that we don’t end up killing our business model as a result. And so we kind of distribute data on Zenodo, which is not ideal. I I tried to upload some data to Zenodo last week for a paper that I’m writing. It took me all day to do because the server was so slow. So that was really suboptimal. But look, there’s There’s that, and then there’s making our data available on Google Earth Engine for researchers as well so that we can of cut out that whole horrible step. But then you’re kind of locked into a platform, so is that necessarily the right approach? We’re trying to just do as many things as we can. Another aspect is making our data available on Esri as well as through Google Earth Engine. So you can kind of access a sort of limited set of data for free, even for commercial use in some cases, so that you can kind of get a taste and see what’s useful and see how it works and build some project types on it. And then come to us when you need to go that kind of last mile and have more climate information or loss information or whatever it might be. It’s kind of like the freemium model of just give people something, a little nibble and then sell them the whole meal kind of thing. So that’s kind of the commercial strategy there. And then the Oasis loss modeling framework is a really interesting. And there’s nothing particularly new to that, really. That’s been going a little bit longer than Fathom has. I think it was 2012, maybe a little bit earlier that that was set up. And that was in response to a lot of the problems in the insurance industry that we’ve sort of been highlighting, where you have two major vendors of catastrophe risk information.

And because they provide the models and the platforms on which you access those models, and they’re hugely complex pieces of software, once you license them, you’re in, you’re locked in. And they’ll probably hike the prices the next year because they know you’re locked in. And that creates a real problem for consuming alternative views of risk, not necessarily multiple views of risk, but yes, multiple views of risk would be nice in these platforms, but really just.

If somebody truly believed that we had a better model than somebody else’s, that doesn’t really matter because the cost of accessing it via a whole different platform and onboarding that is just too massive. So that’s what the Oasis loss modeling framework set up to resolve to create like a common language for cat models to be deployed within and a common set of open data standards for formatting your exposure data. So essentially what buildings am I interested in, or what buildings do I insure, and under what policy terms, and so on. Doing that in a consistent way. Interacting with a model framework that’s vendor agnostic, that’s peril agnostic. There’s everything agnostic. Everything is completely generalized. It’s not like a flood model framework. You provide an intensity value. There could be any unit. You provide an event ID, which could be a flood event or a hurricane doesn’t matter. So it’s all completely interoperable. And then you get your results in a standardized format, and then it will will plug into those workflows. And that’s been a very noble idea. And it’s been completely fundamental to the way that we distribute our models, you know, we we cannot develop our own framework, our own platform, because nobody would, nobody wants a platform just for flood models, it’s just too much of an overhead. So that’s where all of these non-duopoly model vendors play on the Oasis loss modeling framework. And it’s kind of funded by the industry to stimulate greater choice and so on. But it is, you know, it’s had mixed success, shall we say. You know, it isn’t the standard. There are still people using proprietary data formats that don’t play nicely with all these other models. And one could maybe argue deliberately.

I wouldn’t dare be so cynical. And so that’s a fundamental barrier to adoption of more sophisticated tools that you’re just tied in to the two vendors that provide this information. Now, that is starting to change a bit. The two main vendors of catastrophe risk models to the insurance industry and increasingly other sectors as well, are RMS, which is now owned by Moody’s, the credit rating agency, and and what used to be called AIR, which is now owned by Verisk. Those are the two big players. And both of those are now opening up their platforms to Oasis loss modeling framework based models, which could be the game changer we’ve been waiting for. So that alongside your traditional view that you get from the big players, you can also consume any other views without any of those interoperability questions rearing their head as they often do.

And so to kind of stimulate that revolution somewhat, we’ve made our UK flood model completely free on that platform or any of those platforms. So long as you’re consuming it within one of these software as a service type platforms that leverages the Oasis loss modeling framework, even as a company, you can have right business using our UK flood model for free which is a bit of a commercial risk. But we like taking risks. It’s quite good fun. And I think from a commercial perspective, it’s again trying to sort of show people what we have, get people using the data, understanding it, figuring out what they like about it, and then hopefully then selling them other regions or more functionality or whatever that might be. But for now, it’s had pretty interesting response that essentially you can use our flood model to write business in the UK for free and just remove that one last excuse for why you then wouldn’t diversify your views of risk. So early days, we only announced that last month. So I’m personally and professionally fascinated to see where it all goes.

Nissim Lebovits: Yeah, it’s I the the willingness to consolidate on a spec or a standard is it’s fascinating. And when we were when we did the CNG event, the Cloud Native Geospatial Event in London, one of Jed’s colleagues was talking about you know S3 as a utility and and comparing it to USB C and talking about how they’re imperfect formats, if you want to use the word format.

But they both are they’re sort of things that the industry around them have converged upon because generally aligning on this opens up more opportunity than if everybody is playing with their own format, unless you’re Apple, and then you need the EU to step in and sort of go and twist your arm. But yeah, it’s and it it’s interesting to hear how you guys are are sort of using the prospect of of open data to motivate greater interoperability.

Which in turn should I guess open up more more commercial opportunities. That’s that’s a fascinating approach.

Oliver Wing: Well, open data is a tricky thing for a business to figure out how to commercialize in many ways. You know, the sort of, the natural consequence of us choosing not to be part of a university 10, 15 years ago and deciding to build a business because that was the best way of getting money to do the research that we wanted to do. The consequence of that is that even though we’re creating data that should be a public good and should be properly funded by governments and… Democratized, if that’s even the right word, but safely in some way. We can’t make that work. The economics don’t make sense. Where would we get our money from to actually fund all of this work so that we can pay our bills? It’s very difficult for us to do. So even though everybody likes to talk about openness and transparency, there’s always a limit to it. We can’t fully open from you know, fundamental principles, the repositories that build all of this information, because it just kills the business model that actually allows us to create these things. But kind of ideologically, if that’s the right word, it’s something that we want to do more of, because it’s very aligned with our mission. And I think done carefully and deliberately, it can actually make a lot of commercial sense to kind of, you know give people a drip feed them a little bit of free stuff and see if they want the whole thing. But like I say, we’re kind of in an experimental phase of doing that at the minute, is interesting because probably in the circles in which you move and maybe many of the listeners move, open data is just the norm. Everything should be free or everything is free that you would use. And you kind of view proprietary data with contempt, perhaps.

And yet in my world, the fact that we even write down on a piece of paper and share it with the world how we built the thing is a novel and new idea, let alone making it freely accessible and findable and available with coherent standards on metadata and put in the right repositories and so on. There are areas of… Of the commercial world in which we move in which that’s just, yeah, this is all very groundbreaking even though perhaps for those that live this sort of topic every day, it might seem meh.

Nissim Lebovits: You’re you’re reminding me of a paper that Jed likes to reference called I think it’s called Why Should Data Be Free? Don’t you get what you pay for? Which I it’s a r it’s, I mean, yeah, fair point, right? Like I think a lot of us in the open source world come from this open data world, this very idealistic conception of getting this stuff up there, but then you actually do the work. There there’s a there’s a great book called Working in Public that I actually just finished rereading this weekend that’s about open source software.

And this whole the sort of contrast between the ideal that we sort of had in the early internet days of like all these people coming together and hacking out of the like, you know, just out of pure passion for the work and coming together and it being really distributed and collaborative versus the reality of open source software, which is often like one or two maintainers kind of heroically holding up these libraries that support half of the internet. And how the the ideal of openness.

Actually, it makes it very hard to produce a lot of this stuff precisely because somebody has to pay your bills. Like and and also because often when you scale, like the more success you have, it it almost becomes perverse. That like, okay, great, you’ve got a million people who are using this Python package. Now you have a whole bunch of bug reports and stuff that you have to respond to, and you’re not getting like more compensation for it. So that the difficulty of maintenance scales up rapidly.

But you don’t get more daylight hours or more compensation. And so there are these, it is really hard to think about this. And so I think that’s a big part of the motivation behind projects like Source Cooperative, which are trying to think about new ways to build institutions. You know, Source is meant to be literally a cooperative, and it’s meant to be a new way of thinking about how we build institutions that can fund and support open data or alternative models for openness of data. And thinking of open data, I think, is less of a less of a binary and more of a continuum in terms of of forms of openness. But yeah, this it it’s you know, open data is hard and doing it right is is really hard and it’s expensive and it’s time consuming. And other people make a lot of money off of it. So at least a little bit of that should trickle down to the people who are making the the data products to begin with. There’s another question about file formats in in distributions. You know, we’re we’re coming from the cloud native geospatial world.

You’ve told me before that you guys have an API set up and you have some stuff on, I think it’s on S3 or Google Cloud. Are you guys thinking about maybe this is too too data engineering a question for for you? Maybe you’re more on the product side, but are you guys thinking about cloud optimization and file formats and that kind of thing in distribution? What are the what are the ways that people are asking for the data? Is it primarily through API? Do people want you to like ship them a thumb drive with, you know, 10 terabytes of data?

Oliver Wing: Yes. Well, this will partly show up my lack of credentials as a data engineer, but I can definitely speak to some of that. Mean, that we do is, OK, most of the things that we do are cloud-optimized GeoTIFFs. What a beautiful thing they are. And they serve all the purposes that we require of standardized geospatial data right now, because everything comes from the elevation data. And if the elevation data is a grid with some numbers that tell you how high the land is, then our flood model is going to be a grid with some numbers that say how deep the water is on that elevation grid. So it’s all kind of part of the same thing. The vast majority of our revenue comes from people downloading the cloud optimized GeoTIFFs from S3 directly.

And whether that’s downloading or whether it’s a file transfer or an object transfer within the S3, I couldn’t comment. But in essence, they want the data in all of its glorious messiness directly, for the most part. And that’s partly a function of our own biases in terms of the customers that we have. We’re still only a smallish company. And so we’re targeting the kind of tier one players in each space. And these are the teams with are these the businesses with big teams capable of consuming that data and capable of paying for it? So there’s potentially a tier two and three market that we’re not able to access directly in the same way because we’re just using cloud object storage. And you either can access it through downloading it or what have you, or via an API. So we do have an API that, again, connects very easily to Cloud-optimized GeoTIFF. So there’s nothing particularly revolutionary about it. Most people have those sorts of things. And it just means that you can, if you don’t want the whole data set, often for sort of quoting services in the insurance sector, it’s quite popular. You have it less so in the States, actually, don’t you? You still go through agents and brokers and so on. It all seems very archaic to me. But here in the UK, anyway, you just go online and get, you know, 100 quotes for your home insurance policy at the same time on a comparison website. And so that will often involve pinging thousands of APIs to get a really quick response conditional on all of the things that you’ve input. And so we feed into that when it comes to flood risk scoring, for instance. So that’s the kind of architectural side. But beyond that, I kind of deliberately have not much more to say because we then plug into all these other platforms where lots of companies will then go instead to access our insights. So Swiss Re have their own platforms, CatNet and risk data and services for corporates and things like that, which are just geospatial data platforms that do all of that heavy work for us. So you can visualize it in a web browser, run all your analytics within that platform. And the same with, you know, other vendors like MSCI or Climate X or Bloomberg. It’s been a very deliberate strategy to just spread our data in as many places as possible, particularly if somebody else has done all of that labor intensive and difficult kind of customer success and support stuff for actually building the interface between the data and the client. And it works for us because we can just give them the S3 credentials and get back on with building more models.

Nissim Lebovits: Well, I I think that was actually a like great answer. You you did your data engineering credentials proud. Yeah, and I and I think, you know, on the one hand, it’s interesting because it sort of points to the fact that a lot of people need to maybe that we need to find ways to get more people on board with the ease of object storage so that they’re not like downloading a five gigabyte TIFF that they could just stream, you know, a little chunk of. But on the other hand, it speaks to the flexibility of S3 for people to just easily build stuff on top of, even if it’s crappy stuff. That’s that’s their prerogative so that’s a that’s a good argument in favor of it. And I also suspect that a lot of, you know, as we see more and more companies and nonprofits and other institutions that are built on the premise of some kind of global data product, whether that be you know, flooding or building footprints or other things, S3 is going to be an essential part of that because most users are not going to want, you know, the entire product for the entire globe every time. They’re going to want little bits. And so it’s interesting to think about the kind of I I would hope virtuous cycle that that sort of creates of like S3 makes it possible to build global products like this and provide them. And then the fact that they’re out there makes more people, you know, interested in the power of of of object storage.

Oliver Wing: Absolutely. Mean, in the old days where things weren’t and didn’t need to be particularly scalable when we were young and had few clients, we used, we might have used Box or something way back when. Whereas literally here’s a link to the files and you download them. And a lot of our work with state and federal governments in the US, we’ve literally needed to pack a hard drive in our hand luggage and fly it over there.

But that happens less frequently than it used to, but still does sometimes. So yeah, come a long way. And S3 is just a remarkable thing, really.

Nissim Lebovits: Okay, so I’ve I’ve stolen an hour and forty minutes of your time at this point, which I really appreciate. I want to ask you one last question and then we can we can wrap on this. I’ll ask you the question that I kind of initially tried to reel you in with, which is what what makes for good enough data, in your opinion? You know, I I think this is sort of a similar concept to Jed’s idea of a great data product, but we we’ve talked a bit about fit for purpose, adequate. So so when you think of of good enough data in the flood world, what does that look like?

Oliver Wing: Yes, and even though I led with it, I’m unfortunately not going to have a clear answer to it because it totally depends on the use case. And you always have to work back from the use case. And it depends really how much the model is used to make the decision. I still haven’t fully figured out how much does the model actually determine the insurance premium or how much is then, you know a little bit extra or a little bit less to account for uncertainty and then administrative costs and capital loading and everything else that goes on top of it. So ultimately, what makes a model good enough is whether it helps you to make a better decision than you otherwise would have done. And that’s pretty hard to quantify. We have the benefit of being part of Swiss Re where there’s enough data kicking around for you to be able to prove something.

So because they are, you know, kind of by proxy, a model vendor, as well as a risk carrier, we can basically go and use our model and reprice, you know, what if you’d have priced it using our data instead, and go look at some past events and past years. And then you can demonstrate that, you know, and have demonstrated, you know, that your profitability goes up and your risk of solvency goes down.

All the arrows move in the right direction. So does that mean it’s good enough? Or actually the fact that you still made profit last year and remained solvent, was that good enough? It really depends on what is the business outcome that you’re trying to achieve and for what use case. A very, very frustrating answer to an unanswerable question, really. And so we try and provide a sort of I don’t know, maybe a playbook for providing good enough data. We try and set the standard for disclosure of model components and methodologies and the uncertainties associated with them and some guides on how you would interpret uncertainty for different sorts of applications and sort of paradoxically continue to improve the models, even though I personally am quite unsure as to how much better they need to be, as we’ve talked about the real limitations are in how humans are interacting with the models rather than the models themselves. We’ve got a really exciting project related to SWOT, the Surface, Water, and Ocean Topography mission from NASA and the French Space Agency. And the UK Space Agency loves to pin their name to it as well, but they did not fund it very much at all. But anyway, it’s meant to be this three-way thing. And that’s going to provide an incredible array of hydrometric altimetry observations water levels all over the world, which is amazing. Know, a billion dollar project that will have all sorts of benefits far beyond, you know, mere flood modeling. And that will allow us to first and foremost be able to better represent channel bathymetries in our model, which sounds and potentially even is as kind of, you know, esoteric as it may be. You know, this is a component of a model that will be improved by a billion-dollar satellite mission potentially we’ll be able to make more accurate predictions as a result of it. At the minute, we rely on pretty sweeping parameterizations of the shape of rivers. And we assume that they can hold a certain frequency of flow, but also it kind of works. It validates well enough for now. It produces extreme event scenarios that are explainable and seem plausible enough and customers that… Use our data, have seen business outcomes be achieved that otherwise wouldn’t have been achieved. So what is the justification for continuing investment and continue refinement of these tools? Or should we actually just start focusing on, let’s just draw a line, stop all the modeling, and just figure out how to make best use of these tools? Now, I can’t say that that’s ever really going to happen. But if you were to sort of burn everything down and start again would I think maybe a more sort of service oriented or consultancy focused type of approach where you’re actually just working really intimately with the users of these data sets being the biggest priority. Whether that’s a scalable commercial proposition is a different question. Sitting here right now, that probably is the most useful thing that an end user could have is an expert telling them how to use the model rather than making a five-meter resolution one next year instead. That’s my personal view.

Nissim Lebovits: Well, don’t talk yourself out of a business. But yeah, I mean it it is it’s the wicked problem in data, right? That like and and sometimes I feel a little uncomfortable with my own position in this. Like tech is easy, right? Crunching numbers is easy. The really hard work is is getting the policies passed and getting people to shift how they make decisions. And that’s Yeah, that’s that’s hard. And unfortunately it does not scale or get funded very easily. So it’s tough work. Well thank you so much, Ollie. This has been a lot of fun. Really glad we finally got to do this. Any last thoughts before we wrap it up?

Oliver Wing: Yeah, exactly.

No, I think I’ve said enough incriminating things over the past hour and 45, so best hold my tongue.

Nissim Lebovits: Okay, cool. Well, pleasure talking with you. Thanks for for making the time.

Oliver Wing: Thanks a lot, and speak soon.

Featuring:

Oliver Wing Nissim Lebovits