MELODIES team wins second prize at #FloodHack!

On the 16th and 17th of January, 2016 four members of MELODIES (Bethan, Conrad, Gerardo and Maik) took part in the #FloodHack hackathon organised by ECMWF and the JRC to extend the capabilities of the Global Flood Awareness System (GloFAS) and test novel technologies. As we have been developing lots of relevant new technologies in the MELODIES project, we felt we were ready for the challenge.

The day started with a series of talks about GloFAS and the challenges that the team at the JRC and ECMWF faced. GloFAS ingests forecasts generated by ECMWF and passes them through a river discharge model in order to estimate the discharge in major rivers around the world. This global model produces output on a 0.1 degree grid (approximate 10km). The discharge is typically output as a “function of flood frequency” and a once-in-five-year event is generally taken to suggest that the river will potentially breach its banks and cause a flood event.

Following the talks, we did some brainstorming around the possible things that our combined skills and experience would allow us to do. Having discounted the ideas that were too ambitious and those that were too mundane, we settled on an idea that would allow users to interact with GloFAS data in a whole new way.

In terms of global models, a 0.1 degree resolution is pretty good these days, but as a flooding extent of less than 10m2 can lead to serious infrastructural problems, it is still too coarse to really say anything about the likely pattern of flooding on a human scale. Using CoverageJSON, ncWMS, and a 120m resolution Digital Elevation Model (DEM), we decided to develop a web app that would simulate potential flooding at a finer spatial resolution than GloFAS was providing. And thus FloodIT was born!

There followed approximately 24 hours of intense discussions and coding to achieve what we set out to do.

Our plan was to simulate flooding by applying a locally computed threshold in metres to the DEM data for a given area and everything above this threshold would be marked as flooded on a map. Since we wanted the user interface to be dynamic and reactive, we had to do the thresholding and drawing directly inside the browser. The user could then change this threshold using a slider and get immediate feedback as the flooded area grew or shrank in response - as though they were flooding the area themselves. This meant that our web app had to have access to the actual DEM data values in order to use those for subsequent processing.

Given the amount of web APIs and data formats available for geospatial data, it can be a daunting task to choose the right combination. One way to get started is to characterize the data that should be served, in our case the DEM. A DEM has a rigid structure, a geospatial longitude-latitude grid where each grid cell has a data value - the height in metres. One of our requirements was that the format should be compact and easy to read from web apps, which typically means using either plain JSON or a decoding library for more efficient binary formats. Ideally, the format should also be self-describing and include all necessary metadata.

We decided to use CoverageJSON, a format that is currently being developed within MELODIES for exactly these purposes. Storing the whole DEM as a single CoverageJSON file on a server and loading it fully in our web app was not an option since the data volume would be way too high and browsers couldn’t handle that. Fortunately we had another tool at our disposal for solving that problem: ncWMS.

ncWMS is a server software that can expose data stored in netCDF-CF files as WMS image layers, supporting many projections and functionality for resampling and subsetting. The output formats are the standard image formats: PNG, JPEG, GIF, etc. How does that help us though? In a recent experiment, CoverageJSON was added as an additional output format. Even though WMS is not meant for exposing actual data there is some beauty to the simplicity of having to replace just the “format” query parameter of a map image URL from “image/png” to “application/prs.coverage+json” in order to switch from colours to data values. Lack of CoverageJSON support in WCS servers made this our preferred option.


Now that we had a way to flexibly access the DEM data in a convenient format in our web app we started implementing logic, user interface elements, and map visualization. Having JavaScript libraries already available for handling CoverageJSON data (covjson-reader and leaflet-coverage) we could save valuable time and focus on adding features and providing a smooth user experience. In particular, we were able to reuse the visualization features of leaflet-coverage by representing a flood simulation for a given threshold as a grid similar to the DEM data. Each grid cell would then be assigned to one of two categories: water or no water.

In order to link our demo with the data that was output from GloFAS, we also took advantage of recent data that was held in Rasdaman as part of the EarthServer project and queried this via the WCPS extension of WCS using JavaScript. We used “csv” as the only available textual response format (@Rasdaman devs: We’d like to have JSON please. Thanks!). The forecasted values of each of the 51 ECMWF ensemble members were taken from Rasdaman as well as the 2-, 5- and 20-year flood thresholds. From these we could use JavaScript to calculate how many ensemble members were predicting floods of each severity. The values were then used to colour the simulated ‘flood waters’ in accordance to the severity of flooding expected. We could also show on-the-fly how many ensemble members were predicting high flooding, medium, low or no flooding at all.

This gif and all of our code are available on GitHub: https://github.com/neothemachine/floodit

We completed our demo and presented our work back to the groups and a panel of judges. Everything went smoothly and we were able to demonstrate all the aspects of our demo and answer the judge’s questions. Then we waited for the judges to deliberate for about an hour to come to their final decisions. We amused ourselves by chatting and taking photos - we hadn’t really had much sleep by this stage, you can probably tell from the photos below!

Finally the winners were announced and we were delighted to be given the runner-up prize of £300 in vouchers. The winning team had not developed a working demo, but had focused their attention on a specific aspect of user need – the timeliness of response – and for that idea they were worthy winners. But ours was definitely the coolest!

Once again, to have a look at our code, you can check out Maik’s GitHub repository (though be aware of hacky code!) and there is more information on the Devpost and Hackpad pages that we used to publish and share information during the hackathon. It was a great experience and opportunity to put some of our developed tools into action. We would recommend it to anyone (five stars!).

 

 

Add new comment