A Pox on Both Your Houses--Where to Get COPDEM


(post updated, most recently 30 January)

The Copernicus DEM (CopDEM) is the best of the free global one arc second DEMs (Guth & Geoffroy, 2001; Bielski and others, 2003).  FABDEM and Diluvium DEM are beyond the scope of this discussion, but I think there are caveats about what they do well and how they degrade CopDEM for some applications..

So where you do get CopDEM, and if the supplier changes the DEM is it still CopDEM or should users be warned about the changes?  The are in fact changes in some sources for CopDEM, some of which are probably benign, and some of which are potentially serious.

Excuse me if this gets technical, but you need to understand what is being transformed before you get CopDEM. If you want just the conclusion, use the cookie cutter CopDEM exports from OpenTopography with caution, and it should have a clear indication that it changes elevations.

IMPETUS for this WORK

A colleague, Sebastiano Treviasani, reported a shift in CopDEM downloaded from OpenTopography compared with CopDEM obtained from ESA (FIgure 1), which did not occur in ALOS.  ALOS is pixel-is-area, which is a fundamental encoding difference from CopDEM.  The colleague uses ArcGIS and R map packages, while my work below (Figure 2) uses MICRODEM, so we are convinced the problem is with the data and not the software.

Figure 1.  Difference map for a region in the Italian Alps for CopDEM from ESA versus Open Topography.  Map from Sebastiano Trevisani.


METHOD & RESULTS

I will consider two sources from COPDEM from which I obtained four DEMs covering one tile (N00E009).

  • Creodias (https://explore.creodias.eu/search).  This is the official source, and while it has improved dramatically over time, the system is still not ideal.  For each tile you get 5 folders, 19 files, and 105 MB of data when unzipped.  The actual DEM is 50.6 MB, so the metadata about doubles the size of the download.  For a reason I could not figure out, the system provides two versions, but it is not easy to determine why there are two versions and what might be different about them.  There are different versions of CopDEM (but only some tiles), and tracking down that metadata is not easy.
  • OpenTopography (https://opentopography.org/ ) has two ways to download OpenTopography data.  Both provide a single geotiff, with no tile metadata, but a smaller download and an arguarly simpler user interface.
    • The cookie cutter method.  I entered the four corners for the tile. The geotiff is about 28 MB.  File names do not reflect the data set, and are the same for every download of a single data series.
    • The batch download. The geotiff is about 26 MB.
Differences among the files:
  • Changes that should not affect users (Table 1)
    • Compression.  Creodias does not compress within the Geotfff, but OpenTopography does.  This should have no effect on users.
    • Removing overlap points.  CopDEM inherits the DTED format, which repeats columns and rows between adjacent cells, leading to 3601x3601 pixels in a tile.  OpenTopography removes the overlap pixels, shown by the 3600x3600 size of the cell.  This should not affect users.
    • Clipping out just the land part of the tile, which OpenTopography does for the cookie cutter methods.
    • Stating the vertical datum (EGM2008) in the metadata for the geotiff.  OpenTopography does not have this in its downloads.
  • Changes that do affect users, perhaps seriously 
    • Change the pixel representation from pixel-is-point to pixel-is-area.  If the corner coordinates are shifted by 1/2 pixel (as is done for ASTER), this would not shift the pixels and the DEMs would line up perfectly.  This was not done for the cookie cutter exports from OpenTopography, with the result that there is a pixel shift.  I am not not sure how or why this happens, but a colleague using different software confirms that he also sees it.  This is effectively a reinterpolation of the DEM, and 25% of the point are more than a meter high, and 25% are a meter low, with a clear relationship with aspect (points shift up or down the slope).
    • Changing the horizonal (longitudinal spacing) to one arc sec for all latitudes.  While that does not affect this tile, it does affect other tiles and will change things like slope.  It is unclear what impact this will have, but it will result in changing characteristics of derived characteristics computed from the DEM.

Table 1. Characteristics of the four DEMs

DEM PIXEL_IS NUM_COL NUM_ROW
Creodias_1_Copernicus_DSM_10_N00_00_E009_00_DEM Point 3601 3601
Creodias_2_Copernicus_DSM_10_N00_00_E009_00_DEM Point 3601 3601
OT_cookie_cutter_output_COP30 Area 2513 3600
OT_ batch download Copernicus_DSM_COG_10_N00_00_E009_00_DEM Point 3600 3600


Table 2 shows a correlation matrix between the four DEMs.  Three correlate perfectly; the grids overlay perfectly.  The cookie cutter DEM has a lower correlation (it's high but DEMs, even the very low quality ASTER, always have high correlations).  Computing the correlations requires interpolation because the cookie cutter export from OpenTopography does not line up with the other three DEMs.


Table 2.  Corrrelation matrix for the four DEMs.

DEM/GridCreodias 1 Creodias 2 OT cookie cutter OT Batch download
Creodias 1 Copernicus DSM 10 N00 00 E009 00 DEM1.0000001.0000000.9991421.000000
Creodias 2 Copernicus DSM 10 N00 00 E009 00 DEM1.0000001.0000000.9991421.000000
OT cookie cutter output COP300.9991420.9991421.0000000.999142
OT batch download Copernicus DSM COG 10 N00 00 E009 00 DEM1.0000001.0000000.9991421.000000




Figure 2.  Difference map, original CopDEM and shifted version from OpenTopgraphy.  Differences greater than one meter highlighted in red and green; about 25% of the points in the DEM appear in each color.

Recommendation


Any redistribution of a global DEM, like CopDEM, should give "identical" results compared to the official release.  This could be tested with map algebra and difference grid between the two DEMs, perhaps allowing  some tolerance for floating point arithmetic.  I suggest that something like 1 cm would be appropriate.  In this case, the "CopDEM" from the cookie cutter output of OpenTopography has 50% of the pixels failing a 1 m tolerance, so it should be advertised as a modification of CopDEM, with metadata explaining what was changed and why.

What Happened

Sebastiano Trevisani suggested that OpenTopography used nearest neighbor interpolation, which prompted me to make the  maps below.  Note that the cookie cutter DEM from OpenTopography has all grid postings shifted a half second to the west compared to their correct placement.  Whether this is in fact from interpolation, or from an inadvertent shift in the grid's x corner coordinate, will be up to the bug hunters to figure out from the code.


Figure 3.  Mismatch between the grids for CopDEM from ESA (right) and cookie cutter Open Topography (left) which is offset 1/2 pixel to the west.  




I will update this if I get an explanation from OpenTopography.  In the meantime I recommend getting CopDEM from Creodias.


References

  • Bielski, C.; López-Vázquez, C.;  Grohmann, C.H.; Guth. P.L.; and the TMSG DEMIX Working Group, 2023. DEMIX Method Ranks COPDEM, and FABDEM as Top 1” Global DEMs: https://arxiv.org/abs/2302.08425v5   (current version 5; version 6 should be in press "soon")
  • Guth, P. L., & Geoffroy, T. M. (2021). LiDAR point cloud and ICESat-2 evaluation of 1 second global digital elevation models: Copernicus wins. Transactions in GIS, 25, 2245– 2261. https://doi.org/10.1111/tgis.12825 



Comments

Post a Comment

Popular posts from this blog

Connect the data points or not?

Image
  This graph shows 6 terrain parameters, which are only very indirectly related, plotted against the FUV (fraction of unexplained variance, 1-r squared).  It shows one test DEM (satellite radar derived) versus a reference DTM (lidar derived).  Low value is best, so ELEV (elevation) agrees very closely, whereas TPI does not. Because they are not related, some of my colleagues prefer the version on the bottom. I prefer the one on top because: With the lines, it is clear that the urban category is the best (largest FUV).  With the points, it is much less obvious. Despite the attempt to use both colors and different symbols, it is very hard to see all the values when they are close (say with RUFF, roughness).  With the lines, that is clear. Where lines cross, the relative rankings change and it's clear (say shrubs, between HILL, hillshade, and SLOPE). It might not be clear that all the categories are in fact present for the ELEV. As long at the graph is explained, I...
Read more

3D view of La Palma

Image
  This shows the Sentinel-2 image overlaid on the Copernicus DEM. The ash clouds and clouds are plastered on the terrain, since the program has no information of their altitudes and only has the DEM.
Read more

La Palma Eruptions viewed with MICRODEM

Image
A few years ago we spent spring break in the Canary Islands.  We were going to go to La Palma, but instead if Santa Cruz de la Palma, we wound up at Las Palmas to Gran Canaria.  We would up having a great time, but I don't have any pre-eruption pictures to go with the recent activity. Figure 1 shows the Sentinel-2 image from 26 August 2021, before the resumption of volcanic activity.  This is false color; the red is really near infrared (NIR) which is not the standard thermal imagery, but just light we cannot see and which vegetation reflects strongly.  This has 10 m pixels. Figure 2 shows the Landsat 8 image from September 26, showing the same color scheme but with pan sharpening to achieve 15 m pixels. There is ash trailing to the NE, and a lava flow going down the hill to west, with smoke or steam rising.  The lava flow itself is a thin red line, which is NIR energy which will be a mixture of reflected sunlight and heat emitted by the lava witch might be at 8...
Read more