3

I'm working with image super-resolution on terrain height data. Currently, I'm cutting the input data into smaller pieces (20 x 20 rather than 10800 x 10800).

After the upscale 20x20 -> 40x40, piecing the resulting images together causes artifacts along the borders of the smaller pieces. This is because the outputs of the model don't flow well together (we are using the output to run simulations, so it's important that they fit seamlessly).

Is there a recommended way of stitching these smaller images together? Or is there a better approach to this problem than using these smaller cut-down images?

jrdzha
  • 49
  • 1
  • Can you describe in more detail the method you're using to do the super resolution? Is it supervised machine learning? If so, what are the training data? – bogovicj Jan 22 '20 at 12:11
  • Yes this is supervised machine learning using satellite DTM (digital terrain model) data. I'm using high resolution DTM (say 30m x 30m per pixel resolution), and downsampling (say 90m x 90m per pixel resolution) to produce the training set. The goal is to be able to upscale other 90m x 90m per pixel resolution DTM where the higher resolution doesn't exist. – jrdzha Jan 22 '20 at 17:00

1 Answers1

1

You're probably seeing those artifacts because your model doesn't see those pixels immediately outside your tile and so can't know how to "blend" things. (I'm assuming your tiles have a stride equal to the input size)

A typical approach I've seen used (and used myself) is, at inference time, to keep only a central portion of each tile and then to have overlapping windows so you can "fill the space".

The more overlap (and less of the output you use) the less apparent the artifacts will be (or have been in my experience), but more computation is needed.

Here's what I mean in cartoon form:

Current situation(as I assume):

input tile1 #####----------

input tile2 -----#####-----

output tile1 ##########--------------------

output tile2 ----------##########----------

My suggestion:

Here, for the output # means the value there is copied to the output, and * means that the model makes a prediction there but it's not used.

input tile1 #####----------

input tile2 ---#####-------

output tile1 **######**--------------------

output tile2 ------**######**--------------

If you go with this idea, you could even simplify the model so that it doesn't attempt to make predictions at locations you would ignore anyway.

And please comment if things need clarification.

Community
  • 1
bogovicj
  • 699
  • 3
  • 11