Windows keeps the last few background pictures in a history list on the Personalization > Background page, and Settings does not offer a Delete button for them. You have two options:
Option 1 - push the unwanted image out of the history. The Background page shows the 5 most recently used pictures. Set 5 other pictures as your background one after another, and the unwanted one will drop off the list on its own.
Option 2 - clear the history with a registry edit. This removes all remembered pictures at once:
- Press Windows key + R, type regedit, and press Enter.
- Go to HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Wallpapers
- Delete the values named BackgroundHistoryPath0 through BackgroundHistoryPath4.
- Close Registry Editor, then open Settings > Personalization > Background again. The history thumbnails will be gone, while your current wallpaper stays as it is.
Note that this only removes the thumbnails from the picker. If you also want to delete the actual image file, delete it from wherever it is saved, such as your Pictures folder. Be careful in Registry Editor and only delete the values listed above.