1
Click your Start Button, type regedit and hit Enter to open the Registry Editor.
Click View and make sure 'Address Bar' is turned on.
Paste this into the Address Bar at the top and hit Enter.
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Wallpapers
On the right hand side you will see a list with entries named BackgroundHistoryPathX
Double click the entry for the file you want to remove.
Note down the file location of the image file and close the dialog.
Then right click and Delete the entry.
2
If you want to delete the actual image file, in File Explorer, go to the folder you noted down and delete the file form there.
3
In File Explorer, paste this into the Address bar and press Enter.
%AppData%\Microsoft\Windows\Themes
In the resulting folder, delete the TranscodedWallpaper file and also the CachedFiles folder.
4
Restart (not shut down) your PC to apply the change, that should remove all trace of that image from your PC.