To manually delete the ‘Refresh’ button, click on the VBA Design mode button.
Click on Excel Options
Click on Customize, Developer Tab, Design Mode and then Add
Now click once on the Design Mode icon at the top (highlighted below), then move cursor and click once on the 'refresh button' and hit delete. Then click on the Design Mode button again.