The tl;dr for this is: yes, you can just upgrade Prysm by stopping your services, copying the new binaries and then restarting your services.
I had to do this recently as the Altair fork required all clients to be running V2.
Unfortunately, the official docs on upgrading only provide instructions based on three different ways of installing: Prysm.sh, Docker and Bazel. I did not set up Prysm via any of these routes. As I mentioned in my original article, I followed Somer Esat’s instructions, which involved simply downloading binaries and copying them to /usr/local/bin.