Flashing and Programming
On a hosted system, "running the program" means handing a file to a loader. Here it means writing your image into non-volatile memory inside the chip and then resetting it, using a second piece of hardware that talks to the silicon over a two-wire debug port. That second piece of hardware is doing considerably more than copying bytes: it halts the core, drives the flash controller through a sequence the reference manual specifies, verifies, and releases reset.
SWD, JTAG, and GDB
The thing that surprises people about on-chip debugging is how little of it is software. GDB does not single-step your firmware; the processor single-steps it, because Armv7-M defines a halting-debug mode with its own registers, comparator hardware for breakpoints, and comparator hardware for watchpoints. GDB is a client. OpenOCD is a translator. The debug capability is silicon that was on the die before you wrote anything, and it is finite in a way that software breakpoints on a hosted system are not.