Use a flat-head screwdriver or pry bar to remove loose pavers.
Add coarse sand in the spot where the brick used to be.
Tamp down the sand and replace the pavers. Make sure the brick is level.
Once all pavers are in place, put some sand on top of the brick and sweep it to help further lock the brick in place (Images 1 and 2).