You get on seahorses by swimming up and right-clicking on them. You can make them move by singing - sing in the direction you want the horse to move. It's harder to control but moves much faster.
If you get stuck to a wall, just click away from the wall and you'll push off.
I don't remember how to set markers. I had trouble with that at first and just decided to ignore that feature. Once I finally figured it out, I found that I didn't need them, and never actually consulted the ones I set.
Your list of songs will grow throughout the game. The window that shows you your songs will only show the ones you've obtained thus far.
The shield reduces damage you take from physical attacks (I think). However, its most useful function is to repel projectile attacks, most of which bounce off it harmlessly.
The bind song is found in the verse cave, which is located somewhere south of your home. You need to figure out how to open a door in the cave to get it. The clue is located elsewhere in the cave. Hint: it involves singing.
Once you get the bind song, you'll be able to move rocks, get energy form, and by that time you should hopefully be well on your way.