Yes, this is public! =)

The "scary warning" is how the web interface works. If you notice the URL bar, you'll see you're going to a different service. With clients like Pumpa and Dianara you can just enter their address without giving any permissions to other servers.

Also, to follow someone who just followed you, you can simply go to your Followers tab and follow from there.