It's not true that the population is shrinking, in fact it's growing daily and the game is running smoothly. Negative nancies will always critique a game in beta as if it's already released.
That being said, RvR and castle siege are both really diverse and dynamic. This game has serious potential, and the GM team is working hard for that kind of change. Relax play and enjoy.
WHUT?! Do you know what you're saying?
As Sehlyet said, the ACTIVE (or used to be so) population is shrinking daily. Yes there are new players coming, trying the game but the issue here is that the old players have no interesting things to do while waiting for the next big patch or some new events or whatever that makes them still hooked-up to the game. Well, my friendlist is always full of offline players most of them are levels 40-42 who don't go online for about a week and counting, not mentioning my inactive guildmates and my friends' guildmates...there are so many of them.
And as you've said that this game has serious potential, I agree on that. That's why I'm still playing, hoping things will soon get better, before the server become a ghost town (or continent) even before the official release date.