So I got this reply from one of the developers. I think it's ok to share here, as it casts light on the situation.
Hi Orri!
Sorry! Right now the iOS app is behind our Android one in terms of features, so it can only control videos that are
already playing! I can see how that'd be confusing if you have both apps.
We're working on updating the iOS app, but with our small team it could be a while before it catches up with the
Android one!
Cheers!
Michael