If I understand, it is because the user does not have sufficient privileges and that the privileges should be somehow given during the installation process of the application? (if that is right, how can I test it meantime?)
Yes, correct. You can use TA_USER instead of TA_SYSTEM if you can't get or don't want to get the user permission.
Am i obliged to have some kind of installation to somehow get the privileges working right for operation. Hope this makes sense?
Only if you want to store the "trial data" and "activation data" (with CheckAndSavePKey) on a system level rather than a user level. It's entirely up to you.
Also do you have detailed documentation for the mac and xojo. I don't see anything like what apparently exists for Windows..
Yes, see: Using TurboActivate with Xojo (Real Studio / Real Basic).