I bought the solder mask on ebay. This is a little guide:

1. Apply the solder mask on the PCB. I put some paint on to the PCB and spread it equally over the whole PCB. A better method is to use screen printing.
2. Dry it in an oven at 150 °C for 7.5 minutes. Don't use the oven for food afterwards.
3. Mount the mask on the PCB and expose to UV light for about 3 minutes.
4. Clean the PCB with Aceton or something equal.
5. Expose the PCB to UV light again to harden the solder mask for about 5 minutes.

Note: My result wasn't very good, because the paint wasn't spread equally. The time in step 3 was too short. At the area exposed to UV light, it wasn't hard enough.

When you use screen printing to spread the paint and adapt the time in step 3, you could get a quiet good result.