Move cmake guards to a simple cmake conditional

This pipeline shows that this finally works, there are other errors in the other pipeleines unrelated to this.