First, think about how it satisfies your needs and the problem it resolves. Then consider the cost, integration potential, customer service, and frequency of software updates.
For me, outside of the obvious answers, the most important factor is scalability. I don't want to end up in a situation where a SaaS isn't able to tolerate my required workload anymore. This causes a lot of unnecessary stress and time to quickly switching to a different solution.
I am actually launching my own SaaS right now on PH: https://producthunt.com/posts/ezml. Iwould really appreciate the PH community's feedback on it. Especially with relation to the factors stated in this discussion :)
FirstHR