You have to ask yourself, does your site have a lot of content that you can actually benefit from a dynamically driven backend, rather than static HTML?
In terms of ease, for example you stating you find HTML is easy to code, a CMS is not a replacement for HTML so that's not directly comparable. To create a design with WP as the backend, you still have to write out the HTML/CSS, then cut up the code to fit WP. This step isn't bypassed because you're using WP. It's an extra step.
I am web developer. I work with diffent CMS and shopping carts (Joomla, Magento, WordPress, Drupal, PrestaShop, Zen Cart, osCommerce, VirtueMart and HikaShop(Joomla eCommerce extensions), TomatoCart). I want to say if you are going to use CMS you need to know php, even if you are going to use WordPress. I prefer CMS websites, because it give more features for their owners.