Don't worry. This situation is common.
Your machine is an ATX one.
When you shutdown Windows, it turns off, but it stands by.
This means that you need to disconnect it from the power supply or push the on/off button on the power supply, to turn it off.

You can see the leds staying on on some keyboards. That's absolutely natural. It's common.


------------------
"Matter is passive. In spite of its power, it can't be controlled without the human mind."
Sokrates