I had: serial.serialutil.SerialException: could not open port 'COM4': OSError(22, 'A device which does not exist was specified.', None, 433)

COM4 was show on my device manager. I booted several times, and still had problems with the serial port. I used PUTTY and I was able to use COM4, and see the output from the Arduino, but still got the error above.

I fixed my problem by opening the serial monitor, and then closing it.