The answer is that you can not go backwards in operating systems. HTC purposely made sure people can not go from ICS to gb or any similar downgrade.
However, with the recently discovered juopunut bear S-off procedure, s-off can be accomplished.
on a phone with s-off, one can ruu the operating system to previous versions. So, with s-off one should be able to downgrade from ICS to GB.