I have also faced this problem before while installing Redhat 8 in GUI mode.
But I had o successfully installed it the text mode.
For this, when booted from first CD ,
write
: linux text
at the boot screen.
All the steps are as normal installation. and after installtion system will be booted in GUI mode if configured to do so.
sometimes, GUI installation cause problem due to some problem in RAM, but we can avoid that error if we install it the text mode.