Borderless fullscreen windowed mode can be used by setting ScreenMode = 2 in eduke32.cfg. I think it should be selectable as a third mode with the Fullscreen menu option. "No", "Yes" and "Borderless". A lot of recent PC games have that option in their menus. It's nice for quick alt tabbing and you get Window's triple buffered Vsync which can offer less display lag.
