Why the GBCHawk colors look so weird? It doesn't look bright and colorful like VBA.
Is there any reason why the encodes and screenshots look so dark with an option to change that?
There isn't a 'best' palette.
Reason is that if you play a GBC game on a GBC, there is a relationship between the colours as chosen by the software and the colours as perceived by our eyes, due to how the pixel elements react to the chosen software colour, the quality of the screen, the lighting, etc.
If you play that same GBC game on a GBA, or a GBA SP, or an emulator, or a gameboy player via your gamecube, depending on your TV, etc. Then the relationship is different even though the software colours remain the same - you'll perceive them as brighter or darker, more or less washed out, etc etc based on what you're viewing it ON.
And now we have to reproduce that sensation of viewing it in the emulator Bizhawk. Well, what relationship do we choose? We don't even know what your monitor is like (as the emulator writers), so there isn't one true objective palette that looks the most like GBC, one true objective palette that looks the most like GBA SP, etc etc. Not only do you have to pick a target, but you have to guess how to best approximate it by guessing what your monitor is like.
Looks more like the real console.
While GB color display depends heavily on ambient lighting, there's no question that the screen does NOT put out colors that look anything like the "Vivid" options in Bizhawk.