Posts for ProcyonSJJ

Experienced Forum User
Joined: 10/10/2013
Posts: 38
Just wanted to say, congrats on this, this is pretty impressive work. I never envisioned going as far with the 3D effect as you have, and given all the technical constraints imposed by the original hardware, your algorithm for detecting foreground from background is really impressive. I wish you the best of luck with your project, and in finding opportunities to improve it further, and congrats on all the buzz you're getting. It's well deserved.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
mudlord wrote:
All you are doing is getting pixeldata from the raw video buffer, getting pixel colors for each cube, and then rendering each cube. Sounds super simple to me.
In addition to all of rpahut's comments, you're missing the point. It's not about complexity. It's simply about the concept. Yes, it's ridiculously simple. But no one (to my knowledge, which may be woefully incomplete) has ever done it before. And judging by the general reaction to the YouTube video, this "super simple" concept has garnered a lot of attention, interest, and enthusiasm. Sometimes, it's not about the technical complexity, and merely about someone coming along and implementing an idea for the first time. The first implementation always sucks in comparison to what's ultimately possible, mine is no exception.
mudlord wrote:
And rendering each cube on its own is criminally inefficient. Modern GL offers much nicer things like instanced/batched rendering, etc. And per pixel lighting.
Congratulations, you're much smarter than me. You must deserve some kind of award. I'm just a tinkerer, I'm not a perfectionist coder. I found one way that I could implement an idea that I had in my head. If you have a better way to do it, I'd really like to know about it. That's why I started this thread, so that people with better ideas than my own could share them, not so they could be a dick about it.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
Just a quick update: I'm not dead. Been unfortunately busy with Real Life stuff, so I haven't been able to dedicate as much time to this as I'd like. Rpahut has been incredibly helpful offering modifications to the Windows code that provide for SDL and OpenGL support. Once this hurdle is complete, the rest of the work should be relatively downhill, since the existing voxel code should work in Windows with very few modifications. The real heavy lifting will just be creating a dialog to tweak all of the settings. Would love to know the following: 1) Is anyone else out there willing to help us get this off the ground faster and 2) Are there any features that people would like to see in future versions of this project? Thanks very much.
Post subject: Introducing FCE3D
Experienced Forum User
Joined: 10/10/2013
Posts: 38
It probably doesn't need much introduction to the members of this forum, but I'm making it official. I am maintaining my own fork of FCEUX with the Voxel Engine that I developed. You can find the crappy web page dedicated to it at http://home.comcast.net/~plotor/fce3d.html. Somehow the video I posted got picked up by some bloggers and went viral. The overwhelming response gave me a second wind and prompted me to get cracking on the Windows side of things. Steps to take: 1) Get OpenGL working in Windows. (Will likely be the kind of option where you must exit the emulator and restart if you toggle it.) 2) Get the existing engine running in Windows. 3) Further progress: Start playing with the background/foreground distinction and add depth layers. If anyone is interested in helping, I'd welcome to assistance. I'm currently tackling step 1.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
It's a new day, and I figured I'd attempt a rational response to Zeromus. I understand completely that a voxel engine is by no means central to the emulation of the NES, let alone any system. It's a completely "extra-curricular" (for lack of a better expression) feature for an emulator to have, and certainly does not belong in any emulator core. However, I would also argue that neither should a lua scripting interpreter be considered core or integral to an emulator. Is it cool? Yes. Is it necessary? No. So the only arbiter of what side-projects make it into an emulator are the official maintainers of the emulator itself. And then it becomes a question of how well those side-projects line up with the maintainers' priorities. In this case, a lua interpreter obviously lines up extremely well with Zeromus' priorities, and the voxel engine does not line up at all. That's fine, and I even anticipated that this might be the case. That being said, the way you (Zeromus) went about expressing that was so insulting to the point of being hurtful. Rather than recognizing the passion that I had for the idea, and recognizing the support that others were giving it, you flat out dismissed it by saying that I didn't go about in the manner that you would have preferred and that "this didn't cut the mustard," an opinion that is not shared by everyone. So you minimized the accomplishment, and added insult to injury by calling the approach I took inefficient. I never claimed to have come up with the best solution possible, and my hope was that by sharing the code publicly, it could be improved upon by others who were more knowledgeable about how to squeeze more performance out of OpenGL than I. I'm not even going to address the issue of how this feature is in no way suitable for a language meant for scripting as opposed to rendering. No license on the source code? No problem, work with me and we'll slap one on it. I'm not writing any of this to try and convince you that my work should be as equally included in FCEUX as the lua interpreter. Quite the contrary, I'm very content to maintain it on my own. The reason I'm writing this is for the unlikely chance that it actually causes you to pause and reflect on the way in which you deal with other people, particularly people whose interests and desire to contribute do not necessarily line up precisely with your interests. I doubt that you care, but each post you make creates an impression of who you are. And if I were a person who had not already made the choice to contribute something to your project, I would seriously reconsider whether I wanted to associate with you based on how you've reacted to this. And that next person might have come up with something that you _did_ appreciate. But in all likelihood, you'll simply read this, shrug, and maintain your position, which is your prerogative. I'm getting off my soapbox now. I chose FCEUX because it already contained the necessary OpenGL support and it was within my ability to write the necessary C++ code for. I wonder if I might have received a warmer reception from the authors of other emulators.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
Well, I'm about as genuinely discouraged about this as I've ever been, so... thanks for that. No, I'm not going to bother writing this in a language that serves no purpose to any other project. The whole point of this was to write an engine that would be relatively easy to port to other emulators. I've done that. Zeromus, if you want to write it in lua, go nuts, I have zero interest in doing that. I've got the code properly submitted to the branch, but I think I might just be better off forking the whole project and maintaining it as my own one-off. I appreciate the help and support that many of you provided. I hope that those of you who wished to experience this for yourselves get the chance to do so.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
Patashu, I give you high marks for making complete sense, every one of those reasons is valid and I can't argue against them. That being said, I would be completely fine if the code behind the voxel engine remained in a branch, so that at least anyone who was curious about the effect could have access to it and be able to enjoy it. I just don't see how I could have possibly done this in lua and gotten anywhere near the same level of performance out of it. (That and I just have a much better grasp of C++ over lua.) Anyway, I submitted the code into a branch that punkrockguy created for me so now, theoretically, he and everyone has access to the code if they would like to sync to that revision. I'll provide more details once I hear back from prg and can confirm that he got everything working.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
I kind of don't understand this attitude zeromus. I mean, far be it from me to argue with you, but I don't really think the inclusion of this in the windows version would "pollute" the source as you say. The voxel code would still live in its own cpp/hpp files, it's just a matter of providing the address of the DirectDraw surface buffer to OpenGL so that it can render directly to the surface. The final call is up to you, but I would imagine you'd at least want to see how the source code turns out before making a final judgement call.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
Hey there. We're still in the process of getting it folded into the source control. At first it will be a distinct branch, and then it will be migrated into the trunk. Until I or someone else gets around to adding OpenGL support to the DirectDraw version (aka the Windows version), it will remain an SDL only option. That being said, I believe you can compile the SDL version in Windows (someone correct me if I'm wrong), you just have to enable different compiler options. Long story short, once all the kinks have been ironed out in the original release, I'll see if I can't get it running in Windows without too much hair pulling.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
OK, I think the code is finally complete as far as I am concerned without people actually testing it and finding problems and/or suggesting new features. I managed to get mouse control working very well in full screen, so all of the controls in windowed mode are available. On top of that, I was able to fix a bug in the SDL code with the zapper crosshairs. They would only update when the trigger was pressed, so you didn't actually know where you were firing. So I got them moving around again. I will PM punkrockguy and hopefully we can get this code out to everyone to test out. Stay tuned.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
Just a quick update, I'm still working on this, although I'm having trouble finding time. My two current stumbling blocks are: 1) Still no mouse control in full screen, and 2) waiting for punkrockguy to create a branch for me to contribute the code on the SVN. Hope to be able to get this to everyone soon.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
ProcyonSJJ wrote:
OK, I gave it the old college try, but I must be very tired, or I'm missing something.
I was very tired. It helps not to use a uint8 to hold values that could potentially be higher than 256, like the count of each color per frame. Switched it to a uint16, and everything works great. I'll add it as a third option to the voxel dialog.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
AnS wrote:
No, it can be very cheap. You see, the pixels are not in RGB, they use indexed colors. So you can just create an arrayColors[256], then every frame you first clear the array, then loop through all 61440 bytes of XBuf (in PAL, in NTSC there will be only 256x224=57344 pixels) and increase the arrayColors[XBuf]++. There, you have the array of weights, findmax and here is your color.
OK, I gave it the old college try, but I must be very tired, or I'm missing something. The pertinent portions of my code looks a little like this:
	// Find the most frequent color before the rendering loop begins.
	// (Assumes arrayColors was filled in on the previous frame.)
	maxColorIx = 0;
	maxColorCount = arrayColors[0];
	for ( paletteIx = 1 ; paletteIx < 256; paletteIx++ )
	{
		if (maxColorCount <  arrayColors[paletteIx])
		{
			maxColorIx = paletteIx;
			maxColorCount = arrayColors[paletteIx];
		}
	}

	// Now blank out the count.
	memset (arrayColors, 0, 256*sizeof(short));

	// Set the new clear color
	uint8 r, g, b;
	FCEUD_GetPalette(maxColorIx, &r, &g, &b);

	clearcol[0] = (GLfloat)r / 256.0f;    
	clearcol[1] = (GLfloat)g / 256.0f;    
	clearcol[2] = (GLfloat)b / 256.0f;
   
	glClearColor(clearColor[0], clearColor[1], clearColor[2], 0.0f);
 
	for ( yloop = 0; yloop < 256; yloop++ )
	{
		for ( xloop = 0; xloop < 256; xloop++ )
		{
			arrayColors[Xbuf[yloop*256 + xloop]]++;
			
			// draw voxels, etc.
		}
	}
but the clear colors I'm getting in no way correspond to the most prevalent color. I think the problem might be with what I'm passing to FCEUD_GetPalette. Any ideas?
Experienced Forum User
Joined: 10/10/2013
Posts: 38
AnS wrote:
I've noticed on the video that you're drawing the whole 256x256 buffer of pixels, so sometimes there's black tiles at the bottom of the screen. You should only draw those scanlines that usual renderer of FCEUX draws, e.g. scanlines 0-239 in PAL and 8-231 in NTSC (in Windows version these numbers are customizable).
Got it, I will work on correcting the buffer extents. I'm using considerably less information to initialize my OpenGL settings than the original OpenGL mode, so perhaps it's time to take more of them into account.
AnS wrote:
The Megaman example indeed shows the big problem with the current method of removing voxels. How about this idea: each frame you pass through the pixels array and find the color which the majority of pixels use, then use it instead of background color. This would probably make Iceman stage (and many other games) look decent.
Some thoughts on that: I definitely like that idea, since I think it would yield considerably better results, but I can't imagine that would be cheap. You'd have to query the color of over 65k pixels, tally them, and then identify the one with the highest count. That could be done a) in the PPU or b) in the voxel renderer with a one frame delay (e.g. use the current clear color for a frame, and determine if that clear color needs to change next frame.) I would also like to implement a "click and stick" option, sort of along the lines you proposed, where a user clicks on a color and it determines which palette color was chosen by the user. The thing is though, it's a little harder to determine which "pixel" was clicked on when we're looking at voxels at an arbitrary camera angle. It might need a different interface like a palette selection window or something. All food for thought. Punkrockguy contacted me about getting the code uploaded as a branch off of the SVN trunk, so I will start working with him on that and keep you updated as to the code's availability.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
Lol... can't follow my own advice. Got Kazam working by switching the encoder. Enjoy, and please let me know what you all think! http://www.youtube.com/watch?v=mCeFpLMn3sc
Experienced Forum User
Joined: 10/10/2013
Posts: 38
@Patashu: I gather most of those are Windows solutions, correct? I'm not running Windows, I'm on Ubuntu 13.10, with an nVidea GeForce GTX 670, and the latest Bumblebee drivers. I've isolated the problem strictly to file I/O. The more I change games, the more likely the recording is going to crash. So I have two choices: Take separate videos of the games I intended to demo, or find a better recorder :P Edit: Sadly, I'm going to bail on this tonight. I'll probably just take short videos of the games I wanted to demo. It's just not how I hoped to debut this.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
Ugh, every single f-ing screen recorder I tried crashes. I tried Kazam, RecordMyDesktop, VokoScreen, and RecordItNow. Every single one of them dies on me mid-way through. I don't know if it's because I'm tapping the 3D acceleration and there's not enough processing power left to buffer the video or what. Tempted to just hold the camera on my phone up to the screen, but I'm afraid that will give a crummy image. Does anyone have any other ideas?
Experienced Forum User
Joined: 10/10/2013
Posts: 38
HUGE progress tonight with one very tiny accomplishment: I got the mouse polling to work without having to press the mouse button. It was an aspect of GTK that I was very unfamiliar with, mouse polling is initially disabled until you make a call like:
gtk_widget_set_events(evbox, GDK_POINTER_MOTION_MASK);
Took a lot of Googling but I finally figured it out. As a plus, I also got scrolling working, so zooming in and out works great! Now I really feel like I have good control over the camera. (As a side note, this could lead to the addition of a crosshair in the SDL version for Zapper games, should anyone be inclined to add one.) Anyway, current limitations thus far are: 1) I realize now that the resolution of the window is determined by the X and Y scales in the configuration. The only reason I've been testing in a 1024 x 768 window is because I had X and Y set to 4.0 and 3.0 respectively. That may not be ideal for viewing the voxels, but I'm not sure what else to base the window size upon. AnS, do you have any thoughts about this? 2) Mouse/Camera controls work great in window mode, not so much in full screen mode. I need to tackle that side of things as well. 3) The aforementioned issue with the lighting and general look of the voxels, but my hope is that once the code is released into the wild, someone with more OpenGL experience will be able to enhance the look of things.[/code] Edit: I absolutely have to get some screen video recording software on here and start taking movies of this stuff to put on YouTube. I will get working on that ASAP.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
AnS wrote:
I mean, use one color as the fill, but remove voxels of both colors.
But if we did that, in the SMB3 example, the stage would not end in black, it would continue to be the color of the sky.
AnS wrote:
That's too bad. I hoped the engine could gain some popularity among casual players.
I didn't mean to sound so pessimistic, I think it certainly could be ported to Windows specific code relatively easily. On the one hand, SDL and GTK libraries are available for Windows, so that's one route. On the other hand, OpenGL can play just as nicely with DirectDraw as it can with SDL, so that's always a possibility. That would make the port even simpler. I think it can and will happen, it's just a matter of finding the right assistance.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
AnS wrote:
Actually, even they use BG color, just not as a "skybox" fill.
Ah, you are correct. Well, you'll be able to see in the options below that I worked out a way to make the color selection configurable.
AnS wrote:
So, I guess most of time you'll need to filter out voxels of two colors: of the BG color and of the predominant color.
I don't see how that's possible to do. Only one color can be used as the fill color. In the SMB3 example, you pretty much have to go with the sky color and then run into a "sea" of black voxels at the end (unless you use my original method of choosing a sample coordinate, which allows the fill color to be more dynamic based on what's happening on the screen.)
AnS wrote:
BTW, the predominant color is better stored not as some coordinates of a pixel, but as an index of the palette entry (e.g. PALRAM[6]). Of course user should be able to define this index by simply clicking screen (then the program retrieves the pixel at these coordinates and finds out which palette entry the pixel uses at the moment).
I like this idea, and would like to try to implement it after "version 1" of this is released.
AnS wrote:
This is not easily doable, as it requires a certain code injection in the core (ppu.cpp). Once you successfully implement everything else, I can take the risk of carefully doing this.
I figured as much. Again, this would be more a of "version 2" idea. Here's a screenshot of the completed voxel config dialog. Enough options for everyone? Everything is plugged in and working. Though, truth be told, Transparency sucks. I don't know if it's due to the way I set OpenGL or what, but all it seems to do is reduce the quality of the picture. It doesn't do what I hoped it would do, which would be to make the voxels more "crystalline" in appearance. As for wave choices, you can pick Horizontal, Vertical, and both \ and / diagonals. I'm still using my cheesy pre-mapped wave method. Derakon, I'll get circle waves in for v2. This thing is nearly complete. At the moment my three biggest hurdles are: 1) lack of solid control over polling the mouse, to allow for better camera control. 2) too many hard-coded values for the screen size. I've been doing 100% of my testing in 1024x768 windowed, and haven't tried anything else. I need to start making the code more flexible, and to read the values that are currently being used for screen resolution, etc. 3) Providing a good way for everyone to access the source code. BTW, AnS, I started to look over the Windows specific code, and I realize that it's using DirectDraw, but not Direct3D, which makes perfect sense. But the lack of a proper 3D interface upon which to implement this means it may forever be an SDL-only feature. My knowledge of D3D has always been poor, and substantially weaker than my knowledge of OpenGL. A Windows solution may only come about through a willing volunteer who can undertake the port.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
AnS wrote:
Are you sure you were retrieving the right data?
You know what, I might not have done it correctly the first time. I gave it another shot, and it seems to be working better this time. That being said, it's still a case of being great for some games, and not for others. For example, most Mega Man games just leave the BG color black. So even when I play, say, the first Mega Man, the boss selection screen is rendered as one giant blue block against a black background. When I select Bomb Man, whose stage is predominantly blue sky, it's the same effect. Whereas the Super Mario Bros. series uses it correctly. I guess Nintendo just used the feature more consistently than other companies. So AnS' technique works better for some side scrollers (SMB, Gradius) and worse for others (Mega Man), which means I'll probably still need to find a way to implement both solutions (system BG color vs coordinate sample) and have it configurable to the user. Unfortunately, games like Zelda 1 and map-heavy RPGs like Final Fantasy or Dragon Warrior just don't make suitable candidates for this effect. I was wondering, however, about another possible experimental mode which I'll describe below. Current issues At the moment, my biggest problem is tracking the mouse. This is important to have full control over the camera. Right now I can only reposition the camera when I hold a mouse button down. I was hoping to read the mouse whether the button was pressed or not, so that the camera could be moved around on the fly. I added:
    g_signal_connect(G_OBJECT(evbox), "motion-notify-event", G_CALLBACK(handleMouseMove), NULL);
to sdl/gui.cpp but like I said, it will only work if a mouse button is held. Punkrockguy, if you know anything about this, please let me know. Also, I need to start talking with you about how to host the code so that others can start playing with this and beta testing it. For now, a little sign of progress. It's not much, but it's coming along: Experimental thought For games like Zelda, and others that have a top-down or birds eye view, I was thinking of another way to deal with the 3D effect. I imagine the emulator must know when it is rendering a background object versus a player or enemy sprite. If that is the case, it might be possible to have a secondary 256*256 buffer that serves as a depth buffer. Even if it's just 0 for bg and 1 for fg, that would be enough info to let the renderer know that a fg object should be drawn at a depth closer to the camera, so that it "floats" above the background. Any idea if that's easily doable? Without screwing up the emulation too much, because this is the only feature that could possibly take advantage of it.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
AnS wrote:
Wait, are you still using that wrong method of getting background color, or sample means something else?
Yeah, I meant to update that point: I tried the method you mentioned AnS, but unfortunately, it didn't work. Not because you gave me the wrong information, but rather because I guess developers didn't use that information correctly. For example, in Super Mario Bros., the background fill color is black even when most of the background sky is blue. So unfortunately, it didn't work in many cases. It could still be a possible setting though. In case people don't want to try and pick a coordinate, they could try their luck with the system BG color, and see if it matches and produces a good result. In my experience though, if the background wasn't strictly black, most games didn't really do a good job setting that color.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
Ilari wrote:
Not everyone has wide display (and browser set to wide enough), the effect if it isn't is very annoying.
Understood. Just a quick update, I have a beta version of the code ready to go. It correctly incorporates itself into the rest of the SDL code, and has a single checkbox to turn the voxel engine on and off. Ultimately, I would like to have a separate dialog where you can set various settings including:
    Background sample coordinates Voxel size Voxel spacing Wave style
That will be in version two, but the main goal tonight was just to make sure that I could include the code cleanly into the existing code base without hijacking any methods. So far so good. I went on IRC to see if punkrockguy was around, but no dice.
Experienced Forum User
Joined: 10/10/2013
Posts: 38
feos wrote:
Please unembed that wide picture.
OK, I've done so, but... why? A quick update, got a lot of performance increases just by optimizing the code and pulling out one-time calculations that I stupidly left inserted inside the rendering loop. Wrote an email to an OpenGL guru to ask how to improve the lighting/reflection, and what to do about the moire artifacts that I see when there a huge number of voxels drawn on the screen. Here's another quick snapshot. The effect with SMB3 is weird because most of the sprites are outlined in black. So you don't really get a sense of the lighting. I'd embed the picture like I did the others, but I don't want to violate any rules... if such rules exist? http://i43.tinypic.com/33ll73a.png
Experienced Forum User
Joined: 10/10/2013
Posts: 38
Derakon wrote:
You might not even need the sqrt; I think that removing it would mean that the ripple density would increase as you get further away, though, which is kind of counterintuitive.
I forgot to mention that I'm faking some of the math to prevent performance issues. My method is like so:
// global declaration
#define WAVE_ARRAY_SIZE 24
float wave[] = {0.0f,  3.0f,  6.0f,  8.0f,  9.0f,  9.5f,  10.0f,  9.5f,  9.0f,  8.0f,  6.0f,  3.0f,
                0.0f, -3.0f, -6.0f, -8.0f, -9.0f, -9.5f, -10.0f, -9.5f, -9.0f, -8.0f, -6.0f, -3.0f };

// in the render code
z = voxelDistance + (wave[ ( x + Frames ) % WAVE_ARRAY_SIZE ]);
In order to map the wave properly to a circle, I'd probably still need square and sqrt, or the ripples would come out from the center in a diamond.
Derakon wrote:
Glad to hear you got the performance issues sorted!
Thanks, I was really beginning to wonder what I was doing that was so intensive that a 2012 model GeForce couldn't really handle.