awesome stuff to hear man... keep your updates comming man.
thankyou for the advice about disabling rewind. i turned some features off and i have noices even less slow down than before but still not sound perfect yet.
also a thing to note. it seems the idea of the DMG signing didn't work. my OSX kicked Bizhawk and it still freezes when trying to play. but the trick you gave us works still.
I’m sorry. Really I am... I am so sorry for what I did. Forgive me.
awesome stuff to hear man... keep your updates comming man.
thankyou for the advice about disabling rewind. i turned some features off and i have noices even less slow down than before but still not sound perfect yet.
also a thing to note. it seems the idea of the DMG signing didn't work. my OSX kicked Bizhawk and it still freezes when trying to play. but the trick you gave us works still.
Weird. I ran it directly from inside the .dmg on a different machine and it worked there. :-(
I guess I'll add the applications folder and an arrow to the disk image like some do and just make people "install" it.
I still think the performance issue you're experiencing is a settings problem, I just have to dig around and figure out what settings I was including before that aren't part of the default besides the audio sync and rewind stuff. The SNES core pushes my 9 year old Mac to the limit, and although it can still handle full speed, I noticed behavior similar to what you were reporting when I use default settings. If I use my old config file it doesn't stutter, and it also doesn't stutter if I enable MKV recording while I'm playing. (Which is weird because that should use more CPU)
========================
(2017-05-03) Mac Build of 1.12.2 has been released:
http://projects.sappharad.com/bizhawk_mac/BizHawk_mac_1.12.2.dmg
Same changes as the Windows version, when applicable.
Note: If you downloaded before 2017-05-16, the FFMpeg video writer was broken. Re-download 1.12.2 if you need that feature to work.
macOS specific improvements are:
- Tweaked a minor thing which may result in the appearance of slightly improved performance, but probably makes no difference.
========================
(2017-06-04) Mac Build of 1.13.0 has been released:
http://projects.sappharad.com/bizhawk_mac/BizHawk_mac_1.13.0.dmg
Same changes as the Windows version, when applicable. This probably includes C64 too, although I haven't actually tested it.
I've been trying to compile BizHawk and make it available on Gentoo. I managed to do it with some patches to the code. However, there is a nasty issue with the resource files that I could not solve. I did manage to compile the rest of the program by faking a resource file, but it of course crashes immediately because it tries to load a corrupt file.
The build fails while trying to load some png images. It looks like all TAStudio images are affected, while the controller images are not.
After digging around, I found that the issue is not present in this project, but either in libpng or in Mono's libgdiplus. It is probably in gdiplus, because I can load the TAStudio images just fine, and only the Mono shell throws exceptions. This is very problematic because it affects all Mono applications in Gentoo, and also because Gentoo's version of libgdiplus is the latest one.
Does anyone around here have a clue about what's special with the TAStudio png images, so that I can try to fix the bug?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Fair enough. Even if it's not usable after compilation, I would like to understand the PNG bug. It could be a security issue.
EDIT: I think I found it. It has to do with transparency. Apparently libgdiplus reads transparency information from the header and calls libpng to convert the image from RGB to RGBA, and libpng immediately fails because a certain flag is not set. Thankfully, it doesn't look like a security issue (no OOB access or anything), but still an annoying bug. I'll see what I can do about it.
EDIT 2: I have confirmed that the issue is indeed because of transparency. Apparently, libgdiplus 4.2, the version currently available in Linux distributions, got completely broken with some updates to libpng, and no image that implements transparency with the tRNS scheme doesn't work. Fortunately, the team released a new version one month ago that fixes this, so I'll probably have to update that package to make BizHawk work (actually I also have to fix monodevelop too, which is also broken for totally different reasons). I'll try to do that in the next few days.
EDIT 3: After a ton of hacks in three different Gentoo packages (Linux support for Mono sucks), I finally got it to compile, it doesn't look ugly. And it runs fine (for a while), the biggest problem for me now is that it seems you have a shitload of threads running, and every time one of them crashes for a different reason. It will be very annoying to make it stable.
Sure, just wait a little. The situation of the mono packages in the portage tree is a complete mess. What I did was to clone the repos to my machine and rewrite the ebuilds, I am in contact with the maintainer already and am preparing a pull request so that things are fixed upstream and people can test it.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
You're amazing, sir!
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
More good news, the instability was just because I was stupid and did not install OpenAL. After doing that, it's running fine now :)
EDIT: Actually, I spoke too soon. For some reason, it only runs stable when I launch it as root (!), and it gives me a bunch of cryptic symbols when I try to load a GBC ROM. I'm investigating...
EDIT2: Discovered what's wrong with strace, the stupid program tries to create some fonts in the folder where the Xorg is, which obviously can only be done as root. More hacks...
EDIT3: It seems there is no way around this one. Mono's implementation does not support forms being created in multiple threads. Is it doable to create all GUI stuff in only one thread?
EDIT4: Sorry for editing this post like crazy, but I am pretty convinced now that some architecture changes are needed to make the code compatible with Linux. The main issue here is that in Linux, graphics and keyboard input are both handled by the X server, and the horrible thing is that Mono accesses X in a completely thread-unsafe way, that's why you cannot create two WinForms in different threads. But for Bizhawk in particular, things are worse, because it polls keyboard input through OpenTK, which in the case of Linux, also passes through X, and I saw that the code creates a thread for that too.
If you just run the code compiled with my simple patches, sometimes it will start and sometimes it won't. That's because, upon startup, Bizhawk initializes the input polling thread and later draws the form for the main window, and since this is done in different threads, race conditions can happen. I managed to make it always start by only initializing the keyboard input after the window is drawn, but that doesn't solve everything. Because when any other dialog is opened, there's still a possibility of a race condition.
The only way to make sure it will work in Linux is to use a single a single thread to read keyboard input and draw all the forms...
It is possible, but unfortunately it doesn't solve the problem. One case is the following: a thread is creating a form inside X, gets preempted by the kernel, and another one that polls input starts executing, getting also preempted by the kernel before it can finish cleaning its variables. Then the initial thread gets executed with dirty variables and a race condition still happens.
The quickest way I see to fix this would be to make a lock for everything that accesses X and wrap all form creation and input polling code in a function that acquires the lock, that would serialize the access to X even though the program is multithreaded.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Man please ask this on forums, on stackoverflow, anywhere. We need this resolved for the science and for the real portability. Did the commits in the PR I linked help? Maybe the guy who made it has some ideas.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I'll try to do that in the next few days. For now, I am making my patches available on Github. Instructions to build are in the README. Good luck getting the packages, though, it's not trivial, and even fixing them on Gentoo is annoying. Also, I simply disabled input polling altogether to make it stop crashing.
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
Sounds like a design problem in Mono, did you report a bug? As feos said, take it to stack overflow as well.
For the README, the repo can't be cloned without git-svn? That's incredibly weird... Also, you may gain from linking to the libraries git repos if you need to build from source. I am pretty used to have to find things but a lot of newer Linux users aren't.
Sounds like a design problem in Mono, did you report a bug? As feos said, take it to stack overflow as well.
Well, if you go to their page, they state explicitly that you shouldn't call GUI code in different threads, so they are well aware of it. I think we are at that fine line between what is a bug and a feature :)
Anyway, I just sent a massive commit changing the code. Now it works fine. There are some NullPointerExceptions because of some stuff I didn't compile yet, but most of the front end is working.
What I did essentially was define a lock for everything that needs to access X. Then I told the input polling thread to acquire and release the lock. After that, I modified the custom FileDialog class to also use the lock, created a SafeForm and SafeMessageBox that only show themselves inside the lock and that did it. Now, to add stuff, one needs to extend SafeForm and/or call SafeMessageBox. It doesn't look that bad.
Warepire wrote:
For the README, the repo can't be cloned without git-svn? That's incredibly weird...
That's because you need to create a source file that contains the revision number and gets compiled in the about box. If you don't clone with git svn, the generation of this file will fail, and some DLL's will complain about that number missing.
Warepire wrote:
Also, you may gain from linking to the libraries git repos if you need to build from source. I am pretty used to have to find things but a lot of newer Linux users aren't.
Thanks for the suggestion, I will do that. For now, libgdiplus 5.4 is here.
Compile it with ./autogen.sh && make && sudo make install
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
p4wn3r wrote:
Warepire wrote:
For the README, the repo can't be cloned without git-svn? That's incredibly weird...
That's because you need to create a source file that contains the revision number and gets compiled in the about box. If you don't clone with git svn, the generation of this file will fail, and some DLL's will complain about that number missing.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Do the tools like tastudio launch?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
The tools where the menu entries are not greyed out are launching. TAStudio is unavailable at the moment, I will try to get a core compiled and see if it works.
Wow, I completely missed all of this stuff because I don't check the forum daily anymore. Congrats on getting it working on Linux again.
I knew about the GDI+ problem, because about a month ago I tried to see if it would still build on Linux but after omitting those PNG's completely I ran into some other problem and I don't even remember what that was anymore. I'd love to pull in your changes if we can keep everything without breaking the macOS build. I'll take a look at that over the next few days.
I only look at this for a couple hours each week, and at the moment I'm trying to figure out how I can continue even offering macOS support in the next major release. The problem is that in the experimental branch they're going 64-bit only, and Mono's WinForms implementation doesn't support 64-bit on macOS. I've got 2 options at this point...
1. Ditch the WinForms UI and continue developing the Eto one. (Check that out by the way, it's in the /BizHawk.Client.EtoHawk/ directory, but last time I tried it on Linux it was crashing. That might've been the OpenAL problem you mentioned.) I don't like this idea much because it's not fun to rebuild the UI from scratch when I can use WinForms for free
2. Port Mono WinForms to 64-bit macOS. This actually seems like the easier option, but I don't know the internal OS windowing stuff at all so it's a huge learning curve.
And this is assuming that I can even get the new method of hooking up native cores to work with Mono.
I'm really glad that someone who cares about Linux and knows what they're doing is actually taking a look. I've left some comments on your commit, it looks pretty destructive right now and I think we can come up with something that won't cause as many merge conflicts in the future.
p4wn3r wrote:
feos wrote:
Do the tools like tastudio launch?
The tools where the menu entries are not greyed out are launching. TAStudio is unavailable at the moment, I will try to get a core compiled and see if it works.
TASStudio doesn't work on the mono branch and won't without some effort. There are custom controls in TASStudio that make native Windows wrapper calls to directly to GDI. All of that stuff would need to be re-written in pure C# without using the calls that were needed for those controls to function well in the first place. See CustomControls/GDIRenderer.cs
I stubbed out some of the native stuff for the custom list view control so that you could playback and record TAS input files, but any work to get TASStudio working outside of Windows will have to be completely new.
Thanks for the comments, Sappharad. I suck at setting up builds using IDE's, so I didn't have any idea of what I was doing when I changed those .csproj, if you could make a pull request, I'll sure review it fast.
As for the large commit, I am going to revert. It simply doesn't work. After closing wiindows about ten times it almost always crashes. This is probably because of the garbage collector, it must be accessing the X server when it's finalizing the objects. Mono probably has a way to stop the GC from crashing other stuff when GUI is accessed, but if another thread is also using X, it has no way to know. Since the GC is pretty much unpredictable, I don't think there is a way to circumvent Mono's limitation at all. All X things should be done in a single thread.
About the UI, is there any possibility of using another toolkit, like Gtk#? It doesn't seem to have the same limitations in Mono that WinForms has.
Thanks for the comments, Sappharad. I suck at setting up builds using IDE's, so I didn't have any idea of what I was doing when I changed those .csproj, if you could make a pull request, I'll sure review it fast.
As for the large commit, I am going to revert. It simply doesn't work. After closing wiindows about ten times it almost always crashes. This is probably because of the garbage collector, it must be accessing the X server when it's finalizing the objects. Mono probably has a way to stop the GC from crashing other stuff when GUI is accessed, but if another thread is also using X, it has no way to know. Since the GC is pretty much unpredictable, I don't think there is a way to circumvent Mono's limitation at all. All X things should be done in a single thread.
About the UI, is there any possibility of using another toolkit, like Gtk#? It doesn't seem to have the same limitations in Mono that WinForms has.
Based on the fact that you're still deciding what to do, I'll wait until you've found an approach that you like. :-)
Regarding other toolkit, this is why I mentioned the /BizHawk.Client.EtoHawk/ solution in my previous post. Eto Forms uses GTK on Linux. It provides a cross-platform UI API very similar to WinForms. I started building the Eto UI in March of 2016, but BizHawk is so big and the WinForms UI has so much that I abandoned it about a year ago. Before I stopped you could load roms, configure controllers & play games with it, although I hadn't integrated OpenGL yet at that point so it was just outputting to a bitmap. Eto seemed like a good way to go forward at the time, because the actual BizHawk.Emulation.* projects will build fine on Mono so we could eventually start working on that directly in master without worrying about Windows and benefit from any core changes there.
The current EtoHawk code no longer builds on MacOS, due to some API changes after Microsoft bought Xamarin and started migrating MonoDevelop into Visual Studio for Mac, but it should still build on Windows and Linux. (I need to fix the macOS side)
I'm not suggesting EtoHawk as the best option going forward, but when I was looking for options with a native UI back in late 2015 and saw GTK# someone mentioned Eto Forms on IRC and I liked that API better along with the fact that it was native across Windows, Linux and Mac.
If you're interested in building a brand new cross platform UI, I'm willing to help. But the main reason I kept porting the WinForms one is because of all of the features. I just want to use BizHawk to play games and sometimes watch videos posted on the site, so I don't really have any personal interest in developing the TAS tools.
Based on the fact that you're still deciding what to do, I'll wait until you've found an approach that you like. :-)
Well, I have tested things extensively now, and I think I'll stick to the single threaded approach. I don't think the code will lose much responsiveness without multi-threading, but sure, wait a little because things might still break.
Sappharad wrote:
Regarding other toolkit, this is why I mentioned the /BizHawk.Client.EtoHawk/ solution in my previous post. Eto Forms uses GTK on Linux. It provides a cross-platform UI API very similar to WinForms.
Well, these things hardly have a clear answer, but I suggested GTK because I am more familiar with it, and with the small experience I have with toolkits, it's the one that gave me less work across different platforms. I have never heard of Eto Forms, I'll take a look.
Sappharad wrote:
If you're interested in building a brand new cross platform UI, I'm willing to help. But the main reason I kept porting the WinForms one is because of all of the features. I just want to use BizHawk to play games and sometimes watch videos posted on the site, so I don't really have any personal interest in developing the TAS tools.
I see. I think the TAS tools are the selling point of the emulator, though. Without them, there isn't much incentive to use the other ones that offer better cross-platform support.
EDIT: I successfully compiled gambatte (and blip_buf, which it depends on), and I loaded a ROM. Sound and video works, but input doesn't. I don't think it's because of me messing up with the keyboard input thread, because if I try to load a movie, it has no input either.
Also, as expected from Sappharad's comments, TAStudio just crashes the program.
EDIT 2:
OK, here's a summary of what I have now. All input problems are solved. I can play games fine and watch movies.
All pure C# cores, including NESHawk, are working. Gambatte and QuickNES are also working. I think the other cores should work too, I just need to set up a build environment for them.
Some tools are not working. RAM Search, Cheats and Trace Logger don't work (call user32.dll, which doesn't exist in Linux), TAStudio and New Hex Editor don't work (need gdi32.dll). Lua doesn't work (the C# interface is Windows exclusive).
I have just finished replacing all code that calls this with message boxes explaining it's not implemented. I'll try to get more cores compiled and release a binary if there's any interest.
EDIT 3:
After talking to natt and adelikat, I think it makes more sense to port the cores using the new waterbox system. Since that will take a while, here is the compiled program. Let me know if you find something that's not working.
It's probably just me as I am not an experienced programmer or Linux user by any means, but I don't see a Linux executable in the compiled program link above. Am I missing something?
And thank you very much for looking into this, I really appreciate it!
Hello, juef. BizHawk is a C# application, and thus is run on a virtual machine, so there is no linux executable in the usual ELF64 format. To run it, you need to install mono, and launch it from the terminal with "mono EmuHawk.exe", or create a launcher that does that when you click it. I can provide a launcher if needed, but this depends on the distribution and desktop environment you're using.
EDIT: If you don't want to explicitly input the mono EmuHawk.exe command, I can provide a fast solution if you use GNOME. Create a file named EmuHawk in the directory you extracted the binary:
That will work if you extracted the files in a folder named BizHawk in your home folder. If not, change the path above. Give it execute permission by running chmod +x ~/BizHawk/EmuHawk
Now, if you use GNOME, create a bizhawk.desktop file in ~/.local/share/applications with:
Change your username/path in the Exec line (here it needs to be the absolute path, don't use ~), then reload the shell with Alt+F2, type r, then Enter. After that, you can search for BizHawk in the gnome shell and it will appear like any other app. That should work with Ubuntu Unity, too. For KDE and XFCE, I never used them, so maybe someone can give instructions in other replies.
Oh, I see! I have never used C# or mono before, but this couldn't be more straightforward. It seems to run fine here, thank you very much! I'm using xfce but I won't need a shortcut, thanks very much anyway for the detailed explanation, I really appreciate it.
EDIT 2:
OK, here's a summary of what I have now. All input problems are solved. I can play games fine and watch movies.
All pure C# cores, including NESHawk, are working. Gambatte and QuickNES are also working. I think the other cores should work too, I just need to set up a build environment for them.
Some tools are not working. RAM Search, Cheats and Trace Logger don't work (call user32.dll, which doesn't exist in Linux), TAStudio and New Hex Editor don't work (need gdi32.dll). Lua doesn't work (the C# interface is Windows exclusive).
I have just finished replacing all code that calls this with message boxes explaining it's not implemented. I'll try to get more cores compiled and release a binary if there's any interest.
EDIT 3:
After talking to natt and adelikat, I think it makes more sense to port the cores using the new waterbox system. Since that will take a while, here is the compiled program. Let me know if you find something that's not working.
