Joined: 11/27/2017
Posts: 86
Location: United Kingdom
If I remember correctly, FFVII is the same on each disc just with differing fmv.
Perhaps whilst TASing the disc isn't swapped properly after a load state?
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
Spikestuff is correct about the sbi files though. If there is one for your PAL game on the internet, you WILL need it - whether the game boots or not.
Without this circumvention, these games are programmed to screw up later on in the game, or in some cases become ultra-hard or completely unplayable.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
I have no experience with C64 tape formats at all, but the file format specification (here https://ist.uwaterloo.ca/~schepers/formats/T64.TXT ) doesnt look that bad.
I'll try and take a look at the C64 core when I have some time and see if .T64 support will be easy enough to implement.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
This page shows the FBM file format:
http://tasvideos.org/EmulatorResources/Fbarr/FBM.html
There used to be a page at TASVideos that shows the FR file format:
https://web.archive.org/web/20170619115714/http://tasvideos.org/EmulatorResources/Fba/FR.html
If both of these pages are accurate (I don't know much about either format so couldnt really tell you), you could try changing byte 004 from 0 to 2 (the file header format version number) and renaming the file?
There will be differences in the save state chunks by the looks of it, but not sure if/how this will affect things. I mean, its storing a minimum FBA required version, so you would think this wouldnt matter?
Anyway, thats my 2 cents. I really dont know anything more.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
Yes, by crickey!
Obviously I am somewhat biased, but even though you picked a REALLY long speccy game for the first TAS, it remains entertaining throughout.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
GJTASer2018 wrote:
.... and to anyone who is wondering what the deal with the colors in this game, it's a result of the limitations of the Spectrum itself. You watch the footage of ANY ZX Spectrum game, you'll see the same kind of things happening.
Colour clash was always a big thing on the spectrum, due to it's attribute-based video system. So software developers either had to ensure that all 8x8 screen pixels in the video field stayed the same colour (very hard with any kind of animation), or did incredibly complex stuff with updating video memory in relation to the CRT beam position in the TV to get around it.
So basically, not many did this (although you do see this quite a bit in later scene demos and in certain games) although many did go for monochrome animated areas with coloured HUDs.
More info for those who are not bored to death already :)
https://en.wikipedia.org/wiki/Attribute_clash
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
The use of cracked games is an obviously murky one. Not being a TASer I don't really have an opinion on this.
Now, IPF files.
* .ipf images. This stores the results of reading the disk several times, but it's a proprietary, closed source format made by the Software preservation society.
The format source is available under a rather unusual “SPS Freeware License Agreement”. Reading through this agreement is quite confusing though. I was planning on supporting this format for ZXHawk eventually, although by writing my own IPF image parser rather than using their binaries. If I can make it work that would mitigate any weird licensing issues.
As far as I know, it is not deterministic and cannot be used in a rerecording emulator.
I don't believe it this matters if we are writing our own IPF parsing tools. On a real disk, the data is written in such a way that errors are returned on successive reads, corrupting certain return bytes in certain sectors. I imagine IPF will just be storing multiple copies of sectors with CRC errors (much like the EDSK format), then the emulator (or library) will return a random 'mutiple copy sector; on each read.
On ZXHawk, I overcame this by making the 'random' choice of duplicate sector deterministic. It just has an internal counter that increments and that is used to select the duplicate sector data. Because copy protection is generally performing successive reads of a sector looking for specific differences, this method works quite well. If you only have 3 copies within the image and are choosing truly at random there is a good chance that the protection check will fail every so often.
Not to mention that we couldn't claim that everyone dumped the original disks themselves: the tool to dump disks in this format isn't public.
This point is moot.
This is the tool:
https://kryoflux.com/?page=download
The hardware floppy controller is purchasable and the software is freely downloadable for non-commercial use.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
It could be your system (although I'm not much good at comparing proccessors).
The saturn core comes from mednafen, which does have reasonably hefty requirements. From the mednafen website:
Mednafen's Sega Saturn emulation is extremely CPU intensive. The minimum recommended CPU is a quad-core Intel Haswell-microarchitecture CPU with a base frequency of >= 3.3GHz and a turbo frequency of >= 3.7GHz(e.g. Xeon E3-1226 v3), but note that this recommendation does not apply to any unofficial ports or forks, which may have higher CPU requirements.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
Nothing. Its pretty much ready for an initial release imo.
I just figured you guys had some kind of release schedule and we were waiting for that to come around....
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
So, on the surface it has been all quiet on the spectrum front for the last month or so.
What has actually been happening, is that we have been soldiering on away from the master branch working on emulation timings (among other things).
I am happy to say that ZXHawk in the dev build is now passing all the Z80, contention and floating bus tests that we have been able to throw at it. It is also running every one of the technically advanced scene demos (that require a precise timing implementation) that I can find, perfectly.
Alyosha should take most of the credit for much of this work (indeed ALL of the Z80 work) and we are now at a stage where making the core ZX Spectrum machine emulation within Bizhawk more accurate is going to be extremely difficult. Which is a good place to be :)
As usual, please let me know if you come across anything untoward.
Thanks,
-Asni
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
This came after spec256:
https://sites.google.com/site/ulaplus/home
I know some of the other emulators support it. If it ever does get on the todo list for zxhawk, it will probably be pretty far down.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
phoenix1291 wrote:
I didn't think about it, but yes, I have rewind enabled, and that's probably what's causing the performance issues on some games and those OSD issues.!
Latest dev build now only reloads the tape and disk devices when a state is loaded, not saved.
Can you test this when you have a chance and see whether your performance issues with rewind enabled are solved?
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
Spectrum 48k (NTSC)
Just a 48k where the ULA outputs at 60FPS, sold in Chile:
https://faqwiki.zxnet.co.uk/wiki/NTSC_Spectrum
I don't know if there were any games produced exclusively for it, so I don't know if I will ever implement it.
Spectrum +3e
This was a homebrew modification to a +2a or +3 based on updated ROM chips:
http://www.worldofspectrum.org/zxplus3e/whatis.html
It's main benefit was the possibility of an IDE interface to connect HDD or CompactFlash drive. Again, I'm not sure there is much benefit in emulating this in bizhawk at the moment
Timex Official Cloneshttps://en.wikipedia.org/wiki/List_of_ZX_Spectrum_clones#Official_clones
There are a few benefits to these models, but compatibility with spectrum games always varied.
Pentagon Unofficial Clones and Scorpian ZS-256https://en.wikipedia.org/wiki/Pentagon_(computer)https://en.wikipedia.org/wiki/List_of_ZX_Spectrum_clones#Scorpion_ZS-256
I might get to these eventually. They use a clone of the beta disc interface that was a 3rd party peripheral for the original spectrums, and have different RAM options. They also use TR-DOS for this (you may see many TRD or SCL disk images around, thats what these relate to):
https://en.wikipedia.org/wiki/TR-DOS
In the first instance though, I am going to focus on getting the official machines released in the UK emulated properly (and perhaps as many of the peripherals as I can). I am not overly bothered about the clones at the moment, but they may come eventually.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
GJTASer2018 wrote:
This sounds like a mistake anyone could make really. Perhaps there should be a warning about enabling rewind for ZXHawk in a platform-specific documentation page?
The issue is that zxhawk *should not* be reloading the disk or tape device every time a save is made.
It's an oversight rather than being by design :)
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
Ok, I'll take a look soon. Although I think I might now what is causing it.
Whenever you save or load state, the disk or tape is re-initialised in the drive/datacorder and this triggers the OSD message. This process is necessary when loading savestates but not saving them.
I would wager that you have rewind enabled or something so it is constantly saving savestates to disk. This would also explain the performance issues you were having with tape games.
I'll try and sort this out tomorrow:
* Disable media re-insertion on save state
I believe that should solve quite a few issues for you.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
With ZXHawk OSD verbosity set to medium (not full) you should only see the disk inserted messages...
I probably need to provide a another option for tape though that does similar. There is a lot of noise otherwise.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
phoenix1291 wrote:
Interesting the Disk Image Manager, I see that it allows conversion between different file formats, does it work well? Because for the Apple 2, with software like dsk2nib to convert nib files to dsk (because the Apple 2 core doesn't support nib files), it never worked for me, it must be missing sectors when converting.
I have no idea about this. I have only used it to examine dsk structures.
*EDIT: I lie, I do. You will be out of luck doing anything with appleII dsks I think. The Amstrad CPCEMU dsk format is valid for spectrum and amstrad. The AppleII dsk format is something completely different.
phoenix1291 wrote:
So I tried the latest version with the disk switch, with Out Run, again. It worked very well, no problem.
However, (even though I know that being able to change disks via the menu like on the C64hawk core will probably be implemented later) there are two things I noticed, the first of which I'm not 100% sure (I'd have to recheck) is that to switch to disk B, you have to choose "previous disk" and not "next disk".
This works fine for me. Perhaps you have a duplicate in your bindings that is clashing? Next and Prev disk will loop round when it reaches the array boundary on either side.
phoenix1291 wrote:
And the second thing is that if the ZXhawk internal OSD is not displayed, even if the Bizhawk OSD is displayed, no message appears to say if the disk change has been done and which disk is inserted.
Again, this is working for me. Check menu ZX Spectrum > Non-Sync settings and make sure OSD Message Verbosity is at least set to 'Medium'.
Also try binding a key for 'Get Disk Status' as well and see if this displays.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
The latest dev build now has the ability to switch between disks that have been supplied to the ZXHawk +3 core in the same manner that you would multiple tapes. This currently is done by setting up keyboard shortcuts for this (a new tab is now available in controller configuration).
Note: Try to only do this when the game asks you to, or if you are at the initial menu screen. Until this is implemented fully, expect unpredictable results if you swap disks whilst the emulated drive is doing stuff.
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
phoenix1291 wrote:
Great!
Thank you for all your work and information.
Are there any games or protections in particular to test?
It's really hard to know. +3 disk protections are very badly documented, and it looks like even the search on WoS is only dealing with protection schemes on tapes (so the lists I posted previously are probably incorrect). If you aim for Ocean, US Gold, Gremlin and Virgin releases there is more chance than none that you will come across some kind of protection scheme, although it doesnt appear to be documented anywhere which one.
I have been using this initially: http://www.softsea.com/review/Disk-Image-Manager.html
It detects (or in some cases has a best guess) at many of the common copy protection schemes for spectrum and amstrad dsk games.
Although to be honest the Speedlock +3 protection scheme is the only one i've come across that required a bit of hacky work (the CPU gets the controller to read the same sector or sector multiple times, and because of data CRC errors (partially weak sectors) particular bytes within this sector will return different values on each read).
Most of the other protections appear to rely on things like over/undersized sectors, DAMs (deleted address marks) and incorrect sector ID information. So it has been a case of getting the UPD765 controller emulated properly to handle this rather than using anything hacky.
phoenix1291 wrote:
I think I'll make a list on google docs or something similar, so everyone can put the games that are problematic or not. (if you think it might be useful?).
It would be really useful during this dev process (and if you have/can get a github account) if you could post a comment(s) on the current ZXHawk bug issue with any games you find that don't load/have problems etc.
https://github.com/TASVideos/BizHawk/issues/1158
This is primarily where I am working from with regards to bugfixing/development.
phoenix1291 wrote:
Do you know if now with your modifications these last days the support of the dsk format is accurate, for loading, "data processing" etc?
Yes, I believe the dsk format support is done (for reading at least). I may have to expand the underlying structure that the dsk image is parsed into eventually to get better geometry and physical layout information (which I think will be important when it comes to timing), but otherwise the dsk format is a fairly simple format to understand.
As I mentioned previously, timing is not accurate at all yet.
As far as "data processing" within the floppy disk controller, I believe this should be fairly accurate, although tweaks will need to be made is games do not load. And obviously as also stated there are no write or seek commands implemented in the FDC yet (although I havent found a game that uses them yet).
The UPD floppy disk controller has the capacity to be quite a bit more complex, but the Spectrum does NOT use a great deal of the functionality. For example, DMA is not used and there is no interrupt line connected between the ULA and FDC.
This means that from a data-level, everything is pretty simple. Pretty much between every read and write request the CPU makes of the controller, the CPU has to also read the controller's status register. This tells the CPU what to do next. This is a very deterministic process and is why we can get games loading properly with NO emulated drive timing at all. In a real spectrum if the disk drive is not ready (maybe it is still seeking/recalibrating, or data hasnt yet been read from the disk) the main status register is updated on the FDC to inform the CPU to 'come back later' the next time that particular piece of Z80 code cycles around.
This gives a couple of (seemingly) simple timing implementation options when i get round to it.
Anyway, enough (potentially uninteresting) rambling :)
Joined: 11/27/2017
Posts: 86
Location: United Kingdom
phoenix1291 wrote:
I keep testing games.
OutRun works very well (the disk version, the tzx versions I tested seemed to load indefinitely), but there is no function yet to change the disk it seems to me?
Correct, I was going to do that once I got my head out of all the different copy protections.
...Which, I *THINK* I have done in the latest dev release just now :)
If you find any dsk games that don't load, please let me know.
Anyway, I should hopefully have multiple disks supported at some point tomorrow. Initially, it will be the same deal as the tapes. You create an xml using the multi-disk bundler tool, then there will be configurable keys for 'Next Disk' and 'Previous Disk'. When asked to change disks (or turn the disk over) you just cycle through till you get to the one you want. I guess this could also be menu driven as well eventually (like the c64 core).
Anyways... tomorrow :)
*edit: and I just read your post properly. It should be fairly easy to make disk/tape changing menu options. I will get this done sooner or later.