⮜ Blog

⮜ List of tags

Showing all posts tagged gaiji-

📝 Posted:
🚚 Summary of:
💰 Funded by:
🏷 Tags:
rec98+ th04+ th05+ gaiji- pipeline+ position-independence+

Wouldn't it be a bit disappointing to have TH05 completely position-independent, but have it still require hex-editing of the original ZUN.COM to mod its gaiji characters? As in, these custom "text" glyphs, available to the PC-98 text RAM:

Especially since we now even have a sprite converter… the lack of which was exactly 📝 what made rebuilding ZUN.COM not that worthwhile before. So, before the big release, let's get all the remaining ZUN.COM sub-binaries of TH04 and TH05 dumped into .ASM files, and re-assembled and linked during the build process.

This is also the moment in which Egor's 2018 reimplementation of O. Morikawa's comcstm finally gets to shine. Back then, I considered it too early to even bother with ZUN.COM and reimplementing the .COM wrapper that ZUN originally used to bundle multiple smaller executables into that single binary. But now that the time is right, it is nice to have that code, as it allowed me to get these rebuilds done in half a push. Otherwise, it would have surely required one or two dedicated ones.

Since we like correctness here, newly dumped ZUN code means that it also has to be included in the RE% baseline calculation. This is why TH04's and TH05's overall RE% bars have gone back a tiny bit… in case you remember how they previously looked like :tannedcirno: After all, I would like to figure out where all that memory allocated during TH04's and TH05's memory check is freed, if at all.

Alright, one half of a push left… Y'know, getting rid of those last few PI false positives is actually one of the most annoying chores in this project, and quite stressful as well: I have to convince myself that the remaining false positives are, in fact, not memory references, but with way too little time for in-depth RE and to denote what they are instead. In that situation, everyone (including myself!) is anticipating that PI goal, and no one is really interested in RE. (Well… that is, until they actually get to developing their mod. But more on that tomorrow. :onricdennat:) Which means that it boils down to quite some hasty, dumb, and superficial RE around those remaining numbers.

So, in the hope of making it less annoying for the other 4 games in the future, let's systematically cover the sources of those remaining false positives in TH05, over all games. I/O port accesses with either the port or the value in registers (and thus, no longer as an immediate argument to the IN or OUT instructions, which the PI counter can clearly ingore), palette color arithmetic, or heck, 0xFF constants that obviously just mean "-1" and are not a reference to offset 0xFF in the data segment. All of this, of course, once again had a way bigger effect on everything but an almost position-independent TH05… but hey, that's the sort of thing you reserve the "anything" pushes for. And that's also how we get some of the single biggest PI% gains we have seen so far, and will be seeing before the 100% PI mark. And yes, those will continue in the next push.

Alright! Big release tomorrow…

📝 Posted:
🚚 Summary of:
P0076, P0077
222fc99...9ae9754, 9ae9754...f4eb7a8
💰 Funded by:
[Anonymous], -Tom-, Splashman
🏷 Tags:
rec98+ th02+ th03+ th04+ th05+ resident+ gaiji- tcc+

Well, that took twice as long as I thought, with the two pushes containing a lot more maintenance than actual new research. Spending some time improving both field names and types in 32th System's TH03 resident structure finally gives us all of those structures. Which means that we can now cover all the remaining decompilable ZUN.COM parts at once…

Oh wait, their main() functions have stayed largely identical since TH02? Time to clean up and separate that first, then… and combine two recent code generation observations into the solution to a decompilation puzzle from 4½ years ago. Alright, time to decomp-

Oh wait, we'd kinda like to properly RE all the code in TH03-TH05 that deals with loading and saving .CFG files. Almost every outside contributor wanted to grab this supposedly low-hanging fruit a lot earlier, but (of course) always just for a single game, while missing how the format evolved.

So, ZUN.COM. For some reason, people seem to consider it particularly important, even though it contains neither any game logic nor any code specific to PC-98 hardware… All that this decompilable part does is to initialize a game's .CFG file, allocate an empty resident structure using master.lib functions, release it after you quit the game, error-check all that, and print some playful messages~ (OK, TH05's also directly fills the resident structure with all data from MIKO.CFG, which all the other games do in OP.EXE.) At least modders can now freely change and extend all the resident structures, as well as the .CFG files? And translators can translate those messages that you won't see on a decently fast emulator anyway? Have fun, I guess 🤷‍

And you can in fact do this right now – even for TH04 and TH05, whose ZUN.COM currently isn't rebuilt by ReC98. There is actually a rather involved reason for this:

  • One of the missing files is TH05's GJINIT.COM.
  • Which contains all of TH05's gaiji characters in hardcoded 1bpp form, together with a bit of ASM for writing them to the PC-98's hardware gaiji RAM
  • Which means we'd ideally first like to have a sprite compiler, for all the hardcoded 1bpp sprites
  • Which must compile to an ASM slice in the meantime, but should also output directly to an OMF .OBJ file (for performance now), as well as to C code (for portability later)
  • The custom build system I've been using since mid-August has some declarations for OMF .OBJ files, but it needs maybe 1 or 2 more weeks of polish to be shipped
  • Which I won't put in as long as the backlog contains actual progress to drive up the percentages on the front page.

So yeah, no meaningful RE and PI progress at any of these levels. Heck, even as a modder, you can just replace the zun zun_res (TH02), zun -5 (TH03), or zun -s (TH04/TH05) calls in GAME.BAT with a direct call to your modified *RES*.COM. And with the alternative being "manually typing 0 and 1 bits into a text file", editing the sprites in TH05's GJINIT.COM is way more comfortable in a binary sprite editor anyway.

For me though, the best part in all of this was that it finally made sense to throw out the old Borland C++ run-time assembly slices 🗑 This giant waste of time became obvious 5 years ago, but any ASM dump of a .COM file would have needed rather ugly workarounds without those slices. Now that all .COM binaries that were originally written in C are compiled from C, we can all enjoy slightly faster grepping over the entire repository, which now has 229 fewer files. Productivity will skyrocket! :tannedcirno:

Next up: Three weeks of almost full-time ReC98 work! Two more PI-focused pushes to finish this TH05 stretch first, before switching priorities to TH01 again.