Oh look, it's another rather short and straightforward boss with a rather
small number of bugs and quirks. Yup, contrary to the character's
popularity, Mima's premiere is really not all that special in terms of code,
and continues the trend established with
📝 Kikuri and
📝 SinGyoku. I've already covered
📝 the initial sprite-related bugs last November,
so this post focuses on the main code of the fight itself. The overview:
The TH01 Mima fight consists of 3 phases, with phases 1 and 3 each
corresponding to one half of the 12-HP bar.
📝 Just like with SinGyoku, the distinction
between the red-white and red parts is purely visual once again, and doesn't
reflect anything about the boss script. As usual, all of the phases have to
be completed in order.
Phases 1 and 3 cycle through 4 danmaku patterns each, for a total of 8.
The cycles always start on a fixed pattern.
3 of the patterns in each phase feature rotating white squares, thus
introducing a new sprite in need of being unblitted.
Phase 1 additionally features the "hop pattern" as the last one in its
cycle. This is the only pattern where Mima leaves the seal in the center of
the playfield to hop from one edge of the playfield towards the other, while
also moving slightly higher up on the Y axis, and staying on the final
position for the next pattern cycle. For the first time, Mima selects a
random starting edge, which is then alternated on successive cycles.
Since the square entities are local to the respective pattern function,
Phase 1 can only end once the current pattern is done, even if Mima's HP are
already below 6. This makes Mima susceptible to the
📝 test/debug mode HP bar heap corruption bug.
Phase 2 simply consists of a spread-in teleport back to Mima's initial
position in the center of the playfield. This would only have been strictly
necessary if phase 1 ended on the hop pattern, but is done regardless of the
previous pattern, and does provide a nice visual separation between the two
main phases.
That's it – nothing special in Phase 3.
And there aren't even any weird hitboxes this time. What is maybe
special about Mima, however, is how there's something to cover about all of
her patterns. Since this is TH01, it's won't surprise anyone that the
rotating square patterns are one giant copy-pasta of unblitting, updating,
and rendering code. At least ZUN placed the core polar→Cartesian
transformation in a separate function for creating regular polygons
with an arbitrary number of sides, which might hint toward some more varied
shapes having been planned at one point?
5 of the 6 patterns even follow the exact same steps during square update
frames:
Calculate square corner coordinates
Unblit the square
Update the square angle and radius
Use the square corner coordinates for spawning pellets or missiles
Recalculate square corner coordinates
Render the square
Notice something? Bullets are spawned before the corner coordinates
are updated. That's why their initial positions seem to be a bit off – they
are spawned exactly in the corners of the square, it's just that it's
the square from 8 frames ago.
Mima's first pattern on Normal difficulty.
Once ZUN reached the final laser pattern though, he must have noticed that
there's something wrong there… or maybe he just wanted to fire those
lasers independently from the square unblit/update/render timer for a
change. Spending an additional 16 bytes of the data segment for conveniently
remembering the square corner coordinates across frames was definitely a
decent investment.
When Mima isn't shooting bullets from the corners of a square or hopping
across the playfield, she's raising flame pillars from the bottom of the playfield within very specifically calculated
random ranges… which are then rendered at byte-aligned VRAM positions, while
collision detection still uses their actual pixel position. Since I don't
want to sound like a broken record all too much, I'll just direct you to
📝 Kikuri, where we've seen the exact same issue with the teardrop ripple sprites.
The conclusions are identical as well.
Mima's flame pillar pattern. This video was recorded on a particularly
unlucky seed that resulted in great disparities between a pillar's
internal X coordinate and its byte-aligned on-screen appearance, leading
to lots of right-shifted hitboxes.
Also note how the change from the meteor animation to the three-arm 🚫
casting sprite doesn't unblit the meteor, and leaves that job to
any sprite that happens to fly over those pixels.
However, I'd say that the saddest part about this pattern is how choppy it
is, with the circle/pillar entities updating and rendering at a meager 7
FPS. Why go that low on purpose when you can just make the game render ✨
smoothly ✨ instead?
So smooth it's almost uncanny.
The reason quickly becomes obvious: With TH01's lack of optimization, going
for the full 56.4 FPS would have significantly slowed down the game on its
intended 33 MHz CPUs, requiring more than cheap surface-level ASM
optimization for a stable frame rate. That might very well have been ZUN's
reason for only ever rendering one circle per frame to VRAM, and designing
the pattern with these time offsets in mind. It's always been typical for
PC-98 developers to target the lowest-spec models that could possibly still
run a game, and implementing dynamic frame rates into such an engine-less
game is nothing I would wish on anybody. And it's not like TH01 is
particularly unique in its choppiness anyway; low frame rates are actually a
rather typical part of the PC-98 game aesthetic.
The final piece of weirdness in this fight can be found in phase 1's hop
pattern, and specifically its palette manipulation. Just from looking at the
pattern code itself, each of the 4 hops is supposed to darken the hardware
palette by subtracting #444 from every color. At the last hop,
every color should have therefore been reduced to a pitch-black
#000, leaving the player completely blind to the movement of
the chasing pellets for 30 frames and making the pattern quite ghostly
indeed. However, that's not what we see in the actual game:
Nothing in the pattern's code would cause the hardware palette to get
brighter before the end of the pattern, and yet…
The expected version doesn't look all too unfair, even on Lunatic…
well, at least at the default rank pellet speed shown in this
video. At maximum pellet speed, it is in fact rather brutal.
Looking at the frame counter, it appears that something outside the
pattern resets the palette every 40 frames. The only known constant with a
value of 40 would be the invincibility frames after hitting a boss with the
Orb, but we're not hitting Mima here…
But as it turns out, that's exactly where the palette reset comes from: The
hop animation darkens the hardware palette directly, while the
📝 infamous 12-parameter boss collision handler function
unconditionally resets the hardware palette to the "default boss palette"
every 40 frames, regardless of whether the boss was hit or not. I'd classify
this as a bug: That function has no business doing periodic hardware palette
resets outside the invincibility flash effect, and it completely defies
common sense that it does.
That explains one unexpected palette change, but could this function
possibly also explain the other infamous one, namely, the temporary green
discoloration in the Konngara fight? That glitch comes down to how the game
actually uses two global "default" palettes: a default boss
palette for undoing the invincibility flash effect, and a default
stage palette for returning the colors back to normal at the end of
the bomb animation or when leaving the Pause menu. And sure enough, the
stage palette is the one with the green color, while the boss
palette contains the intended colors used throughout the fight. Sending the
latter palette to the graphics chip every 40 frames is what corrects
the discoloration, which would otherwise be permanent.
The green color comes from BOSS7_D1.GRP, the scrolling
background of the entrance animation. That's what turns this into a clear
bug: The stage palette is only set a single time in the entire fight,
at the beginning of the entrance animation, to the palette of this image.
Apart from consistency reasons, it doesn't even make sense to set the stage
palette there, as you can't enter the Pause menu or bomb during a blocking
animation function.
And just 3 lines of code later, ZUN loads BOSS8_A1.GRP, the
main background image of the fight. Moving the stage palette assignment
there would have easily prevented the discoloration.
But yeah, as you can tell, palette manipulation is complete jank in this
game. Why differentiate between a stage and a boss palette to begin with?
The blocking Pause menu function could have easily copied the original
palette to a local variable before darkening it, and then restored it after
closing the menu. It's not so easy for bombs as the intended palette could
change between the start and end of the animation, but the code could have
still been simplified a lot if there was just one global "default palette"
variable instead of two. Heck, even the other bosses who manipulate their
palettes correctly only do so because they manually synchronize the two
after every change. The proper defense against bugs that result from wild
mutation of global state is to get rid of global state, and not to put up
safety nets hidden in the middle of existing effect code.
The easiest way of reproducing the green discoloration bug in
the TH01 Konngara fight, timed to show the maximum amount of time the
discoloration can possibly last.
In any case, that's Mima done! 7th PC-98 Touhou boss fully
decompiled, 24 bosses remaining, and 59 functions left in all of TH01.
In other thrilling news, my call for secondary funding priorities in new
TH01 contributions has given us three different priorities so far. This
raises an interesting question though: Which of these contributions should I
now put towards TH01 immediately, and which ones should I leave in the
backlog for the time being? Since I've never liked deciding on priorities,
let's turn this into a popularity contest instead: The contributions with
the least popular secondary priorities will go towards TH01 first, giving
the most popular priorities a higher chance to still be left over after TH01
is done. As of this delivery, we'd have the following popularity order:
TH05 (1.67 pushes), from T0182
Seihou (1 push), from T0184
TH03 (0.67 pushes), from T0146
Which means that T0146 will be consumed for TH01 next, followed by T0184 and
then T0182. I only assign transactions immediately before a delivery though,
so you all still have the chance to change up these priorities before the
next one.
Next up: The final boss of TH01 decompilation, YuugenMagan… if the current
or newly incoming TH01 funds happen to be enough to cover the entire fight.
If they don't turn out to be, I will have to pass the time with some Seihou
work instead, missing the TH01 anniversary deadline as a result.Edit (2022-07-18): Thanks to Yanga for
securing the funding for YuugenMagan after all! That fight will feature
slightly more than half of all remaining code in TH01's
REIIDEN.EXE and the single biggest function in all of PC-98
Touhou, let's go!
📝 7 pushes to get Konngara done, according to my previous estimate?
Well, how about being twice as fast, and getting the entire boss fight done
in 3.5 pushes instead? So much copy-pasted code in there… without any
flashy unused content, apart from four calculations with an unclear purpose. And the three strings "ANGEL", "OF",
"DEATH", which were probably meant to be rendered using those giant
upscaled font ROM glyphs that also display the
STAGE # and
HARRY UP strings? Those three strings
are also part of Sariel's code, though.
On to the remaining 11 patterns then! Konngara's homing snakes, shown in
the video above, are one of the more notorious parts of this battle. They
occur in two patterns – one with two snakes and one with four – with
all of the spawn, aim, update, and render code copy-pasted between
the two. Three gameplay-related discoveries
here:
The homing target is locked once the Y position of a snake's white head
diamond is below 300 pixels.
That diamond is also the only one with collision detection…
…but comes with a gigantic 30×30 pixel hitbox, reduced to 30×20 while
Reimu is sliding. For comparison: Reimu's regular sprite is 32×32 pixels,
including transparent areas. This time, there is a clearly defined
hitbox around Reimu's center pixel that the single top-left pixel can
collide with. No imagination necessary, which people apparently
📝 still prefer over actually understanding an
algorithm… Then again, this hitbox is still not intuitive at all,
because…
… the exact collision pixel, marked in
red, is part of the diamond sprite's
transparent background
This was followed by really weird aiming code for the "sprayed
pellets from cup" pattern… which can only possibly have been done on
purpose, but is sort of mitigated by the spraying motion anyway.
After a bunch of long if(…) {…} else if(…) {…} else if(…)
{…} chains, which remain quite popular in certain corners of
the game dev scene to this day, we've got the three sword slash
patterns as the final notable ones. At first, it seemed as if ZUN just
improvised those raw number constants involved in the pellet spawner's
movement calculations to describe some sort of path that vaguely
resembles the sword slash. But once I tried to express these numbers in
terms of the slash animation's keyframes, it all worked out perfectly, and
resulted in this:
Yup, the spawner always takes an exact path along this triangle. Sometimes,
I wonder whether I should just rush this project and don't bother about
naming these repeated number literals. Then I gain insights like these, and
it's all worth it.
Finally, we've got Konngara's main function, which coordinates the entire
fight. Third-longest function in both TH01 and all of PC-98 Touhou, only
behind some player-related stuff and YuugenMagan's gigantic main function…
and it's even more of a copy-pasta, making it feel not nearly as long as it
is. Key insights there:
The fight consists of 7 phases, with the entire defeat sequence being
part of the if(boss_phase == 7) {…}
branch.
The three even-numbered phases, however, only light up the Siddhaṃ seed
syllables and then progress to the next phase.
Odd-numbered phases are completed after passing an HP threshold or after
seeing a predetermined number of patterns, whatever happens first. No
possibility of skipping anything there.
Patterns are chosen randomly, but the available pool of patterns
is limited to 3 specific "easier" patterns in phases 1 and 5, and 4 patterns
in phase 3. Once Phase 7 is reached at 9 HP remaining, all 12 patterns can
potentially appear. Fittingly, that's also the point where the red section
of the HP bar starts.
Every time a pattern is chosen, the code only makes a maximum of two
attempts at picking a pattern that's different from the one that
Konngara just completed. Therefore, it seems entirely possible to see
the same pattern twice. Calculating an actual seed to prove that is out
of the scope of this project, though.
Due to what looks like a copy-paste mistake, the pool for the second
RNG attempt in phases 5 and 7 is reduced to only the first two patterns
of the respective phases? That's already quite some bias right there,
and we haven't even analyzed the RNG in detail yet…
(For anyone interested, it's a
LCG,
using the Borland C/C++ parameters as shown here.)
The difficulty level only affects the speed and firing intervals (and
thus, number) of pellets, as well as the number of lasers in the one pattern
that uses them.
After the 📝 kuji-in defeat sequence, the
fight ends in an attempted double-free of Konngara's image
data. Thankfully, the format-specific
_free() functions defend against such a thing.
And that's it for Konngara! First boss with not a single piece of ASM left,
30 more to go! 🎉 But wait, what about the cause behind the temporary green
discoloration after leaving the Pause menu? I expected to find something on
that as well, but nope, it's nothing in Konngara's code segment. We'll
probably only get to figure that out near the very end of TH01's
decompilation, once we get to the one function that directly calls all of
the boss-specific main functions in a switch statement. Edit (2022-07-17):📝 Only took until Mima.
So, Sariel next? With half of a push left, I did cover Sariel's first few
initialization functions, but all the sprite unblitting and HUD
manipulation will need some extra attention first. The first one of these
functions is related to the HUD, the stage timer, and the
HARRY UP mode, whose pellet pattern I've
also decompiled now.
All of this brings us past 75% PI in all games, and TH01 to under 30,000
remaining ASM instructions, leaving TH03 as the now most expensive game to
be completely decompiled. Looking forward to how much more TH01's code will
fall apart if you just tap it lightly… Next up: The aforementioned helper
functions related to HARRY UP, drawing the
HUD, and unblitting the other bosses whose sprites are a bit more animated.
Alright, onto Konngara! Let's quickly move the escape sequences used later
in the battle to C land, and then we can immediately decompile the loading
and entrance animation function together with its filenames. Might as well
reverse-engineer those escape sequences while I'm at it, though – even if
they aren't implemented in DOSBox-X, they're well documented in all those
Japanese PDFs, so this should be no big deal…
…wait, ESC )3 switches to "graph mode"? As opposed to the
default "kanji mode", which can be re-entered via ESC )0?
Let's look up graph mode in the PC-9801 Programmers' Bible then…
> Kanji cannot be handled in this mode.
…and that's apparently all it has to say. Why have it then, on a platform
whose main selling point is a kanji ROM, and where Shift-JIS (and, well,
7-bit ASCII) are the only native encodings? No support for graph mode in
DOSBox-X either… yeah, let's take a deep dive into NEC's
IO.SYS, and get to the bottom of this.
And yes, graph mode pretty much just disables Shift-JIS decoding for
characters written via INT 29h, the lowest-level way of "just
printing a char" on DOS, which every printf()
will ultimately end up calling. Turns out there is a use for it though,
which we can spot by looking at the 8×16 half-width section of font ROM:
The half-width glyphs marked in red
correspond to the byte ranges from 0x80-0x9F and 0xE0-0xFF… which Shift-JIS
defines as lead bytes for two-byte, full-width characters. But if we turn
off Shift-JIS decoding…
(Yes, that g in the function row is how NEC DOS
indicates that graph mode is active. Try it yourself by pressing
Ctrl+F4!)
Jackpot, we get those half-width characters when printing their
corresponding bytes. I've
re-implemented all my findings into DOSBox-X, which will include graph
mode in the upcoming 0.83.14 release. If P0140 looks a bit empty as a
result, that's why – most of the immediate feature work went into
DOSBox-X, not into ReC98. That's the beauty of "anything" pushes.
So, after switching to graph mode, TH01 does… one of the slowest possible
memset()s over all of text RAM – one printf(" ")
call for every single one of its 80×25 half-width cells – before switching
back to kanji mode. What a waste of RE time…? Oh well, at least we've now
got plenty of proof that these weird escape sequences actually do
nothing of interest.
As for the Konngara code itself… well, it's script-like code, what can you
say. Maybe minimally sloppy in some places, but ultimately harmless.
One small thing that might not be widely known though: The large,
blue-green Siddhaṃ seed syllables are supposed to show up immediately, with
no delay between them? Good to know. Clocking your emulator too low tends
to roll them down from the top of the screen, and will certainly add a
noticeable delay between the four individual images.
… Wait, but this means that ZUN could have intended this "effect".
Why else would he not only put those syllables into four individual images
(and therefore add at least the latency of disk I/O between them), but also
show them on the foreground VRAM page, rather than on the "back buffer"?
Meanwhile, in 📝 another instance of "maybe
having gone too far in a few places":
Expressing distances on the playfield as fractions of its width
and height, just to avoid absolute numbers? Raw numbers are bad because
they're in screen space in this game. But we've already been throwing
PLAYFIELD_ constants into the mix as a way of explicitly
communicating screen space, and keeping raw number literals for the actual
playfield coordinates is looking increasingly sloppy… I don't know,
fractions really seemed like the most sensible thing to do with what we're
given here. 😐
So, 2 pushes in, and we've got the loading code, the entrance animation,
facial expression rendering, and the first one out of Konngara's 12
danmaku patterns. Might not sound like much, but since that first pattern
involves those
blue-green diamond sprites and therefore is one of the more complicated
ones, it all amounts to roughly 21.6% of Konngara's code. That's 7 more
pushes to get Konngara done, then? Next up though: Two pushes of website
improvements.
Well, make that three days. Trying to figure out all the details behind
the sprite flickering was absolutely dreadful…
It started out easy enough, though. Unsurprisingly, TH01 had a quite
limited pellet system compared to TH04 and TH05:
The cap is 100, rather than 240 in TH04 or 180 in TH05.
Only 6 special motion functions (with one of them broken and unused)
instead of 10. This is where you find the code that generates SinGyoku's
chase pellets, Kikuri's small spinning multi-pellet circles, and
Konngara's rain pellets that bounce down from the top of the playfield.
A tiny selection of preconfigured multi-pellet groups. Rather than
TH04's and TH05's freely configurable n-way spreads, stacks, and rings,
TH01 only provides abstractions for 2-, 3-, 4-, and 5- way spreads (yup,
no 6-way or beyond), with a fixed narrow or wide angle between the
individual pellets. The resulting pellets are also hardcoded to linear
motion, and can't use the special motion functions. Maybe not the best
code, but still kind of cute, since the generated groups do follow a
clear logic.
As expected from TH01, the code comes with its fair share of smaller,
insignificant ZUN bugs and oversights. As you would also expect
though, the sprite flickering points to the biggest and most consequential
flaw in all of this.
Apparently, it started with ZUN getting the impression that it's only
possible to use the PC-98 EGC for fast blitting of all 4 bitplanes in one
CPU instruction if you blit 16 horizontal pixels (= 2 bytes) at a time.
Consequently, he only wrote one function for EGC-accelerated sprite
unblitting, which can only operate on a "grid" of 16×1 tiles in VRAM. But
wait, pellets are not only just 8×8, but can also be placed at any
unaligned X position…
… yet the game still insists on using this 16-dot-aligned function to
unblit pellets, forcing itself into using a super sloppy 16×8 rectangle
for the job. 🤦 ZUN then tried to mitigate the resulting flickering in two
hilarious ways that just make it worse:
An… "interlaced rendering" mode? This one's activated for all Stage 15
and 20 fights, and separates pellets into two halves that are rendered on
alternating frames. Collision detection with the Yin-Yang Orb and the
player is only done for the visible half, but collision detection with
player shots is still done for all pellets every frame, as are
motion updates – so that pellets don't end up moving half as fast as they
should.
So yeah, your eyes weren't deceiving you. The game does effectively
drop its perceived frame rate in the Elis, Kikuri, Sariel, and Konngara
fights, and it does so deliberately.
📝 Just like player shots, pellets
are also unblitted, moved, and rendered in a single function.
Thanks to the 16×8 rectangle, there's now the (completely unnecessary)
possibility of accidentally unblitting parts of a sprite that was
previously drawn into the 8 pixels right of a pellet. And this
is where ZUN went full and went "oh, I
know, let's test the entire 16 pixels, and in case we got an entity
there, we simply make the pellet invisible for this frame! Then
we don't even have to unblit it later!"
Except that this is only done for the first 3 elements of the player
shot array…?! Which don't even necessarily have to contain the 3 shots
fired last. It's not done for the player sprite, the Orb, or, heck,
other pellets that come earlier in the pellet array. (At least
we avoided going 𝑂(𝑛²) there?)
Actually, and I'm only realizing this now as I type this blog post:
This test is done even if the shots at those array elements aren't
active. So, pellets tend to be made invisible based on comparisons
with garbage data.
And then you notice that the player shot
unblit/move/render function is actually only ever called from the
pellet unblit/move/render function on the one global instance
of the player shot manager class, after pellets were unblitted. So, we
end up with a sequence of
which means that we can't ever unblit a previously rendered shot
with a pellet. Sure, as terrible as this one function call is from
a software architecture perspective, it was enough to fix this issue.
Yet we don't even get the intended positive effect, and walk away with
pellets that are made temporarily invisible for no reason at all. So,
uh, maybe it all just was an attempt at increasing the
ramerate on lower spec PC-98 models?
Yup, that's it, we've found the most stupid piece of code in this game,
period. It'll be hard to top this.
I'm confident that it's possible to turn TH01 into a well-written, fluid
PC-98 game, with no flickering, and no perceived lag, once it's
position-independent. With some more in-depth knowledge and documentation
on the EGC (remember, there's still
📝 this one TH03 push waiting to be funded),
you might even be able to continue using that piece of blitter hardware.
And no, you certainly won't need ASM micro-optimizations – just a bit of
knowledge about which optimizations Turbo C++ does on its own, and what
you'd have to improve in your own code. It'd be very hard to write
worse code than what you find in TH01 itself.
(Godbolt for Turbo C++ 4.0J when?
Seriously though, that would 📝 also be a
great project for outside contributors!)
Oh well. In contrast to TH04 and TH05, where 4 pushes only covered all the
involved data types, they were enough to completely cover all of
the pellet code in TH01. Everything's already decompiled, and we never
have to look at it again. 😌 And with that, TH01 has also gone from by far
the least RE'd to the most RE'd game within ReC98, in just half a year! 🎉
Still, that was enough TH01 game logic for a while.
Next up: Making up for the delay with some
more relaxing and easy pieces of TH01 code, that hopefully make just a
bit more sense than all this garbage. More image formats, mainly.
from the bottom of the playfield within very specifically calculated
random ranges… which are then rendered at byte-aligned VRAM positions, while
collision detection still uses their actual pixel position. Since I don't
want to sound like a broken record all too much, I'll just direct you to
📝 Kikuri, where we've seen the exact same issue with the teardrop ripple sprites.
The conclusions are identical as well.
Three gameplay-related discoveries
here:
Thankfully, the format-specific
blue-green diamond sprites and therefore is one of the more complicated
ones, it all amounts to roughly 21.6% of Konngara's code. That's 7 more
pushes to get Konngara done, then? Next up though: Two pushes of website
improvements.