⮜ Blog

⮜ List of tags

Showing all posts tagged
and

📝 Posted:
🚚 Summary of:
P0203, P0204
Commits:
4568bf7...86cdf5f, 86cdf5f...0c682b5
💰 Funded by:
GhostRiderCog, [Anonymous], Yanga
🏷 Tags:

Let's start right with the milestones:


So, how did this card-flipping stage obstacle delivery get so horribly delayed? With all the different layouts showcased in the 28 card-flipping stages, you'd expect this to be among the more stable and bug-free parts of the codebase. Heck, with all stage objects being placed on a 32×32-pixel grid, this is the first TH01-related blog post this year that doesn't have to describe an alignment-related unblitting glitch!

That alone doesn't mean that this code is free from quirky behavior though, and we have to look no further than the first few lines of the collision handling for round bumpers to already find a whole lot of that. Simplified, they do the following:

pixel_t delta_y_between_orb_and_bumper = (orb.top - bumper.top);
if(delta_y_between_orb_and_bumper <= 0) {
	orb.top = (bumper.top - 24);
} else {
	orb.top = (bumper.top + 24);
}

Immediately, you wonder why these assignments only exist for the Y coordinate. Sure, hitting a bumper from the left or right side should happen less often, but it's definitely possible. Is it really a good idea to warp the Orb to the top or bottom edge of a bumper regardless?
What's more important though: The fact that these immediate assignments exist at all. The game's regular Orb physics work by producing a Y velocity from the single force acting on the Orb and a gravity factor, and are completely independent of its current Y position. A bumper collision does also apply a new force onto the Orb further down in the code, but these assignments still bypass the physics system and are bound to have some knock-on effect on the Orb's movement.

To observe that effect, we just have to enter Stage 18 on the 地獄/Jigoku route, where it's particularly trivial to reproduce. At a 📝 horizontal velocity of ±4, these assignments are exactly what can cause the Orb to endlessly bounce between two bumpers. As rudimentary as the Orb's physics may be, just letting them do their work would have entirely prevented these loops:

One of at least three infinite bumper loop constellations within just this 10×5-tile section of TH01's Stage 18 on the 地獄/Jigoku route. With an effective 56 horizontal pixels between both hitboxes, the Orb would have to travel an absolute Y distance of at least 16 vertical pixels within (56 / 4) = 14 frames to escape the other bumper's hitbox. If the initial bounce reduces the Orb's Y velocity far enough for it to not manage that distance the first time, it will never reach the necessary speed again. In this loop, the bounce-off force even stabilizes, though this doesn't have to happen. The blue areas indicate the pixel-perfect* hitboxes of each bumper.
TH01 bumper collision handling without ZUN's manual assignment of the Y coordinate. The Orb still bounces back and forth between two bumpers for a while, but its top position always follows naturally from its Y velocity and the force applied to it, and gravity wins out in the end. The blue areas indicate the pixel-perfect* hitboxes of each bumper.

Now, you might be thinking that these Y assignments were just an attempt to prevent the Orb from colliding with the same bumper again on the next frame. After all, those 24 pixels exactly correspond to ⅓ of the height of a bumper's hitbox with an additional pixel added on top. However, the game already perfectly prevents repeated collisions by turning off collision testing with the same bumper for the next 7 frames after a collision. Thus, we can conclude that ZUN either explicitly coded bumper collision handling to facilitate these loops, or just didn't take out that code after inevitably discovering what it did. This is not janky code, it's not a glitch, it's not sarcasm from my end, and it's not the game's physics being bad.

But wait. Couldn't these assignments just be a remnant from a time in development before ZUN decided on the 7-frame delay on further collisions? Well, even that explanation stops holding water after the next few lines of code. Simplified, again:

pixel_t delta_x_between_orb_and_bumper = (orb.left - bumper.left);
if((orb.velocity.x == +4) && (delta_x_between_orb_and_bumper < 0)) {
	orb.velocity.x = -4;
} else if((orb.velocity.x == -4) && (delta_x_between_orb_and_bumper > 0)) {
	orb.velocity.x = +4;
}

What's important here is the part that's not in the code – namely, anything that handles X velocities of -8 or +8. In those cases, the Orb simply continues in the same horizontal direction. The manual Y assignment is the only part of the code that actually prevents a collision there, as the newly applied force is not guaranteed to be enough:

An infinite loop across three bumpers, made possible by the edge of the playfield and bumper bars on opposite sides, an unchanged horizontal direction, and the Y assignments neatly placing the Orb on either the top or bottom side of a bumper. The alternating sign of the force further ensures that the Orb will travel upwards half the time, canceling out gravity during the short time between two hitboxes.
With the unchanged horizontal direction and the Y assignments removed, nothing keeps an Orb at ±8 pixels per frame from flying into/over a bumper. The collision force pushes the Orb slightly, but not enough to truly matter. The final force sends the Orb on a significant downward trajectory beyond the next bumper's hitbox, breaking the original loop.

Forgetting to handle ⅖ of your discrete X velocity cases is simply not something you do by accident. So we might as well say that ZUN deliberately designed the game to behave exactly as it does in this regard.


Bumpers also come in vertical or horizontal bar shapes. Their collision handling also turns off further collision testing for the next 7 frames, and doesn't do any manual coordinate assignment. That's definitely a step up in cleanliness from round bumpers, but it doesn't seem to keep in mind that the player can fire a new shot every 4 frames when standing still. That makes it immediately obvious why this works:

The green numbers show the amount of frames since the last detected collision with the respective bumper bar, and indicate that collision testing with the bar below is currently disabled.

That's the most well-known case of reducing the Orb's horizontal velocity to 0 by exactly hitting it with shots in its center and then button-mashing it through a horizontal bar. This also works with vertical bars and yields even more interesting results there, but if we want to have any chance of understanding what happens there, we have to first go over some basics:

However, if that were everything the game did, kicking the Orb into a column of vertical bumper bars would lead them to behave more like a rope that the Orb can climb, as the initial collision with two hitboxes cancels out the intended sign change that reflects the Orb away from the bars:

This footage was recorded without the workaround I am about to describe. It does not reflect the behavior of the original game. You cannot do this in the original game.
While the visualization reveals small sections where three hitboxes overlap, the Orb can never actually collide with three of them at the same time, as those 3-hitbox regions are 2 pixels smaller than they would need to be to fit the Orb. That's exactly the difference between using < rather than <= in these hitbox comparisons.

While that would have been a fun gameplay mechanic on its own, it immediately breaks apart once you place two vertical bumper bars next to each other. Due to how these bumper bar hitboxes extend past their sprites, any two adjacent vertical bars will end up with the exact same hitbox in absolute screen coordinates. Stage 17 on the 魔界/Makai route contains exactly such a layout:

The collision handlers of adjacent vertical bars always activate in the same frame, independently invert the Orb's X velocity, and therefore fully cancel out their intended effect on the Orb… if the game did not have the workaround I am about to describe. This cannot happen in the original game.

ZUN's workaround: Setting a "vertical bumper bar block flag" after any collision with such a bar, which simply disables any collision with any vertical bar for the next 7 frames. This quick hack made all vertical bars work as intended, and avoided the need for involving the Orb's X velocity in any kind of physics system. :zunpet:


Edit (2022-07-12): This flag only works around glitches that would be caused by simultaneously colliding with more than one vertical bar. The actual response to a bumper bar collision still remains unaffected, and is very naive:

These conditions are only correct if the Orb comes in at an angle roughly between 45° and 135° on either side of a bar. If it's anywhere close to 0° or 180°, this response will be incorrect, and send the Orb straight through the bar. Since the large hitboxes make this easily possible, you can still get the Orb to climb a vertical column, or glide along a horizontal row:

Here's the hitbox overlay for 地獄/Jigoku Stage 19, and here's an updated version of the 📝 Orb physics debug mod that now also shows bumper bar collision frame numbers: 2022-07-10-TH01OrbPhysicsDebug.zip See the th01_orb_debug branch for the code. To use it, simply replace REIIDEN.EXE, and run the game in debug mode, via game d on the DOS prompt. If you encounter a gameplay situation that doesn't seem to be covered by this blog post, you can now verify it for yourself. Thanks to touhou-memories for bringing these issues to my attention! That definitely was a glaring omission from the initial version of this blog post.


With that clarified, we can now try mashing the Orb into these two vertical bars:

At first, that workaround doesn't seem to make a difference here. As we expect, the frame numbers now tell us that only one of the two bumper bars in a row activates, but we couldn't have told otherwise as the number of bars has no effect on newly applied Y velocity forces. On a closer look, the Orb's rise to the top of the playfield is in fact caused by that workaround though, combined with the unchanged top-to-bottom order of collision testing. As soon as any bumper bar completed its 7 collision delay frames, it resets the aforementioned flag, which already reactivates collision handling for any remaining vertical bumper bars during the same frame. Look out for frames with both a 7 and a 1, like the one marked in the video above: The 7 will always appear before the 1 in the row-major order. Whenever this happens, the current oscillation period is cut down from 7 to 6 frames – and because collision testing runs from top to bottom, this will always happen during the falling part. Depending on the Y velocity, the rising part may also be cut down to 6 frames from time to time, but that one at least has a chance to last for the full 7 frames. This difference adds those crucial extra frames of upward movement, which add up to send the Orb to the top. Without the flag, you'd always see the Orb oscillating between a fixed range of the bar column.
Finally, it's the "top of playfield" force that gradually slows down the Orb and makes sure it ultimately only moves at sub-pixel velocities, which have no visible effect. Because 📝 the regular effect of gravity is reset with each newly applied force, it's completely negated during most of the climb. This even holds true once the Orb reached the top: Since the Orb requires a negative force to repeatedly arrive up there and be bounced back, this force will stay active for the first 5 of the 7 collision frames and not move the Orb at all. Once gravity kicks in at the 5th frame and adds 1 to the Y velocity, it's already too late: The new velocity can't be larger than 0.5, and the Orb only has 1 or 2 frames before the flag reset causes it to be bounced back up to the top again.


Portals, on the other hand, turn out to be much simpler than the old description that ended up on Touhou Wiki in October 2005 might suggest. Everything about their teleportations is random: The destination portal, the exit force (as an integer between -9 and +9), as well as the exit X velocity, with each of the 📝 5 distinct horizontal velocities having an equal chance of being chosen. Of course, if the destination portal is next to the left or right edge of the playfield and it chooses to fire the Orb towards that edge, it immediately bounces off into the opposite direction, whereas the 0 velocity is always selected with a constant 20% probability.

The selection process for the destination portal involves a bit more than a single rand() call. The game bundles all obstacles in a single structure of dynamically allocated arrays, and only knows how many obstacles there are in total, not per type. Now, that alone wouldn't have much of an impact on random portal selection, as you could simply roll a random obstacle ID and try again if it's not a portal. But just to be extra cute, ZUN instead iterates over all obstacles, selects any non-entered portal with a chance of ¼, and just gives up if that dice roll wasn't successful after 16 loops over the whole array, defaulting to the entered portal in that case.
In all its silliness though, this works perfectly fine, and results in a chance of 0.7516(𝑛 - 1) for the Orb exiting out of the same portal it entered, with 𝑛 being the total number of portals in a stage. That's 1% for two portals, and 0.01% for three. Pretty decent for a random result you don't want to happen, but that hurts nobody if it does.

The one tiny ZUN bug with portals is technically not even part of the newly decompiled code here. If Reimu gets hit while the Orb is being sent through a portal, the Orb is immediately kicked out of the portal it entered, no matter whether it already shows up inside the sprite of the destination portal. Neither of the two portal sprites is reset when this happens, leading to "two Orbs" being visible simultaneously. :tannedcirno::onricdennat:
This makes very little sense no matter how you look at it. The Orb doesn't receive a new velocity or force when this happens, so it will simply re-enter the same portal once the gameplay resumes on Reimu's next life:

And that's it! At least the turrets don't have anything notable to say about them 📝 that I haven't said before.


That left another ½ of a push over at the end. Way too much time to finish FUUIN.exe, way too little time to start with Mima… but the bomb animation fit perfectly in there. No secrets or bugs there, just a bunch of sprite animation code wasting at least another 82 bytes in the data segment. The special effect after the kuji-in sprites uses the same single-bitplane 32×32 square inversion effect seen at the end of Kikuri's and Sariel's entrance animation, except that it's a 3-stack of 16-rings moving at 6, 7, and 8 pixels per frame respectively. At these comparatively slow speeds, the byte alignment of each square adds some further noise to the discoloration pattern… if you even notice it below all the shaking and seizure-inducing hardware palette manipulation.
And yes, due to the very destructive nature of the effect, the game does in fact rely on it only being applied to VRAM page 0. While that will cause every moving sprite to tear holes into the inverted squares along its trajectory, keeping a clean playfield on VRAM page 1 is what allows all that pixel damage to be easily undone at the end of this 89-frame animation.

Next up: Mima! Let's hope that stage obstacles already were the most complex part remaining in TH01…

📝 Posted:
🚚 Summary of:
P0147
Commits:
456b621...c940059
💰 Funded by:
Ember2528, -Tom-
🏷 Tags:

Didn't quite get to cover background rendering for TH05's Stage 1-5 bosses in this one, as I had to reverse-engineer two more fundamental parts involved in boss background rendering before.

First, we got the those blocky transitions from stage tiles to bomb and boss backgrounds, loaded from BB*.BB and ST*.BB, respectively. These files store 16 frames of animation, with every bit corresponding to a 16×16 tile on the playfield. With 384×368 pixels to be covered, that would require 69 bytes per frame. But since that's a very odd number to work with in micro-optimized ASM, ZUN instead stores 512×512 pixels worth of bits, ending up with a frame size of 128 bytes, and a per-frame waste of 59 bytes. :tannedcirno: At least it was possible to decompile the core blitting function as __fastcall for once.
But wait, TH05 comes with, and loads, a bomb .BB file for every character, not just for the Reimu and Yuuka bomb transitions you see in-game… 🤔 Restoring those unused stage tile → bomb image transition animations for Mima and Marisa isn't that trivial without having decompiled their actual bomb animation functions before, so stay tuned!

Interestingly though, the code leaves out what would look like the most obvious optimization: All stage tiles are unconditionally redrawn each frame before they're erased again with the 16×16 blocks, no matter if they weren't covered by such a block in the previous frame, or are going to be covered by such a block in this frame. The same is true for the static bomb and boss background images, where ZUN simply didn't write a .CDG blitting function that takes the dirty tile array into account. If VRAM writes on PC-98 really were as slow as the games' README.TXT files claim them to be, shouldn't all the optimization work have gone towards minimizing them? :thonk: Oh well, it's not like I have any idea what I'm talking about here. I'd better stop talking about anything relating to VRAM performance on PC-98… :onricdennat:


Second, it finally was time to solve the long-standing confusion about all those callbacks that are supposed to render the playfield background. Given the aforementioned static bomb background images, ZUN chose to make this needlessly complicated. And so, we have two callback function pointers: One during bomb animations, one outside of bomb animations, and each boss update function is responsible for keeping the former in sync with the latter. :zunpet:

Other than that, this was one of the smoothest pushes we've had in a while; the hardest parts of boss background rendering all were part of 📝 the last push. Once you figured out that ZUN does indeed dynamically change hardware color #0 based on the current boss phase, the remaining one function for Shinki, and all of EX-Alice's background rendering becomes very straightforward and understandable.


Meanwhile, -Tom- told me about his plans to publicly release 📝 his TH05 scripting toolkit once TH05's MAIN.EXE would hit around 50% RE! That pretty much defines what the next bunch of generic TH05 pushes will go towards: bullets, shared boss code, and one full, concrete boss script to demonstrate how it's all combined. Next up, therefore: TH04's bullet firing code…? Yes, TH04's. I want to see what I'm doing before I tackle the undecompilable mess that is TH05's bullet firing code, and you all probably want readable code for that feature as well. Turns out it's also the perfect place for Blue Bolt's pending contributions.

📝 Posted:
🚚 Summary of:
P0111, P0112
Commits:
8b5c146...4ef4c9e, 4ef4c9e...e447a2d
💰 Funded by:
[Anonymous], Blue Bolt
🏷 Tags:

Only one newly ordered push since I've reopened the store? Great, that's all the justification I needed for the extended maintenance delay that was part of these two pushes 😛

Having to write comments to explain whether coordinates are relative to the top-left corner of the screen or the top-left corner of the playfield has finally become old. So, I introduced distinct types for all the coordinate systems we typically encounter, applying them to all code decompiled so far. Note how the planar nature of PC-98 VRAM meant that X and Y coordinates also had to be different from each other. On the X side, there's mainly the distinction between the [0; 640] screen space and the corresponding [0; 80] VRAM byte space. On the Y side, we also have the [0; 400] screen space, but the visible area of VRAM might be limited to [0; 200] when running in the PC-98's line-doubled 640×200 mode. A VRAM Y coordinate also always implies an added offset for vertical scrolling.
During all of the code reconstruction, these types can only have a documenting purpose. Turning them into anything more than just typedefs to int, in order to define conversion operators between them, simply won't recompile into identical binaries. Modding and porting projects, however, now have a nice foundation for doing just that, and can entirely lift coordinate system transformations into the type system, without having to proofread all the meaningless int declarations themselves.


So, what was left in terms of memory references? EX-Alice's fire waves were our final unknown entity that can collide with the player. Decently implemented, with little to say about them.

That left the bomb animation structures as the one big remaining PI blocker. They started out nice and simple in TH04, with a small 6-byte star animation structure used for both Reimu and Marisa. TH05, however, gave each character her own animation… and what the hell is going on with Reimu's blue stars there? Nope, not going to figure this out on ASM level.

A decompilation first required some more bomb-related variables to be named though. Since this was part of a generic RE push, it made sense to do this in all 5 games… which then led to nice PI gains in anything but TH05. :tannedcirno: Most notably, we now got the "pulling all items to player" flag in TH04 and TH05, which is actually separate from bombing. The obvious cheat mod is left as an exercise to the reader.


So, TH05 bomb animations. Just like the 📝 custom entity types of this game, all 4 characters share the same memory, with the superficially same 10-byte structure.
But let's just look at the very first field. Seen from a low level, it's a simple struct { int x, y; } pos, storing the current position of the character-specific bomb animation entity. But all 4 characters use this field differently:

Therefore, I decompiled it as 4 separate structures once again, bundled into an union of arrays.

As for Reimu… yup, that's some pointer arithmetic straight out of Jigoku* for setting and updating the positions of the falling star trails. :zunpet: While that certainly required several comments to wrap my head around the current array positions, the one "bug" in all this arithmetic luckily has no effect on the game.
There is a small glitch with the growing circles, though. They are spawned at the end of the loop, with their position taken from the star pointer… but after that pointer has already been incremented. On the last loop iteration, this leads to an out-of-bounds structure access, with the position taken from some unknown EX-Alice data, which is 0 during most of the game. If you look at the animation, you can easily spot these bugged circles, consistently growing from the top-left corner (0, 0) of the playfield:


After all that, there was barely enough remaining time to filter out and label the final few memory references. But now, TH05's MAIN.EXE is technically position-independent! 🎉 -Tom- is going to work on a pretty extensive demo of this unprecedented level of efficient Touhou game modding. For a more impactful effect of both the 100% PI mark and that demo, I'll be delaying the push covering the remaining false positives in that binary until that demo is done. I've accumulated a pretty huge backlog of minor maintenance issues by now…
Next up though: The first part of the long-awaited build system improvements. I've finally come up with a way of sanely accelerating the 32-bit build part on most setups you could possibly want to build ReC98 on, without making the building experience worse for the other few setups.

📝 Posted:
🚚 Summary of:
P0034, P0035
Commits:
6cdd229...6f1f367, 6f1f367...a533b5d
💰 Funded by:
zorg
🏷 Tags:

Deathbombs confirmed, in both TH04 and TH05! On the surface, it's the same 8-frame window as in most Windows games, but due to the slightly lower PC-98 frame rate of 56.4 Hz, it's actually slightly more lenient in TH04 and TH05.

The last function in front of the TH05 shot type control functions marks the player's previous position in VRAM to be redrawn. But as it turns out, "player" not only means "the player's option satellites on shot levels ≥ 2", but also "the explosion animation if you lose a life", which required reverse-engineering both things, ultimately leading to the confirmation of deathbombs.

It actually was kind of surprising that we then had reverse-engineered everything related to rendering all three things mentioned above, and could also cover the player rendering function right now. Luckily, TH05 didn't decide to also micro-optimize that function into un-decompilability; in fact, it wasn't changed at all from TH04. Unlike the one invalidation function whose decompilation would have actually been the goal here…

But now, we've finally gotten to where we wanted to… and only got 2 outstanding decompilation pushes left. Time to get the website ready for hosting an actual crowdfunding campaign, I'd say – It'll make a better impression if people can still see things being delivered after the big announcement.