Hacking DOS programs with DOSBox debugger

Introduction

So, I’ve decided that I want to crack the Descent II setup program and remove the CD check. I prefer not to have to search for my CD and put it in the drive every time I want to play a game, or run its setup program.

In the age of DOS gaming, CD-based copy protection was usually simple – the program would try to access the CD drive during its startup, and abort if it did not find what it expected to be there (the game disk). Sometimes it would still need to access actual data on the disk later on, but many games (Descent II included) allowed a full installation, where all the game content would be copied onto the hard drive, to speed things up. Thus, the CD check served the purpose of copy protection only (before duplicating CDs became easy), but, unfortunately, also acted as a hassle to legitimate owners, by forcing the CD to be always in the drive to play the game, or in this case, even to run the setup program to configure sound and what not.

Most programs thus protected would have the following type of code somewhere:

1. Check For presence of game CD in drive
2. If Not Found – Abort
3. Else – Continue execution

The simplest way to defeat such a protection, with no changes to the rest of the program, is to find the place where it happens, and modify the code so that regardless of the outcome of the check, the same “Continue execution” code path is taken. Code branching in higher languages is eventually translated to jump instructions in assembly / machine code, so “cutting off” one of the branches is done either by replacing a conditional jump (JZ/JE/JNZ/JNE/etc.) with an unconditional one (JMP), if the jump leads to the desired path, or, if it would lead to the undesired path, removing it altogether by placing a no-operation (NOP) CPU instruction. Alternatively, it may be possible to find the call to the routine which performs the CD check, and just replace it with NOPs (assuming that it eventually returns to the same place in the code and has no side effects).

Initial ideas (did not work)

The difficult part is finding where this code resides in the game executable. As I do not have much experience debugging x86 assembly, I wanted to try a shortcut first – I had a working “no-CD” crack for the Descent II game executable – DESCENT2.EXE (which I found elsewhere), so the first idea was:

1. Let’s assume the CD check code was shared between DESCENT2.EXE and SETUP.EXE
2. Compare the cracked and uncracked version of DESCENT2.EXE and find the place where it was changed (hopefully the change is of the simple nature I described above)
3. Try to find the equivalent code in SETUP.EXE, by searching for the same instructions sequences (represented by HEX strings) that exist in the surroundings of the ‘cracked’ area in DESCENT2.EXE). Hopefully it will be found and in a unique location (if it’s not unique, it’s possible to try to patch each location, one at a time, until one works).

This technique is frequently successful between different versions of the same program, as the copy protection routines and surrounding code rarely change. I used it in the past to infer the no-CD crack for some games, for which I had a cracked different-version executable as reference. Unfortunately, in this case, it quickly became clear that it will not work, as no similar instruction sequences were found in SETUP.EXE.

The next idea was to try to look into the disassembly of DESCENT2.EXE and SETUP.EXE, using IDA Free interactive disassembler, which has the ability to display nice graphs of how the code flow jumps between chunks of assembly, and correlate it to a HEX sequence view of the program. Both DESCENT2.EXE and SETUP.EXE are DOS/4GW-bound programs, which means they contain a 16-bit stub, just calling DOS/4GW (which is a separate executable, DOS4GW.EXE in the game directory), and a 32-bit “linear” executable part, which DOS/4GW calls back into. To allow IDA to properly figure out what’s going on in the 32-bit code (where all the program business logic is likely to be), the executable first needs to be unbound. The alternative DOS extender, DOS/32 Advanced, provides a utility to do just that.

I hoped that by viewing the linear executables in IDA, I’d be able to find patterns in the code around the CD check of DESCENT2.EXE that will look like something in SETUP.EXE, structure-wise. No obvious patterns were found (probably it was naive to think so in the first place). I thought I could perhaps trace back the code flow from the “program exit” points (B4 4C CD 21 – MOV AH, 4Ch; INT 21h) until I stumble on something that looks like it might be related to the CD check, but did not get far: even though there was only one such point in the linear executable, the places where it was called from quickly became too numerous. Besides, at this point I realized that I’m analyzing assembly code (which I hoped to avoid), and once I am doing that, I might as well use a debugger and try to step through the program as it executes.

Debugging in the DOS environment

Not having access (at the moment) to any real hardware running DOS, I tried using a debugger inside DOSBox. Debugging 32-bit DOS/4GW-bound programs is also, apparently, not always straightforward. First I tried using the Borland Turbo Debugger (TD.EXE). I could run the program in it, and step through some of the startup code. However, at some point, I reach a procedure call that, if I stepped over, would cause the program to resume execution outside of the debugger control (returning only upon termination), and if stepped into, would reach a bizarre code chunk with a few instructions that would repeat, and never continue. I assume that this is the place where control should transfer to DOS4/GW and back to the linear executable, and the debugger just does not handle that well. After learning that the Watcom debugger with the RSI trap (WD.EXE -TR=RSI) is what should be used to debug DOS4/GW programs, I tried that, but could not get it to work properly inside DOSBox.

I decided not to waste too much time on this, and switch to using the built-in DOSBox Debugger. DOSBox builds with debug enabled can be downloaded here, and a guide is available here.

DOSBox Debugger – initial experiments

With a debug-enabled DOSBox, the status window turns into a DOSBox Debugger window, which shows some low-level details about every program run inside DOSBox, such as files it attempts to open, and interrupts it triggers. It also has an interactive mode, invoked by the built-in debug command (which is not available in the standard build). During interactive debug, you can see disassembly, registers and memory contents, set breakpoints and step through the instructions of the emulated program execution. Note, that to step instruction-by-instruction, core=normal setting is required in DOSBox (if the core is in dynamic mode, only the first instruction of every 32-instruction block can be stopped at).

Just running SETUP.EXE with the debugger in the background immediately provided some useful insight:

Program output

F:\DESCENT2>setup
DOS/4GW Professional Protected Mode Run-time  Version 1.97
Copyright (c) Rational Systems, Inc. 1990-1994
Error: Cannot find a CD-ROM drive.

Debugger output

MISC:MSCDEX: INT 2F 1500 BX= 0000 CX=0000

The error message is due to no CD drives present in the DOSBox environment. Running the program with a CD drive mounted (but without the game CD) yields a different result.

Program output

F:\DESCENT2>setup
DOS/4GW Professional Protected Mode Run-time  Version 1.97
Copyright (c) Rational Systems, Inc. 1990-1994
Error: Descent II CD must be present to run SETUP.

Debugger output

MISC:MSCDEX: INT 2F 1500 BX= 0000 CX=0000
MISC:MSCDEX: INT 2F 1501 BX= 0000 CX=0000

It is thus evident, that there are at least two different checks performed by the setup program – one for the presence of a CD drive, and another one for the presence of the actual game disk. Both are implemented using Interrupt 0x2F – the DOS multiplex interrupt, which provides a general means for programs to know the availability and state of various DOS kernel services and TSRs. The exact functionality is determined by the sub-function number (in AX), and possibly additional parameters (in BX, CX, and so forth). The same registers can be used to return information from the interrupt handler to the caller. In this case, AH=15 indicates MSCDEX, and AL specifies the request. AL=00 returns the number of CD-ROM drive letters in BX, and AL=01 returns a list of the actual devices into the buffer specified by ES:BX. Naturally, if there are no CD drives, there is no point in the second call.

The high-level plan

As discussed above, there are two ways of defeating the check – you can either skip the call to it, or ignore its return value. The latter is typically easier, since it requires setting a breakpoint on the check routine, and tracing the code that follows, rather than going through the entire program assembly to find the code that jumps into the check routine. The DOSBox Debugger makes it especially easy, since it can set breakpoints on particular interrupts with specific sub-functions. In this case we want to trap interrupt 0x2F, with AH=15:

debug setup.exe    (from DOSBox command-line)
bpint 2f 15        (from DOSBOx debug window)

It should be noted, that SETUP.EXE, or DESCENT2.EXE for that matter, being protected mode 32-bit applications, do not call INT 2F directly, using instead the DPMI real-mode interrupt simulation routine – INT 31h, Function 0300h. However, it does not matter to DOSBox, which can still trap the specific underlying real-mode interrupt.

Had SETUP.EXE been a pure assembly program, it would have likely tested and reacted on the return values of the MSCDEX interrupts immediately, but obviously this is not the case. The values returned by the interrupt are passed through multiple wrappers and function calls before being handled by the “business logic”, so to speak. It may take going through dozens or hundreds (perhaps even thousands) of lines of assembly to get to that point. I figured out that the easiest approach would be the following:

1. Launch two instances of DOSBox; one will simulate a “good” environment (Descent II CD available), the other will simulate a “bad” environment (no Descent II CD or no CD-ROM drive at all).
2. Debug SETUP.EXE in parallel in both environments, placing breakpoints on the MSCDEX interrupts. Step through the assembly in parallel, looking for the earliest place where the executions diverge.
3. Look for the relevant instruction causing the divergence, and see if it can be patched so that the code path of the ‘good’ environment is taken in the ‘bad’ case as well.

Defeating the check for the number of CD drives

As was observed, SETUP.EXE errors out differently in case there are no CD drives mounted, versus the case of Descent II CD not being in the drive. This is different from DESCENT2.EXE, which performs both tests, but presents the same error message in either case. This suggests that in order to support an environment with no CD drives at all, at least two places in the code must be changed. I decided to start with the “number of drives” check, since it happens earlier, and started stepping through the code in parallel, after breaking on the first interrupt (INT 2Fh, AX=1500).

Like every debugger, the DOSBox one allows you to either step over an instruction (F10) or trace into it (F11). The latter steps into function calls and repeat string operations (REP/REPE/REPZ/REPNE/REPNZ), the former does not. One does not know in advance whether it’s important to trace into a CALL or not, since the called routine may transfer control somewhere else entirely, and that may be where the critical decisions are made. For example, these are the first 16 instructions executed, after the debugger breaks on INT 2Fh, AX=1500:

0657:1FBB CD2F       int 2F
0657:1FBD 9C         pushf 
0657:1FBE FA         cli
0657:1FBF 2E8F06A41F pop word cs:[1FA4]
0657:1FC4 681E02     push 021E
0657:1FC7 17         pop ss
0657:1FC8 368B269609 mov sp,ss:[0996]
0657:1FCD 6660       pushad
0657:1FCF 06         push es
0657:1FD0 1E         push ds
0657:1FD1 16         push ss
0657:1FD2 1F         pop ds
0657:1FD3 8BEC       mov bp,sp
0657:1FD5 8B4638     mov ax,[bp+38]
0657:1FD8 2EA3BB1F   mov cs:[1FBB],ax
0657:1FDC E8B1E2     call 00000290 ($-1d4f)

The last of these, call 00000290, never returns: the program errors out if I step over it, so I need to step into it. But I don’t necessarily want to step into every single CALL, and go through hundreds of unrelated instructions. There is a simple trick to speed-up the process: every time a CALL is reached, put a breakpoint on it (F9), then step over it (F10). If it returns, disable the breakpoint, and continue debugging; if it transfers control and does not return (program will either error out or display the main setup screen), run the debugger again, reach the breakpoint that was just placed, and now step into it (F11).

With this approach, it did not take long to get to the IRET that exits the real-mode interrupt handler and the RET from the DPMI interrupt table. Now we are getting close to the business logic. A few more function returns and we get to the first divergence point:

0180:1A1475 F745BCFFFF0000  test dword [ebp-0044],0000FF
0180:1A147C 750C            jne 001A148A ($+c)
0180:1A147E C745ECFFFFFFFF  mov dword [ebp-0014],FFFFFF
0180:1A1485 E990010000      jmp 001A161A ($+190)
0180:1A148A 8B45BC          mov eax,[ebp-0044]
0180:1A148D 8945F0          mov [ebp-0010],eax
0180:1A1490 B805000000      mov eax,00000005
0180:1A1495 E88C9A0000      call 001AAF26 ($+9a8c)
0180:1A149A 8945F8          mov [ebp-0008],eax
0180:1A149D 837DF800        cmp dword [ebp-0008],0000

The two blue lines check if any of the lower 8 bits at a particular offset [EBP-44] are set. If so, they jump to the green line, where execution continues. If no bits are set, the test result is zero, so JNE/JNZ is not taken, and the next jump (red line) takes us somewhere else in the code path. The good case (CD drive available) takes the jump to the green line; the bad case (no CD drive), takes the red line jump, leading to the “display error and terminate” code path.

The trivial solution is to replace the JNE instructon (75 0C) with the unconditional JMP (EB 0C). Opening up SETUP.EXE in a HEX editor I searched for the sequence of bytes F7 45 BC FF FF 00 00 75 0C. Fortunately, it is found, in a single location. After replacing the “75” with “EB”, I tested the executable again, and find out that it successfully “passed” the number of drives check, and stopped at the second MSCDEX interrupt (INT 2Fh, AX=1501). Success!

Defeating the check for the Descent II CD

Unfortunately, this is just half the job, and not the important half. Any system that has a CD-ROM drive, even if there is no disk in it, would not fail here anyway. To defeat the actual CD protection, the tracing process had to be repeated from the second MSCDEX interrupt (INT 2Fh, AX=1501). Again, running two sessions in parallel, one with the CD in drive, one without, we get to the first difference in execution:

0180:1A1500 8B45FC       mov eax,[ebp-0004]
0180:1A1503 3B45F0       cmp eax,[ebp-0010]
0180:1A1506 7C0D         jl 001A1515 ($+d)
0180:1A1508 E9FC000000   jmp 001A1609 ($+fc)
0180:1A150D 8B45FC       mov eax,[ebp-0004]
0180:1A1510 FF45FC       inc dword [ebp-0004]
0180:1A1513 EBEB         jmp short 001A1500 ($-15)
0180:1A1515 8B45FC       mov eax,[ebp-0004]
0180:1A1518 8D0480       lea eax,[eax+eax*4]
0180:1A151B 0345F8       add eax,[ebp-0008]

Looks a lot like the previous one: two values are compared – [EBP-4] and [EBP-10]. If the first one is lower, execution jumps to the green line and continues – this happens in the good case. Otherwise, it takes the red jump somewhere else – this happens in the bad case. The same idea from last time (replacing JL with JMP) seems natural, but, surprisingly (or not), in this case it fails. Apparently, this code is executed as part of a loop, and the loop termination criteria relies on other data. Just replacing the single instruction at 180:1A1506 leads to an infinite loop in case the CD is not present – obviously not what we wanted. Examining the code showed that the conditional jumps depend on values of CPU registers (EAX-EDX) that are affected by CALLs made from inside the block, and differ in the ‘good’ and ‘bad’ case.

Not wanting to dig too deep into the code, I managed to “force” the ‘bad’ case to behave like the ‘good’ case by forcefully setting the instruction pointer using DOSBox Debugger‘s SR EIP <value> command, and eventually got it to load the main setup screen. Success! However, there were 3 or 4 different places where I had to apply this technique, which means – a whole lot of instructions that I’d have to replace with NOPs. That’s not entirely terrible (since it still has to be done once), but I hoped for a “cleaner” solution. It is reasonable that the code detecting the game CD checks a few parameters, but usually there would be a higher-level function somewhere calling that code, reporting either “success” or “failure”. So somewhere there should be a single condition test, which determines the code path.

Thus, I decided to trace the above code block until the next function return (RET instruction), stepping over any CALLs. Hopefully, both the good and bad case would return to the same place, and hopefully the “single test” would be somewhere nearby. Indeed that turned out to be the case. Within a few tens of instructions, both debuggees executed a RET that jumped to the following block:

0180:1A0750 A332BD1D00      mov [001DBD32],eax
0180:1A0755 833D32BD1D00FF  cmp dword [001DBD32],FFFF
0180:1A075C 7518            jne 001A0776 ($+18)
0180:1A075E B8BB801D00      mov eax,001D80BB
0180:1A0763 50              push eax
0180:1A0764 E858E50000      call 001AECC1 ($+e558)
0180:1A0769 83C404          add esp,0004
0180:1A076C B8FFFFFFFF      mov eax,FFFFFFFF
0180:1A0771 E909CF0000      jmp 001AD67F ($+cf09)
0180:1A0776 833D32BD1D00FE  cmp dword [001DBD32],FFFE
0180:1A077D 7518            jne 001A0797 ($+18)
0180:1A077F B8DF801D00      mov eax,001D80DF
0180:1A0784 50              push eax
0180:1A0785 E837E50000      call 001AECC1 ($+e537)
0180:1A078A 83C404          add esp,0004
0180:1A078D B8FFFFFFFF      mov eax,FFFFFFFF
0180:1A0792 E9E8CE0000      jmp 001AD67F ($+cee8)
0180:1A0797 833D32BD1D00FD  cmp dword [001DBD32],FFFD
0180:1A079E 7518            jne 001A07B8 ($+18)
0180:1A07A0 B804811D00      mov eax,001D8104
0180:1A07A5 50              push eax
0180:1A07A6 E816E50000      call 001AECC1 ($+e516)
0180:1A07AB 83C404          add esp,0004
0180:1A07AE B8FFFFFFFF      mov eax,FFFFFFFF
0180:1A07B3 E9C7CE0000      jmp 001AD67F ($+cec7)
0180:1A07B8 A032BD1D00      mov al,[001DBD32]
0180:1A07BD 0440            add al,40
0180:1A07BF A236BD1D00      mov [001DBD36],al
0180:1A07C4 EB19            jmp short 001A07DF ($+19)

In this block, there are three conditional jumps that take place after comparing the data at [1DBD32] to FFFF (-1), FFFE (-2), or FFFD (-3). These are typical values for error codes. If any one of these succeeds, the corresponding JNE is not taken, causing a jump to [1AD67F]. That jump eventually triggers the program exit, with the appropriate error message (which appears to be generated by loading its address into EAX and calling [1AECC1]; note that the addresses are different each time).

If none of the comparisons succeeds, the code continues at 1A07B8 (the green line), which is the normal execution path. In the parallel debug, the “good” case continued normally, while the “bad” case failed the 3rd check (comparison to FFFD) and errored out.

Thus, the first attempt to disable this behavior was to replace the JNE at [1A079E] to JMP, which is another single byte replacement. Finding it in the executable was a bit trickier, though. It seems that the actual location of the error code in memory is modified at run-time, so the string 32 BD 1D (corresponding to [1DBD32]) does not ever appear. Instead, I should try to search just for the fixed part of the opcodes: the CMP instruction always compares to FFFD, so we can expect FD to always be there, and JNE uses a fixed offset, so 75 18 should also appear. Fortunately, the sequence FD 75 18 appeared only once; The next byte was B8, corresponding to MOV EAX,…, and the surroundings also looked familiar, which told me that I’m at the right place. Replacing the “75” with “EB”, again, did the trick. The setup program now works, regardless of whether the Descent II CD is in the drive, or whether there is a drive at all.

Out of curiosity, I decided to check what the other error messages were. Again, using the debugger to force EIP to skip the jump instructions, I saw that the first error message (corresponding to FFFF) is “Cannot find a CD-ROM drive”, and the second (corresponding to FFFE) is “Not enough real mode memory”. Obviously we should not mess with the second one, but the first one seems to be exactly the one we got before patching the first check. This suggests, that the termination path for the case when there is no CD at all also passes though this block, and that we can disable it patching the instruction at 0180:1A075C instead of the one at 0180:1A147C. That turned out to be accurate, even though there is no practical value to this – it is simply replacing one patched byte with another.

I went through all the features the setup program one by one, to verify that there are no other checks, and that no functionality is impaired by the absence of the CD (except, naturally, testing CD audio playing ability, which just fails silently, without crashing). In the end, SETUP.EXE is cracked and the CD protection is removed with just two bytes changed. Only basic understanding of x86 assembly was required for the process.

Summary

The steps outlined here may take some time to figure out the first time, but once you’ve got the main idea down, they can be applied to most CD protection cases for most DOS games. Generally, not just CD protection can be skipped this way, but also manual-based copy protection (testing for specific words), as well as some other checks that may be redundant (for instance, certain applications may test for specific versions of DOS, or specific environment variables, and refuse to run, even if their actual operation would not be impaired. In certain cases, there may be simpler solutions, not requiring debugging the program, but it’s a good tool to master for the more complicated scenarios.

Debugging DOS programs using the DOSBox Debugger presents numerous advantages over using standalone DOS-based debuggers, either in DOSBox or on real hardware: it is possible to launch multiple sessions in parallel on the same physical system; there is no need to worry about the app or the debugger crashing the machine; there is no need to deal with DOS/4GW stuff, 16-bit/32-bit or real-mode/protected-mode transitions. There are many more features to the debugger that were not utilized in this scenario, but can be easily inferred from the guide linked above, or from DOSBox Debugger‘s internal HELP command.

The debug process can be greatly sped up if you have the ability to step through the “good” and the “bad” cases in parallel and can place a breakpoint close to the start of the diverging code paths. Otherwise, you may need to step blindly through a lot of code, or require much better understanding of assembly and DOS application internals. In this case, DOSBox Debugger‘s ability to set breakpoints on specific interrupts came very handy.

Finally, I want to make it clear that I do not in any way endorse using the techniques outlined here to crack programs that you do not own and are not legally allowed to use, or for any other malicious intent. To me this was just a fun exercise, a way to learn some tricks in debugging DOS applications, and a means to defeat a minor inconvenience of having to have my Descent II CD around me when I want to play or configure the game, at least on my real DOS system. Admittedly, it is not such an issue when using DOSBox, since the CD can just be mounted as an image off the hard drive.

Update (2021)

I recently went through another hacking exercise, this time for no-CD patches for Rayman, which taught me a few additional tricks in assembly, debug DOS programs, and DOSBox debugger specifically. You can read about it here.