 |
|
|
|
|
|
|
|
C
H A P T E RT
H R E E
STORM
|
|
|
|
|
The
Storm library, or Storm for short, is an operating system unto itself. It
is a gargantuan library of functions, and may be one of the most
complicated shared libraries ever to not be made by Microsoft. It contains
enough functions to make calling the native API almost unnecessary. In
fact, this was one of its two purposes for being. Storm contains all the
reusable functions written by Blizzard - such as the MPQ functions. But it
also wraps OS specific calls, such as those in the GDI, DirectX, QuickDraw,
etc. The reason for this is simple - to ease porting from one OS to
another. After all, why spend thousands of man-hours rerouting thousands
of OS function calls in the Windows source to Mac equivalents; why not
just have all the calls go to a single function, and change that one
function when the time to port to another OS comes?
By
the version of Storm that ships with Starcraft (the version that I will
refer to primarily in this document) has accumulated about 275 exported
functions (functions which it makes available to anyone who wants
them). You see, when a new game that uses Storm is released, the old
functions in Storm aren't updated. Rather, the new functions are added on
to the old version. This assures backward compatibility with games written
for an older version of Storm.
These
275 functions are grouped into about 20 function sets (commonly referred
to as subsystems on Windows, and managers on the Mac. I will
call them subsystems in this document). A partial list of these is shown
below:
- Memory Subsystem - Routines for
common memory functions, including allocating new memory, freeing
allocated memory, filling memory, and more. That's right, Storm does
its own memory management, including built-in error-checking and other
robust features. This subsystem is equivalent to the functions with
the 'mem' prefix on PCs.
- String Subsystem - Functions to
work with strings, such as copying, merging, searching, etc. These
functions are, for the most part, equivalent to the 'str' functions.
- File Subsystem - Functions for
accessing the file system. These functions have the capability to read
from (but not write to) either straight data files on disk, or MPQ
archives. The MPQ reading capability aside, the manner in which these
functions work is highly OS specific.
- Network Subsystem - Functions to
access remote computer system, through IPX, modem, TCP/IP, and direct
cable. Functions for communicating with the server and/or other
players during gameplay. Uses highly OS specific calls.
- Error Subsystem - Functions for
trapping and dealing with errors. Most of these functions do not have
any OS equivalents.
- Registry Subsystem - Functions to
save persistent data to the computer. Uses the Registry on Windows
systems, and the Preferences system on Macs.
- Bitmap Subsystem - Functions for
loading and displaying bitmaps from files. Uses OS specific calls.
Now,
it would be quite easy to fill a full-size book with documentation for
every one of these functions. Unfortunately, only about 40 of these have
been identified and documented so far (although I believe that every
useful MPQ function has been identified), and I don't have near
enough time to take my dissassembler and debugger to every one of the rest
(such a task would take months even if worked on diligently). Besides,
this document is specifically made for MPQs. Consequently, only MPQs will
be discussed.
|
|
|
|
Using
the Storm API |
|
|
|
|
WARNING: THE REST OF
THIS CHAPTER IS SPECIFIC TO THE WINDOWS PLATFORM!
Click
here to download the Storm Interface Library for Windows
As I said before, the
functions in Storm are available to anyone who wants to use them. However,
Blizzard isn't just going to hand them over without a fight.
To
summarize two days I spent recently: Storm uses a downright depraved
method to lock out hackers like us from using it. I spent at least 10
hours working hard trying to disarm the stupid thing, a task of which I'm
proud to have succeeded in. But, I'll spare you the rather lengthy and
complicated details of what I did, and get right down to business.
The
culmination of my efforts matching wits with Mike O'Brien is what I call
the Storm Interface Library. It is composed of a header file and an import
library, which I synthesized with great effort to disarm the Storm
booby-traps (download the Library from the link above). If you recall from
DLLs 101, an import library contains the import tables that are used when
a program is compiled to connect the program to a DLL. What this means is
that all you have to do is include Storm.lib (in the Storm Interface
Library) with the modules to link in your program and #include the
Storm.h header file and you're good to go. Kinda makes me mad that I had
to go to all that work to make the Storm Interface Library, and you can
use it so easily :-p
And
now, without further ado, let us get on to the Storm functions themselves.
|
|
|
|
Opening
an MPQ Archive - SFileOpenArchive |
|
|
|
|
BOOL WINAPI
SFileOpenArchive(LPCSTR lpFileName,
DWORD dwMPQID, DWORD dwUnknown,
HANDLE *lphMPQ);
|
|
Parameter |
What it is |
|
|
lpFileName |
[in] A pointer to NULL
terminated string that holds the path of the MPQ to open. SFileOpenArchive
will crash if this is NULL. |
|
|
dwMPQID |
[in] An ID value that is
saved internally in Storm for the MPQ. What this parameter is used
for is not clear at this time. |
|
|
dwUnknown |
Unknown. Should always be
NULL. |
|
|
lphMPQ |
[out] Pointer to a HANDLE
variable that, upon successful completion, will hold the HANDLE
of the MPQ. SFileOpenArchive will fail if this is NULL. |
|
Before
you can read from an MPQ, you have to open it. For this, you must use SFileOpenArchive.
It will open an archive and give you a HANDLE that you can later
use for calls to SFileOpenFileEx and SFileCloseArchive.
The
first parameter, lpFileName, is simply the name of the MPQ to be
open, and must not be NULL. The second parameter, dwMPQID is
the ID that Storm will assign to the MPQ internally. This does not change
the MPQ, and it is unclear as to what it actually does do. The
third parameter, dwUnknown, is just that, and, for all we know,
unused; it may, therefore, be ignored. The final parameter, lphMPQ,
is a pointer to a HANDLE that you must have declared previously. If
SFileOpenArchive completes successfully, this HANDLE will be
that of the MPQ.
If
SFileOpenArchive succeeds, its return value will be nonzero. But,
there are several situations which may cause SFileOpenArchive to
fail. If it does fail, it will return with a value of FALSE. In
this situation, you can call GetLastError to get information about
why it failed. If lpFileName is a 0-length string or lphMPQ
is NULL, GetLastError will return ERROR_INVALID_PARAMETER.
If the file lpFileName does not exist, GetLastError will
return ERROR_FILE_NOT_FOUND. In some very rare cases, GetLastError
may return some other cryptic error value.
|
|
|
|
Closing
an Archive - SFileCloseArchive |
|
|
|
|
BOOL WINAPI
SFileCloseArchive(HANDLE hMPQ);
|
|
Parameter |
What it is |
|
|
hMPQ |
[in] The HANDLE of
the MPQ to close, which was acquired earlier with SFileOpenArchive.
SFileCloseArchive will fail (or worse) if this is NULL
or a HANDLE not obtained with SFileOpenArchive. |
|
Once
you have opened an MPQ archive, you must remember to close it when you're
done! SFileCloseArchive is the natural compliment of SFileOpenArchive,
closing an archive that was opened earlier.
As
with SFileOpenArchive, SFileCloseArchive returns a nonzero
value to indicate success, and FALSE if it should fail. However, in
this case, GetLastError wouldn't provide any useful information.
But, there should only be one reason for SFileCloseArchive to fail
- if the hMPQ parameter is invalid - so you can only assume that
that was the reason.
|
|
|
|
Opening
a File Inside an MPQ - SFileOpenFileEx |
|
|
|
|
BOOL WINAPI
SFileOpenFileEx(HANDLE hMPQ,
LPCSTR lpFileName, DWORD dwSearchScope,
HANDLE *lphFile);
|
|
Parameter |
What
it is |
|
|
hMPQ |
[in] The HANDLE of
the MPQ previously opened with SFileOpenArchive that contains
the file you want to open. SFileOpenFileEx will fail or crash
if this is NULL or a HANDLE not obtained with SFileOpenArchive. |
|
|
lpFileName |
[in] A pointer to a NULL
terminated string that holds name of the file within the MPQ to be
opened. SFileOpenFileEx will crash if this is NULL. |
|
|
dwSearchScope |
[in] Specifies where to
look for a file when opening a file on the hard drive. Must be NULL
when working with MPQs. |
|
|
lphFile |
[out] A pointer to a HANDLE
variable that, upon successful completion, will hold the HANDLE
of the requested file. SFileOpenFileEx will fail if this is NULL. |
|
Just
because you have opened an MPQ with SFileOpenArchive doesn't mean
you can start reading from it immediately. Remember, MPQs are nothing more
than archives - they contain other files. Before you can read anything
from an MPQ, you must open one (or more) of the files inside. SFileOpenFileEx
is the function you will use for this task; it will open the requested
file in an MPQ and return a HANDLE to it.
Once
again, SFileOpenFileEx will return a nonzero value on success and FALSE
on failure, and you can call GetLastError to get the reason why. If
lpFileName is a 0-length string or lphFile is NULL, GetLastError
will return ERROR_INVALID_PARAMETER. If the file does not exist in
the MPQ, GetLastError will report ERROR_FILE_NOT_FOUND. In
some rare cases, GetLastError may report ERROR_FILE_INVALID,
and on extremely rare occasions, it may return some other obscure error
value.
IMPORTANT
NOTE: When you call SFileCloseArchive to close an MPQ, all the
open files inside that MPQ are also closed, and the HANDLEs you
received from SFileOpenFileEx become invalid. If you call SFileReadFile,
SFileGetFileSize, SFileSetFilePointer, or SFileCloseFile
with one of these invalid HANDLEs, the call will fail, and Storm
may even crash.
|
|
|
|
Closing
a File Inside an MPQ - SFileCloseFile |
|
|
|
|
BOOL WINAPI
SFileCloseFile(HANDLE hFile);
|
|
Parameter |
What it is |
|
|
hFile |
[in] The HANDLE of
the file to close, which was acquired earlier with SFileOpenFileEx.
SFileCloseFile will fail (or worse) if this is NULL or
a HANDLE not obtained with SFileOpenFileEx. |
|
Just
like SFileCloseArchive is to SFileOpenArchive, SFileCloseFile
is the natural compliment of SFileOpenFileEx, closing an already
open file.
Also
like SFileCloseArchive, SFileCloseFile will return a nonzero
value on success, FALSE on failure, and GetLastError will
provide exactly zero help. Fortunately, even more than SFileCloseArchive,
SFileCloseFile should only fail if hFile is NULL
or an invalid HANDLE.
|
|
|
|
Reading
from a File in an MPQ - SFileReadFile |
|
|
|
|
BOOL WINAPI
SFileReadFile(HANDLE hFile,
LPVOID lpBuffer, DWORD nNumberOfBytesToRead,
LPDWORD lpNumberOfBytesRead,
LPOVERLAPPED lpOverlapped);
|
|
Parameter |
What it is |
|
|
hFile |
[in] The HANDLE of
the file to read from, which was acquired earlier with SFileOpenFileEx.
SFileReadFile will crash if this is NULL or a HANDLE
not obtained with SFileOpenFileEx. |
|
|
lpBuffer |
[out] A pointer to a
buffer in memory where SFileReadFile will place the data read
from the file. This buffer must be at least as large as nNumberOfBytesToRead.
SFileReadFile will fail if this is NULL. |
|
|
nNumberOfBytesToRead |
[in] The number of bytes
for SFileReadFile to read from the file. SFileReadFile
may crash if this is larger than the size of lpBuffer. |
|
|
lpNumberOfBytesRead |
[out] A pointer to a DWORD
that will hold the number of bytes actually read from the file. The
number of bytes read will never be more than nNumberOfBytesToRead,
but may be less if the number of unread bytes in the file is less
than nNumberOfBytesToRead. It is not recommended to let this
be NULL. |
|
|
lpOverlapped |
[in] A pointer to an OVERLAPPED
structure. This is used for asynchronous reading of files on a disk,
and must be NULL when reading files in MPQs. |
|
Of
course, the whole reason you went to the trouble of opening an MPQ and a
file inside it is to read from the file. Well, once you have obtained a
valid file HANDLE from SFileOpenFileEx, this is the function
for the job. SFileReadFile will read a specified number of bytes,
and then advance the file pointer. This means that if you have a file, and
you call SFileReadFile to read half the file you would get the
first half of the file, but if you called SFileReadFile again you
would get the second half of the file. If you needed to get the first half
again, you would need to call SFileSetFilePointer.
SFileReadFile
will give a nonzero return value on success, or FALSE on failure.
However, you must remember that just because it returned a nonzero value
doesn't mean it actually read anything; it only means that no errors
occurred. If there are less unread bytes (bytes from the file pointer to
the end of the file) in the file hFile than the number requested, SFileReadFile
will read less than the number of requested bytes; and If the file pointer
for the file hFile is at the end of the file, SFileReadFile
will read nothing, set lpNumberOfBytesRead to 0, and return TRUE.
Consequently, it's very important to check lpNumberOfBytesRead.
|
|
|
|
Getting
a File's Size - SFileGetFileSize |
|
|
|
|
DWORD WINAPI
SFileGetFileSize(HANDLE hFile,
LPDWORD lpFileSizeHigh);
|
|
Parameter |
What it is |
|
|
hFile |
[in] The HANDLE of
the file whose size is to be determined. SFileGetFileSize
will crash if this is NULL or a HANDLE not obtained
with SFileOpenFileEx. |
|
|
lpdwFileSizeHigh |
[out] A pointer to a DWORD
that, upon successful completion, will hold the high 32-bits (a DWORD)
of the file's size. However, it is not possible for a file in an MPQ
to be this large (over 4 gigabytes), so this is virtually unused,
and may safely be NULL. |
|
Although
it is entirely possible to read from a file of unknown length without
causing a crash, this is generally considered extremely bad programming
practice, and most programmers prefer to know the size before they start
reading from a file. SFileGetFileSize is the trick here, as it can
retrieve the size of a file already opened with SFileOpenFileEx.
There
are some major trouble spots in SFileGetFileSize. First is the
ambiguous way that it deals with errors. When SFileGetFileSize
succeeds, it returns the file's size (which can be 0!). But, when an error
occurs, it will return 0xffffffff, which is the same thing it would return
for a 4,294,967,295 byte (4 GB). Fortunately, this isn't a big problem, as
you'll probably never actually see a file that large. The second problem,
however, is more dangerous, and is concerned with SFileGetFileSize's
lack of error checking. SFileGetFileSize doesn't check whether hFile
is valid or even NULL. That means that if you give it an invalid HANDLE
for hFile, down goes the program, and down goes the computer. So,
the bottom line is this: use this function with care.
|
|
|
|
Moving
the File Pointer - SFileSetFilePointer |
|
|
|
|
DWORD WINAPI
SFileSetFilePointer(HANDLE hFile,
long nDistanceToMove,
long *lpDistanceToMoveHigh,
DWORD dwMoveMethod);
|
|
Parameter |
What it is |
|
|
hFile |
[in] The HANDLE of
the file whose file pointer is to be moved. SFileSetFilePointer
will crash if this is NULL or a HANDLE not obtained
with SFileOpenFileEx. |
|
|
nDistanceToMove |
[in] The low-order
32-bits of the number of bytes for SFileSetFilePointer to
move the file pointer, with positive numbers moving the pointer
forward and negative numbers moving the pointer backward. This value
can also be 0. |
|
|
lpDistanceToMoveHigh |
[in] A pointer to the
high-order 32-bits of the distance for SFileSetFilePointer to
move the file pointer. But, because MPQs do not support files this
large, this is unused and must be NULL or SFileSetFilePointer
will fail. |
|
|
dwMoveMethod |
[in] Specifies the
relative location the file pointer will be moved to. Must be one of
these following values in Windows.h:
| FILE_BEGIN |
The file pointer
will be set to nDistanceToMove bytes from the beginning
of the file. nDistanceToMove must be positive. |
| FILE_CURRENT |
The file pointer
will be set to nDistanceToMove bytes from the current
location of the file pointer. |
| FILE_END |
The file pointer
will be set to nDistanceToMove bytes from the end of
the file. nDistanceToMove must be negative. |
|
|
Many
times it is sufficient to read through an entire file sequentially - from
the very beginning to the very end. Other times, however, it is necessary
to move straight to some location in the file and read only a small
portion of the file. Reading from an arbitrary location in the file
required that you move move the file pointer, a task performed by SFileSetFilePointer.
The file pointer dictates where the next read operation will read data
from, or the next write operation will write data to in a file; each read
or write operation will then move the file pointer to the end of the
read/written region.
SFileSetFilePointer
does not actually move the file pointer to the absolute position nDistanceToMove
in the file. Rather, SFileSetFilePointer moves the file pointer to
a position relative to either the beginning or end of the file, of the
current file pointer position. For example, let's say you have a file 1000
bytes in length. When the file is first opened, it's file pointer is set
to 0, referring to the very first byte. But, you then read 100 bytes from
the file. You then call SFileSetFilePointer with a nDistanceToMove
of 500. If you set dwMoveMethod
to FILE_BEGIN in the call, the file pointer would be set to 500. If
you had dwMoveMethod as FILE_CURRENT,
the file pointer would be 600, because the file pointer was moved to byte
100 when you read from the file. But, if you set dwMoveMethod
to FILE_END, SFileSetFilePointer would fail, because it
would try to set the file pointer to 1499 (the last byte if the file, 999,
plus 500) which doesn't exist. This system is easy to use if you
understand it.
On
failure, SFileSetFilePointer returns 0xffffffff, which carries the
same pitfalls as SFileGetFileSize. But, on successful completion, SFileSetFilePointer
returns the new absolute position of the file pointer for the file hFile.
This means that you can simply get the current position of the file
pointer by calling SFileSetFilePointer with a nDistanceToMove
of 0, and dwMoveMethod set to FILE_CURRENT. In fact, it is
for this very reason that there isn't a SFileGetFilePosition
function.
Just
like SFileGetFileSize, SFileSetFilePointer will recklessly
use any hFile HANDLE you give it, without performing any
error-checking whatsoever. That means that, once again, it is your
responsibility to make sure it gets a valid file HANDLE, or else
it's good-night-computer.
|
|
|
|
Choosing
a Language - SFileSetLocale |
|
|
|
|
LCID WINAPI
SFileSetLocale(LCID lcNewLocale);
|
|
Parameter |
What it is |
|
|
lcNewLocale |
[in] The language code (LCID)
that SFileSetLocale will make the new default. The following
codes are ones that I've found in Starcraft MPQs:
| 0 |
Language Neutral/English |
| 0x407 |
German |
| 0x409 |
English |
| 0x40a |
Spanish |
| 0x40c |
French |
| 0x410 |
Italian |
| 0x416 |
Portuguese |
|
|
SFileSetLocale
is the functional manifestation of the simple, elegant multilinguality
system Blizzard created in MPQs. Thanks to it, despite the underlying
complexity, a single function call assures that all files read from an MPQ
are of the intended language. It's only parameter is the new language for
the entire Storm MPQ subsystem. It will never fail, and it returns the
language code you gave it, making its return value worthless.
The
way MPQ's multilinguality works is this: Each file in an MPQ has a
language code, and there can be multiple files with the same name as long
as they have different language codes. When a call to SFileOpenFileEx
is made, SFileOpenFileEx looks for a file with the same language
code as the one stored by the last call to SFileSetLocale (or a
language code of 0, if SFileSetLocale has never been called). If a
file with matching language code cannot be found, SFileOpenFileEx
will open the language neutral (having a language code of 0) version of
the file instead.
|
|
|
|
Putting
it All Together - DumbExtractor |
|
|
|
|
Click
here to download DumbExtractor and source code
By now you should have a
pretty good idea of what all the Storm MPQ functions do and how to use
them. But, of course, no good programming chapter would be complete
without one more thing - a sample program!
DumbExtractor
is a simple console application that I whipped up in about 20 minutes,
which can be run simply by double-clicking on it. All it does is start up,
ask you for an archive and file to extract, and extract the specified
file, all with a healthy dose of status reports and comments (I would
suggest you download it from the link above and look it over before you
continue reading this chapter). But don't let its simple operation fool
you; DumbExtractor is as robust as anything, and can handle just about any
error you can throw at it.
DumbExtractor
works in a straightforward way, as it's intended to be as simple and
comprehensible as possible. Out of necessity, it uses the Storm Interface
Library. The first task it performs is to allocate a read buffer.
This is done first because I wanted to get as many things that could go
wrong as possible out of the way before we go bothering the user, even
though there is the slim possibility that the file to extract might be 0
bytes large (in which case we haven't really lost anything in allocating
the memory, since any computer that can't spare 4 MBs of RAM for a couple
seconds is in desperate need of life-support anyway). After making sure we
have memory to work with, DumbExtractor then prompts the user for an
archive and a file to extract, all of which is performed with the simple cout
and cin.
Once
all the preparatory duties are out of the way, we can get into the real
Storm stuff. Our first call is to SFileSetLocale, to set the
language. I suppose this is sort of unnecessary, considering my language
is English, and that is what Storm defaults to anyway. But, it could also
be argued that it's good programming practice, so think what you will
about it. The next thing we will do is to open the requested archive using
SFileOpenArchive, followed by opening the file with SFileOpenFileEx.
Once
we're sure that the file exists in the archive and opened it, we can then
open the output file on the hard drive using the Win32 API function CreateFile.
Then, we get the file's size with SFileGetFileSize. The reason I
waited to check whether the file is not empty (has more than 0 bytes)
until after creating the file on the hard drive despite the fact that
doing so adds something else that could go unnecessarily wrong, is because
even if the file is 0 bytes, it's supposed to be 0 bytes. That
means we have to deliver it as intended - as a 0 byte file.
Finally,
we get to the inner loop, where we marshal the data from the MPQ file to
the output file using SFileReadFile and the Win32 API WriteFile.
Here I use a BOOL variable to remember whether there were any
reading or writing errors. The reason I didn't just end the program right
away when an error occurs is that I didn't want to duplicate any code and
just let things end naturally instead (I'll leave the question of whether
I actually achieved this goal up to the philosophers). That means I need
to wait for the output file to be closed before I can delete it. Finally,
once the extraction is complete, we can shut everything down, closing the
files and MPQ, and freeing the read buffer we allocated.
|
|
|
|
The
Rest is Attitude |
|
|
|
|
Okay, you got me; that is
indeed a quote from book Writing Solid Code, by Microsoft Press (which is
a good book, by the way). Anyway, in this chapter, I've shown you
everything you need in order to make any manner of MPQ extractor; from
DumbExtractor to MPQView, to something much more spectacular, the
foundation - and flexibility - is all there in the simple Storm functions.
A program's personality is not what it does, but what it is - the user
interface. The Storm functions are all there. What you choose to make of
them is just attitude.
|
|
|
|
|
|
|
|
