Inside MoPaQ - Prologue

LEGAL COPYRIGHTS

The MPQ Format

The copyrights to the MPQ format are held by Havas Interactive, Blizzard Entertainment's parent company, all rights reserved

This Article

The copyrights to this document and content are held by Justin Olbrantz(Quantam), all rights reserved. You may freely distribute this document provided that you do not derive profit from the distribution, and that the document remains complete and unchanged. You may quote this document ONLY with my explicit permission. Contact me to obtain permission to quote.
Also, although I would appreciate recognition for your use of this information, I will not be held legally responsible for anything you may do with it. Any way that you misuse this information is your problem, and I will not be responsible for it.

ACKNOWLEDGEMENTS AND CREDITS

Before we get on too far, I want to give credit where credit is due. These people either contributed something to this article or simply deserve mention based on their merits in the MPQ field.

Tom Amigo

Although he didn't actually contribute anything to this article, for the sake of history, I decided to include him here. Tom was the first person to hack the MPQ format and make a Stormless MPQ Editor by that very name.

Bobby Newmark

Bobby and I have been friends since late '98. He was either the 2nd or the 3rd person to hack the MPQ format, and presently maintains a Stormless MPQ viewer named RView, written in Java. I was fortunate to know him while he was doing the hacking, and I learned about 15% of the MPQ format from him.

Ladislav Zezula

Ladislav (I've always wondered if this is a guy or a girl) was either the 2nd or the 3rd person (opposite Bobby Newmark) to hack the MPQ format. In fact, he has had an MPQ format overview page up for quite some time now. I should emphasize that I did *NOT* know about this until after I had done all my hacking. So, he too did not contribute directly to this document, but he is worth mention, nonetheless.

Andrey Lelikov

Andrey hardly needs an introduction. Our association has become almost legendary among the Starcraft customizers who use MPQs. While he never actually worked with the MPQ format directly, he helped me in a number of ways. He took an (extremely) innovative solution to MPQ editing: the MPQ API Library (explained in chapter 4), and taught me a lot about "unorthodox" programming techniques, like the Library uses. Ultimately, he was the one who (unwittingly) prodded me to investigate the MPQ format myself.

INTRODUCTION

As should be self-evident, this article is about the MPQ format, the "Blizzard archive". I hope that I can explain what MPQs are, how they works, and why they do the things they do.
This article is organized in six chapters. The first chapter gives a brief overview and history of the MPQ format. The second covers basic programming concepts that you will need to know before we get into the MPQ format. The third chapter covers the Storm library, and how it can be used to read from MPQs. The fourth explains about the MPQ API Library, and how it provides the capability to edit MPQs. In the fifth chapter, I'll cover the internal format of MPQs, which we'll use in the sixth chapter to view and edit MPQs without Storm or the MPQ API Library.

Why I Wrote this Article

I've been interested in the MPQ format since shortly after Starcraft was released. I poked around a bit, but I just wasn't a good hacker then, and I didn't know what I was doing. Consequently, I didn't get very far. But, with a streak of providential luck, I met Bobby Newmark in a Starcraft hacking BBS. Bobby had been writing a Java archive viewer which could read and extract from the archives of numerous games, and he wanted to add MPQs to it. Unlike me, he was a skilled hacker. He worked slowly (he did have a life and a job), but he hacked away at the MPQ format. Eventually he figured out enough of it to build MPQ support into his viewer. He told me about a third of the format, although I only understood about half that at the time.
About 6 months later, after Bobby and I lost contact, Andrey (commonly know as Lelik) came into the picture. He released a program called MPQ Archiver, the ancestor of my own Mo'PaQ 2000. I was immediately interested in it. At the time, I would have been happy just to use it. But, it had one major deficiency - it couldn't import files that weren't encrypted (this means that Smackers couldn't be imported, as explained in chapter 4). I sent him a message telling him this. His response more than surprised me. He said that he was too busy to do it, and if I wanted it, I'd have to fix it myself - with the source code attached to the e-mail. Okay, then. I hadn't expected this, but I knew a good thing when I saw it. I looked at the code for about a week straight (that's how long it took me to figure out his unorthodox code). Well, eventually, I made the fix. But I didn't stop there. With Lelik's permission, I started redoing MPQ Archiver's shell from scratch, in what eventually evolved into Mo'PaQ 2000.
But things didn't stop even there. After releasing version 1.0 of MPQ2K and several bug-fix versions, I wanted to add renaming and MPQ compacting. I knew from Lelik that these functions didn't exist in Storm or StarEdit (explained in chapter 3 and 4), so they would have to be written from scratch. At the time I thought Lelik would be more qualified to figure out the MPQ format than me. So I asked him, and I waited. And waited. And waited. And finally I got tired of waiting, and I did something crazy. I started hacking the format myself. I really didn't know what I was doing, and I spent the first week getting the hang of it. But after that, things really took off. 6 weeks after I started, I had figured out about 90% of the MPQ format, and was able to write the rename and compact functions.
Anyway, the whole point of this lengthy explanation is that I have worked extensively with MPQs, and I am considered one of the worldwide authorities on MPQs. And, obviously, it is important for people to be able to use and understand the MPQ format, because of the power it provides for customizations. So, I want to use my knowledge to explain the MPQ format to everyone who wants to know.

Who Should Read this Article

There are principally three types of people who ought to read this article. The most obvious is people who want to customize games to play in interesting new ways, such as Campaign Creations and Camelot Systems. These types of customizations have become very popular, but most customizers still don't understand the MPQ format, and the benefits it provides. So, I intend for this article to teach these people the MPQ format so that they can make the best use of it.
Considering my obvious programming slant, it should also be fairly obvious that I also write this for programmers who want to know about the MPQ format so that they can use it to their own liking. Until now, formal documentation for the MPQ format was completely nonexistent. Ladislav's overview gave a little taste, but it didn't provide usable knowledge for programmers. But, I intend this article to be the definitive, comprehensive guide for, among other things, programming the MPQ format.
And finally, the third class is those people who are curious in general. It's far from uncommon for one of my Mo'PaQ 2000 beta testers to ask me something about MPQs that can't be easily answered (by far, the most common is "How come you can't list ALL the files in an MPQ?"). The answers to such questions lie in the fundamental architecture of MPQs, and can't be explained without a lot of background information. So, I hope to provide that information in this article, and answer everyone's problems and curiosities once and for all.

Back to the Table of Contents


LEGAL COPYRIGHTS
	The MPQ Format
		The copyrights to the MPQ format are held by Havas Interactive, Blizzard Entertainment's parent company, all rights reserved
	This Article
		The copyrights to this document and content are held by Justin Olbrantz(Quantam), all rights reserved. You may freely distribute this document provided that you do not derive profit from the distribution, and that the document remains complete and unchanged. You may quote this document ONLY with my explicit permission. Contact me to obtain permission to quote. Also, although I would appreciate recognition for your use of this information, I will not be held legally responsible for anything you may do with it. Any way that you misuse this information is your problem, and I will not be responsible for it.
ACKNOWLEDGEMENTS AND CREDITS
	Before we get on too far, I want to give credit where credit is due. These people either contributed something to this article or simply deserve mention based on their merits in the MPQ field.
	Tom Amigo
		Although he didn't actually contribute anything to this article, for the sake of history, I decided to include him here. Tom was the first person to hack the MPQ format and make a Stormless MPQ Editor by that very name.
	Bobby Newmark
		Bobby and I have been friends since late '98. He was either the 2nd or the 3rd person to hack the MPQ format, and presently maintains a Stormless MPQ viewer named RView, written in Java. I was fortunate to know him while he was doing the hacking, and I learned about 15% of the MPQ format from him.
	Ladislav Zezula
		Ladislav (I've always wondered if this is a guy or a girl) was either the 2nd or the 3rd person (opposite Bobby Newmark) to hack the MPQ format. In fact, he has had an MPQ format overview page up for quite some time now. I should emphasize that I did NOT know about this until after I had done all my hacking. So, he too did not contribute directly to this document, but he is worth mention, nonetheless.
	Andrey Lelikov
		Andrey hardly needs an introduction. Our association has become almost legendary among the Starcraft customizers who use MPQs. While he never actually worked with the MPQ format directly, he helped me in a number of ways. He took an (extremely) innovative solution to MPQ editing: the MPQ API Library (explained in chapter 4), and taught me a lot about "unorthodox" programming techniques, like the Library uses. Ultimately, he was the one who (unwittingly) prodded me to investigate the MPQ format myself.
INTRODUCTION
	As should be self-evident, this article is about the MPQ format, the "Blizzard archive". I hope that I can explain what MPQs are, how they works, and why they do the things they do. This article is organized in six chapters. The first chapter gives a brief overview and history of the MPQ format. The second covers basic programming concepts that you will need to know before we get into the MPQ format. The third chapter covers the Storm library, and how it can be used to read from MPQs. The fourth explains about the MPQ API Library, and how it provides the capability to edit MPQs. In the fifth chapter, I'll cover the internal format of MPQs, which we'll use in the sixth chapter to view and edit MPQs without Storm or the MPQ API Library.
	Why I Wrote this Article
		I've been interested in the MPQ format since shortly after Starcraft was released. I poked around a bit, but I just wasn't a good hacker then, and I didn't know what I was doing. Consequently, I didn't get very far. But, with a streak of providential luck, I met Bobby Newmark in a Starcraft hacking BBS. Bobby had been writing a Java archive viewer which could read and extract from the archives of numerous games, and he wanted to add MPQs to it. Unlike me, he was a skilled hacker. He worked slowly (he did have a life and a job), but he hacked away at the MPQ format. Eventually he figured out enough of it to build MPQ support into his viewer. He told me about a third of the format, although I only understood about half that at the time. About 6 months later, after Bobby and I lost contact, Andrey (commonly know as Lelik) came into the picture. He released a program called MPQ Archiver, the ancestor of my own Mo'PaQ 2000. I was immediately interested in it. At the time, I would have been happy just to use it. But, it had one major deficiency - it couldn't import files that weren't encrypted (this means that Smackers couldn't be imported, as explained in chapter 4). I sent him a message telling him this. His response more than surprised me. He said that he was too busy to do it, and if I wanted it, I'd have to fix it myself - with the source code attached to the e-mail. Okay, then. I hadn't expected this, but I knew a good thing when I saw it. I looked at the code for about a week straight (that's how long it took me to figure out his unorthodox code). Well, eventually, I made the fix. But I didn't stop there. With Lelik's permission, I started redoing MPQ Archiver's shell from scratch, in what eventually evolved into Mo'PaQ 2000. But things didn't stop even there. After releasing version 1.0 of MPQ2K and several bug-fix versions, I wanted to add renaming and MPQ compacting. I knew from Lelik that these functions didn't exist in Storm or StarEdit (explained in chapter 3 and 4), so they would have to be written from scratch. At the time I thought Lelik would be more qualified to figure out the MPQ format than me. So I asked him, and I waited. And waited. And waited. And finally I got tired of waiting, and I did something crazy. I started hacking the format myself. I really didn't know what I was doing, and I spent the first week getting the hang of it. But after that, things really took off. 6 weeks after I started, I had figured out about 90% of the MPQ format, and was able to write the rename and compact functions. Anyway, the whole point of this lengthy explanation is that I have worked extensively with MPQs, and I am considered one of the worldwide authorities on MPQs. And, obviously, it is important for people to be able to use and understand the MPQ format, because of the power it provides for customizations. So, I want to use my knowledge to explain the MPQ format to everyone who wants to know.
	Who Should Read this Article
		There are principally three types of people who ought to read this article. The most obvious is people who want to customize games to play in interesting new ways, such as Campaign Creations and Camelot Systems. These types of customizations have become very popular, but most customizers still don't understand the MPQ format, and the benefits it provides. So, I intend for this article to teach these people the MPQ format so that they can make the best use of it. Considering my obvious programming slant, it should also be fairly obvious that I also write this for programmers who want to know about the MPQ format so that they can use it to their own liking. Until now, formal documentation for the MPQ format was completely nonexistent. Ladislav's overview gave a little taste, but it didn't provide usable knowledge for programmers. But, I intend this article to be the definitive, comprehensive guide for, among other things, programming the MPQ format. And finally, the third class is those people who are curious in general. It's far from uncommon for one of my Mo'PaQ 2000 beta testers to ask me something about MPQs that can't be easily answered (by far, the most common is "How come you can't list ALL the files in an MPQ?"). The answers to such questions lie in the fundamental architecture of MPQs, and can't be explained without a lot of background information. So, I hope to provide that information in this article, and answer everyone's problems and curiosities once and for all.