Paperbound is built on XNA/Monogame, and I (Dan Holbert) am generally a big fan. However, the stock content pipeline that you get with XNA is disadvantageous for many reasons. It requires extra steps that you'd like to otherwise not have to do, it makes adding new assets via a custom game/level editor impossible, and it creates a huge bottleneck, as likely only a programmer is going to be adding items to a Visual Studio project.
Luckily, there's a way to get compiled .xnb content files without muanually adding them to a content project in Visual Studio. The basic idea is to dynamically create a project and spawn an MSBuild process to compile it. Luckily, MSBuild is included in all full .NET Runtimes (thought not the Client Profile, so be careful that your installer installs the full .NET runtime if you want this to work out in the wild). The core of this approach was actually provided by Microsoft in a project called XNAContentCompiler:
Xnacontentcompiler.codeplex.com
XNAContentCompiler is a stand-alone GUI application. It's is a great start, and it's relatively easy to convert it into something more automatic and useful in your game's pipeline. I'll walk you through the changes that I made, issues that I ran into, and how I solved them. By providing you with a run-down of these issues, I hope that that I can help you get this going in no time (instead of the several days it took me).
I'm also providing a modified version of XNAContentCompiler that can be invoked with a command-line parameter to build content for you. You can launch this programmatically from your game/editor via the Process class.
Download Modified XNAContentCompiler
But if you want to do it yourself, here are the steps.
Step 1. Remove GUI references
The one file that we really care about from XNAContentCompiler is ContentBuilder.cs. Unfortunately, it was coded in a manner that couples it to the GUI code in the other files. The main culprit here is using ComboItems for various things. I didn't want unnecessary references to Windows GUI .NET libraries, which is important for Monogame portability. Use some common "built-in" types or create your own. I tested that the refactored code still worked in the original GUI application before trying to bring it into my game's code.
Step 2. Project Settings
I don't know about you, but I want to use the full Shader Model 3.0, and I wanted to use the full .NET 4.0 runtime, so I changed some options in CreateBuildProject().
Step 3A: Add All the Standard Importers/Processors
You have to specify which input file formats you'll support and what the corresponding classes to build them are. The XNAContentCompiler example left out some important ones. In the ContentBuilder() constructor, add the support for the following types.
Step 3B (Optional). Custom content processors.
If you want to use non-MS content processors (either your own or Monogame processors), you'll need to update the list of assemblies in two places.
Note that if you are going to include this code in an executable or library where these are already loaded, you'll want to leave them out of this. Otherwise, you'll get exceptions saying that it can't load the assemblies.
In CreateBuildProject(), you might want to add a targets file. This is necessary for building Monogame shaders, but I didn't need to do this for my custom Model processor.
Also in CreateBuildProject(), you'll want to copy your DLLs to the temporary build directory.
Finally, add your importer and processor to the list.
Step 4. Invoking MSBuild without the GUI application
Extract the minimum code necessary to trigger the build so that you can use it independently of the GUI interface provided by Microsoft.
Note that when songs are compiled, two files are created: the .xnb file and the .wma file. The code needed to be modified to also copy the .wma file back from the temporary directory (more explanation here: Xbox Create Forums).
In addition, compiling Models from .fbx files will create several .xnb files (one for the model and one for each referenced texture), so you want to make sure to copy every .xnb file from the temporary directory.
Step 5. Check file timestamps
You'll want to build the content only when you detect that the source file has changed since you last build that particular piece of content.
Various issues I encountered
Compiling audio files (.mp3 and .wav) throws an exception that you can ignore. This is harmless, and you can set Visual studio to ignore this. Open the Debug->Exceptions dialog and uncheck "Managed Debugging Assistants"->PInvokeStackImbalance
PInvokeStackImbalance was detectedMessage: A call to PInvoke function 'Microsoft.Xna.Framework.Content.Pipeline! Microsoft.Xna.Framework.Content.Pipeline.UnsafeNativeMethods+ AudioHelper::OpenAudioFile' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.
Multithreading Woes
I tried putting all my content loading in a separate loading thread. However, ContentManager does not play well with multithreading. For example, any calls to ContentManager.Load() must happen in your rendering thread, or you will get exceptions. One way around this is to manually load data from a .png or .tga file, wrap that data in a memory stream, and then pass it to your rendering thread so that it can call Texture2D.FromStream(). However, I don't believe you will get texture compression or mipmaps with this approach.
Recursive Assembly References
If you added your own processors via custom DLLs, and you might have run into an issue. If you are going to include ContentBuilder.cs in an executable or library where your custom processor DLLs are already loaded, you'll want to leave them out (ignoring Step 2). Otherwise, you'll get exceptions saying that it can't load the assemblies. If the code is running in a library that is referenced by your importer library, your only option is to spawn a separate process to build the content. It's actually pretty easy and has some benefits.
In your game/editor, you can invoke another process like this:
In order for this to work, you'll need to modify the XNAContentCompiler project to accept a command-line argument.
Other Tips and Tricks
If I put the XNAContentCompiler project in the same solution as my I did my custom processor DLL, XNAContentCompiler failed to load the DLL. I don't know why. Keeping XNAContentCompiler in its own solution resolved this problem. Pathing issues are always a problem. If you are having the build process reference your own executables, make sure that you aren't using absolute paths, because that is almost certain to break on someone else's machine. Instead, you can either (A) set up a post-build step in your DLL's Visual Studio project to copy the resulting DLL to the folder where the ContentBuilder executable is, (B) set up a pre-build step for the ContentBuilder to copy the DLL over to its folder, or (C) using relative pathing from the ContentBuilder to wherever your DLL is.
Another option is to avoid putting the code in your game. Instead, spawn a separate process and wait for it to complete. You can modify the XNAContentCompiler project to take command-line arguments instead of opening a window. This has the following benefits:
1) It's a lot easier to get smooth multithreading when the work is done in an entirely different process.
2) If the content compilation crashes, your game/editor will still run. These processes will not corrupt your game/editor process's memory.
3) It's easy to spawn multiple build jobs in parallel.
It has the following flaws:
1) It's harder to debug.
2) It's foreseeable that you might get lingering ghost processes.Set your content builder project to copy its binary (exe or dll) to your game/editor's directory via a post-build step.
Conclusion
I hope that you're ready to ditch the content project and streamline your game's asset importing pipeline. Please tell me if you have any questions or find any errors in this article.
