Dynamic Asset Management in TopHat

Small update, I made a YouTube video showing off the ability to keep assets in-game updated with their version on the filesystem. I decided to verbally explain the changes as I made them similar to how Wolfire makes their update videos for Overgrowth. I got a lot of positive feedback about it, so I’m probably going to continue this style of video for all future updates.

LED Music Visualizer

I’ve spent the past few days finally working on the music visualizer that I’ve wanted to make for well over a year now. While there are a few issues, it still looks beautiful, so I decided to call the first version of this project “done”.


As the video shows, the visualizer doesn’t require much hardware knowledge. So long as you have the equipment to do some soldering (or at least a friend who does), this project is entirely viable.

Materials

Setup

The Arduino/ShiftyVU setup

The Arduino/ShiftyVU setup

  1. Solder on the header pins that came with the ShiftyVU shield and fit on top of Arudino.
  2. Replace the audio output from your computer with the Y splitter. Plug your normal output as well as the extra cable into the splitter. The extra cable plugs into the ShiftyVU shield.
  3. Desolder the LED strips from each other. They should come in sets of 16 LEDs. Cut each one in half for 8 LEDs per strip.
  4. Lay out the strips where you want them to be and measure how much wire you’ll need. The bars don’t necessarily need to be soldered in the same direction. In my case, every other strip is backwards. This is very easy to account for in software.
  5. Cut and strip all the wire you need. Solder the LED strips together with the wires, making sure you’re always connecting an output to an input (marked by “CO/DO” and “CI/DI” respectively). Here’s the Adafruit article on wiring this particular LED strip.
  6. Optionally, solder the JST SM plug to the input of the first strip. That way you can unplug the LEDs if it’s ever necessary.
    The way I laid out my LED strips

    The way I laid out my LED strips

  7. Plug in the power supply to the 2.1mm jack to screw terminal block converter. Attach a red wire to the + block and a black one to the – block. The red wire can go directly to the +5V on the LEDs, but the black one must be in a common ground with the Arudino. Luckily, the Arduino has 2 GND ports, so plug the black wire to one of those ports, and then run another black wire from the other GND pin to the LED’s GND wire.
  8. Plug the data wire (DI, green wire) into the Arduino’s digital pin 2 and the clock wire (CI, yellow wire) into digital pin 3.
  9. Download the source code from my GitHub repository, follow the installation instructions, then upload it to your Arduino.
  10. Put on some music and stare!

IMG_3258

In the next version of this project, I’m planning on building a small circuit to read the audio in myself. As I explain in the video, the ShiftyVU does some filtering of the audio that doesn’t work well with spectrum analysis, but is good enough to make something visually pleasing.

Source Code

https://github.com/Robmaister/LEDMusicVisualizer - In case you glanced over the setup section.

Progress

Here are some of the pictures I posted on Facebook and the videos I posted on Instagram as I was working on the project.

IMG_3064

IMG_3132

dat VU meter #arduino #daftpunk

Still a VU meter, but making progress... #arduino #musicvisualizer #parovstelar

My project, now with 100% more FFT and a better color gradient algorithm #arduino #colors #musicvisualizer #justice

Now the LEDs are mounted under the desk. It's impossible to tell from the video, but the frosted glass diffuses the light in a very cool way. #arduino #musicvisualizer #colors #digitalism

Reflections on Implementing Cascaded Shadow Maps

So I’ve spent the past few days implementing shadow mapping in TopHat.

Crytek Sponza model with 4096x4096 4-split CSMs, 3x3 Gaussian blurring with PCF.

Crytek Sponza model with 4096×4096 4-split CSMs, 3×3 Gaussian blurring with PCF.

It looks nice, doesn’t it? Well with the settings I had on the screenshot, I was getting about 50fps on my desktop, which has an Intel i7-2600k and a NVIDIA GeForce GTX570. That means that it’s going to be too slow to run on the “average” gaming desktop that we want to target. There’s still a lot of optimization to do, but the simplest solution is to simply use less cascades and smaller shadow maps. In this post I’m going to discuss the process I went through in implementing CSMs, and any issues I ran into along the way.

Why CSMs?

Since the world is procedurally generated, lightmapping would add a rediculous amount of load time if I wanted it to look even halfway decent. It would also consume a ton of memory, which I’d rather save for more important things. So I needed a dynamic shadow mapping algorithm, and one that would work well on a very large scale (about 16km2) with a good amount of detail (objects as small as 1m3 should have shadows). After doing some research, I came across Cascaded Shadow Maps which essentially fits all of my requirements and decided I wanted to implement it.

Gathering Resources

My primary reference was the cascaded shadow map sample in NVIDIA’s OpenGL SDK 10.6. In addition, I kept a copy of the NVIDIA paper and read through the MSDN article on CSMs.

Porting to C#

My game is written in C#, and NVIDIA’s sample was (naturally) written in C++. Normally, porting code between languages isn’t too hard so long as you understand exactly what’s going on in the original version and know how to re-implement it in the target language. The issue with the NVIDIA sample was that it relied on the now-removed OpenGL matrix stacks, and my game manages it’s own matrices separately. So in order to port the sample to C#, I had to keep track of the entire GL_PROJECTION and GL_MODELVIEW stacks in my head throughout the entire draw loop. It didn’t help that matrices were occasionally read out into float arrays and modified them with their own matrix4 class and only sometimes sent back to an OpenGL matrix stack.

Besides that, there weren’t really any issues in porting the code itself. I did end up doing some of the matrix math incorrectly, so the first time I tried running the game with shadows, this was the result:

First attempt

Fixing the Issues

So after a nice break, I came back to the computer and tried to debug the shadows. The first thing I did was shade all the geometry based on which cascade it would read from. As it turns out, that worked just fine. The second thing I did was drawing each shadow cascade on-screen. That’s where I found the first of two bugs, an incorrect light crop/projection matrix.

The way I fixed that was correcting the order of a few of my matrix multiplications and changing the parameters of the look-at matrix. This gave me something resembling a shadow map from the NVIDIA sample, only rotated 90 degrees. For some reason, the NVIDIA sample defines its up axis as the negative X axis, so I went and changed the parameters of my look-at matrix to use the proper up-axis, the Y axis. At that point, I had a shadow map that was looking towards the light instead of from it. I defined my light direction vector as the direction the light is pointing in, and the NVIDIA sample did the opposite of that.

The second bug was an incorrect fragment-to-shadow matrix. After a lot of guessing and trying to follow the sample, I gave up and decided to see if reading the paper would give me any clearer of an understanding of what goes into the matrix. It did. The matrix is simply the inverse view matrix multiplied by the light’s crop/projection matrix multiplied by a “bias” matrix that scales the point into [0; 1] range for easier texture lookup.

Remaining bugs

The only remaining bug that has an impact on shadow map quality is finding the right Z bounds on the light’s orthographic projection matrix. Either part of the terrain gets clipped out and the shadows from a mountain disappear, or I test against all the bounding spheres in the world and the minimum Z is too far away and I lose precision, causing a Peter Panning artifact. The solution to this is to either place several small bounding spheres at the edges of the terrain and at local extrema, or to do some sort of raycasting test against the terrain.

Also, in testing with lower quality shadow maps and less splits, I’m getting a pretty bad amount of shadow acne, something I’ll look into and tweak some values for.

Looking Forward

The first thing I want to change about my shadow maps in the future is the way they’re filtered. Perhaps the option to use PCSS on higher-end machines, maybe replace the 3×3 gaussian blurred PCF filtering with VSM or SAVSM. I have other non-graphics things to tend to for the game right now, but I will eventually write another post about how I fixed the remaining bugs and any additional work I do on shadow mapping.


TopHat 2013-06-29 16.44.07.5504   TopHat 2013-06-30 14.53.33.6017   TopHat 2013-07-03 01.28.03.7231

The Last 6 Months

It’s been a busy few months for me, I haven’t had a whole lot of time to write blog posts as I finished my first semester at RPI, had an action-packed winter break, and completed my first year at RPI. So here’s one really long post about everything I would have posted individually had I the time.


I had two final projects due at the end of last semester that were games. One was for Intro to Game Design, where I chose to focus on the mechanics and technical aspects of the game, the other was for Art for Interactive Media, where we had to focus on the aesthetics, specifically pixel art. The source code for both are available on GitHub, as igd-final and afim-final, respectively. I spent a disproportionate amount of time on these projects, completing the Art for Interactive Media project in a matter of hours but spending several days on the Intro to Game Design one.

igd-final-screenshotMy Intro to Game Design project was a puzzle game with the primary mechanic of failed attempts acting as agents in the current attempt. The game stores all your keypresses (compressed by number of frames held in a run-length encoding fashion) in a lifetime and stores them. When you die, it copies them to a list of previous lives. Every frame the list of lives is iterated over and and the keypresses are applied to a separate Player instance. The previous life Players interact with the world as the current Player would, meaning that changing the state of the puzzle may result in previous lives dying early or being stuck where they wouldn’t have been earlier. The idea of the game is to force the player to die multiple times in order to be able to complete the level. Making the previous lives interact with the world as it is in the current life gives the game an interesting challenge where you have to worry not only about getting your current life to where it needs to go but also to make sure that you don’t mess up the previous lives and get yourself stuck.

I only had time to make 2 very simple levels, but still found it difficult to design levels that were difficult without being diabolical or relying on perfect timing. The goal of the project was to make a game based off one of two (or both) randomly selected words, of which mine were Failure and Propagate. I felt like I succeeded in that goal, as my game requires failure to succeed, and that your failed attempts propagated through the time that you spend solving the puzzle.

Screenshot

My Art for Interactive Media project was a much smaller project in which I attempted to do pixel art. The game is very simple and was made in only a few hours. The point of it is to build a gingerbread house by clicking on tiles in under 30 seconds. You get a score based on how similar your gingerbread house is to a reference one stored in a file. This game could have used a lot of additional work, the most important bit being what happens when you click on an incorrect tile. I should have some logic to push the boundaries of the house further away instead of just making a floating gingerbread block there. I doubt I’ll really have much time to go back and add anything or fix any of the flaws, as there will always be something more important to work on, but you never know, maybe I’ll use it as a way to transition back to Python and pygame in the future or something. I’m not that great at pixel art and I’m fine with that, I’m mostly interested in 3d graphics anyways. If anything, I’d work on generating 3d gingerbread houses or something really cool like that.


Throughout everything, I’ve also been making steady progress on that game I’m working on with friends, which we’re calling TopHat for now. We’ve got a very basic website and some social networking accounts in place (mostly empty) as we’ve begun testing an authentication scheme for the game and for user accounts. We’re still working on lower-level code and world generation, but since multiplayer is an integral part of the game it’s important that we get networking and authentication working early.

On the graphics side, about 6 months ago I did a lot of work abstracting rendering and finally wrote code that controls the directional light and lets you set multiple lights. Here’s a video of that functionality:

Shortly after that, I did some benchmarking, as the game was a bit slow on my laptop, and found out we were vertex processor bound with the terrain alone. So I did some research into terrain LoD algorithms and settled on a quadtree-based solution based loosely off the DICE publication “Terrain Rendering in Frostbite using Procedural Shader Splatting”, mostly around slides 37-40. Our implementation uses a zero-balanced quadtree, their T-junction solution, and a static buffer of vertices. We’re trying to target a minimum of OpenGL 2.1, so we chose not to use the method that relied on vertex texture lookup. At some point in the future we may raise the minimum OpenGL version if we feel enough people have support for newer versions of OpenGL. Our current T-junction implementation uses 9 index buffers per quadtree level, using a static 33×33 grid would be a much cleaner solution if we have vertex texture fetch support. Here’s a video of that algorithm in action:

Most of the work I did over break was networking and other backend stuff that doesn’t have much to show off. There wasn’t a whole lot of work on the game throughout the semester, but I’ve recently modified our terrain generation algorithm and started on road generation. Here’s what the terrain looks like so far:

TopHat 2013-05-20 22.26.31.7624


Starting this last semester, I signed up for the Rensselaer Center for Open Source (RCOS), a form of undergraduate research that allows students to work on their own open source software for either course credit or money. I started off making minor contributions to Kinect Gesture Library, but I plan to start my own project next semester. There really isn’t a whole lot to show for my contributions to RCOS at the moment, but I felt like mentioning this anyways.


Over this last semester I’ve made more significant contributions to OpenTK. Not directly, as the original maintainer has gone missing, the last release being in 2010. Instead, I sent my contributions to andykorth’s github repository where people have started making their contributions to. My contributions so far are mostly related to the math classes. I went through and profiled/optimized a large portion of the Matrix4 class, then added several other Matrix classes to match all the available matrices in GLSL (everything from a 2×2 to a 4×4).

Additionally, I wrote a library called SharpFont, a wrapper around FreeType2 for C# and all other CLI languages. I wrote this library because GDI+ was not giving me enough information about each glyph to be able to pack glyphs into a texture atlas and because Tao.FreeType was old and very lacking. SharpFont exposes an API typical to libraries written in C# that neatly wraps everything up in classes. For this reason it’s become my most popular GitHub repo. As of today 14 users have starred it and there are two forks. OpenRA recently replaced Tao.FreeType with SharpFont, and someone created a NuGet package that, as of today, has 168 downloads.


So that’s about it for now, I’ll try and take the time to post more regularly instead of having these massive post twice annually.

RPI Game Jam 10/27/12 Postmortem

MineEscape is our entry for the RPI game dev club’s game jam on 10/27/2012.

Source code: https://github.com/Robmaister/RPI-Game-Jam-10-27-12
Binary release: https://github.com/Robmaister/RPI-Game-Jam-10-27-12/downloads

Requires OpenAL. If the game crashes at the very beginning, this is probably the issue. Go download it here: http://connect.creativelabs.com/openal/Downloads/oalinst.zip

So now that I’ve got two of these RPI game dev club game jams under my belt, it’s time to get that second postmortem done. The theme was Surprise and Suspense. I’m just going to dive right in…

What Happened?

Like the previous game jam, we spent about an hour coming up with an idea. This time I decided to use C# and OpenTK, a combination I’m much more familiar with than Python and Pygame. I borrowed a teeny tiny amount of code from a previous game jam (open sourced) to avoid having to spend time on boilerplate code that most engines already have. Pretty much just the Texture class, the AudioBuffer class, and the Resources class, the AABB class, and the PhysicsManager class, along with a few method definitions for the IState interface. In total it was about ~650 lines of borrowed code, including blank lines and those bulky C# doc comments. The only functionality I borrowed was basic resource management, light wrapping around some OpenGL/OpenAL functionality, and AABB collision detection.

The first few hours were spent setting things up – getting the camera and player classes to move, defining the proper coordinate spaces, etc. Following that, one of my teammates finishes a rough outline of the first level which I use to set up the map collision code. (The entire map is converted to a bool[][] based on whether or not the alpha value of that pixel is below a certain threshold. Each entity’s axis-aligned bounding box is checked against the map for any pixels that are considered solid.)

Following that, I start work on the flashlight/shadowing. The way it works is that I draw the area of the flashlight to the stencil buffer (along with an alpha-tested version of the player) and then draw a fullscreen quad with a slightly transparent black texture everywhere except for the parts of the stencil map I wrote to. The results were alright, there was no penumbra (the edges of the flashlight were very harshly aliased) and no falloff towards the end of the light.

At the same time my teammates were working on detailing the level more and making sprites. I got a bit of work done in the ways of text rendering and menus, then grabbed the new version of the map and the sprites and incorporated them in. There were some minor issues with the level collision map, due to the layering effect/transparency that took a little bit of time to find a hack around. Towards the end of the night, we got a second level done and I took that in and set up some code to progress through levels. This also involved setting up entity interactivity (some hacks to detect collision between the player and the entities in the world) and implement the core mechanic – turn on the generator to turn on the lights in the level and make all the enemies disappear, then use the newly-activated lift to get up closer to the surface.

Before I went to bed that night I also added all of the menus and polished the transitions with some nice fading effects.

A scaled-down version of the first level.

The next morning I woke up at about 9:15am and walked over to Sage 2510 (where the game jam was taking place) and continued working on the menus and polishing the game a bit until my teammates arrived. When that happened I imported the third completed level that he had finished later on the previous night and put together AI for goblins and possessed pickaxes. AI was very simply a radius check and some movement code, and it worked well enough. Following that, I spent a little while trying to get OpenAL’s positional audio to work, but I couldn’t manage it. I’m still not entirely sure why it wasn’t working, but thinking about it now the scale was probably too large for the units OpenAL assumed, but I made all the sources relative to the listener and it got the job done.

With only a few hours left, we started work on the 4th and final level and the “end” level in which you simply walk towards a bright exit out of the cave. We finished that and added a bit of polish in the last remaining hour we had. The final game still had a handful of item placement bugs and dying would send you back to the first level instead of just the last level you were on, which we found out while presenting the game…

What Went Right?

  • Execution. We thought of the idea and got something done within the 24 hours with enough time to eat/sleep. We were not rushing any features in the last hour, unlike the previous game jam and we had time to do a little bit of polish (not as much as I had hoped for.
  • Communication. Again, comparing this to the previous jam, my team communicated much more effectively and we were much more efficient in this way. There were some minor communication issues with the first level’s collision map (and spawn point info), but after that it was all smooth sailing.
  • I used a language/libraries I was already familiar with but still learned something new (stencil buffer flashlight). I didn’t have to look up any language features like the previous game jam. The code was cleaner and more organized because of this.
  • I let the artists do the art (mostly). There were a few things that didn’t look too great as they were getting imported, but since I was getting psd’s, it was very easy to go in and adjust the map quickly. The most time I spent on art was cleaning up the player sprite and making it look like his headlamp was emitting the light.

What Went Wrong?

  • Like all game jam entries, there were features that we didn’t have time to bring into the game. What went wrong here was that the main balancing mechanic was not implemented. We originally wanted enemies to back off when the light was shined towards them, but never got a chance to implement it. This should have been done very early on but never happened.
  • The game was too simple, again. The feedback we got from the judges was very useful and highlighted that the game was too easy once the generator was on. Some additional puzzle elements would have made the game a lot better.

How Will I Fix My Mistakes?

  • Implement core mechanics earlier, then work on the polishing and other effects. That way we won’t run into this issue of completely forgetting a mechanic that would have added a bit of balance to the game.
  • Aim higher and focus the programming early on. A simple inventory and locked door/key puzzle system would have taken the game a lot further. We might not have had as much polish (or a 4th level), but the game would have been a lot more interesting with this.

Worst Hack?

Continuing with the organization of the previous postmortem, I’ll post the worst hack in the game’s source code. A lot of the hacking in this game was just making variables public to save some time in writing wrapper properties, and there weren’t really too many terrible hacks since I’m more familiar with C# and have a better idea of how to structure game code in it. The biggest hack I had to make, though, was one dealing with the flow of game states. I wrote a basic state manager really early on that kept track of a state stack (very similar to the article I wrote earlier on the subject), but the way I got the game to restart after winning was by spawning a main menu state under the next menu after the main menu was popped off the stack:

startMenu.OnPopped +=
	(s, e) =>
	{
		startMenu.Reset();
		StateManager.PushState(startMenu); //reset and loop
		StateManager.PushState(objectiveMenu);
	};

This meant that for a while the game could not be exited with Esc, since that would just pop states off the stack, so the fix was to write a method that would pop all the states off without giving it a chance to spawn the temporary new start menu. So I added a bool to StateManager’s StateAction class for clearing the entire stack knowing that the method immediately after update would check for an empty stack and exit the game, giving it no time to actually push the new stack on.

else if (action.clearing)
{
	while (stack.Count > 0)
	{
		stack.Pop().OnUnload(new EventArgs());
	}
}

This was also what caused the death-returning-to-main-menu bug. We had simply forgotten to set up the lose menu state to load a new GameState with the same level, and the game quitting to desktop would have been a clearer indication of this forgetfulness than returning to the main menu, which we simply overlook since most of the testing was done on the first level.