Deep Sea Adventure: Dev Diary

[triplefox]

I thought I would keep some notes on my progress with this project for fun and posterity. I plan to update "occasionally."

I have been working on this for about three evenings now. First a little bit about the concept and goals, and then onto the technical fun.

The basic idea of the Deep Sea series, which will now become a trilogy (with absolutely no plot relationship), is that you pilot a submarine with some cool and highly unrealistic abilities through undersea caves filled with various enemies. In the first game you used mini-subs with smaller size and special abilities to collect bombs planted by evil terrorists; when you had collected them all the path to a final boss fight opened up. In the second game had you do treasure-hunting to pay off a debt, with a "town" screen that gave you opportunities to buy equipment and engage in minimalistic dialogue trees.

For this game I'd like to bring in all the features of the previous games, with a tighter interface, and expand the content to have many levels and a real plotline. I also want to have a really, really nice level editor for it. It'll be an open source game once I have a good demo ready - everyone is welcome to contribute.

===============

On the first day I familiarized myself a little with Love and set down some ideas about how I was going to structure the game engine. With each project I try to learn a little something and try to take a fresh approach to the architecture. I have never really focused on low-level details, primarily interesting myself in ways to make gameplay easier to create and the logic more reliable. This time around my focus was on the entity model. I have concluded that notions of an Entity class and Entity objects as a first-class data structure are nonsensical. Not that they don't work, but naive uses of this approach lead to a "god class" where Entity is expected to do almost everything, and subclasses of Entity do a tiny bit extra.

So modern game architectures are heading towards a compositional approach. The gist of this is that the Entity is nothing more than a container for some component structures: for example, basic x/y coordinates and velocity vector can be encapsulated in a component called SimpleMovable. The sprite data and anims can be encapsulated in a Display2d component. Collision is contained in Collision components - perhaps you want to use a different type of collision for the scene background vs. individual entities.

Lua makes it a lengthy, but relatively straightforward process, to do this. "entities" is a table with an incrementing index, each entry containing a table with component names. Each component contains its own name and index number so that components can access each other's data.

Some bookkeeping for adds and deletes is done per-component; my Display2d components live in a table separate from entities, as well as on the entity table. That way is more efficient for blasting out draw updates than querying every entity for their display component. I don't like the bookkeeping, though. I wonder if there is a way I can eliminate the bulky code it produces in each component.

A "gotcha" with this strategy, which I spent this evening resolving, are that when you start adding and deleting entities you get gaps in your tables, and you can't eliminate gaps without reordering indexes and opening another can of worms. table.insert and ipairs(table) don't seem to deal with gaps properly, so I ended up not using those functions.

Along the way I did some helper functions for loading data. Resources are accessed through a table for each type of resource(gfx, sfx, bgm) and I recursively traverse directories with the same names to autogenerate resource names like "/bg/tile/tile0", for example. Each name is assigned an index so that table lookups for using resources are mostly integers, not strings. (faster) I would also like to improve this a little to condense multiple images into anims automatically.



Here you can see the test case for the engine as-is, using the tile tutorial graphics. The top number is frame rate, the other is the number of sprites. The things I wanted to test are:
-Basic sprite drawing
-Performance of entity system
-Adding and removing many entities
-Basic usage of components

So in the test, I create 1000 entities with a SimpleMovable and a Display2d, with random positions, sprites, and directions. The main update loop processes each one naively and tells them to bounce off the edges by reversing velocity. The display loop reads from the table of all Display2d components, gets the associated SimpleMovable from the Display2d id, and then combines the data to place the sprite in the right position. Last, I had the update loop add three and delete three entities each frame, to prove that I can dynamically add and remove entities safely.

To cover mass data-manipulation functionality well, there are four general cases to handle:
0 - no data available
1 - a single instance(usually your first test)
>1 - groups require more logic than singular instances
lots - how well it can scale

Failing to cover any of these means you get bugs later on when you use your data differently. Hence why I focused on adding and removing entities even though there's no gameplay. This is the time when it's easiest to do so. And -- working with thousands of entities here quickly revealed problems in my original add and delete functions.

In the commercial project I've been working on, the engine has to work on the Wii, with tight memory constraints - 88MB total RAM. Compare this with my laptop's 1GB, and the faster processor too, and it becomes plain that we operate in entirely different worlds. While the engine I'm setting up in Love has ways to be generally optimized -- in its current state it is not too difficult to understand and it is open to localized optimizations.

For example, if I wanted to reproduce the current test with better performance, it could be embedded entirely in a single component. I plan to do such things for the major abstractions of the game like maps.

[mike]

My god. I sat down to read this wall of text (I'm thinking that most of the forum goers were thinking "tl;dr" right about here) and I was impressed. It's strange how you encapsulated the concept of thinking yourself to a programming conclusion so well. Setting up the desired action, going over the idea, trying some things and then settling on a way that works. Brilliant. I want to read what you come up with next, as it is relevant to my interests (the actual development process is actually more exciting than the end product, in my eyes).

[triplefox]

The last few weeks I have been thinking about tools. Not just for this project, but in general - I have another project coming up which may sideline Deep Sea for a while, but could also help it - this project is to set up a pipeline that allows construction of a Flash game every week, within only a few hours of spare time. These games will have minimal art, but should also contain enough depth to be commercially viable. I really like the thought of doing something in a short period of time. Deep Sea is a meant as a "big project" with different goals, and my interest is low right now, but I may drift back to it over time. As you will see I have a plan that may help both projects.

This image shows some various ways in which one could design tools.

tooldesign.png (33.68 KiB) Viewed 10161 times

The first model is the naive one - you have some tool, which does everything requiring custom data, and it produces data for the game. Then the game uses the data. In practice this model only works for games with simple design goals - where there is low extensibility. But almost everyone uses an external image editor, text editor, etc. at some point. So they are actually using the second model.

In the second model, instead of one tool there are several, and they each produce some data which the game combines at runtime. This is the traditional process for most games.

The third model takes this to a new level, by encouraging the same tools to be used across many projects. For my game-per-week project I am using Tile Studio. This program lets you write code generating templates (with a somewhat obscure markup language) so that instead of writing a data importer, you have only to paste the output into your code. The distinction between this and the second model is that the data is never particularly targeted towards an engine until the compilation step. The planned process for game-per-week is:

pipeline.png (61.03 KiB) Viewed 10160 times

This is a new thing for me, and I don't completely like it as-is. It's a large process and hard to "hold in your head." But it has a huge benefit - I can work on different stages at any time, and test them incrementally. To explain it in more detail:

The graphics and collision info of a map are defined in the Tile Studio project. I wrote a second editor in Python that adds additional information on top of the Tile Studio data, so that I can precisely specify particular entity instances - recalling that entities in my engine model are compositions of various components, I plan a DSL for specifying named archetypes with default combinations of components and values, which the entity placer tool can then override. I also plan a few more DSLs for things like cinematic scripting, AI, and whatever comes up. The sound management events are one such example.

The distinction between haXe and AS3 comes from some clumsiness in using only haXe: most of the world doesn't use it. But I find its features highly useful. AS3, on the other hand, is well-supported by the commercial world, with debugging support, library code, etc., and the last time I tried going purely haXe I found myself in want of those things. So the haXe code never deals directly with input/output stuff, it only passes and receives abstract events. This is an extension of a system I came up previously for simplifying my game code...

If you look at very old games from the 1980s or prior, they tend to have a one-to-one conflation of graphics and gameplay. The memory used to display a map or sprites on the screen is the same memory used to detect collisions. Only after technology advanced a little was it feasible to use collision boxes and other useful abstractions.

I bring this up because the same kind of conflation can exist in writing game loops, only it happens as an unintended mistake of the programmer, not as a memory-saving device. When you write a game loop you have three general things going on:

-Getting input and timer events from the platform
-Processing the events
-Drawing to the screen

But, if you have a complex point-and-click GUI in your game, for example, this loop can become hard to maintain and unexpected behavior may result because your game interface breaks down, forgets to draw things, changes game state in one place but not another, etc. So I add two steps:

-Getting input and timer events from the platform
-Running the low-level input and timer events through an interface which turns them into high-level equivalents(select the hex at these coordinates, play the game for one frame, etc.)
-Processing the events
-Sending draw events back to the interface in another high-level form(this unit's sprite has moved 50% into the next hex on this frame)
-Drawing to the screen

For games where user control is more limited and predictable one can do without this, but the benefits I've gotten from using such a system are enormous. Since you can change the interface at will, you can run a debug console which plays the game through command-line input, lets you play back game recordings and filter through the data, etc.. Bugs in the interface will not have consequences on the intended gameplay, making it easier to know which is at fault. This is a powerful tool to cut debug time.

To turn the game-per-week pipeline into one for Love and Deep Sea means swapping out the haXe and AS3 code for Lua equivalents. That's a big job, but it's a task that can, like the initial implementation, also be approached incrementally. And since Deep Sea is a different game and not a port of an existing one, I can use old code as a model, but it doesn't have to be exactly the same.