Is this a good ECS like approach?

[ivan]

What do you do if you have a system that requires two components, like "position" and "velocity?"

There is a saying in programming: "data that changes together - belongs together". So ideally, the components should be combined/organized appropriately. Like for example "width" and "height" in this case don't mean much without a position.

What do you do about systems that require a component *not* be present?

Can't think of an example to this. Generally, you want the critical systems like drawing and collisions to use just 1 component. Higher level systems (pathfinding or AI) may access 2 or more components, but that code would usually run less often.

This seems like a lot of trouble to go through in the name of performance, have you actually perf'd this against the "simple approach?" Note that if components are primitive values they don't create *any* extra tables in the "simple approach" (obviously). Although, entities will need to be tables, where I guess they can just be ids in your example. I'm skeptical about pairs ruining everything, though.

Sure, I've used this approach and it works pretty good.
It's not perfect, but the main advantage is that you can easily iterate all entities that have a specific component.
This way you don't have to loop over all the entities in each one of your systems.

[airstruck]

There is a saying in programming: "data that changes together - belongs together". So ideally, the components should be combined/organized appropriately. Like for example "width" and "height" in this case don't mean much without a position.

Sure, that might be ideal, but doesn't seem realistic. I don't know if you're suggesting that position and velocity should be parts of the same component, but surely things can have a position without having a velocity, and something like a motion system would need both. Not to mention, only the position would be changing in that system, not the velocity (so they don't change together, or belong together, really).

It's pretty common for systems to need position and at least one other component as far as I know, for example a firing system needs to know where the thing is so the bullets can come from the right place.

Can't think of an example to this. Generally, you want the critical systems like drawing and collisions to use just 1 component. Higher level systems (pathfinding or AI) may access 2 or more components, but that code would usually run less often.

I can't think of one offhand either, but every ECS implementation I've seen has a way of doing it, so I assume someone must be using it.

It's not perfect, but the main advantage is that you can easily iterate all entities that have a specific component.
This way you don't have to loop over all the entities in each one of your systems.

Yeah, that is a nice benefit. I guess if most of your systems don't apply to most of your entities, it might make a noticeable difference. I'm still not sold on pairs, since it won't JIT and is just dog slow compared to ipairs or numeric for (as of last time I checked, anyway). I'd probably be willing to overlook the somewhat awkward getComponent stuff if things could be done without pairs somehow, but as it stands I'm not convinced the added cruft is worth whatever performance gains you might get out of it. ECS is about convenient design anyway, not raw performance, right? Really, in scenarios where most of the systems *don't* apply to most of the entities or where performance is a huge concern, maybe ECS should just be off the table altogether.

[Plu]

I can see a few advantages of using the more explicit method Tjakka uses over using table fields and functions.

Primarily, It allows you to specify which components must go together. A component in his setup must have both x and y, while in yours you could have only one of the two, which serves no purpose and almost certainly means some kind of bug. Additionally, it allows you to assert the presence of another component if it's required because componentB makes no sense to have without componentA; this again lets you spot issues sooner and without having to debug.
Also, it prevents name collisions. You have made "x", "y", "w" and "h" components implicitly, and now have to take care everywhere that these values on the table represent a component. There is no system that will warn you if you make create a system that uses "x" for something else, whereas with a definer function it would immediately warn you that a named component already exists.

Ultimately, like Tjakka said, once your project starts having dozens of components I think it'll help that you'll have a good overview of all your components and what they do, a single point of entry for initialization and destructor functions, and a few methods that help you catch when you're trying to re-use an existing name.

[raidho36]

I just doodled this little thing here, it has huge memory overhead, but it should work fast (the overhead memory isn't actually used most of the time) and it's more of a proof of concept than anything. But I think it could be adapted to real ECS system.

Note that there is no real reason to use ECS other than for performance benefits, and there would be none by default, i.e. if you use Lua tables. E.g. you can just create a list of drawable objects and stick everything that's drawable into it, and then render it using duck typing. It wouldn't be different from iterating over components that reside in their own component-specific lists.

Code: Select all

local StructArray = setmetatable ( { }, { __call = function ( class, ... ) return class.new ( ... ) end } )
StructArray.__index = StructArray

function StructArray.new ( field, ctype, length )
	local self = setmetatable ( { }, StructArray )
	self.allocated = length
	self.field = field
	self.ctype = ctype
	self.length = 0
	self.empty = { }
	self.positions = { }
	self.references = { }
	self.array = ffi.C.malloc ( ffi.sizeof ( ctype ) * length )
	return self
end

function StructArray:insert ( object )
	if self.positions[ object ] then return end
	
	-- reallocate array if it's too short
	if self.length == self.allocated then
		self.allocated = self.allocated * 2
		self.arr = ffi.C.realloc ( self.array, ffi.sizeof ( self.ctype ) * self.allocated )
	end
	-- check for unused blocks
	local position = self.empty[ #self.empty ]
	if position then
		self.empty[ #self.empty ] = nil
	else
		-- use last block
		position = self.length
		self.length = self.length + 1
	end
	-- record object
	self.positions[ object ] = position
	self.references[ position ] = object
	return position
end

function StructArray:remove ( object )
	-- find object position
	local position = self.positions[ object ]
	if not position then return end
	-- remove object record
	self.positions[ object ] = nil
	self.references[ position ] = nil
	-- add unused block record
	self.empty[ #self.empty + 1 ] = position
end

function StructArray:pack ( )
	while #self.empty > 0 do
		-- find position to which relocation is possible
		local hole = self.empty[ #self.empty ]
		self.empty[ #self.empty ] = nil
		-- relocate data from the end of array
		self.length = self.length - 1
		self.array[ hole ] = self.array[ self.length ]
		-- update object record
		local object = self.references[ self.length ]
		self.references[ self.length ] = nil
		self.references[ hole ] = object
		self.positions[ object ] = hole
		-- update object's field record
		object[ self.field ] = hole
	end
end

[airstruck]

Note that there is no real reason to use ECS other than for performance benefits

I'm not sure where you got that idea. As I understand it ECS is all about SRP/SoC, composition, clean design essentially; any performance gains are incidental and there's almost definitely going to be a way to achieve better performance without ECS in any scenario (but without any of the benefits).

Do you have any reference you can cite about "no real reason to use ECS other than for performance benefits?" I've never heard of this, all the literature on ECS I've ever seen has talked about advantages from a design perspective, and when performance is mentioned, it's mentioned as a *disadvantage* (for example, see the Wikipedia entry or pretty much any other article on the subject).

[raidho36]

Performance really depends on whether or not you can minimize cache misses. This is primarily achieved with localized memory space access, elimination of remote jumps. Simplest example is using a struct vs. multiple arrays per each struct field, when you iterate over structs or over fields. Since fields as arrays are disjoint, there will be long jumps when you access different fields of the same object, increasing cache misses. And vice versa, if you access individual fields, having them bundled with other fields (that you don't care for) in an object increases jump distance when you go for the next field, increasing cache misses. But if you don't design your code to accommodate for hardware quirks and use too high level abstractions, you won't achieve high performance either way. E.g. if you create individual component objects on heap and put them in a list. I didn't read the wikipedia article but one has to doubt their aptitude and truthfulness and do their own research, since they have a long standing history of misrepresenting facts, so that wouldn't matter.

We already had that argument and my point is that code purity is tertiary concern, the first priority is stability and performance, secondary being your ability to write it fast. But that's just my opinion.

[airstruck]

I asked if you had any reference you could cite to back up your claim that "there is no real reason to use ECS other than for performance benefits," I guess the short answer is "no?"

I mean, if the only thing someone cares about is performance, then I guess the only reason they'll do anything is for performance, but that doesn't make statements like the above true. You might as well just say "I only care about performance" and be done with it. The "real reason" the rest of the world uses patterns like ECS has nothing to do with performance. That's not just me saying that; seriously, just look at any article on the subject.

It's not about "code purity," it's really about the fact that systems are highly reusable (because of their singular purpose), for example a gravity system or motion system or whatever can be used in many different games as long as the relevant components have the same meaning (which, in my opinion, they should). You don't achieve this kind of reuse by accident or by just worrying about performance above all else, you achieve it through thoughtful design. You may not think code reuse is important (or even acknowledge its existence), but I think this is a clear case where it can boost productivity across multiple projects.

@Plu, I see your point, if that kind of stuff is important to you it's definitely a benefit. I tend not to care about that kind of "training wheels" type of code, so it just feels like extra cruft to me; if I did care I'd probably be using a strongly-typed strictly class-based language like Java or something. I do think there could be some merit to something like "component schemas," but I'm not convinced this is the best way to do it. It's also easy enough to go around the safety mechanisms and create more problems, which is actually one of the concerns I had in mind earlier. I mean, if supplying "x" and forgetting "y" is a concern, then removing "x" or "y" the usual way and forgetting to use the safety mechanism should be just as much of a concern, shouldn't it?

[raidho36]

Common programming knowledge and few simple test cases support the idea that if your sequentially access resources that are located in closer proximity, then execution performance increases due to better CPU cache performance. YMMW, but the penalty for not hitting even L3 cache can be 3-5 times the total execution time. Hitting L2 cache very often can give another 3-5 times boost compared to L3, and hitting L1 cache will give yet another 3-5 times boost. You may dismiss this as anecdotal evidence but it works like this on any CPU that has cache, particularly on any x86-based CPU, so absolute majority of desktop computers. I wouldn't know of any modern CPU that didn't had a CPU cache, in fact.

I also wouldn't know of any credible figure or organization that would say this explicitly, so I wouldn't be able to make argumentum ad verecundiam. So you'll just have to take my word on it.

[airstruck]

I do see what you're getting at, and I think you could probably achieve it with something like Ivan's approach (if you can ditch pairs somehow), maybe combined with something like your approach (if you don't care about mobile; FFI's going to slow you down there). My point is, you can't just unequivocally say things like "there is no real reason to use ECS other than for performance benefits." If you think there's no other reason for you personally to use it, that's fine, but it's misleading to make a statement like that when there's an overwhelming amount of evidence indicating that there are real reasons most people use it that have nothing to do with performance.

Honestly, I think at some point you'll appreciate the real purpose of patterns like this, even if right now you only see it as a potential way to optimize performance. I'm not saying this to be a dick or anything, I really think you have an interesting and probably valuable point of view on it, but I don't think you've fully realized the purpose of this pattern. In any case, when there's a huge difference between your personal opinion and what the rest of the world thinks, the responsible thing to do would be to make it clear that the idea you're presenting is just your own opinion, and isn't commonly accepted as being true, and maybe share the other side of the story, and if you believe it's in error somehow, maybe explain why you think that, if you can give some substantial evidence to support it.

[raidho36]

Well I did said it was just my opinion.