Question about table in memory

[DIMMaX]

Hi! And sorry for my bad english :/
in code: -- 1.088 (time ) \ 230 (mem size (mb) in proc. manager) \ 197078 (gcinfo() size in kb)

Code: Select all

for i = 1 , 3000000 do
	local x = math.random(1,100)
	local y = math.random(1,100)
	local z = math.random(10,100)

	-- testtable[i] = {x, y} --  1.088 \ 230 \ 197078
	-- testtable[i] = {x, y, z} --  1.1 \ 256 \ 220516
	-- testtable[i] = {[0] = z, x, y} --  1.120 \ 230 \ 197080 -- ???
	-- testtable[i] = {[0] = x, y} --  1.090 \ 230 \ 197078 -- ???
	  
	-- testtable[i] = {[0] = x} --  1.025 \ 230 \ 197078
	-- testtable[i] = {[0] = x,nil} --  0.940 \ 211 \ 173642 -- ???
	-- testtable[i] = {[0] = x} --  1.000 \ 230 \ 197078
	-- testtable[i] = {[0] = x, [1] = y} --  1.000 \ 394 -- ???
end

and question - WHY? O_O
e.g - i can store {[0] = z, x, y} at same size of {x, y} ?
and:

Code: Select all

	-- testtable[i] = {[0] = x} --  1.025 \ 230 \ 197078
	-- testtable[i] = {[0] = x,nil} --  0.940 \ 211 \ 173642 -- ???

OMG ! {[0] = x,nil} - smaller ! WHY?

Example 2 -- I do not understand this:

Code: Select all

test_2 = {}

for i = 1 , 1000000 do
	-- local tpl = {[0] = "node",[1] = "label x "..i,[2] = 100200} -- / 152
	-- local tpl = {[0] = "node",[1] = "label x ",[2] = 100200} -- / 88
	
	-- local tpl = {[0] = "node",[1] = i,[2] = 100200} -- / 88
	-- local tpl = {[0] = i,[1] = "node",[2] = 100200} -- / 125 ???
	
	-- local tpl = {[0] = "label x "..i,[1] = "node",[2] = 100200} -- / 183 ???
	-- local tpl = {"node","label x "..i,100200} -- / 167  ???
	
	-- local tpl = {[0] = "node",i,100200} -- / 88
	-- local tpl = {"node",i,100200} -- / 102
	-- local tpl = {i,"node",100200} -- / 102  
	
	-- local tpl = {["n"] = "node",["l"] = "label x "..i,["h"] = 100200} -- / 220
	-- local tpl = {"node @ label x "..i, 100200} -- / 157
	-- table.insert(test_2, tpl)
end

{[0] = "node",[1] = i,[2] = 100200} and {[0] = i,[1] = "node",[2] = 100200}
- different size, but:
{"node",i,100200} and {i,"node",100200} - some size O_O

and last, if {[0] = "node",i,100200} smaller, than {"node",i,100200}, then why testtable = {[0] = x, y} - 197078 and testtable = { x, y} - 197078 -- some size ?

[raidho36]

Lua tables are pre-warmed to 4 slots in hash storage and 4 slots in array storage. Anything smaller than that doesn't save memory. When keys are added, entire storage is resized if it's not big enough, to the next power of two, i.e. doubled. Relocation of the storage gets more expensive with bigger size, so this strategy saves performance greatly for big tables, which are common in Lua. Setting key to nil erases it, so the storage may be shrunk to save memory. Next, only consecutive integer keys greater than zero go into array storage, everything else goes to hash storage - including key "0". So when you set the empty table's array element to nil, that triggers shrinking, but you key 0 went to hash part. That didn't save a whole lot of memory, hence pre-warming doesn't detract much memory-wise while saving on storage relocation for small tables which are also common in Lua.

LuaJIT may pre-warm to 2 slots, which would explain your dynamic. Also there are string producing code in there, each unique string is stored separately, however any amount of them is referenced. One shared string takes less memory than unique strings​. Which makes comparing strings as fast as comparing numbers, but makes creation of new strings slow.

[DIMMaX]

thanks for the explanation, but...

Code: Select all

	-- local tpl = {[0] = "node",[1] = i,[2] = 100200} -- / 88
	-- local tpl = {[0] = i,[1] = "node",[2] = 100200} -- / 125 ???

magick zero index?

and if i undestand, if i use zero index

Code: Select all

	-- testtable[i] = {x, y, z} --  1.1 \ 256 \ 220516
	-- testtable[i] = {[0] = z, x, y} --  1.120 \ 230 \ 197080

i save memory?

[Sir_Silver]

I see code and I see writing, but I don't really see a question here.

[DIMMaX]

-- local tpl = {[0] = "node",[1] = i,[2] = 100200} -- why it use 88 mb, and this:
-- local tpl = {[0] = i,[1] = "node",[2] = 100200} -- use 125 mb ???

[0] = "node",[1] = i -- not identical -- [0] = i,[1] = "node" ?

{x, y} , and {x, y, z}, and {[0] = z, x, y} -- lua create table with reserve memory at zero index?

[DIMMaX]

-- testtable = {x, y} -- 1.088 \ 230 \ 197078
-- testtable = {[0] = z, x, y} -- 1.120 \ 230 \ 197080

testtable = {x, y, z} -- 1.1 \ 256 \ 220516
testtable = {[0] = vvv, x, y, z} -- 1.3 \ 257 \ 220511

zero index is exist? lua reserve memory, but not use it?

[raidho36]

The zero index puts value into hash part, because only consecutive indices greater than zero go into array part. Array and hash part of the table are totally separate, and yes array access is faster if you're concerned with performance. Tables' storage is pre-warmed to store some amount of values (seems like 2 for LuaJIT), so you can put that amount of variables in each part without increase of memory consumption.

Also, the OS can choose not to immediately yield memory to the program when it's requested, as long as it's not actually used. Hence you can allocate several gigabytes worth of memory in C, but as long as you don't use it, it would not put a dent on the program's memory use.

[DIMMaX]

i use - collectgarbage("collect") + gcinfo() and collectgarbage("count")
{[0] = x, y} (197 mb) - zero index not change array to hash table, becose {["x"] = x, y} (314mb)

and why - {x, y} - not eq - {[1] = x, [2] = y}, ? -- 197mb and 267mb

http://lua-users.org/wiki/TablesTutorial

Table constructors can contain a comma separated list of objects to create an "array":

> t = {"a", "b", "c"}
> = t[1]
a
> = t[3]
c

This is a syntax shortcut for:

> t = {[1]="a", [2]="b", [3]="c"}
> = t[1]
a
> = t[3]
c