Page 1 of 1

Question about table in memory

Posted: Sat May 06, 2017 9:41 am
by DIMMaX
Hi! And sorry for my bad english :/
in code: -- 1.088 (time ) \ 230 (mem size (mb) in proc. manager) \ 197078 (gcinfo() size in kb)

Code: Select all

for i = 1 , 3000000 do
	local x = math.random(1,100)
	local y = math.random(1,100)
	local z = math.random(10,100)

	-- testtable[i] = {x, y} --  1.088 \ 230 \ 197078
	-- testtable[i] = {x, y, z} --  1.1 \ 256 \ 220516
	-- testtable[i] = {[0] = z, x, y} --  1.120 \ 230 \ 197080 -- ???
	-- testtable[i] = {[0] = x, y} --  1.090 \ 230 \ 197078 -- ???
	  
	-- testtable[i] = {[0] = x} --  1.025 \ 230 \ 197078
	-- testtable[i] = {[0] = x,nil} --  0.940 \ 211 \ 173642 -- ???
	-- testtable[i] = {[0] = x} --  1.000 \ 230 \ 197078
	-- testtable[i] = {[0] = x, [1] = y} --  1.000 \ 394 -- ???
end
and question - WHY? O_O
e.g - i can store {[0] = z, x, y} at same size of {x, y} ?
and:

Code: Select all

	-- testtable[i] = {[0] = x} --  1.025 \ 230 \ 197078
	-- testtable[i] = {[0] = x,nil} --  0.940 \ 211 \ 173642 -- ???
OMG ! {[0] = x,nil} - smaller ! WHY?

Example 2 -- I do not understand this:

Code: Select all

test_2 = {}

for i = 1 , 1000000 do
	-- local tpl = {[0] = "node",[1] = "label x "..i,[2] = 100200} -- / 152
	-- local tpl = {[0] = "node",[1] = "label x ",[2] = 100200} -- / 88
	
	-- local tpl = {[0] = "node",[1] = i,[2] = 100200} -- / 88
	-- local tpl = {[0] = i,[1] = "node",[2] = 100200} -- / 125 ???
	
	-- local tpl = {[0] = "label x "..i,[1] = "node",[2] = 100200} -- / 183 ???
	-- local tpl = {"node","label x "..i,100200} -- / 167  ???
	
	-- local tpl = {[0] = "node",i,100200} -- / 88
	-- local tpl = {"node",i,100200} -- / 102
	-- local tpl = {i,"node",100200} -- / 102  
	
	-- local tpl = {["n"] = "node",["l"] = "label x "..i,["h"] = 100200} -- / 220
	-- local tpl = {"node @ label x "..i, 100200} -- / 157
	-- table.insert(test_2, tpl)
end
{[0] = "node",[1] = i,[2] = 100200} and {[0] = i,[1] = "node",[2] = 100200}
- different size, but:
{"node",i,100200} and {i,"node",100200} - some size O_O

and last, if {[0] = "node",i,100200} smaller, than {"node",i,100200}, then why testtable = {[0] = x, y} - 197078 and testtable = { x, y} - 197078 -- some size ?

Re: Question about table in memory

Posted: Sat May 06, 2017 5:17 pm
by raidho36
Lua tables are pre-warmed to 4 slots in hash storage and 4 slots in array storage. Anything smaller than that doesn't save memory. When keys are added, entire storage is resized if it's not big enough, to the next power of two, i.e. doubled. Relocation of the storage gets more expensive with bigger size, so this strategy saves performance greatly for big tables, which are common in Lua. Setting key to nil erases it, so the storage may be shrunk to save memory. Next, only consecutive integer keys greater than zero go into array storage, everything else goes to hash storage - including key "0". So when you set the empty table's array element to nil, that triggers shrinking, but you key 0 went to hash part. That didn't save a whole lot of memory, hence pre-warming doesn't detract much memory-wise while saving on storage relocation for small tables which are also common in Lua.

LuaJIT may pre-warm to 2 slots, which would explain your dynamic. Also there are string producing code in there, each unique string is stored separately, however any amount of them is referenced. One shared string takes less memory than unique strings​. Which makes comparing strings as fast as comparing numbers, but makes creation of new strings slow.

Re: Question about table in memory

Posted: Sat May 06, 2017 8:22 pm
by DIMMaX
thanks for the explanation, but...

Code: Select all

	-- local tpl = {[0] = "node",[1] = i,[2] = 100200} -- / 88
	-- local tpl = {[0] = i,[1] = "node",[2] = 100200} -- / 125 ???
magick zero index?

and if i undestand, if i use zero index

Code: Select all

	-- testtable[i] = {x, y, z} --  1.1 \ 256 \ 220516
	-- testtable[i] = {[0] = z, x, y} --  1.120 \ 230 \ 197080
i save memory?

Re: Question about table in memory

Posted: Sat May 06, 2017 8:42 pm
by Sir_Silver
I see code and I see writing, but I don't really see a question here.

Re: Question about table in memory

Posted: Sat May 06, 2017 10:21 pm
by DIMMaX
-- local tpl = {[0] = "node",[1] = i,[2] = 100200} -- why it use 88 mb, and this:
-- local tpl = {[0] = i,[1] = "node",[2] = 100200} -- use 125 mb ???

[0] = "node",[1] = i -- not identical -- [0] = i,[1] = "node" ?

{x, y} , and {x, y, z}, and {[0] = z, x, y} -- lua create table with reserve memory at zero index?

Re: Question about table in memory

Posted: Sat May 06, 2017 10:31 pm
by DIMMaX
-- testtable = {x, y} -- 1.088 \ 230 \ 197078
-- testtable = {[0] = z, x, y} -- 1.120 \ 230 \ 197080

testtable = {x, y, z} -- 1.1 \ 256 \ 220516
testtable = {[0] = vvv, x, y, z} -- 1.3 \ 257 \ 220511

zero index is exist? lua reserve memory, but not use it?

Re: Question about table in memory

Posted: Sat May 06, 2017 11:02 pm
by raidho36
The zero index puts value into hash part, because only consecutive indices greater than zero go into array part. Array and hash part of the table are totally separate, and yes array access is faster if you're concerned with performance. Tables' storage is pre-warmed to store some amount of values (seems like 2 for LuaJIT), so you can put that amount of variables in each part without increase of memory consumption.

Also, the OS can choose not to immediately yield memory to the program when it's requested, as long as it's not actually used. Hence you can allocate several gigabytes worth of memory in C, but as long as you don't use it, it would not put a dent on the program's memory use.

Re: Question about table in memory

Posted: Sat May 06, 2017 11:36 pm
by DIMMaX
i use - collectgarbage("collect") + gcinfo() and collectgarbage("count")
{[0] = x, y} (197 mb) - zero index not change array to hash table, becose {["x"] = x, y} (314mb)

and why - {x, y} - not eq - {[1] = x, [2] = y}, ? -- 197mb and 267mb

http://lua-users.org/wiki/TablesTutorial
Table constructors can contain a comma separated list of objects to create an "array":

> t = {"a", "b", "c"}
> = t[1]
a
> = t[3]
c

This is a syntax shortcut for:

> t = {[1]="a", [2]="b", [3]="c"}
> = t[1]
a
> = t[3]
c