sock.lua - A simple networking library for LÖVE

[klis]

@Ikoroth thank you for reminding me of xpcall. Wondering why didn't I think about this before. LOL! This will solve my problem.

[4aiman]

Could anyone tell me how to un-freeze the app while it is sending a file?

Some info on how I'm doing that first, the symptoms after that. (see at the bottom of the post, please)

What I'm doing is letting a client to download some files from the server.
The logic is like this (S-Server, C-client):

• C - Hi!
  S - Hello. Care to introduce yourself?
  C - Sure! Name's 4aiman, my pass is *****.
  S - Ok, let me check... Yep, there is such user with that credentials. Come on in
  S - You know, I have some files here... like... thousand or so. I'll send them to you and you'll check if you need any of them, m'kay?
  C - Ok!
  S - Here's the 1st filie's hash. Do you need it?
  C - Let me see... there's no file with that hash on my side. I'll take this one
  S - Ok, sending file.
  S - And here's some info on it.
  C - Got file!
  C - Got Info.
  C - Ok, I'm ready for another one!
  S - Here's the 2nd filie's hash. Do you need it?
  ...

The server's work with the file list is done with the code as ugly as this (only much more nested tables).

Code: Select all

if current_file_ord and files_to_receive then -- we're getting  current_file_ord file out of files_to_receive
   if current_file_ord < files_to_receive then -- do we have anything to send?
      local rec = sending_list[current_file_ord] -- get a table with info on current file. The table is pre-generated elsewhere
      local filename = rec.filename
      local prefix   = rec.prefix
      local file = love.filesystem.newFileData(prefix.."/"..filename) -- get contents of the current file
      local hash = md5.sumhexa(file:getString()) -- yet again my thanks to Kikito
      sending_to:send("do you need this file?", hash)
   end
end

The code is bogus.
It's here only to show what I'm doing.
Note, that the code above is inside the love.update func.
Right before dealing with the list the server:update() gets called (dunno if that matters to any of fellow form users).

Now, if the client has checked the hash and it appears it actually don't have one (or the one it has produce a wrong md5 hash) then it says "Yep, I need that file":

Code: Select all

client:send('I need a file', hash)

The server has it's own answer to that message.

Code: Select all

server:on("I need a file", function(data, client) -- remember, that data contains an md5 hash
   if not (client_status[client] == 'I need a file') then -- check the status of a client, maybe it had requested that file long ago
      client_status[client] = 'I need a file'	
      local path, type = hashes[data].path, hashes[data].type -- server has all info on hashes pre-generated
      local file = love.filesystem.newFileData(path)
      client:send("here's your file", file) -- the file
      client:send("here's your file", {filename=file:getFilename()}) -- the info on that file
   end
end)

The code again is bogus too.
Note two messages: one with the file itself and one with the info. Probably should send info ahead...
Anyways, client determines what he got (actual file data or just some info) and uses that info to cache a file.


And the last piece of info and code to show how is that I know something's off:
I have this line

Code: Select all

runtime = (runtime or 0) + dt

inside the love.update method and this line

Code: Select all

love.graphics.print("Uptime:\t".. dt_to_time(runtime),10,30)

inside the love.draw
All that dt_to_time does is converting dt sum to a "00h 00m 00s 000ms" format.



================ the problem I'm trying to solve =================

The problem is whenever server sends a file of more than 100Kb I notice that the dt increases up to ~1.5 seconds for a file of 3-5Mb. (The uptime drawn on the screen freezes and after ~1.5 seconds updates.)
For testing purposes I *do* run client and server within one lua file. So I assume that for ~0.7 sec the server sends and for ~0.7 sec the client receives a file. Maybe that's not how it works, but my point here is that the load is spread between two.

Now what I'm concerned about is what If there would be yet another client?
I assume that during those 1.5 seconds any new client won't be able to do anything with the server as it is too busy.
Or that is not the case?

I'm very confused on this matter since I have never ever tried to implement my own client-server protocol and simultaneous multi-user downloads.

Phew... isn't that a long post?
Thanks in advance!

[zorg]

In contrast, a very short answer:
You're dealing with blocking function calls, i.e. ones that don't return control until they are finished.
I'd use a separate thread for sending or receiving large files, though that will mean the logic will get more complicated as well.

[4aiman]

Thanks, zorg!

So, theoretically, if instead of sending from love.update I will create a new thread and somehow manage to maintain communication with a client... Gotta try

Sorry for digressing, but is it "normal practice" to create a new server in a new thread, tell client to connect there to get files and then (after the client got everything) re-connect it to the main app?
I really don't know whether there are more aesthetic ways to do just that.

[Nuthen224]

While putting the networking in another thread seems like a just and noble thing to do, I have an alternate suggestion.

Enet automatically splits the data you want to send into smaller packets. When you are sending them with the "reliable" mode, it ensures that each of these packets is received, and in order too. If any of them drop it has to resend it. I can only imagine that with a 100kb file it is split into tons of packets - all which need to be guaranteed to have been received and sorted into the right order by enet - and it will pause until the sending is complete.

My suggestion is perhaps you could split the file up into smaller segments on your own, perhaps 1kb segments or something, and only send 1kb of data or so per update tick. Keep track of what belongs to what file. If you keep them reliable then they will be received in the correct order. Then assemble them once they are all received, or write to a file as each is received. That could perhaps alleviate this screen freezing by smoothing out the data bottleneck.

[zorg]

My suggestion is perhaps you could split the file up into smaller segments on your own, perhaps 1kb segments or something, and only send 1kb of data or so per update tick. Keep track of what belongs to what file. If you keep them reliable then they will be received in the correct order. Then assemble them once they are all received, or write to a file as each is received. That could perhaps alleviate this screen freezing by smoothing out the data bottleneck.

So instead of enet chopping your data up, do it yourself? That's +1 thing you need to keep track of, and it doesn't guarantee that enet won't chop it up even more still. It's worth trying though, it's just a bit weird.

[Nuthen224]

Hey, that's networking, it's weird all the way down

[4aiman]

Nuthen224, zorg, thanks!
I've been able to offload all the gimme-those-files stuff to a separate thread.
Threads appeared not as bad as those seemed to be.
If only I could kill one... but that's for a different thread

[Sir_Silver]

I believe that, once you've yielded the coroutine, you just need to free any references to it, and it will become garbage collected.

[4aiman]

I believe that, once you've yielded the coroutine...

I'm using love.thread rather than coroutines.
Actually, there's no way to kill a server too - no server:stop() or any other equivalent.
Not sure nilling will do the job and underlying eNet will acknowledge the fact that the nilled server is no more.
So, now there're 2 things I need to be able to kill in reasonable time (of several minutes perhaps?)