Running Python helper scripts in bundled game

[meteopath]

I finished my game. It works on my computer. But my attempts to bundle it in Xcode fail. This isn't surprising because I haven't put in the time to learn Xcode yet. So go learn Xcode. I thought I'd ask for help first because I'm concerned that the file design I've chosen for my game is a fatal flaw, and that I would be better off taking a different path to deployment, whether that means rewriting code and changing the file design or abandoning Xcode altogether and trying instead to deploy my game as a web app.

I'd really appreciate any advice or suggestions.

The basic problem with my file design is that the LOVE game relies on data provided by two Python helper scripts. The LOVE game and the Python files must be able to communicate while the game is running. This works when the game is unzipped via io.popen. It fails when the game is zipped.

The solutions seem to be either finding a way to enable Python-Love communication in a bundle or to rewrite the Python files (and, unfortunately, the functions in third-party libraries the Python files rely on) into Lua. That kind of translation seems beyond me right now. But if it is the way to go, I could certainly try to learn and figure it out. So any perspective would be great on what my simplest path forward is.

I posted the game on github. I think the game runs if it's cloned. If you try it and it doesn't, please let me know. The Readme has the same description of my problem with a little more detail.

https://github.com/delightfuldonut/FirstPersonFranz

[ReFreezed]

It looks like converting the Python code to Lua/LÖVE (or vice versa) is pretty much out of the question - it would be the "best" option. But anyway, the solution should just be to have the Python scripts (and whatever files they themselves use) outside the .love file, or possibly copy the files from the zip to the save folder on startup and running them from there (may be a bad idea if there are many or big files).

[meteopath]

I'll give it a shot. The idea sounds great.

My Python scripts aren't big. I don't know about the libraries. I'm not sure how much of gutenberg and PIL I'd have to copy. I guess I'd need a Python interpreter too. I could try creating a venv folder and copying that. Thanks very much for the reply. I hadn't thought of that. Also for the perspective on how hard rewriting the library functions would be.

[ReFreezed]

The only files that need to be in the .love file are main.lua and conf.lua. Literally everything else can be next to the .love file or in a subfolder - no copying needed. I don't really have any input on how the Python files should be executed, at least in macOS. I'd imagine you could put the Python binaries somewhere in the .app bundle? Or simply have separate .app bundles for the game and the Python interpreter (or one for the LÖVE part of the game and one with *all* the Python stuff).

I haven't used Python that much, but having a quick look specifically at all the library files in the repo and all the import statements and stuff just makes me think trying to convert everything would be a bad headache for a long time.

[meteopath]

Your earlier suggestion actually seems to work, at least in a simple test run. I put the python files and libraries and a Python interpreter in a venv folder next to main.lua. I put a copy command (cp -R) at the top of main.lua:

Code: Select all

function bashEscape(s)
    return "'" ..s:gsub("'", "'\\''") .. "'"
end

local path = love.filesystem.getSaveDirectory()
local handle = io.popen('cp -R venv '..bashEscape(path)) 
local result = handle:read("*a")
handle:close()

And then lower I had main.lua send another command thru io.popen to call the necessary function in the Python script, and it worked. The Python script (now a copy in the save folder) took the command and the arguments and outputted a JSON file to the save folder. I can't tell you how happy I am about this. I've been so worried that I wouldn't be able to figure it out. Still have to run a test with a zipped .love file and then try it with the whole game. But it looks very promising. So thank you again. I'll update this with the results, whatever they are. But I'm feeling very good about it!

[ReFreezed]

I'm not sure I explained things properly, or if I'm missing something here. If the Python stuff isn't inside the .love/zip file then you can just run Python from there with no file copying needed (see love.filesystem.getSourceBaseDirectory/love.filesystem.getSource). This would certainly be the best option. But if the Python stuff is inside the .love/zip file (next to main.lua) then you have to use love.filesystem functionality to copy the files out of the .love/zip file (to the save folder) before other programs (executed with io.popen/os.execute) can access them.

[meteopath]

It's me being uncomfortable with Xcode and wanting to simplifying things as much as possible before moving my code into Xcode. I've messed around with Xcode a couple of times in the last few weeks and been pretty overwhelmed by all the terms and settings. I have tried the file structure you suggest, figuring out the right paths between getSourceBaseDirectory and getSource so that I got it to work on my computer: a .love file (without the Python scripts) in a folder next to my Python scripts. It worked on my computer. But I've tried it in Xcode, and it fails. This was definitely because I don't understand Xcode, to the point where I probably didn't know how to even phrase my problem correctly in a google search because I couldn't find anything helpful in the results...just a lot of 'Python and Xcode, don't do it.' So it is good to hear that once I learn it a little, I might be able to design a file structure in Xcode to get my files talking like they should. But for now my plan is to first try doing a zipped file with everything, including the copy commands at the top of main.lua, because once I have that single .love file, I can follow the bundling instructions step-by-step without having to freelance at all in Xcode. After that maybe I can look into a more efficient non-copy way. I really appreciate the attention. I'll update this with the results, and if they aren't good I'll probably have more questions and be a little more Xcode literate so that I make sense.

[ReFreezed]

Ok, I see. Unfortunately I don't have any experience with Xcode at all so I can't help with that. My very ignorant question would be why you're using Xcode at all, but I'll assume you have reasons for that.

[pgimeno]

I see one of the files (gifMaker.py) is a GIF optimizer and encoder. I have already written a GIF decoder, which by the way, I see you have included in the repo (vgif.love). I could consider writing a GIF encoder, I didn't think there would be demand for that.

I'm wondering what you're trying to do with the GIF encoder, specifically whether a regular, unoptimized GIF would suffice. I haven't tried to run your program, so I don't know what the project is about.

Also I'm curious about how you're using vgif if you're using it at all. It strikes me as odd that you've included the .love binary in the repo rather than adapting the Lua source (which is also included) for use with your program. Even more strange is that you're requiring gifload without using it, without even storing the returned function anywhere, so it's useless to require it.

About the other fiile (bookParse.py) I couldn't make sense of its purpose on a cursory read.

I have the strong impression that you're using the wrong tools for the purpose you're seeking. Perhaps invoking something like ffmpeg is a better idea than using Python for certain things. Can you explain what your game is about, why it needs a gif encoder and viewer, and what does bookParse.py do?

[meteopath]

I think an unoptimized GIF encoder would work for me here because the optimizer I was using (gifsicle) never took more than 10 percent off the user-generated GIFs made by gifMaker.py. This is a screen shot my game at the point where the user chooses to make a gif and selects the text to go into the gif:



The user-generated gif is the little typewriter (here, at 4x speed) in the pop-up window above (or below at the very bottom. I think I messed up the formatting of this comment).

I think the optimizer never did much because my drawings had no gradients and only a few colors. And still, the user-gernerated gifs are very heavy. The one above is 3.9MB. I never really figured out why. I got the idea it was the use of a font. And it didn't seem like it was caused by the specific font I'm using, which is a normal .ttf of about 400kb. I was disappointed by how heavy all the gifs were, but the gifMaker is not really core to my game, so I decided it was ok that way. But I would like to learn a little more about it at some point.

The vgif.love does get used in my game. It's what opens the gif in the screenshot above. It is called in my infoPopup.lua file. lines 73-84:

Code: Select all

function openGif(params)
    if pcall(doPopen2,params) then
        print('command sent via doPopen')
    else
        print('*****************doPopen failed*********************')
    end
end

function doPopen2(path)
    local pathToSaveFolder = love.filesystem.getSaveDirectory()..'/userGifNew.gif'
    local path2 = love.filesystem.getSource( )
    local handle = io.popen('love '..bashEscape(path2)..'/vgif.love '..bashEscape(pathToSaveFolder))
    local result = handle:read("*a")
    handle:close()
end

I was trilled to find it a few weeks ago. I don't remember why displaying the gif was tricky or why I thought it was tricky. But thank you for that. It was just the right thing and easy to install. I guess I can take the gifLoad require and the library out of the code. I tried to clean my code up before posting, but I think I have a lot of dead stuff in there.

bookParse.py is important. It is essential, while the gifMaker is just a nice-to-have. I wrote bookParse.py in Python because I needed the help of a Python library (c-w/gutenberg on gitHub), specifically the function load_etext called on bookParse.py, line 309, here:

Code: Select all

    while True:
        try:
            text = strip_headers(load_etext(ids[j]).strip())
        except: 
            j = j+1
            continue

This function takes a book_id-number as argument and downloads a plain text version of that book from a Project Gutenberg library at http://gutenberg.readingroo.ms. The indexing scheme of the readingroo.ms website is explained in the c-w library. So I think theoretically I could replicate the load_etext function. The problem is it's just beyond me right now. I spent a little time a month or two ago trying to figure out lua socket, and I never got any of the test cases to work. As for the rest of bookParse.py, I'm pretty sure I could rewrite it in Lua. It's just a ton of regular expressions and matching to do things like find the start of the main text and find chapter breaks and account for the wide variety of formatting in the incoming texts.

And as for the main problem of this thread, I might be ok. ReFreezed's suggestion to try copying the Python-related things into the saved folder seems to work. I haven't tested it in Xcode, but I did do a test on my computer today zipping it all, Python files included, into a .love file with that cp -R venv command at the top of main.lua. It ran just fine when I ran it from the command line. Oddly, though, it failed to run when I tried running it by double-clicked the icon. Not sure why. So more tests to do, but pretty optimistic.

I don't want to make this too long. But thank you for asking about the game and for suggesting ffmpeg. The game is basically a first-person shooter for writers. You get the view you'd get sitting down to a typewriter. You press the keys and the typewriter goes, revealing the text of a random book. You can also exit the typewriter and read the book with a conventional page format. You can flip through the pages or jump from one book to the next. There is a metadata view for details of all the books presented. Here a couple screenshots actually taken today from that successfully zipped .love file. User navigation is done by keyboard shortcuts: