I am trying to get the origin in a pixel shader in screen space but the origin is always the same value (I think it is always 0,0 but opengl makes it hard to tell)
No, the origin rarely changes in Löve 11.x because most transforms are done by the CPU, not the GPU. See the note in TransformMatrix here: Shader_Variables (it applies to TransformProjectionMatrix as well).
I believe it's a consequence of overzealous autobatching. Apparently it's done so the same batch can be used between matrix changes, yet I don't think it's common to keep drawing the same texture between matrix changes.
That is pretty weird since I am using the transform projection on the vertices and that has an effect, so I was expecting I was missing something basic
If you leave only the projection and remove the transform (i.e. use only ProjectionMatrix), you'll probably obtain the same result. That means that the transform is a unit matrix and therefore has no effect.
The projection matrix is necessary to transform screen coordinates to normalized device coordinates (ranging from -1 to 1 in all axes). I don't think there's anything that changes it, so it's likely not considered for autobatching. I haven't looked into the related source, though, so I may be wrong.
In 11.x I'm not sure if it's possible directly. Maybe if you render a spritebatch, not sure. Anyway, if you use the Löve transformation object (e.g. via cam11) you can pass the transform matrix to the shader; see Transform:getMatrix and Shader:send.