Q: How to access get_disc_threshold() and other global functions?

[LeslieGerman]

Hi John,

I wanted to use the get_disc_threshold() function.
But vips.get_disc_threshold () does not exist.
How can I use it and other global functions?

They seem like not "exposed" to Lua.
Or do I misunderstand something?

Thanks.

[jcupitt]

Hi @LeslieGerman, no, sorry, that function is not wrapped by lua-vips. You would need to write a little FFI code and call it yourself.

What are you trying to achieve? Perhaps there is some other way to do what you want.

[LeslieGerman]

What are you trying to achieve?

Well, first just experimenting...

I want to achieve something like the abandoned vips streaming branch would do, but with manual-coding on top of the existing vips API.
I don't need to handle *nix file descriptors, we have our own special stream classes.

Basically our framework would call the following filter function, passing bytes-sequences of a JPEG, GIF etc. image in chunks:

inputImage = nil
headerChunks = ""

function filter(chunk)
	local ret = ""
	if chunk == nil then
		close_processing(inputImage)
	else
		if inputImage == nil then
			headerChunks = headerChunks .. chunk
			inputImage = detect_type(headerChunks)
		else
			ret = transformed_image(inputImage, headerChunks, chunk)
		end
	end
	return ret
end

The filter would return the bytes of the transformed image in chunks, too.
Maybe somewhere another vips image have to be created, too, for the transformed image.

Is it possible to implement this on top of the current API?

(vips is a great lib, BTW :) )

[LeslieGerman]

Just a note:
I'm aware that not all image types are streamable, like you mentioned it here.

[jcupitt]

Ah OK. No, sorry, I don't think that can be done. You have to read the whole of the compressed image into memory.

The true-streams idea should probably be resurrected :( It's not actually a huge amount of work, you only need to add support to the streamable formats, it's just a question of someone finding the time.

[LeslieGerman]

Hi John,
How much effort do you think it would take to finish and merge the streaming branch?

It seems to me the task consists of 2 steps:

1. finish the implementation
2. merge it to master

Regarding 1:
What is not complete yet in terms of functionality and features?
I mean what is still to be implemented?
Do you already have tests for the new features, classes, etc.?

Regarding 2:
Rather technical, but maybe time-consuming, since the master diverged a lot - like you mentioned in one of your comments.
Maybe it's not possible to do it without knowing the code at some level, because of the likely merge conflicts.

And the important question for me:
Having the streaming features finished, would it be possible to use it in the way I described above in my comment? I.e. without the file-descriptors, but using some API functions.

Thanks.

[jcupitt]

libvips is a "pull" system, so you can't push bytes into it as they arrive, instead you'd need to attach a callback to libvips to grab another chunk of input when it's needed.

The streaming branch added a couple of simple classes which you could put the callbacks into. The API would look something like:

input_steam = libvips.new_input_stream()
input_stream.fetch_bytes = function () my_own_stream_object:fetch() end
image = Image.new_from_stream(input_stream)

now libvips will call the fetch_bytes method whenever it needs more data. In turn, this would call the fetch method on your stream class, which would presumably read another X bytes (up to 4096 bytes perhaps?) and return them as a string.

There's something similar to for output streams.

Anyway, I'd guess a couple of weeks work.