Array encoding/decoding

[greg-rychlewski]

Closes #7

ltree introduced a binary protocol a couple years back. If you take advantage of it then you can get arrays for free.

since binary encoding/decoding is only available in pg 13+ i changed this to text encoding

[greg-rychlewski]

cc @warmwaffles in case you're still interested

[warmwaffles]

I am still VERY interested in that.

[greg-rychlewski]

@warmwaffles there's something else I can do, if the maintainer of this is busy with other things

i can make a package just for the binary protocol and then you'd just have to change [EctoLtree.Postgrex.Lquery, EctoLtree.Postgrex.Ltree] below to whatever my module names are

Postgrex.Types.define(
  MyApp.PostgresTypes,
  [EctoLtree.Postgrex.Lquery, EctoLtree.Postgrex.Ltree]
)

then it should just work™. if you're interested i can make a package quickly

[warmwaffles]

Would be nice if this was just upstreamed into the postgres ecto adapter 😄

[greg-rychlewski]

I'll give it a shot :). I think the stance in the past was to use packages for non-builtin data types. But this one is really simple so hopefully it's ok.

[warmwaffles]

@greg-rychlewski the funny part is that the postgres ltree extension is in the documentation for custom adapters.

[greg-rychlewski]

Here goes nothing: elixir-ecto/postgrex#636

[greg-rychlewski]

I just found out this only works for pg 13+. The text protocol is the only one prior to that.

[greg-rychlewski]

You could do arrays for the text protocol too but you'd have to create another extension for ltree/lquery arrays. i.e. type: _ltree and type: _lquery. And then this would have to convert the Elixir list to the postgres array text protocol, i.e. {'this.is.path1, 'this.is.path2'}

If that's preferable for you all, I could submit a PR for that instead.

[warmwaffles]

Ooof, I can't use this in our production use case at the moment. We are on PG12, but are working towards migrating to PG14.

[greg-rychlewski]

Ok I'll change this PR to use text protocol and then Postgrex can handle the binary protocol.

[greg-rychlewski]

@warmwaffles I got the text protocol working, you just have to add these 2 new extensions:

EctoLtree.Postgrex.LqueryArray, EctoLtree.Postgrex.LtreeArray

I'm not sure if the decoding will fail in certain situations though. It assumes that commas aren't allowed in any of the individual elements. I'm not familiar enough with either data types to know if that's possible though.

I tried some queries like this and it failed in psql:

select '{*.path1.*,path2.*{2, 5}}'::lquery[];

If there are some cases where commas are allowed we just have to make the decoding smarter. If someone can find a query that breaks I can fix it.

[greg-rychlewski]

BTW, the reason you need a completely different extension for the text protocol is because arrays are still just strings but formatted a certain way. It's not like the binary protocol where you can say "I have a collection of these other things that already exist".