[PPC0036] Add document about Viral Value Magic - #90
Conversation
|
|
||
| * When calculating a new value as a result of one *or more* input values that have these annotations attached, the `infect` function on each annotation of each such value is invoked to copy it into the resulting destination. It may be that the result ends up with more than one annotation attached to it. | ||
|
|
||
| * When a container variable is `local`ised, magic annotations of this shape are not copied into the new temporary container. |
There was a problem hiding this comment.
this behavior doesn't match with what i'd have in my head - i'd imagine that unless mutated, a local-ised copy would continue being the same thing.
perhaps our mental models are not aligned - could you spell out specifically why you want this behavior?
There was a problem hiding this comment.
We may need to have a sharper distinction between value and container magic before trying to make value magic viral.
There was a problem hiding this comment.
When you local a variable, the internals of how it works means that you get a new SV that gets moved into place for where that variable was visible. This means that magics that ought to be considered part of the "container" (things like the special behaviours of the various dollar-punctuation variables) needs to be copied onto that new SV as well, so that "the SV visible as that variable" retains that behaviour.
Whereas, any behaviours that are part of the value currently stored in the SV should not get copied into that new one; they belong with the value rather than the variable that contains it.
There was a problem hiding this comment.
We may need to have a sharper distinction between value and container magic before trying to make value magic viral.
Perhaps so. I try to make more of a distinction of that in Magic-v2 (see the corresponding PPC0035). With v1 you basically just distinguish by being PERL_MAGIC_ext vs PERL_MAGIC_extvalue and that has effects on localisation, whereas magic-v2 has a specific MGv2f_CONTAINER flag to properly say "this relates to the container, not the value stored in it".
There was a problem hiding this comment.
I think here an example would paint a thousand words.
There was a problem hiding this comment.
Ah so in actual fact it turns out that literally today, I reworked the localisation stuff anyway. Rather than the "container" flag, I got rid of that in favour of having a localise trigger function on the (scalar|array|hash) variable magic tables. By default it does nothing, so the magic disappears on localisation (i.e. matches the PERL_MAGIC_extvalue behaviour), but if present that is invoked so you can do something different. A handy function is also provided to put in there directly if you just want to copy the magic, so if you set that it's basically like having PERL_MAGIC_ext behaviour.
|
|
||
| struct ScalarValueMagicFunctions { | ||
| ..., /* existing fields, where shape == MGv2s_SCALARVALUE */ | ||
| void (*infect)(pTHX_ SV *ssv, MAGIC *smg, SV *dsv, MAGIC *dmg); |
There was a problem hiding this comment.
I'm sure we can find a better name than that.
There was a problem hiding this comment.
Likely so; this was just a unique name that came to mind that would be pretty easy to search-replace later on.
There was a problem hiding this comment.
Maybe propagate is better? It's a slight overlap with the thing that exception objects do on rethrow...
|
|
||
| Normally, this section is used to explain around any potential security *problems* that might be introduced by the changes proposed by such a document. As is typical when considering adding new low-level abilities that modules can build on top of, the answer is usually "none foreseen, they would have to be considered by the modules being built". | ||
|
|
||
| But a much better answer here is to observe that this proposed mechanism adds a great ability to introduce new *solutions* to security problems. As perhaps was already hinted at in the introduction, the mechanism proposed here acts similar to but is more flexible and powerful than perl's existing "taint" behaviour, which was primarily intended for its security-related purposes. In practice, that mode is seldom used, because of its lack of flexibility. Taint mode just stores a single boolean annotation about any particular value ("is this considered tainted?"), which affects certain entirely-internal functions and operations. In contrast, this proposal would easily permit the creation of a more flexible set of policies; see the "Supertaint" example below. |
There was a problem hiding this comment.
I think it's very different from taint, I'm not sure it can actually replace that logic; taint works much more indirect that this would.
|
Strong support for this proposal! I am currently working on building modules to elevate Perl as a first-class participant in the Google Cloud Platform (GCP) space. For modern enterprise cloud customers, automatic Data Lineage and observability are critical requirements, driven centrally by GCP Dataplex. The "Viral Value Magic" concept is exactly what Perl needs to stay competitive in this space. Currently, tracking data moving through a Perl script (e.g., pulling from BigQuery, manipulating it, and uploading to Cloud Storage) requires verbose, intrusive instrumentation. Having this automatic propagation built directly into the core interpreter would allow Perl to natively produce high-fidelity OpenLineage traces (including column-level lineage) effortlessly. These traces could be pushed directly to the Dataplex Data Lineage API, making Perl significantly more viable for modern, cloud-native data pipelines alongside peers like Python or Java. Since the implementation is essentially ready, I strongly urge the inclusion of this PPC. |
5 participants
No description provided.