Fix compiler support for additional CPAN suites

dev/modules/want.md

@@ -229,15 +229,26 @@ The port is "done enough" when:
 
 ## Progress Tracking
 
-### Current Status: scoping / design
+### Current Status: JSONP compatibility subset implemented; A1 remains planned
 
 ### Completed Steps
-- [ ] Design doc reviewed
+- [x] Design doc reviewed (2026-07-29)
+- [x] Added a focused pure-Perl compatibility subset for JSONP (2026-07-29)
+  - Implements `OBJECT`, `RVALUE`, `BOOL`, scalar/list/void queries, and
+    conservative non-lvalue defaults.
+  - File: `src/main/perl/lib/Want.pm`
 - [ ] A1 proof-of-concept on a feature branch
 - [ ] Class::Accessor::Lvalue passes
 - [ ] ActiveResource passes (sans network I/O)
 - [ ] Want.pm shim merged
 
+### Next Steps
+
+1. Add the per-call-frame lvalue context described in A1.
+2. Replace the conservative `OBJECT`/`RVALUE` answers with compiler-provided
+   call-site metadata.
+3. Add `want_basics.t` and validate Class::Accessor::Lvalue.
+
 ### Open Questions
 - A1 vs A2 vs B: confirm A1 (runtime hook + Perl shim) is the right
   starting point.

docs/about/changelog.md

@@ -2,6 +2,28 @@
 
 Release history of PerlOnJava. See [Roadmap](roadmap.md) for future plans.
 
+## Work in progress
+
+- Add compatibility modules for `Socket6`, `Email::Address::XS`, and the
+  JSONP-used subset of `Want`; add a Java `Net::Gen` XS bridge for Net-ext.
+- Bugfix: subroutine return values are rvalue copies instead of aliases to
+  reusable scalar containers.
+- Bugfix: interpreter `map` now inherits the caller's scalar/list context.
+- Bugfix: valid UTF-8 octets in interpolated source strings compile without
+  `use utf8` while retaining byte-string semantics.
+- Bugfix: inherited AutoSplit forward declarations now participate in method
+  resolution, allowing parent `.al` methods to load before child `AUTOLOAD`
+  fallbacks.
+- Bugfix: nested typeglob hash/code dereferences and numeric or byte-string
+  `AUTOLOAD` method names retain their Perl symbol-table semantics.
+- Bugfix: IPv6 bind/listen/send/receive, socket-name packing, and
+  `getnameinfo` work through Java NIO; UDP sends to wildcard local addresses
+  are routed consistently.
+- Bugfix: generated `jperl` shebang commands work in piped opens without being
+  misclassified as shell syntax.
+- CPAN: `HTML::Diff`, `Graph::PetriNet`, `MIME::Lite`,
+  `File::Path::Tiny`, `Net::UDP`, `IO::Socket::INET6`,
+  `Email::Sender`, and `JSONP` pass their installation test suites.
 
 ## v5.44.0: Named Parameters in Signatures
 

docs/reference/bundled-modules.md

@@ -7,6 +7,15 @@ Additional pure-Perl modules can be installed from CPAN with
 This page lists every bundled module, grouped by category, and documents
 modules that have **external requirements** or **special instructions**.
 
+Recent CPAN compatibility additions include:
+
+| Module | Implementation | Notes |
+|--------|----------------|-------|
+| `Socket6` | Pure Perl over Java-backed `Socket` | Legacy Socket6 flat-result API and IPv6 sockaddr helpers |
+| `Email::Address::XS` | Pure Perl compatibility subset | Address parsing/object API used by `Email::Sender` |
+| `Want` | Pure Perl compatibility subset | Non-lvalue predicates used by `JSONP`; full op-tree introspection is not yet available |
+| `Net::Gen` | Java XS bridge | Net-ext constants and sockaddr helpers used by `Net::UDP` |
+
 ---
 
 ## Modules with External Requirements

docs/reference/feature-matrix.md

@@ -710,6 +710,8 @@ The `:encoding()` layer supports all encodings provided by Java's `Charset.forNa
 - ✅  **IO::Socket** module.
 - ✅  **IO::Socket::INET** module.
 - ✅  **IO::Socket::UNIX** module.
+- ✅  **Socket6** compatibility module backed by the core `Socket` IPv6 implementation.
+- ✅  **Net::Gen** XS compatibility bridge for the Net-ext socket modules.
 - ✅  **IO::Zlib** module.
 - ✅  **List::Util**: module.
 - ✅  **MIME::Base64** module
@@ -732,7 +734,12 @@ The `:encoding()` layer supports all encodings provided by Java's `Charset.forNa
 - ✅  **Time::Local** module.
 - ✅  **UNIVERSAL**: `isa`, `can`, `DOES`, `VERSION` are implemented. `isa` operator is implemented.
 - ✅  **URI::Escape** module.
-- ✅  **Socket** module: socket constants and functions (`pack_sockaddr_in`, `unpack_sockaddr_in`, `sockaddr_in`, `inet_aton`, `inet_ntoa`, `gethostbyname`).
+- ✅  **Socket** module: IPv4/IPv6 socket constants and functions, including
+  sockaddr packing, address presentation conversion, `getaddrinfo`, and
+  `getnameinfo`.
+- ⚠️  **Want** compatibility subset: scalar/list/void and the non-lvalue
+  predicates needed by JSONP; full lvalue/op-tree introspection remains planned.
+- ✅  **Email::Address::XS** compatibility subset used by Email::Sender.
 - ✅  **Unicode::UCD** module.
 - ✅  **XSLoader** module.
 - 🚧  **DynaLoader** placeholder module.

src/main/java/org/perlonjava/backend/bytecode/BytecodeCompiler.java

@@ -2682,7 +2682,18 @@ void handleGeneralArrayAccess(BinaryOperatorNode node) {
      */
     void handleGeneralHashAccess(BinaryOperatorNode node) {
         // Compile the left side (the expression that should yield a hash or hashref)
-        node.left.accept(this);
+        // `${EXPR}{key}` applies the hash subscript directly to EXPR. It must
+        // not first compile `${EXPR}` as an independent scalar dereference:
+        // when EXPR is a typeglob, that would select its SCALAR slot instead
+        // of the HASH slot. This mirrors the JVM backend's `${BLOCK}{}` path.
+        if (node.left instanceof OperatorNode sigilNode
+                && sigilNode.operator.equals("$")
+                && sigilNode.operand instanceof BlockNode block
+                && block.elements.size() == 1) {
+            compileNode(block.elements.getFirst(), -1, RuntimeContextType.SCALAR);
+        } else {
+            node.left.accept(this);
+        }
         int baseReg = lastResultReg;
 
         // Compile the key expression (right side)

src/main/java/org/perlonjava/backend/bytecode/CompileBinaryOperatorHelper.java

@@ -284,7 +284,7 @@ private static int compileBinaryOperatorSwitch(BytecodeCompiler bytecodeCompiler
                 bytecodeCompiler.emitReg(rd);
                 bytecodeCompiler.emitReg(rs2);       // List register
                 bytecodeCompiler.emitReg(rs1);       // Closure register
-                bytecodeCompiler.emit(RuntimeContextType.LIST);  // Map always uses list context
+                bytecodeCompiler.emit(bytecodeCompiler.currentCallContext);
             }
             case "grep" -> {
                 // Grep operator: grep { block } list

src/main/java/org/perlonjava/frontend/parser/CoreOperatorResolver.java

@@ -130,7 +130,11 @@ public static Node parseCoreOperator(Parser parser, LexerToken token, int startI
             case "method" -> parseAnonymousMethodExpression(parser, startIndex);
             case "q", "qq", "qx", "qw", "qr", "tr", "y", "s", "m" ->
                     OperatorParser.parseSpecialQuoted(parser, token, startIndex);
-            case "dump", "dbmclose", "dbmopen" ->
+            // CORE::dump is an obsolete process/core-dump primitive. Parse it
+            // as a harmless false value so legacy modules can compile guarded
+            // diagnostics without terminating the JVM.
+            case "dump" -> new NumberNode("0", parser.tokenIndex);
+            case "dbmclose", "dbmopen" ->
                     throw new PerlJavaUnimplementedException(parser.tokenIndex, "Not implemented: operator: " + token.text, parser.ctx.errorUtil);
             case "format" ->
                 // Format statements should be handled by StatementResolver, not as operators

src/main/java/org/perlonjava/frontend/parser/StringDoubleQuoted.java

@@ -146,6 +146,29 @@ static Node parseDoubleQuotedString(EmitterContext ctx, StringParser.ParsedStrin
         var isRegex = !parseEscapes;
         if (CompilerOptions.DEBUG_ENABLED) ctx.logDebug("parseDoubleQuotedString isRegex:" + isRegex);
 
+        // StringParser represents non-utf8 source text as UTF-8 octets stored in
+        // Java chars. Decode those octets while tokenizing interpolation syntax;
+        // otherwise continuation bytes (for example 0x80 in a curly quote) are
+        // interpreted as source tokens and rejected. Literal segments are encoded
+        // back to octets by StringSegmentParser.flushCurrentSegment().
+        if (!ctx.symbolTable.isStrictOptionEnabled(org.perlonjava.runtime.perlmodule.Strict.HINT_UTF8)
+                && !ctx.compilerOptions.isUnicodeSource
+                && !ctx.compilerOptions.isByteStringSource) {
+            byte[] octets = new byte[input.length()];
+            boolean allOctets = true;
+            for (int i = 0; i < input.length(); i++) {
+                char ch = input.charAt(i);
+                if (ch > 0xff) {
+                    allOctets = false;
+                    break;
+                }
+                octets[i] = (byte) ch;
+            }
+            if (allOctets) {
+                input = new String(octets, java.nio.charset.StandardCharsets.UTF_8);
+            }
+        }
+
         // Tokenize the string content
         var lexer = new Lexer(input);
         var tokens = lexer.tokenize();

src/main/java/org/perlonjava/frontend/parser/StringSegmentParser.java

@@ -192,6 +192,17 @@ protected void addStringSegment(Node node) {
     protected void flushCurrentSegment() {
         if (!currentSegment.isEmpty()) {
             String value = currentSegment.toString();
+            if (currentSegmentHasSourceNonAscii
+                    && !ctx.symbolTable.isStrictOptionEnabled(HINT_UTF8)
+                    && !ctx.compilerOptions.isUnicodeSource
+                    && !ctx.compilerOptions.isByteStringSource) {
+                byte[] utf8 = value.getBytes(java.nio.charset.StandardCharsets.UTF_8);
+                StringBuilder octets = new StringBuilder(utf8.length);
+                for (byte b : utf8) {
+                    octets.append((char) (b & 0xff));
+                }
+                value = octets.toString();
+            }
             boolean forceByteString = shouldForceByteStringLiteral(value);
             addStringSegment(new StringNode(value, false, forceByteString, tokenIndex));
             currentSegment.setLength(0);

src/main/java/org/perlonjava/frontend/parser/Variable.java

@@ -1018,6 +1018,18 @@ public static Node parseBracedVariable(Parser parser, String sigil, boolean isSt
             bracedVarName = IdentifierParser.parseComplexIdentifierInner(parser, true);
         }
 
+        if (bracedVarName != null) {
+            // In a dereference such as `%{ $ {*$glob}{Keys} }`, the `$`
+            // starts a nested scalar expression; it is not the name of a
+            // global hash `%$`. Fall back to block parsing so the glob-slot
+            // expression remains intact.
+            if ((sigil.equals("%") || sigil.equals("@") || sigil.equals("&"))
+                    && bracedVarName.equals("$")) {
+                bracedVarName = null;
+                parser.tokenIndex = savedIndex;
+            }
+        }
+
         if (bracedVarName != null) {
             // Check if this might be ambiguous with an operator
             boolean isAmbiguous = isAmbiguousOperatorName(bracedVarName);

src/main/java/org/perlonjava/runtime/io/PipeInputChannel.java

@@ -5,6 +5,7 @@
 import org.perlonjava.runtime.runtimetypes.RuntimeIO;
 import org.perlonjava.runtime.runtimetypes.RuntimeScalar;
 import org.perlonjava.runtime.runtimetypes.RuntimeScalarCache;
+import org.perlonjava.runtime.operators.SystemOperator;
 
 import java.io.*;
 import java.nio.charset.Charset;
@@ -33,7 +34,7 @@ public class PipeInputChannel implements IOHandle {
     /**
      * Pattern to detect shell metacharacters
      */
-    private static final Pattern SHELL_METACHARACTERS = Pattern.compile("[*?\\[\\]{}()<>|&;`'\"\\\\$\\s]");
+    private static final Pattern SHELL_METACHARACTERS = Pattern.compile("[*?\\[\\]{}()<>|&;`'\"\\\\$]");
 
     /**
      * The external process
@@ -237,7 +238,8 @@ private void startProcess(String command) throws IOException {
      * @throws IOException if an I/O error occurs starting the process
      */
     private void startProcessDirect(List<String> commandArgs) throws IOException {
-        ProcessBuilder processBuilder = new ProcessBuilder(commandArgs);
+        ProcessBuilder processBuilder = new ProcessBuilder(
+                SystemOperator.resolveCommandForProcessBuilder(commandArgs));
         setupProcess(processBuilder);
     }
 

src/main/java/org/perlonjava/runtime/io/PipeOutputChannel.java

@@ -5,6 +5,7 @@
 import org.perlonjava.runtime.runtimetypes.RuntimeIO;
 import org.perlonjava.runtime.runtimetypes.RuntimeScalar;
 import org.perlonjava.runtime.runtimetypes.RuntimeScalarCache;
+import org.perlonjava.runtime.operators.SystemOperator;
 
 import java.io.*;
 import java.nio.charset.Charset;
@@ -52,7 +53,7 @@ public class PipeOutputChannel implements IOHandle {
     /**
      * Pattern to detect shell metacharacters
      */
-    private static final Pattern SHELL_METACHARACTERS = Pattern.compile("[*?\\[\\]{}()<>|&;`'\"\\\\$\\s]");
+    private static final Pattern SHELL_METACHARACTERS = Pattern.compile("[*?\\[\\]{}()<>|&;`'\"\\\\$]");
 
     /**
      * The external process
@@ -139,7 +140,8 @@ private void startProcess(String command) throws IOException {
      * @throws IOException if an I/O error occurs starting the process
      */
     private void startProcessDirect(List<String> commandArgs) throws IOException {
-        ProcessBuilder processBuilder = new ProcessBuilder(commandArgs);
+        ProcessBuilder processBuilder = new ProcessBuilder(
+                SystemOperator.resolveCommandForProcessBuilder(commandArgs));
         setupProcess(processBuilder);
     }
 

src/main/java/org/perlonjava/runtime/io/SocketIO.java

@@ -517,7 +517,9 @@ public RuntimeScalar listen(int backlog) {
                 }
 
                 // Create a new ServerSocketChannel and bind to the same address
-                serverSocketChannel = ServerSocketChannel.open();
+                serverSocketChannel = protocolFamily != null
+                        ? ServerSocketChannel.open(protocolFamily)
+                        : ServerSocketChannel.open();
                 // Transfer non-blocking mode from the original socket
                 if (!blocking) {
                     serverSocketChannel.configureBlocking(false);
@@ -1045,26 +1047,35 @@ public RuntimeScalar getpeername() {
      */
     private RuntimeScalar packSockaddrIn(InetSocketAddress address) {
         try {
-            byte[] sockaddr = new byte[16];
+            byte[] ipBytes = address.getAddress().getAddress();
+            if (ipBytes.length == 16) {
+                byte[] sockaddr = new byte[28];
+                sockaddr[0] = 0;
+                sockaddr[1] = (byte) org.perlonjava.runtime.perlmodule.Socket.AF_INET6;
+
+                int port = address.getPort();
+                sockaddr[2] = (byte) ((port >> 8) & 0xFF);
+                sockaddr[3] = (byte) (port & 0xFF);
+                System.arraycopy(ipBytes, 0, sockaddr, 8, 16);
+
+                int scopeId = address.getAddress() instanceof Inet6Address inet6
+                        ? inet6.getScopeId()
+                        : 0;
+                sockaddr[24] = (byte) ((scopeId >> 24) & 0xFF);
+                sockaddr[25] = (byte) ((scopeId >> 16) & 0xFF);
+                sockaddr[26] = (byte) ((scopeId >> 8) & 0xFF);
+                sockaddr[27] = (byte) (scopeId & 0xFF);
+                return new RuntimeScalar(new String(sockaddr, StandardCharsets.ISO_8859_1));
+            }
 
-            // Family: AF_INET = 2 (network byte order)
+            byte[] sockaddr = new byte[16];
             sockaddr[0] = 0;
-            sockaddr[1] = 2;
-
-            // Port (network byte order - big endian)
+            sockaddr[1] = (byte) org.perlonjava.runtime.perlmodule.Socket.AF_INET;
             int port = address.getPort();
             sockaddr[2] = (byte) ((port >> 8) & 0xFF);
             sockaddr[3] = (byte) (port & 0xFF);
-
-            // IP address (4 bytes)
-            byte[] ipBytes = address.getAddress().getAddress();
             System.arraycopy(ipBytes, 0, sockaddr, 4, 4);
 
-            // Padding (8 bytes of zeros)
-            for (int i = 8; i < 16; i++) {
-                sockaddr[i] = 0;
-            }
-
             return new RuntimeScalar(new String(sockaddr, StandardCharsets.ISO_8859_1));
         } catch (Exception e) {
             return scalarUndef;

src/main/java/org/perlonjava/runtime/mro/InheritanceResolver.java

@@ -433,7 +433,16 @@ public static RuntimeScalar findMethodInHierarchy(String methodName, String perl
 
                 if (GlobalVariable.existsGlobalCodeRef(normalizedClassMethodName)) {
                     RuntimeScalar codeRef = GlobalVariable.getGlobalCodeRef(normalizedClassMethodName);
-                    if (!RuntimeCode.isCodeDefined(codeRef)) {
+                    // A forward declaration is a real method-table entry even
+                    // before it has a body. AutoSplit relies on this: the stub
+                    // in a parent class must win method lookup so invoking it
+                    // reaches that package's AUTOLOAD and corresponding .al
+                    // file. Ignore only incidental undefined code slots that
+                    // were never declared.
+                    boolean isDeclaredForward = codeRef != null
+                            && codeRef.value instanceof RuntimeCode code
+                            && code.isDeclared;
+                    if (!RuntimeCode.isCodeDefined(codeRef) && !isDeclaredForward) {
                         continue;
                     }
                     cacheMethod(cacheKey, codeRef);