Skip to content

RAT-532: Bump tika.version from 3.3.2 to 4.0.0 and migrate charset detection #714

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dependabot wants to merge 4 commits into
base: master
Choose a base branch
from
Open

RAT-532: Bump tika.version from 3.3.2 to 4.0.0 and migrate charset detection #714

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f1523bb
Bump tika.version from 3.3.2 to 4.0.0
dependabot[bot] Aug 24, 2026
af87ca1
RAT-532: Upgrade to Tika 4.0.0
ottlinger Aug 24, 2026
922cb2f
Log as warning if no encoding found
ottlinger Aug 27, 2026
5043b79
Add test for character detection for empty contents and 'invalid data'
ottlinger Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions apache-rat-core/src/it/resources/ReportTest/RAT_81/verify.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ NodeList nodeList = XmlUtils.getNodeList(doc, xPath, "/rat-report/resource[@name
assertEquals(1, nodeList.getLength())
node = nodeList.item(0)
attributes = node.getAttributes()
assertEquals("IBM500", attributes.getNamedItem("encoding").getNodeValue())
// pre-Tika4: recognized as IBM500 instead of IBM1047
assertEquals("IBM1047", attributes.getNamedItem("encoding").getNodeValue())
assertEquals("text/plain", attributes.getNamedItem("mediaType").getNodeValue())
assertEquals("STANDARD", attributes.getNamedItem("type").getNodeValue())
nodeList = XmlUtils.getNodeList(node, xPath, "license")
Expand All @@ -45,7 +46,7 @@ nodeList = XmlUtils.getNodeList(doc, xPath, "/rat-report/resource[@name='/UTF8.t
assertEquals(1, nodeList.getLength())
node = nodeList.item(0)
attributes = node.getAttributes()
assertEquals("ISO-8859-1", attributes.getNamedItem("encoding").getNodeValue())
assertEquals("windows-1252", attributes.getNamedItem("encoding").getNodeValue())
assertEquals("text/plain", attributes.getNamedItem("mediaType").getNodeValue())
assertEquals("STANDARD", attributes.getNamedItem("type").getNodeValue())
nodeList = XmlUtils.getNodeList(node, xPath, "license")
Expand Down
53 changes: 37 additions & 16 deletions apache-rat-core/src/main/java/org/apache/rat/analysis/TikaProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import java.nio.charset.Charset;
import java.nio.charset.UnsupportedCharsetException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import org.apache.rat.api.Document;
Expand All @@ -32,19 +33,29 @@
import org.apache.rat.document.guesser.NoteGuesser;
import org.apache.rat.utils.DefaultLog;
import org.apache.tika.Tika;
import org.apache.tika.detect.DefaultEncodingDetector;
import org.apache.tika.detect.EncodingDetector;
import org.apache.tika.detect.EncodingResult;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.txt.CharsetDetector;
import org.apache.tika.parser.txt.CharsetMatch;
import org.apache.tika.parser.ParseContext;

/**
* A wrapping around the Tika processor.
*/
public final class TikaProcessor {

/** The Tika parser */
/** The Tika parser. */
private static final Tika TIKA = new Tika();

/** The Tika encoding detector. */
private static final EncodingDetector ENCODING_DETECTOR = new DefaultEncodingDetector();

/** Due to performance reasons we do not read the whole file for charset detection (RAT-494). */
private static final int BYTES_FOR_CHARSET_DETECTION = 256;

/** A map of mime type string to non-BINARY types.
* "text" types are already handled somewhere else
* BINARY unless listed here
Expand Down Expand Up @@ -165,21 +176,31 @@ public static String process(final Document document) throws RatDocumentAnalysis
* @throws IOException on IO error.
* @throws UnsupportedCharsetException on unsupported charset.
*/
private static Charset detectCharset(final InputStream stream, final DocumentName documentName) throws IOException, UnsupportedCharsetException {
final int bytesForCharsetDetection = 256;
CharsetDetector encodingDetector = new CharsetDetector(bytesForCharsetDetection);
encodingDetector.setText(stream);
CharsetMatch charsetMatch = encodingDetector.detect();
if (charsetMatch != null) {
try {
return Charset.forName(charsetMatch.getName());
} catch (UnsupportedCharsetException e) {
DefaultLog.getInstance().warn(String.format("Unsupported character set '%s' in file '%s'",
charsetMatch.getName(), documentName));
throw e;
static Charset detectCharset(final InputStream stream, final DocumentName documentName) throws IOException, UnsupportedCharsetException {
stream.mark(BYTES_FOR_CHARSET_DETECTION);
try {
byte[] sample = stream.readNBytes(BYTES_FOR_CHARSET_DETECTION);
if (sample.length == 0) {
DefaultLog.getInstance().debug(String.format("No contents in file '%s'", documentName));
return null;
}

Metadata metadata = new Metadata();
ParseContext parseContext = new ParseContext();

try (TikaInputStream tis = TikaInputStream.get(sample, metadata)) {
List<EncodingResult> results = ENCODING_DETECTOR.detect(tis, metadata, parseContext);

if (results.isEmpty()) {
DefaultLog.getInstance().warn(String.format("No encoding found for file '%s'", documentName));
return null;
}
Comment on lines +194 to +197

@Claudenw Claudenw Aug 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code does not do the same thing. the debug should be a warning.

And what happend to unsupported character sets?

@ottlinger ottlinger Aug 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not have an explicit test for unsupported character sets. Tika handles this internally and returns no charset. If no charset is returned RAT will mark as UNKNOWN if I'm not too mistaken.

Do you have an example file that triggers this exception?

@ottlinger ottlinger Aug 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried adding "random bytes" but Tika still reports a probabilistic value and I'm unable to provide an input that yields an empty result in order to test RAT's behaviour.


return results.get(0).getCharset();
}
} finally {
stream.reset();
}
return null;
}

/**
Expand Down
24 changes: 15 additions & 9 deletions apache-rat-core/src/main/java/org/apache/rat/api/Document.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,14 @@

import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.nio.charset.Charset;
import java.util.SortedSet;

import org.apache.rat.analysis.TikaProcessor;
import org.apache.rat.document.DocumentName;
import org.apache.rat.document.DocumentNameMatcher;
import org.apache.tika.parser.txt.CharsetDetector;

/**
* The representation of a document being scanned.
Expand Down Expand Up @@ -104,19 +105,24 @@ public boolean equals(final Object obj) {
}

/**
* Reads the contents of this document.
* Reads the contents of this document and
* relies on the charset detection of the underlying Tika processor.
*
* @return <code>Reader</code> not null
* @throws IOException if this document cannot be read.
*/
public Reader reader() throws IOException {
final int bytesForCharsetDetection = 256;
CharsetDetector charsetDetector = new CharsetDetector(bytesForCharsetDetection);
// RAT-494: Tika's CharsetDetector.getReader() may return null if the read can not be constructed due to I/O or encoding errors
Reader result = charsetDetector.getReader(TikaProcessor.markSupportedInputStream(inputStream()), getMetaData().getCharset().name());
if (result == null) {
throw new IOException(String.format("Can not read document `%s`", getName()));
final Charset charset = getMetaData().getCharset();
if (charset == null) {
throw new IOException(
String.format(
"No charset detected for document `%s`",
getName()));
}
return result;

return new InputStreamReader(
TikaProcessor.markSupportedInputStream(inputStream()),
charset);
}

/**
Expand Down
4 changes: 2 additions & 2 deletions apache-rat-core/src/test/java/org/apache/rat/ReporterOptionsProvider.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -866,7 +866,7 @@ private void styleSheetTest(final Option option) {
TextUtils.assertContainsExactly(1, "?????: 1 ", actualText);
break;
case XML:
TextUtils.assertContainsExactly(1, "<resource encoding=\"ISO-8859-1\" mediaType=\"text/plain\" name=\"/stylesheet\" type=\"STANDARD\">", actualText);
TextUtils.assertContainsExactly(1, "<resource encoding=\"windows-1252\" mediaType=\"text/plain\" name=\"/stylesheet\" type=\"STANDARD\">", actualText);
break;
case UNAPPROVED_LICENSES:
TextUtils.assertContainsExactly(1, "Files with unapproved licenses:" + System.lineSeparator() + " /stylesheet", actualText);
Expand Down Expand Up @@ -928,7 +928,7 @@ protected void xmlTest() {
assertThat(output.getStatistic().getCounter(ClaimStatistic.Counter.UNAPPROVED)).isEqualTo(1);
output.format(config);
String actualText = baos.toString(StandardCharsets.UTF_8);
TextUtils.assertContainsExactly(1, "<resource encoding=\"ISO-8859-1\" mediaType=\"text/plain\" name=\"/stylesheet\" type=\"STANDARD\">", actualText);
TextUtils.assertContainsExactly(1, "<resource encoding=\"windows-1252\" mediaType=\"text/plain\" name=\"/stylesheet\" type=\"STANDARD\">", actualText);

try (InputStream expected = StyleSheets.getStyleSheet("xml").ioSupplier().get();
InputStream actual = config.getStyleSheet().get()) {
Expand Down
22 changes: 11 additions & 11 deletions apache-rat-core/src/test/java/org/apache/rat/ReporterTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,30 +207,30 @@ void testXMLOutput() throws Exception {
Map<String, Map<String, String>> expected = new HashMap<>();
expected.put("/.hiddenDirectory", mapOf("isDirectory", "true", "mediaType", "application/octet-stream",
"type", "IGNORED"));
expected.put("/ILoggerFactory.java", mapOf("encoding", "ISO-8859-1", "mediaType", "text/x-java-source",
expected.put("/ILoggerFactory.java", mapOf("encoding", "windows-1252", "mediaType", "text/x-java-source",
"type", "STANDARD"));
expected.put("/Image.png", mapOf("mediaType", "image/png", "type", "BINARY"));
expected.put("/LICENSE", mapOf("encoding", "ISO-8859-1", "mediaType", "text/plain", "type", "NOTICE"));
expected.put("/NOTICE", mapOf("encoding", "ISO-8859-1", "mediaType", "text/plain", "type", "NOTICE"));
expected.put("/Source.java", mapOf("encoding", "ISO-8859-1", "mediaType", "text/x-java-source",
expected.put("/LICENSE", mapOf("encoding", "windows-1252", "mediaType", "text/plain", "type", "NOTICE"));
expected.put("/NOTICE", mapOf("encoding", "windows-1252", "mediaType", "text/plain", "type", "NOTICE"));
expected.put("/Source.java", mapOf("encoding", "windows-1252", "mediaType", "text/x-java-source",
"type", "STANDARD"));
expected.put("/Text.txt", mapOf("encoding", "ISO-8859-1", "mediaType", "text/plain",
expected.put("/Text.txt", mapOf("encoding", "windows-1252", "mediaType", "text/plain",
"type", "STANDARD"));
expected.put("/TextHttps.txt", mapOf("encoding", "ISO-8859-1", "mediaType", "text/plain",
expected.put("/TextHttps.txt", mapOf("encoding", "windows-1252", "mediaType", "text/plain",
"type", "STANDARD"));
expected.put("/Xml.xml", mapOf("encoding", "ISO-8859-1", "mediaType", "application/xml",
expected.put("/Xml.xml", mapOf("encoding", "windows-1252", "mediaType", "application/xml",
"type", "STANDARD"));
expected.put("/buildr.rb", mapOf("encoding", "ISO-8859-1", "mediaType", "text/x-ruby",
expected.put("/buildr.rb", mapOf("encoding", "windows-1252", "mediaType", "text/x-ruby",
"type", "STANDARD"));
expected.put("/dummy.jar", mapOf("mediaType", "application/java-archive",
"type", "ARCHIVE"));
expected.put("/generated.txt", mapOf("encoding", "ISO-8859-1", "mediaType", "text/plain",
expected.put("/generated.txt", mapOf("encoding", "windows-1252", "mediaType", "text/plain",
"type", "IGNORED"));
expected.put("/plain.json", mapOf("mediaType", "application/json",
"type", "BINARY"));
expected.put("/sub/Empty.txt", mapOf("encoding", "UTF-8", "mediaType", "text/plain",
expected.put("/sub/Empty.txt", mapOf("encoding", "windows-1252", "mediaType", "text/plain",
"type", "STANDARD"));
expected.put("/tri.txt", mapOf("encoding", "ISO-8859-1", "mediaType", "text/plain",
expected.put("/tri.txt", mapOf("encoding", "windows-1252", "mediaType", "text/plain",
"type", "STANDARD"));

File output = new File(tempDirectory, ".rat/testXMLOutput");
Expand Down
37 changes: 28 additions & 9 deletions apache-rat-core/src/test/java/org/apache/rat/analysis/TikaProcessorTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,13 @@
import org.apache.rat.document.DocumentName;
import org.junit.jupiter.api.Test;

import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.nio.charset.Charset;
import java.nio.charset.MalformedInputException;
import java.nio.charset.StandardCharsets;
import java.util.Collections;
Expand All @@ -40,6 +42,7 @@
import java.util.Objects;
import java.util.SortedSet;

import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;

Expand All @@ -52,7 +55,7 @@ public class TikaProcessorTest {
* @see <a href="https://issues.apache.org/jira/browse/RAT-81">RAT-81</a>
*/
@Test
public void RAT81() {
void RAT81() {
// create a document that throws a MalformedInputException
Document doc = mkDocument(new InputStream() {
@Override
Expand All @@ -64,7 +67,7 @@ public int read() throws IOException {
}

@Test
public void UTF16_input() throws Exception {
void UTF16_input() throws Exception {
Document doc = mkDocument(Resources.getResourceStream("/binaries/UTF16_with_signature.xml"),
DocumentNameMatcher.MATCHES_ALL);
TikaProcessor.process(doc);
Expand All @@ -80,48 +83,48 @@ private FileDocument mkDocument(String fileName) throws IOException {
}

@Test
public void UTF8_input() throws Exception {
void UTF8_input() throws Exception {
FileDocument doc = mkDocument("/binaries/UTF8_with_signature.xml");
TikaProcessor.process(doc);
assertEquals(Document.Type.STANDARD, doc.getMetaData().getDocumentType());
}

@Test
public void RAT178Test() {
void RAT178Test() {
FileDocument doc = new FileDocument(new File("/not_a_real_file"), DocumentNameMatcher.MATCHES_ALL);
assertThrows(RatDocumentAnalysisException.class, () ->TikaProcessor.process(doc));
}

@Test
public void missNamedBinaryTest() throws Exception {
void missNamedBinaryTest() throws Exception {
FileDocument doc = mkDocument("/binaries/Image-png.not");
TikaProcessor.process(doc);
assertEquals(Document.Type.BINARY, doc.getMetaData().getDocumentType());
}

@Test
public void plainTextTest() throws Exception {
void plainTextTest() throws Exception {
FileDocument doc = mkDocument(Resources.getExampleResource("exampleData/Text.txt"));
TikaProcessor.process(doc);
assertEquals(Document.Type.STANDARD, doc.getMetaData().getDocumentType());
}

@Test
public void emptyFileTest() throws Exception {
void emptyFileTest() throws Exception {
FileDocument doc = mkDocument(Resources.getExampleResource("exampleData/sub/Empty.txt"));
TikaProcessor.process(doc);
assertEquals(Document.Type.STANDARD, doc.getMetaData().getDocumentType());
}

@Test
public void javaFileWithChineseCharacters_RAT301() throws Exception {
void javaFileWithChineseCharacters_RAT301() throws Exception {
FileDocument doc = mkDocument("/tikaFiles/standard/ChineseCommentsJava.java");
TikaProcessor.process(doc);
assertEquals(Document.Type.STANDARD, doc.getMetaData().getDocumentType());
}

@Test
public void testTikaFiles() throws RatDocumentAnalysisException {
void testTikaFiles() throws RatDocumentAnalysisException {
File dir = new File("src/test/resources/tikaFiles");
Map<String, Document.Type> unseenMime = TikaProcessor.getDocumentTypeMap();
ClaimStatistic statistic = new ClaimStatistic();
Expand All @@ -144,6 +147,22 @@ public void testTikaFiles() throws RatDocumentAnalysisException {
}
}

@Test
void testDetectionOfInvalidData() throws IOException {
byte[] invalidData = new byte[] {
0x00, (byte) 0xFF, 0x00, (byte) 0xFE,
0x01, (byte) 0x80, 0x00, 0x7F
};
// as Tika works with a probabilistic encoding detection it does not return NO encoding
assertThat(TikaProcessor.detectCharset(new ByteArrayInputStream(invalidData), null)).isEqualTo(Charset.forName("Windows-1258"));
}

@Test
void testEmptyFileEncoding() throws IOException {
byte[] empty = {};
assertThat(TikaProcessor.detectCharset(new ByteArrayInputStream(empty), null)).isNull();
}

/**
* Build a document with the specific input stream
* @return a document with the specific input stream
Expand Down
2 changes: 1 addition & 1 deletion apache-rat-plugin/src/it/it1/verify.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Document document = XmlUtils.toDom(new FileInputStream(f))
XPath xPath = XPathFactory.newInstance().newXPath()

XmlUtils.assertAttributes(document, xPath, "/rat-report/resource[@name='/src.apt']",
mapOf("encoding", "ISO-8859-1", "mediaType", "text/plain", "type", "STANDARD" ))
mapOf("encoding", "windows-1252", "mediaType", "text/plain", "type", "STANDARD" ))

XmlUtils.assertAttributes(document, xPath, "/rat-report/resource[@name='/src.apt']/license[@id='MyLicense']",
mapOf("approval", "true", "family", "YAL ", "name", "Yet another license" ))
Expand Down
4 changes: 2 additions & 2 deletions apache-rat-plugin/src/test/java/org/apache/rat/mp/RatCheckMojoTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ void it1() throws Exception {
mapOf("mediaType", "application/octet-stream", "type", "IGNORED"));

XmlUtils.assertAttributes(document, xPath, "/rat-report/resource[@name='/pom.xml']",
mapOf("mediaType", "application/xml", "type", "STANDARD", "encoding", "ISO-8859-1"));
mapOf("mediaType", "application/xml", "type", "STANDARD", "encoding", "windows-1252"));
}

private static Map<String, String> mapOf(String... parts) {
Expand Down Expand Up @@ -291,7 +291,7 @@ void it5() throws Exception {
XmlUtils.assertAttributes(document, xPath, "/rat-report/resource[@name='/pom.xml']",
"mediaType", "application/xml", "type", "IGNORED", "isDirectory", "false");
XmlUtils.assertAttributes(document, xPath, "/rat-report/resource[@name='/src/main/java/nl/basjes/something/Something.java']",
"mediaType", "text/x-java-source", "type", "STANDARD", "encoding", "ISO-8859-1");
"mediaType", "text/x-java-source", "type", "STANDARD", "encoding", "windows-1252");
XmlUtils.assertAttributes(document, xPath, "/rat-report/resource[@name='/src/main/java/nl/basjes/something/Something.java']/license",
"approval", "true", "family", ILicenseFamily.makeCategory("CC"), "id", "CC-BY-NC-ND", "name",
"Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International");
Expand Down
2 changes: 1 addition & 1 deletion apache-rat-tasks/src/test/resources/antunit/report-normal-operation.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ SPDX-License-Identifier: Apache-2.0
</path>
</pathconvert>
<property name="expectedOutputXML"
value='&lt;resource encoding="ISO-8859-1" mediaType="application/xml" name="/report-normal-operation.xml" type="STANDARD"' />
value='&lt;resource encoding="windows-1252" mediaType="application/xml" name="/report-normal-operation.xml" type="STANDARD"' />
<property name="expectedOutputXML2" value='family="AL "' />
</target>

Expand Down
2 changes: 1 addition & 1 deletion pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ agnostic home for software distribution comprehension and audit tools.
<assertj.version>4.0.0-M1</assertj.version>
<!-- this is the target Java version that this project supports at runtime -->
<javaVersion>17</javaVersion>
<tika.version>3.3.2</tika.version>
<tika.version>4.0.0</tika.version>
<mockito.version>5.23.0</mockito.version>
<!-- This is the version of Maven required to use the RAT Maven Plugin -->
<mavenMinVersion>3.9</mavenMinVersion>
Expand Down
3 changes: 3 additions & 0 deletions src/changes/changes.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,9 @@ in order to be properly linked in site reports.
</release>
-->
<release version="1.0.0-SNAPSHOT" date="xxxx-yy-zz" description="Current SNAPSHOT - release to be done">
<action issue="RAT-532" type="add" dev="pottlinger">
Update to Tika 4.0.0: new charset detection logic in Tika returns different values compared to 3.x before, such as windows-1252 instead of ISO-8859-1.

@Claudenw Claudenw Aug 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is an error. We have some windows-1252 files but most are ISO-8859-1

I think for our purpose we can label windows-1252 as ISO-8859-1. I need to check the list that is returned from the new Tika and see if it includes ISO-8859-1 as one of the encodings. I think we should select ISO over windows when we have the option. This PR needs investigation and work.

@ottlinger ottlinger Aug 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Tika4 logics is to return the "best" charset. In contrast to version 3.x this changed into windows-1252. The new implementation returns the first hit. Personally I wouldn't want to introduce new logics on the RAT-side to generalise into ISO-8859-1 and would take the change as tika-induced and document it in our changelog.

@ottlinger ottlinger Aug 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity I added some logging locally in TikaProcessor to just list the encodings:

[INFO] Running org.apache.rat.analysis.AnalyserFactoryTest
INFO: >>> /jira/RAT147/windows-newlines.txt.bin has encodings: [UTF-8@1.00[DECLARATIVE]]
INFO: >>> /Users/me/creadur-rat/apache-rat-core/target/test-classes/examples/exampleData/Text.txt has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /Users/me/creadur-rat/apache-rat-core/target/test-classes/examples/exampleData/sub/Empty.txt has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /jira/RAT147/unix-newlines.txt.bin has encodings: [UTF-8@1.00[DECLARATIVE]]
INFO: >>> /Users/me/creadur-rat/apache-rat-core/target/test-classes/examples/exampleData/Text.txt has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /Users/me/creadur-rat/apache-rat-core/target/test-classes/examples/exampleData/sub/Empty.txt has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /Users/me/creadur-rat/apache-rat-core/target/test-classes/examples/exampleData/Text.txt has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /Users/me/creadur-rat/apache-rat-core/target/test-classes/examples/exampleData/LICENSE has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /Users/me/creadur-rat/apache-rat-core/target/test-classes/examples/exampleData/Text.txt has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /Users/me/creadur-rat/apache-rat-core/target/test-classes/examples/exampleData/sub/Empty.txt has encodings: [windows-1252@0.10[STATISTICAL]]

....

[INFO] Running org.apache.rat.report.xml.XmlReportFactoryTest
INFO: >>> /ILoggerFactory.java has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /LICENSE has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /NOTICE has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /Source.java has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /Text.txt has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /TextHttps.txt has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /Xml.xml has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /buildr.rb has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /generated.txt has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /sub/Empty.txt has encodings: [windows-1252@0.10[STATISTICAL]]
INFO: >>> /tri.txt has encodings: [windows-1252@0.10[STATISTICAL]]

It seems that ISO is not recognized anymore with Tika 4.x.

</action>
<action issue="RAT-553" type="add" dev="pottlinger" due-to="Guillaume Nodet">
Fix NPE with parallel builds in SCM ignore parsers.
</action>
Expand Down
Loading